Version Changelog
Bug fixes
Fix an issue where the terms file could contain tokens named eval or arguments, which isn't allowed in strict mode.
The Rollup plugin will now ignore sources starting with null characters in its resolve hook.
Bug fixes
Fix an issue in TreeCursor.iterate that made it iterate through siblings of the current node.
Bug fixes
Fix a bug in TreeCursor.matchContext where, if the context fell partially in a buffer node and partially in tree nodes, it could return incorrect results.
Bug fixes
Give Tag objects an optional string name for debugging, and use it in their toString method.
@lezer/lr 1.4.2 (2024-07-27)
Bug fixes
Fix a tree corruption issue caused by lookahead records in some circumstances.
Bug fixes
Fix a bug that broken references to tokens in local token groups.
@lezer/lr 1.4.1 (2024-05-30)
Bug fixes
Fix a bug where incremental parsing could inappropriately reuse a node when it ended in a repeat or optional part, and was followed by a sequence of skipped nodes longer than 25 characters.
Bug fixes
Include type declarations for the Rollup plugin.
Named or @export-ed specialized tokens are now available in the terms file.
New features
The generator now emits a warning when rules generate a lot of different variants (usually due to a combinatory explosion of ? and | operators).
@lezer/lr 1.4.0 (2024-01-22)
New features
The new InputStream.acceptTokenTo method allows a tokenizer to create a token that ends at some absolute position.
Bug fixes
Fix a bug where getChild and getChildren would, if the first and second arguments matched the same node, return incorrect results.
Bug fixes
Fix an issue where the generator could output invalid JavaScript when a specialization used a string that started with a number.
Adjust TypeScript output to compile with recent tsc versions. Add a test for zero-length node mounts
New features
Support an exportName option to the Rollup plugin.
New features
The new NodeProp.isolate can be used by parser to signal that a given node should be treated as isolated when it comes to bidirectional text rendering.
Bug fixes
Fix a crash that could happen in mixed-language parsing when mounting a tree to a zero-length node.
New features
The new highlightCode function provides a higher-level interface for emitting highlighted code.
Bug fixes
Fix a bug where resolveStack could sometimes yield the same node multiple times.
Allow mixed-parsing trees to be mounted for zero-length nodes.
Fix a bug in mixed-language parsing that could some parts of an inner language to be missed when incrementally reparsing from a stopped parse.
In Tree.build, when given an extremely deeply nested tree structure, flatten it instead of overflowing the stack.
@lezer/lr 1.3.14 (2023-11-02)
Bug fixes
Further reduce maximum parse depth—the old limit could still cause the tree building function to overflow the stack on V8 when all rules on the stack produce actual visible tree nodes.
@lezer/lr 1.3.13 (2023-10-03)
Bug fixes
The parser will now emit an error node when it reaches the end of the top rule but not the end of the input, and restarts parsing the top rule.
Fix an issue that could, when error-recovery happened after a skipped token, cause the parser to emit nodes whose start position was after their end position.
@lezer/lr 1.3.12 (2023-09-22)
Bug fixes
Reduce tree depth limit to avoid V8 overflowing its stack.
New features
The new Tree.resolveStack method returns an iterator through the nodes covering a position, including those from overlays not active at that point.
@lezer/lr 1.3.11 (2023-09-19)
Bug fixes
When parsing input ranges with gaps, don't treat the token after a gap as covering the entire gap.
Bug fixes
Fix a quadratic complexity in state merging.
Bug fixes
Fix a build issue that made the ES version of the Rollup plugin fail to load.
New features
The new typeScript option to buildParserFile (and --typeScript option to lezer-generator) makes the tool emit TypeScript code.
Bug fixes
Make the package usable in TypeScript with node16/nodenext resolution.
@lezer/lr 1.3.10 (2023-08-17)
Bug fixes
Make this package usable in TypeScript setups using node16/nodenext resolution.
Bug fixes
Fix a regression in the build process that caused the Rollup plugin to not be part of the npm package.
Bug fixes
Make this package usable in TypeScript setups with node16/nodenext resolution.
New features
BuildOptions.contextTracker now takes a function, so that the code that produces it has access to the term IDs.
@lezer/lr 1.3.9 (2023-07-03)
Bug fixes
Fix another issue in the way local token groups and input gaps interact.
@lezer/lr 1.3.8 (2023-07-03)
Bug fixes
Fix an issue that caused incorrect trees to be emitted when a forced reduction happened inside a skipped term.
@lezer/lr 1.3.7 (2023-06-22)
Bug fixes
Fix a bug where the parser could skip characters while tokenizing in a local token group.
New features
The test utilities can now be imported as "@lezer/generator/test".
Bug fixes
Fix a bug where precedences specified for local tokens were not properly applied.
Bug fixes
Tree.iterate now properly includes anonymous nodes when IterMode.IncludeAnonymous is enabled.
@lezer/lr 1.3.6 (2023-06-02)
Bug fixes
Fix an issue where context-tracking parsers, in specific circumstances, could create incorrectly nested trees.
Bug fixes
Fix a bug in that could cause it to add random highlighting to the text between parts of an overlaid tree.
@lezer/lr 1.3.5 (2023-05-25)
Bug fixes
Fix an issue where some grammars could make the parser unable to force reductions on invalid input, leading to trees that were missing some wrapping nodes around the broken input.
Bug fixes
Inheritable (/...) styles no longer apply across mounted subtrees.
Bug fixes
Make sure the Rollup plugin imports the rest of the library using a full file path.
Make --help show the correct executable name.
@lezer/lr 1.3.4 (2023-04-24)
Bug fixes
Fix tokenizing of @else tokens that span separate input ranges.
Bug fixes
Make sure TypeScript declaration file has the same name as the ES module to avoid some TypeScript resolution issues.
@lezer/lr 1.3.3 (2023-02-02)
Bug fixes
Fix a crash when parsing a non-atomic skippable at the top of a nested parse.
Fix a bug that caused the tree-depth limit on left-associative rules to be applied to rules generated for repeat operators.
@lezer/lr 1.3.2 (2023-01-31)
Bug fixes
Place a limit on the amount of (non-recovering) parse stacks active at the same time to avoid exponential slowdown on inputs that nest ambiguous constructs.
Reduce the maximum left-associative nesting depth, since the old value could still cause Chrome (but not Firefox) to overflow its call stack.
Bug fixes
Make sure require isn't used as an identifier in generator output.
@lezer/lr 1.3.1 (2023-01-13)
Bug fixes
Fix a bug that could corrupt the parse stack when parsing a non-single-token skippable expression from the initial parse state.
Bug fixes
Fix an invalid optimization that sometimes led to incomplete token precedence tables, leading to incorrect tokenization.
@lezer/lr 1.3.0 (2023-01-09)
Bug fixes
Avoid the problem of huge numbers of repeated left-associative operator applications creating a stack too deep to recurse on by detecting such parses and force-reducing out of them.
New features
Add support for grammars that use @local tokens declarations.
Bug fixes
Fix a bug where the error to use () to denote empty options in a choice expression was signalled even when that is what the input did.
New features
Grammars can now declare @local tokens blocks defining all the tokens that may appear in a set of state, and allowing @else tokens that match everything else.
Bug fixes
Fix a bug where the highlighting over overlaid ranges falling within a highlighted node in the parent tree was sometimes broken.
Bug fixes
Fix a bug in mixed parsing that would sometimes produce invalid trees.
@lezer/lr 1.2.5 (2022-11-16)
Bug fixes
Fix a bug that made Stack.canShift incorrectly return true in some circumstances.
Bug fixes
Fix broken Rollup plugin due to a bad import.
Bug fixes
Fix a bug where the _ notation in tokens would match only half of a surrogate pair.
Fix a bug that caused invalid tokenizer data to be generated for character ranges ending in \uffff.
@lezer/lr 1.2.4 (2022-11-05)
Bug fixes
Fix a tokenizer bug that broke tokenizing of the character \uffff.
Bug fixes
Fix an issue where unmodified tags were treated as having higher specificity than modified ones when computing precedence for tags with multiple modifiers.
Bug fixes
Make sure all highlighting is applied even to nodes with no associated highlight tags.
New features
The new getStyleTags function can be used to query the style tags that match a given syntax nodes.
Bug fixes
Fix a bug that could cause incremental parsing to incorrectly reuse nodes inside nested mixed parses.
@lezer/lr 1.2.3 (2022-08-16)
Bug fixes
Fix a regression in the way tokenizers read input when parsing multiple separate ranges.
@lezer/lr 1.2.2 (2022-08-12)
Bug fixes
Fix an issue that could cause the tokenizer to crash when tokens ended directly on a gap in the input ranges.
@lezer/lr 1.2.1 (2022-08-03)
Bug fixes
Include LRParser.deserialize in the TypeScript declarations.
Fix a bug that broke ParserConfig.specializers (requires @lezer/generator 1.1.1).
Bug fixes
Emit enough information about external specializers to make ParserConfig.specializers fixable.
@lezer/lr 1.2.0 (2022-07-04)
New features
External specializers can now be replaced when reconfiguring an LRParser.
@lezer/lr 1.1.0 (2022-06-27)
New features
Support the output produced by the new @eof marker in lezer-generator.
New features
Things that used to be written like std.digit are now written @digit. The old notation will remain available until a breaking release.
The new @eof marker can be used in tokens to match the end of the input.
New features
First stable version.
@lezer/lr 1.0.0 (2022-06-06)
New features
First stable version.
New features
First stable version.
New features
First stable version.
Bug fixes
Declare matchContext as a property of SyntaxNodeRef.
@lezer/lr 0.16.3 (2022-05-11)
Bug fixes
Don't crash when a non-Node environment defines a process global that doesn't have an env property.
Fix a bug that could cause partially-parsed nodes to be reused when they ended in skipped nodes, when parsing incrementally.
@lezer/lr 0.16.2 (2022-04-21)
Bug fixes
Fix an issue that could make the parser merge adjacent error nodes even when they had different parent nodes.
@lezer/lr 0.16.1 (2022-04-21)
Bug fixes
Fix an issue that caused error nodes created for inserted tokens to sometimes end up in the wrong parent node.
Breaking changes
The mixed parsing interface now passes SyntaxNodeRef values instead of TreeCursor instances around.
Creating a cursor at a given position is now done with Tree.cursorAt, not cursor.
Getting a cursor from a syntax node is now done with a method (cursor) rather than a getter.
Tree.fullCursor was removed and replaced by the IncludeAnonymous iteration mode.
The optional arguments to enter tree traversal methods were replaced with a single mode argument.
Tree.iterate now passes node objects, rather than type/from/to as separate arguments, to its callback functions.
New features
The new SyntaxNodeRef type provides an interface shared by SyntaxNode and TreeCursor.
TreeCursor instances now how have an iterate method that calls a function for each descendant of the current node.
The new matchContext method on SyntaxNode and TreeCursor provides a convenient way to match the names of direct parent nodes.
Cursors can now be passed flags to control their behavior.
@lezer/lr 0.16.0 (2022-04-20)
Bug fixes
Avoid the need for generated parser files to directly import @lezer/common.
Breaking changes
First numbered release.
Bug fixes
Fix an issue in the tokenizer for grammars that could cause it to run very slowly on some inputs.
New features
A grammar can now include an @external propSource a from "b" declaration to import a programmatically defined node prop source.
Bug fixes
Work around a TypeScript issue that caused it to infer return type any for resolve and resolveInner.
Fix a bug in incremental mixed-language parsing where it could incorrectly add a parse range twice, causing a crash in the inner parser.
@lezer/lr 0.15.8 (2022-02-04)
Bug fixes
Fix a bug that caused reductions that didn't consume anything to sometimes end up outside their parent node in the tree.
Bug fixes
Fix a bug where explicitly specified token precedences were sometimes not properly enforced.
@lezer/lr 0.15.7 (2022-01-21)
Bug fixes
Fix a bug that could cause some kinds of composite skipped expressions to not be parsed correctly.
Bug fixes
Fix a bug that caused some kind of skip rules (those ending in something optional) to not work correctly.
@lezer/lr 0.15.6 (2022-01-11)
Bug fixes
Make sure tree depth does not grow unbounded, so that recursive tree traversal is safe from overflowing the stack.
Be less agressive about pruning long-running GLR parse splits.
Bug fixes
Fix a bug where nested parsing would sometimes corrupt the length of parent nodes around the nesting point.
@lezer/lr 0.15.5 (2021-12-01)
Bug fixes
Fix a bug that caused node lookahead to be one less than it should be.
Fix an issue that could cause the parser to, when recovering, parse beyond the position given to stopAt.
New features
SyntaxNode now has a resolveInner method (analogous to Tree.resolveInner).
Bug fixes
Full tree cursors no longer automatically enter mounted subtrees.
Fix a bug where a nested parser would not re-parse inner sections when given fragments produced by a parse that finished the outer tree but was stopped before the inner trees were done.
Bug fixes
Fix a bug that could cause incorrectly structured nodes to be created for repeat rules, breaking incremental parsing using the resulting tree.
Bug fixes
Fix an issue in parseMixed where parses nested two or more levels deep would use the wrong document offsets.
Bug fixes
Fix a null-dereference crash in mixed-language parsing.
@lezer/lr 0.15.4 (2021-09-27)
Bug fixes
Fix a mistake in the way forced reductions are checked.
@lezer/lr 0.15.3 (2021-09-24)
Bug fixes
Fix crashes or infinite recursion caused by applying forced reductions in situations where they were not valid.
Bug fixes
Fix an infinite recursion caused by some kinds of (obscure) token state machines.
New features
Syntax node objects now have a method enterUnfinishedNodesBefore to scan down the tree for nodes that were broken off directly in front of a given position (which can provide a more accurate context that just resolving the position).
Bug fixes
Fix a bug that could lead to spurious 'inconsistent skip sets' errors.
Fix a bug that caused an unescaped '-' at the start or end of a character set to silently be converted to a nonsensical character.
Fix a confusing behavior where literal tokens declared in the @tokens block didn't get names when they started with a lower-case character.
New features
Top rules may now be defined inside @skip scopes.
The parser no longer treats an empty position in a choice operator as the empty expression, but requires an explicit () marker (to avoid a common mistake).
Bug fixes
parseMixed will now scan children not covered by the ranges of an eagerly computed overlay for further nesting.
@lezer/lr 0.15.2 (2021-08-31)
Bug fixes
Fix a bug where the parse position could get corrupted during a parse, leading to crashes or nonsensical output.
@lezer/lr 0.15.1 (2021-08-16)
Bug fixes
Fix an inconsistency in the .d.ts emitted by the build.
New features
LRParser.configure now takes a contextTracker option to replace the context tracker used by the parser.
Bug fixes
Fix an issue where parseMixed could create overlay mounts with zero ranges, which were useless and confused CodeMirror's highlighter.
Bug fixes
Fix a bug that would cause enter to return incorrect results when called entering children in a buffer with .
Bug fixes
Fix a bug where parseMixed could crash by dereferencing null.
Breaking changes
The module name has changed from lezer-tree to @lezer/common.
TreeBuffers no longer accept a node type.
Tree.balance no longer takes a buffer size as argument.
NodeProp.string, NodeProp.number, and NodeProp.flag have been removed (the thing they provided is trivial to write by hand).
A node's context hash is now stored in the NodeProp.contextHash prop.
Reused nodes passed to Tree.build must now be Tree instances (not tree buffers).
Parser is now an abstract class that all parser implementations must extend, implementing the createParse method.
The PartialParse interface changed to make multi-pass parsers possible.
Parser.startParse now takes different arguments (input, fragments, ranges) instead of (input, startPos, context).
The Input interface has changed (to become chunk-based and more low-level). A single Input object is now shared between outer and inner parses.
stringInput is no longer exported (Parser methods will automatically wrap strings when appropriate).
Bug fixes
Fix a bug in TreeFragment.applyChanges that prevented some valid reuse of syntax nodes.
Fix a bug where reused nodes could incorrectly be dropped by Tree.build.
New features
Node props can now be per-node, in which case they are stored on Tree instances rather than NodeTypes.
Tree nodes can now be replaced with other trees through NodeProp.mountedTree.
Tree.resolveInner can now be used to resolve into overlay trees.
SyntaxNode objects now have a toTree method to convert them to a stand-alone tree.
Tree.balance now accepts a helper function to create the inner nodes.
Tree cursors' next/prev methods now take an enter argument to control whether they enter the current node.
SyntaxNode and TreeCursor now have an enter method to directly enter the child at the given position (if any).
Tree.iterate callbacks now get an extra argument that allows them to create a SyntaxNode for the current node.
The parsing interface now supports parsing specific, non-contiguous ranges of the input in a single parse.
The module now exports a parseMixed helper function for creating mixed-language parsers.
@lezer/lr 0.15.0 (2021-08-11)
Breaking changes
The module's name changed from lezer to @lezer/lr.
The Parser class was renamed to LRParser (Parser is now the abstract class that all parsers extend).
Nested parsing is no longer handled inside the LR parser (instead, it is done by using parseMixed from the @lezer/common package as a wrapper).
External tokenizers are passed different arguments and get a different input stream abstraction (to make accidental uncontrolled lookahead and lookbehind, which were easy ways to break incremental parsing, harder).
Drops support for Stack.startOf (which is no longer useful without lookbehind).
Context trackers also get passed different arguments, following the changed input stream format.
This package no longer re-exports bindings from @lezer/common.
Bug fixes
Slightly prefer deleting token to inserting them to avoid far-fetched parses.
Fix a problem where node reuse didn't take the amount of look-ahead done by the tokenizer into account, and could reuse nodes whose content would tokenize differently due to changes after them.
Track tokenizer lookahead to fix a problem where some incremental parses would produce incorrect results.
New features
LR parsers now support a wrapper configuration option that allows you to inject additional logic by wrapping the PartialParse object it returns.
Breaking changes
The module's name changed from lezer-generator to @lezer/generator.
Nested parsers can no longer be specified in the grammar (they must now be set up programmatically).
Bug fixes
Fix an issue where newlines in string tokens could silently corrupt the token.
Handle alternative output file extensions more gracefully.
New features
@export props may now have a value to set a specific export name.
lezer 0.13.5 (2021-05-14)
Bug fixes
Fix a bug with overeager reuse of nodes on change boundaries.
Bug fixes
Don't add inline rules to the terms file (since they may not be uniquely identified by name).
Generate more minimal state machines for the tokenizer.
lezer 0.13.4 (2021-03-03)
New features
Parser instances now have a topNode property that holds the type of the parser's top node.
New features
Add support for context tracking.
lezer 0.13.3 (2021-02-17)
New features
Context trackers can now disable strictness in node reuse.
lezer 0.13.2 (2021-02-17)
New features
Add support for context trackers.
New features
Support @context syntax to register a context tracker for a grammar.
Bug fixes
Fix a bug where building a tree from a buffer would go wrong for repeat nodes whose children were all repeat nodes of the same type.
Bug fixes
Fix an issue where imported identifiers could clash with the export name in generated code.
Breaking changes
NodeType.isRepeated is now called isAnonymous, which more accurately describes what it means.
NodeGroup has been renamed to NodeSet to avoid confusion with NodeProp.group.
The applyChanges method on trees is no longer supported (TreeFragment is now used to track reusable content).
Trees no longer have cut and append methods.
New features
It is now possible to pass a node ID to SyntaxNode.getChild/getChildren and NodeType.is. Allow specifying a tree length in Tree.build
Tree.build now allows you to specify the length of the resulting tree.
Tree.fullCursor() can now be used to get a cursor that includes anonymous nodes, rather than skipping them.
Introduces NodeType.define to define node types.
The new TreeFragment type is used to manage reusable subtrees for incremental parsing.
Tree.build now accepts a start option indicating the start offset of the tree.
The Input type, which used to be InputStream in the lezer package, is now exported from this package.
This package now exports a PartialParse interface, which describes the interface used, for example, as return type from Parser.startParse.
lezer 0.13.1 (2020-12-04)
Bug fixes
Fix versions of lezer packages depended on.
lezer 0.13.0 (2020-12-04)
Breaking changes
Parser.group is now called Parser.nodeSet.
Nested parsers now work differently. They don't have to be Lezer parsers, but have to produce object conforoming to the PartialParse interface. The interface with which non-trivial nested parsers are specified also changed—see the NestedParser type.
Parser objects no longer have a topType property (scan their node set for types with isTop set instead).
Parser objects no longer have withProps, withNested, and withTokenizer methods (use configure instead).
Both Parser.parse and Parser.startParse now take an optional start position as second parameter and an optional parse context as third. startParse returns an instance of the PartialParse interface instead of the old ParseContext class (whose name is now used for something else). Parse options are no longer passed to these methods, but configured in advance through Parser.configure.
During incremental parsing, instead of passing a tree as the cache option, reusable subtrees (see TreeFragment from lezer-tree) are now retrieved from the fragments property of the parse context object, if provided.
Parser.parse and Parser.startParse no longer take an options parameter. Instead, bufferLength and strict can be configured with Parser.configure now, and the start position and context are passed as optional arguments.
The InputStream type has been replaced by Input from the lezer-tree package (which has the same interface but a more appropriate name).
New features
The Parser class now has a configure method that is used to create a parser instance with a more specific configuration.
Bug fixes
Fix versions of lezer packages depended on.
Breaking changes
Adjust to the new way nested parsers work in Lezer.
Bug fixes
Top rule node types will now show up in the terms file.
It is no longer allowed for a top rule to share a name with another rule.
lezer 0.12.1 (2020-11-19)
Bug fixes
Fix an infinite loop in incremental parsing when repeatedly reusing a zero-length cached node.
New features
Make NodePropSource a function type.
Bug fixes
Fix a bug that made SyntaxNode.prevSibling fail in most cases when the node is part of a buffer.
Bug fixes
Fix issue where using Tree.append with an empty tree as argument would return a tree with a nonsensical length property.
Breaking changes
Tree.iterate no longer allows returning from inside the iteration (use cursors directly for that kind of use cases).
Subtree has been renamed to SyntaxNode and narrowed in scope a little.
The top, skipped, and error node props no longer exist.
New features
The package now offers a TreeCursor abstraction, which can be used for both regular iteration and for custom traversal of a tree.
SyntaxNode instances have nextSibling/prevSibling getters that allow more direct navigation through the tree.
Node types now expose isTop, isSkipped, isError, and isRepeated properties that indicate special status.
Adds NodeProp.group to assign group names to node types.
Syntax nodes now have helper functions getChild and getChildren to retrieve direct child nodes by type or group.
NodeType.match (and thus NodeProp.add) now allows types to be targeted by group name.
Node types have a new is method for checking whether their name or one of their groups matches a given string.
lezer 0.12.0 (2020-10-23)
Breaking changes
Follow the change from Subtree to TreeCursor in lezer-tree.
The serialized parser format changed.
Stack.startOf now returns null, rather than -1, when it can't find the given element.
New features
Stack.startOf now takes an optional second argument that allows you to select a match beyond the innermost one.
Breaking changes
The serialized parser format changed.
Pseudo-props like name, dialect, inline and dynamicPrec now require an @ in front of them when specified in a rule's prop list.
@export is now specified as a pseudo-prop instead of in front of the rule.
Top rule names are now required.
New features
Rules can now specify an @isGroup pseudo-prop to automatically attach a group name to all the (single) named nodes they produce.
Bug fixes
Fix a crash that could happen when reporting a conflict error.
New features
A @conflict block inside @tokens can now be used to explicitly indicate a conflict between two tokens.
Allow rules to be explicitly inlineable with an [inline] pseudo-prop.
Bug fixes
Fix lezer dependency versions
Breaking changes
Adjust to new output format of repeat rules.
lezer 0.11.2 (2020-09-26)
Bug fixes
Fix lezer depencency versions
lezer 0.11.1 (2020-09-26)
Bug fixes
Fix an infinite loop that was sometimes hit during error recovery.
lezer 0.11.0 (2020-09-26)
Breaking changes
Follow a breaking change in the way repeat nodes are represented.
Support the new action table format that allows sharing between states.
Bug fixes
Fix lezer depencency versions
Breaking changes
Simplify the representation of repeat expressions in the grammar in a way that avoids some spurious conflicts.
The output format has been modified to allow states to share part of their action table for better compression.
Bug fixes
Fix a bug where the state collapsing could introduce GLR parsing in grammars that otherwise didn't require it.
lezer 0.10.4 (2020-09-15)
New features
Parser objects now have a withTokenizer method that can be used to replace external tokenizers.
Bug fixes
Fix a bug where moduleStyle defaulted to "cjs" when using the node API (rather than to "es" as documented).
New features
You can now import "lezer-generator/rollup" to get a rollup plugin that will transform grammars during the build.
Bug fixes
Fix a bug that broke @external prop declarations in grammars.
Bug fixes
Make sure unrelated precedence declarations for non-cyclic overlapping tokens don't end up also defining a relation between those tokens.
lezer 0.10.3 (2020-09-10)
Bug fixes
Fix a bug that caused the value returned by ParseContext.badness to be much higher than intended.
lezer 0.10.2 (2020-09-02)
Bug fixes
Stack.ruleStart will now ignore repeat rules and node-less rules when determining the inner rule.
Work around a failure mode where error-recovery got stuck in an end-of-grammar state and thus could not continue meaningfully parsing anything by restarting such states back to their initial state.
New features
External tokenizers can now provide an extend flag to allow their tokens to be used alongside tokens produced by other tokenizers.
Add support for dynamic precedences.
Bug fixes
Actually reuse parser states when skip rules refer to rules also used in other contexts.
Fix a bug where the automaton generated for skip rules wasn't being compressed.
Properly raise an error when different specializations for the same token are given different names.
Fix a bug that prevented NodeProp.skipped from being properly attached to node types.
Fix a corner-case infinite loop in the state-collapsing algorithm (and speed it up).
Compile + and * operators in a way that is less likely to lead to conflicts.
Emit all shift/reduce and reduce/reduce conflicts in a single run, rather than stopping on the first one.
Emit all overlapping token errors, rather than only the first one.
New features
Inline rules can now be anonymous (with syntax [props..] { body }).
Dynamic precedences can now be associated with productions, which can help pick the preferred GLR parse when a grammar is ambiguous.
Token @precedence declarations can now refer to a parameterized rule by name (without arguments) to indicate that all instances of that rule have a given precedence.
lezer 0.10.1 (2020-08-20)
Bug fixes
Fixes an issue where repeated error recovery could lead to a tree so deep that recursive functions on it would overflow the stack.
Breaking changes
No longer list internal properties in the type definitions.
lezer 0.10.0 (2020-08-07)
New features
Add support for grammar dialects.
Add support for external specializers.
Stacks now have a parser accessor that gets you the active parser instance.
Breaking changes
No longer list internal properties in the type definitions.
Follow changes in the serialized parser format.
The way different tokenizers are combined is now slightly different. The first one to return a token wins, even if that token has no actions in the current state. The old behavior, where further tokenizers are tried until actions are found, can be enabled for a given tokenizer by setting its fallback flag.
Bug fixes
Fix an issue where the output file would in some cases have a superfluous comma.
Bug fixes
Fix a bug in the reuse of compiled rules (which sometimes got confused by inline rules).
The error message for overlapping tokens is a bit more concrete now, including an example of a string that matches both tokens.
New features
Add support for grammar dialects.
Add support for external specializers.
Commas in precedence tag, dialect, or external token lists are now optional.
Breaking changes
Changes the serialized parser format.
New features
The test runner helper now allows tests to pass additional configuration options.
lezer 0.9.1 (2020-06-29)
Bug fixes
Fix accidental use of non-ES5 library methods.
Breaking changes
Drop NodeProp.delim in favor of NodeProp.openedBy/closedBy.
lezer 0.9.0 (2020-06-08)
Breaking changes
Upgrade to 0.9 parser serialization
Breaking changes
The @detectDelim directive now assigns NodeProp.openedBy/closedBy props to the bracket nodes, instead of NodeProp.delim to the parent node.
lezer 0.8.5 (2020-05-01)
Bug fixes
Publish less useless cruft to npm, reducing package size.
Bug fixes
Publish less useless cruft to npm, reducing package size.
lezer 0.8.4 (2020-04-14)
Bug fixes
Fix a bug in Stack.startOf that made it fail to find rules that were actually on the stack in many situations.
Bug fixes
Fix an issue where token groups were inappropriately merged when conflicting tokens didn't appear in overlapping state sets.
Fix an issue where external tokenizers needed for skipped tokens were not properly enabled for some states.
Fix a bug where the tool would sometimes overeagerly merge states, resulting in incorrect output.
Bug fixes
Make dist/test loadable from CommonJS modules again.
Fix a bug that prevented NodeProp.top from being assigned to top rules in most cases.
Bug fixes
Make the package load as an ES module on node
lezer 0.8.3 (2020-04-01)
Bug fixes
Make the package load as an ES module on node
Bug fixes
Fix an issue that broke the bin command.
Bug fixes
Make the package load as an ES module on node
New features
The package now provides an ES6 module.
lezer 0.8.2 (2020-02-28)
New features
The package now provides an ES6 module.
Bug fixes
Fix a bug that caused applyChanges to include parts of the old tree that weren't safe to reuse.
lezer 0.8.1 (2020-02-26)
New features
You can now find the top node type parsed by the parser through its topType property.
Bug fixes
Fix bug that would cause tree balancing of deep trees to produce corrupt output.
New features
Bump version along with the rest of the lezer packages.
lezer 0.8.0 (2020-02-03)
Breaking changes
The serialized parser format changed.
New features
Add support for multiple @top rules through the top parse option.
Breaking changes
Changes the serialized parser format.
New features
Add support for multiple @top rules.
Bug fixes
In applyChanges, make sure the tree is collapsed all the way to the
nearest non-error node next to the change.
lezer 0.7.1 (2020-01-23)
Bug fixes
Tweak recovery cost for forced reductions to prefer those to other recovery strategies.
More agressively reuse cached nodes.
New features
Support Foo(...) syntax in test specs to indicate "ignore the children of this node".
Bug fixes
Fix a bug that prevented balancing of repeat nodes when there were skipped nodes present between the repeated elements (which ruined the efficiency of incremental parses).
New features
TreeBuffer objects now have an iterate function.
Buffers can optionally be tagged with an (unnamed) node type to allow reusing them in an incremental parse without wrapping them in a tree.
Breaking changes
Tree.build now takes its arguments wrapped in an object. It also expects the buffer content to conform to from lezer 0.7.0's representation of repeated productions.
The repeated node prop was removed (the parser generator now encodes repetition in the type ids).
lezer 0.7.0 (2020-01-20)
Breaking changes
This now consumes the adjusted parser output of lezer-generator 0.7.0.
New features
You can now write a node name directly after @top to give your top node type a name.
Breaking changes
Changes the way repeated nodes are represented in the generated parser.
lezer 0.6.0 (2020-01-15)
Bug fixes
Rewrite the way the parser advances and recovers from errors, to more rigorously address a number of infinite loops and poor recovery behaviors.
New features
Parse context objects now have a badness property that you can use to estimate just how poorly the input matches the grammar.
Bug fixes
Fix crash when the top rule had no node name.
Adjust the way states' forced reductions are computed to avoid cycles (where force-reducing multiple times gets you back in your original state).
lezer 0.5.2 (2020-01-09)
Bug fixes
Fix an issue where the parser will sometimes continue, and even pick as result, a parse with error recovery even though there are error-free parses available.
Fix a mistake in our binary heap implementation that would cause stacks to be ordered incorrectly.
Fix an issue where the Stack.startOf method would ignore the top frame of the stack.
lezer 0.5.1 (2020-01-01)
Bug fixes
Fix an issue where the parser would loop infinitely when leaving a nested parse in some circumstances.
Fix an infinite loop on incomplete input at end of file that occurred for some types of mutually recursive rules.
New features
NodeProp.add now also allows a selector object to be passed.
New features
Adds NodeProp.top, which flags a grammar's outer node type.
Breaking changes
Drops the NodeProp.lang prop (superseded by top).
lezer 0.5.0 (2019-10-22)
New features
Parser instances now have a hasNested property that tells you whether they nest grammars.
Bug fixes
Fix issue where serialized parsers didn't have the top node prop set.
New features
The generator now automatically assigns the top node prop to the grammar's top node.
lezer 0.4.1 (2019-10-14)
Bug fixes
Fix an infinite loop where error recovery keeps finding zero-length tokens and imagining it's making progress.
Bug fixes
Export BufferCursor again, which was accidentally removed from the exports in 0.3.0.
Breaking changes
The iterate method now takes an object instead of separate parameters.
lezer 0.4.0 (2019-09-10)
Bug fixes
Don't rely on additional data stored in the parse table during recovery (shrinking the parse tables).
Fix a crash that could occur when starting a nested parse when there were multiple active stacks.
Fix an issue where error nodes would sometimes not be merged.
Don't reuse cached tokens for states that have a different token group.
Breaking changes
The on-disk parse table format changed again.
Bug fixes
Fix bug that made matching single-token skipped expressions unnecessarily expensive.
Breaking changes
Do not emit recovery actions in the parse table anymore.
New features
Introduces node props.
Node types are now objects holding a name, id, and set of props.
Breaking changes
Tags are gone again, nodes have plain string names.
lezer 0.3.0 (2019-08-22)
Bug fixes
Don't treat reused nodes as if they are error terms when merging errors.
Add badness penalty for forced reductions at end of input.
Fix several infinite loops around forced reductions.
Don't move error nodes out of reduces.
New features
Add a Parser.withProps method for extending a parser with new node props.
Breaking changes
Emits lezer-tree 0.3.0 style trees with NodeType type objects.
Parser.deserialize's interface changed (now taking an object rather than a long list of parameters).
Bug fixes
Fix several issues in the way forced reductions were picked, to avoid infinite reduction loops.
New features
Add support for props and custom node names in the grammar notation.
Allow importing of props via @external prop.
Rule capitalization is now relevant (again), only capitalized rules appear in the tree by default.
Breaking changes
Remove support for node tags, tag expressions, the @tags block, and everything else related to tags.
Replace tagged expression syntax with inline rule syntax.
Literal declarations must now go into the @tokens block.
The @detectDelim declaration must now appear at the top level.
The dash in @external-... syntax was dropped—@external is now a separate token.
External grammars that default to null must now have the word empty instead of from "..." (to resolve an ambiguity that syntax introduced).
Bug fixes
Fix incorrect node length calculation in Tree.build.
New features
Tree nodes are now identified with tags.
New Tag data structure to represent node tags.
Breaking changes
Drop support for grammar ids and node types.
lezer 0.2.0 (2019-08-02)
Bug fixes
Don't include lezer-tree inline in dist/index.js.
New features
The output tree now uses tags, rather than term names, to identify its nodes.
Export Tag data structure from lezer-tree.
Support per-grammar global tag suffixes in Parser.deserialize.
Breaking changes
Grammars no longer have ids.
Removes export of allocateGrammarID and TagMap.
Bug fixes
Fix bug where the grammar parser unintentionally required semicolon between rules in skip blocks.
Actually throw an error when detecing a skip inconsistency.
Track skip context more accurately through parse states.
Fix specializing of external tokens.
New features
Add support for tags.
Add @tags blocks, allow tags for literals.
Add @punctuation to succinctly declare punctuation tags.
Add @infer-delim to enable automatic delimiter detection.
Add @all as a way to append tags to all tagged rules in the grammar.
Allow a choice of literals to be passed to @specialize/@extend.
Add dist/test.js with test helper functions.
Breaking changes
Require @ in front of grammar keywords.
Remove support for =-style tag declarations.
Replace tag.foo syntax with colon suffix syntax.
Bug Fixes
Actually include the .d.ts file in the published package.
New Features
First documented release.
lezer 0.1.1 (2019-07-09)
Bug Fixes
Actually include the .d.ts file in the published package.
lezer 0.1.0 (2019-07-09)
New Features
First documented release.
Bug Fixes
Actually include the .d.ts file in the published package.
New Features
First documented release.