The Lezer Parser System
The problem of parsing structured text has resisted one-size-fits all solutions for over 60 years. Lezer isn't such a solution either, but it is a very decent parser generator, especially well suited for use in code editors.
Lezer (the Dutch word for reader, pronounced like “laser”) provides a parser generator that outputs JavaScript modules, which can be loaded to parse code into a non-abstract syntax tree. This tree can then be used to do highlighting and basic semantic analysis.
Features
-
Grammar Driven
Write your grammars declaratively, use a parser generator tool to convert it into an efficient parser.
-
Incremental
Lezer can quickly re-parse documents that were slightly changed compared to a previously parsed version by reusing nodes from the old parse.
-
Error-Insensitive
Being designed for the code editor use case, the parser is equipped with strategies for recovering from syntax errors, and can produce a tree for any input.
-
Fast
By using the LR parsing algorithm, an efficient tokenizer, and data structures optimized for locality, the parser can process its input at a very respectable speed.
-
Lightweight
The parser generator produces compact parse table files which, along with the run-time library, are all you need to ship to clients.
-
Memory-Friendly
The syntax tree representation is designed to be small in memory, packing groups of smaller nodes together in a representation using 64 bits per node.
Grammars
About
Lezer is open source under an MIT license. It is maintained by the CodeMirror team.
Lezer is being developed on GitHub. Contributions are welcome.
Discussing the project, or asking questions, is best done on the forum. Bugs should be reported through the issue tracker. We aim to be an inclusive, welcoming community. To make that explicit, we have a code of conduct that applies to communication around the project.
Links
- Blog post announcing the library
- CodeMirror, the code editor for which Lezer was built