Myrddin Dundragon
a926e08061
I went and enhanced the LexerError to now wrap a Rust source Error. This makes it so we can wrapup the possible IO errors we get when trying to open a file and read its contents. I also added some documentation for all the implemented functions.
Rune
Rune is a high-performance, customizable lexical analysis library written in Rust.
It transforms source files into tokens using a fast, cache-friendly design.
“Turn raw text into structured meaning — like spellcraft for source code.”
Features
- Basic tokenization: Whitespace, text, numbers, symbols, and newlines.
- Flat
TokenStreamdesign: Optimized for speed and cache locality. - Custom transforms: Supply your own function to turn base tokens into domain-specific ones (like Markdown, HTML, or custom Domain Specific Languages).
- Iterators and mutation: Traverse or modify tokens efficiently.
Getting Started
Add to your project
This library is hosted on the CyberMages registry. To add it to a project the CyberMages registry will need to be added to Cargo as per the Cargo Book.
First add the registry to your cargo config.toml file.
[registries.cybermages]
index = "sparse+https://workshop.cybermages.tech/api/packages/CyberMages/cargo/"
Then add this to your Cargo.toml file.
[dependencies]
rune = { version = "0.2.0", registry = "cybermages" }