rune/README.md

Rune

Rune is a high-performance, customizable lexical analysis library written in Rust.
It transforms source files into tokens using a fast, cache-friendly design.

“Turn raw text into structured meaning — like spellcraft for source code.”

Features

• Basic tokenization: Whitespace, text, numbers, symbols, and newlines.
• Flat TokenStream design: Optimized for speed and cache locality.
• Custom transforms: Supply your own function to turn base tokens into domain-specific ones (like Markdown, HTML, or custom Domain Specific Languages).
• Iterators and mutation: Traverse or modify tokens efficiently.

Getting Started

Add to your project

This library is hosted on the CyberMages registry. To add it to a project the CyberMages registry will need to be added to Cargo as per the Cargo Book.

First add the registry to your cargo config.toml file.

[registries.cybermages]
index = "sparse+https://workshop.cybermages.tech/api/packages/CyberMages/cargo/"

Then add this to your Cargo.toml file.

[dependencies]
rune = { version = "0.3.0", registry = "cybermages" }

---

Copyright & License

Copyright 2025 CyberMages LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this library except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS