# Rune

**Rune** is a high-performance, customizable **lexical analysis library** written in Rust.  
It transforms source files into tokens using a fast, cache-friendly design.

> _“Turn raw text into structured meaning — like spellcraft for source code.”_

## Features

- **Basic tokenization**: Whitespace, text, numbers, symbols, and newlines.
- **Flat `TokenStream` design**: Optimized for speed and cache locality.
- **Custom transforms**: Supply your own function to turn base tokens into domain-specific ones (like Markdown, HTML, or custom Domain Specific Languages).
- **Iterators and mutation**: Traverse or modify tokens efficiently.


## Getting Started

### Add to your project
This library is hosted on the CyberMages registry. To add it to a project the CyberMages registry will need to be added to Cargo as per the [Cargo Book][1].

[1]: https://doc.rust-lang.org/cargo/reference/registries.html "The Cargo Book: Registries"

First add the registry to your cargo config.toml file.

```toml
[registries.cybermages]
index = "sparse+https://workshop.cybermages.tech/api/packages/CyberMages/cargo/"
```

Then add this to your Cargo.toml file.

```toml
[dependencies]
rune = { version = "0.2.0", registry = "cybermages" }
```