CyberMages/rune
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

[#2] A data-oriented Lexer.

Browse Source 
I took the Token module from the Arcanum project and brought it over
to here. It was a nice data oriented way of handling the Tokens.

I then created a Lexer that can scan a file or text and allow the
user to transform the scanned tokens before the final token array
is returned. This should allow for more complex and specific tokens
to be created for whatever domain is being targeted.

I also added basic library examples and testing.

Finally, I made sure the documentation generated nicely.

This is now marked as version: 0.1.0
This commit is contained in:
Myrddin Dundragon
2025-04-15 21:17:28 -04:00
parent 0350a151a9
commit acf869efbb
13 changed files with 823 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
Cargo.toml
View File

@ -1,8 +1,8 @@
[package]
name = "rune"
version = "0.0.0"
version = "0.1.0"
edition = "2021"
description = "A simple lexical analysis library."
description = "A lexical analysis library."
repository = "/myrddin/rune"
authors = ["CyberMages LLC <Software@CyberMagesLLC.com>", "Jason Travis Smith <Myrddin@CyberMages.tech>"]
readme = "README.md"