A high-performance tokenizer (BPE, WordPiece, SentencePiece) built with Rust with Python bindings, focused on speed, safety, and resource optimization.

Last push 3 days ago

huggingfacellmmachine-learningopenairusttokenizer

Created

Nov 2025

Total stars

63

Last 7 days

0

Last 30 days

0

Last 12 months

0

README badge

Rusty Stars badge for ml-rust/splintr
[![Rusty Stars](https://rustystars.dev/badge/ml-rust/splintr.svg)](https://rustystars.dev/ml-rust/splintr)