Categories / Artificial Intelligence

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Last push 2 hours ago

bertgptlanguage-modelnatural-language-processingnatural-language-understandingnlptransformers

Created

Nov 2019

Total stars

11,260

Last 7 days

+22

Last 30 days

+83

Last 12 months

+1,144

README badge

Rusty Stars badge for huggingface/tokenizers
[![Rusty Stars](https://rustystars.dev/badge/huggingface/tokenizers.svg)](https://rustystars.dev/huggingface/tokenizers)