huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
nlpnatural-language-processingnatural-language-understandinglanguage-modeltransformersbertgpt
Created
Nov 2019
Total stars
11,258
Last 7 days
0
Last 30 days
0
Last 12 months
0
README badge
Show this repo's rank among all Rust repos. Paste the Markdown into the README.
[](https://rustystars.dev/huggingface/tokenizers)