Categories / Artificial Intelligence
huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
bertgptlanguage-modelnatural-language-processingnatural-language-understandingnlptransformers
Created
Nov 2019
Total stars
11,259
Last 7 days
+22
Last 30 days
+83
Last 12 months
+1,144
README badge
[](https://rustystars.dev/huggingface/tokenizers)