Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.
Created May 2025
Repositories
46
Total stars
47,834
Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.
Created May 2025
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Created Nov 2019
Japanese Input Method System for Linux, macOS, Neural Kana-Kanji Conversion Engine
Created Aug 2024
Language model tokenization at GB/s
Created Nov 2025
Official Rust Implementation of Model2Vec
Created May 2025
Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)
Created Jan 2020
GraphRAG-rs is a high-performance, state-of-the-art Rust implementation of GraphRAG (Graph-based Retrieval Augmented Generation) that builds knowledge graphs from documents and enables natural language querying with configurable entity extraction and local LLM integration
Created Sep 2025
Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.
Created Jun 2024
Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.
Created May 2025
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Created Nov 2019
Language model tokenization at GB/s
Created Nov 2025
Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)
Created Jan 2020
Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.
Created Jun 2024
Natural language detection library for Rust. Try demo online: https://whatlang.org/
Created Nov 2016
Japanese Input Method System for Linux, macOS, Neural Kana-Kanji Conversion Engine
Created Aug 2024
A fast, low-resource Natural Language Processing and Text Correction library written in Rust.
Created Jul 2020
GraphRAG-rs is a high-performance, state-of-the-art Rust implementation of GraphRAG (Graph-based Retrieval Augmented Generation) that builds knowledge graphs from documents and enables natural language querying with configurable entity extraction and local LLM integration
Created Sep 2025
🎤 vibrato: Viterbi-based accelerated tokenizer
Created Jul 2022
Checks all your documentation for spelling and grammar mistakes with hunspell and a nlprule based checker for grammar
Created May 2020
🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer
Created Aug 2021
Official Rust Implementation of Model2Vec
Created May 2025
:spider: The pipeline for the OSCAR corpus
Created Feb 2021
Simple NLP in Rust with Python bindings
Created Nov 2018
A sentence segmentation library with wide language support optimized for speed and utility.
Created Aug 2023
Reliable infrastructure for Sanskrit software.
Created Oct 2022
Distribute and run transformer encoders with a single file.
Created Oct 2025
Rust port of sentence-transformers (https://github.com/UKPLab/sentence-transformers)
Created Apr 2020
Somali NLP Corpus is an open, high-quality Somali text corpus designed to support research and development in Natural Language Processing (NLP), Large Language Models (LLMs), and other AI applications for the Somali language.
Created Jun 2026
The open, machine-readable standard for the Somali language — orthography, grammar, terminology, translation, and AI/benchmark resources, as a versioned, citable RFC-style catalog.
Created Jul 2026
Two-tier hybrid local search for Rust and the fsfs CLI: sub-millisecond first-pass results from a Potion/Model2Vec fast tier, then MiniLM quality refinement. Fuses native pure-Rust BM25 (Quill engine) and semantic vector search with Reciprocal Rank Fusion. Progressive iterator API, f16 SIMD vector index, feature-gated compilation.
Created Feb 2026
A BM25 embedder, scorer, and search engine, written in Rust.
Created Aug 2024
⚡Japanese sentence splitting(日本語文境界判定器), 40–250× faster via a Rust-accelerated Python library with near-perfect API compatibility with megagonlabs/bunkai.
Created Oct 2025
A command line tool for extracting machine learning ready data from software binaries powered by Radare2
Created May 2023
Fast, accurate web content extraction in Rust. ML page-type classification, per-type extraction, confidence scoring. F1=0.966 on ScrapingHub (#1), F1=0.859 across 2,008 annotated pages (1,497 development + 511 held-out test
Created Jan 2026
Fast tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and WordPiece tokenization in JavaScript, Python and Rust.
Created Jul 2023
A HDPSG-inspired symbolic natural language parser written in Rust
Created Oct 2020
fastText model serving service
Created Jan 2018
3DCF / doc2dataset: token-efficient document layer with NumGuard numeric integrity and multi-framework exports for RAG & fine-tuning.
Created Nov 2025
Add Arabic diacritics (tashkeel/harakat) using Rust/Python/C++/WASM and NLP models
Created Mar 2023
World's most accurate and fast procedural English conjugation library
Created Dec 2024