GitHub topics

nlp

13% of these repos are in Artificial Intelligence.

Repositories

46

Total stars

47,834

Trending

memvid

Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.

Created May 2025

ai context embedded faiss knowledge-base
Total17,433
30 days +861
messense

The Jieba Chinese Word Segmentation Implemented in Rust

Created May 2018

chinese-word-segmentation jieba jieba-chinese nlp wasm
Total995
30 days +11
pemistahl

The most accurate natural language detection library for Rust, suitable for short text and mixed-language text

Created Jun 2020

language-classification language-detection language-identification language-processing language-recognition
Total1,154
30 days +10
guillaume-be

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

Created Jan 2020

bart bert deep-learning electra gpt
Total3,091
30 days +8
automataIA

GraphRAG-rs is a high-performance, state-of-the-art Rust implementation of GraphRAG (Graph-based Retrieval Augmented Generation) that builds knowledge graphs from documents and enables natural language querying with configurable entity extraction and local LLM integration

Created Sep 2025

Total540
30 days +7
yobix-ai

Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.

Created Jun 2024

data-pipelines docx etl etl-pipelines extraction
Total1,786
30 days +6

Top

memvid

Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.

Created May 2025

ai context embedded faiss knowledge-base
Total17,433
guillaume-be

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

Created Jan 2020

bart bert deep-learning electra gpt
Total3,091
yobix-ai

Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.

Created Jun 2024

data-pipelines docx etl etl-pipelines extraction
Total1,786
pemistahl

The most accurate natural language detection library for Rust, suitable for short text and mixed-language text

Created Jun 2020

language-classification language-detection language-identification language-processing language-recognition
Total1,154
messense

The Jieba Chinese Word Segmentation Implemented in Rust

Created May 2018

chinese-word-segmentation jieba jieba-chinese nlp wasm
Total995
automataIA

GraphRAG-rs is a high-performance, state-of-the-art Rust implementation of GraphRAG (Graph-based Retrieval Augmented Generation) that builds knowledge graphs from documents and enables natural language querying with configurable entity extraction and local LLM integration

Created Sep 2025

Total540
daac-tools

🎤 vibrato: Viterbi-based accelerated tokenizer

Created Jul 2022

japanese morphological-analysis nlp rust segmentation
Total430
drahnr

Checks all your documentation for spelling and grammar mistakes with hunspell and a nlprule based checker for grammar

Created May 2020

cargo cargo-plugin cargo-spellcheck grammar grammar-mistakes
Total361
akaza-im

Yet another Japanese IME for IBus/Linux

Created Sep 2020

ibus ime nlp rust
Total267
ticki

A blazingly fast phonetic reduction/hashing algorithm.

Created May 2016

nlp
Total220
oscar-project

:spider: The pipeline for the OSCAR corpus

Created Feb 2021

common-crawl commoncrawl corpus-linguistics crawler fasttext
Total178
fbilhaut

Inference engine for GLiNER models, in Rust

Created Jan 2025

information-extraction ner nlp relation-extraction
Total173
ambuda-org

Reliable infrastructure for Sanskrit software.

Created Oct 2022

nlp rust sanskrit
Total144
graniet

Use multiple LLM backends in a single crate, simple builder-based configuration, and built-in prompt chaining & templating.

Created Jan 2025

ai anthropic builder-pattern chatbot llm
Total137
cpcdoy

Rust port of sentence-transformers (https://github.com/UKPLab/sentence-transformers)

Created Apr 2020

bert nlp rust rust-sbert sbert
Total128
nwiizo

日本語文書の自然さと読みやすさを再現可能に診断するRust CLI / Deterministic diagnostics for natural and readable Japanese writing

Created Aug 2026

cli japanese japanese-language lint linter
Total115
somastllc

Somali NLP Corpus is an open, high-quality Somali text corpus designed to support research and development in Natural Language Processing (NLP), Large Language Models (LLMs), and other AI applications for the Somali language.

Created Jun 2026

corpus dataset language-resources llm low-resource-languages
Total115
somastllc

The open, machine-readable standard for the Somali language — orthography, grammar, terminology, translation, and AI/benchmark resources, as a versioned, citable RFC-style catalog.

Created Jul 2026

ai benchmarks cushitic grammar language-standard
Total109
hotchpotch

⚡Japanese sentence splitting(日本語文境界判定器), 40–250× faster via a Rust-accelerated Python library with near-perfect API compatibility with megagonlabs/bunkai.

Created Oct 2025

nlp
Total77
br0kej

A command line tool for extracting machine learning ready data from software binaries powered by Radare2

Created May 2023

binary-analysis data-generation graph-neural-networks machine-learning ml4sec
Total76
Murrough-Foley

Fast, accurate web content extraction in Rust. ML page-type classification, per-type extraction, confidence scoring. F1=0.966 on ScrapingHub (#1), F1=0.859 across 2,008 annotated pages (1,497 development + 511 held-out test

Created Jan 2026

content-extraction machine-learning nlp rust search-engine-optimization
Total71
arosspope

A rust implementation of ELIZA - a natural language processing program developed by Joseph Weizenbaum in 1966.

Created Nov 2017

chatbot eliza linguistics nlp regex
Total65
Systemcluster

Fast tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and WordPiece tokenization in JavaScript, Python and Rust.

Created Jul 2023

bpe nlp nodejs python rust
Total64
vgel

A HDPSG-inspired symbolic natural language parser written in Rust

Created Oct 2020

grammar hacktoberfest linguistics nlp rust
Total63
3DCF-Labs

3DCF / doc2dataset: token-efficient document layer with NumGuard numeric integrity and multi-framework exports for RAG & fine-tuning.

Created Nov 2025

3dcf cli data-pipeline dataset-generation doc2dataset
Total54
mush42

Add Arabic diacritics (tashkeel/harakat) using Rust/Python/C++/WASM and NLP models

Created Mar 2023

arabic diacritics nlp tashkeel
Total54
gold-silver-copper

World's most accurate and fast procedural English conjugation library

Created Dec 2024

conjugator decliner english gamedev inflector
Total52
Miezhiko

アマデウス Memory storage and artificial intelligence system. Complex chatting AI bot for discord, also voice / music, streams trackers, WarCraft3 news, games trackers, various statistics/info grabbers.

Created May 2020

Total51

Related

All topics →