A vector index built on TurboQuant, written in Rust with Python bindings
Created Mar 2026
18% of these repos are in Image processing (Libraries), 18% in LLM Infrastructure.
Repositories
11
Total stars
20,737
A vector index built on TurboQuant, written in Rust with Python bindings
Created Mar 2026
RDNA-native LLM inference engine in Rust.
Created Mar 2026
Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.
Created Jun 2026
SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.
Created Jul 2026
Palette quantization library that powers pngquant and other PNG optimizers
Created Nov 2016
Camelid: a Rust-native local inference backend with evidence-gated model compatibility.
Created May 2026
PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.
Created Jan 2026
A vector index built on TurboQuant, written in Rust with Python bindings
Created Mar 2026
Palette quantization library that powers pngquant and other PNG optimizers
Created Nov 2016
RDNA-native LLM inference engine in Rust.
Created Mar 2026
Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.
Created Jun 2026
PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.
Created Jan 2026
SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.
Created Jul 2026
Camelid: a Rust-native local inference backend with evidence-gated model compatibility.
Created May 2026
🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE, native video/document understanding, long-horizon tool agents, quantization-aware training. Scales: Tiny (25M) to Large (1.3B).
Created Jun 2026
600 KB WASM runtime for Cactus Compute's Needle AI tool-calling models, Needle 3, 2 and 1 from one build. Browser, Cloudflare Workers, Node.js, Python, C FFI and no_std. Token-exact with the JAX reference. No backend, no API key.
Created May 2026
Buffer-first rust dithering and halftoning library.
Created Mar 2026