GitHub topics

quantization

18% of these repos are in Image processing (Libraries), 18% in LLM Infrastructure.

Repositories

11

Total stars

20,737

Trending

RyanCodrai

A vector index built on TurboQuant, written in Rust with Python bindings

Created Mar 2026

ann avx512 embedding embeddings faiss
Total17,502
30 days +131
Dicklesworthstone

Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.

Created Jun 2026

avx512 cpu-inference deepseek-ocr int4 int8
Total347
30 days +13
giannisanni

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.

Created Jul 2026

Total239
30 days +9
ImageOptim

Palette quantization library that powers pngquant and other PNG optimizers

Created Nov 2016

callback conversion image-optimization image-pixels minification
Total940
30 days +5
Epistates

PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.

Created Jan 2026

ai ane apple-silicon deep-learning distillation
Total333
30 days +3

Top

RyanCodrai

A vector index built on TurboQuant, written in Rust with Python bindings

Created Mar 2026

ann avx512 embedding embeddings faiss
Total17,502
ImageOptim

Palette quantization library that powers pngquant and other PNG optimizers

Created Nov 2016

callback conversion image-optimization image-pixels minification
Total940
Dicklesworthstone

Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). Five-model zoo, custom int8 kernels, no ML framework, no Python, no GPU.

Created Jun 2026

avx512 cpu-inference deepseek-ocr int4 int8
Total347
Epistates

PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.

Created Jan 2026

ai ane apple-silicon deep-learning distillation
Total333
giannisanni

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.

Created Jul 2026

Total239
AarambhDevHub

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE, native video/document understanding, long-horizon tool agents, quantization-aware training. Scales: Tiny (25M) to Large (1.3B).

Created Jun 2026

ai-agents candle decoder-only deep-learning dora
Total112
Geekgineer

600 KB WASM runtime for Cactus Compute's Needle AI tool-calling models, Needle 3, 2 and 1 from one build. Browser, Cloudflare Workers, Node.js, Python, C FFI and no_std. Token-exact with the JAX reference. No backend, no API key.

Created May 2026

agent ai browser-ai cloudflare-workers edge-ai
Total109
arizqi

Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.

Created Jul 2026

Total57

Related

All topics →