ferrumox

ferrumox

GitHub

Repositories

2

Total stars

385

Top repositories

Run Qwen 3.8 MAX (2.4T MoE) on a 50GB-RAM consumer machine in pure Rust, experts streamed from disk. Tiny engine, immense model.

Created Jul 2026

Total193

A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.

Created Mar 2026

cuda gguf inference-engine llama-cpp llm
Total192