Categories / LLM Infrastructure
A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.
Created
Mar 2026
Total stars
192
Last 7 days
0
Last 30 days
0
Last 12 months
0
README badge
[](https://rustystars.dev/ferrumox/fox)