Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
Created Feb 2026
Total748
Repositories
3
Total stars
1,013
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
Created Feb 2026
A model-agnostic GPU runtime for shipping models as verified programs.
Created Aug 2026
OpenAI-compatible speech serving in Rust: streaming TTS and full-duplex voice over GPT-Live. 128 concurrent PersonaPlex sessions on one GPU.
Created Sep 2026