A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.
Created Jan 2025
Categories / More categories
Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.
Repos
123
Stars
73,559
Last 30 days
+5,730
A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.
Created Jan 2025
All-in-one platform for search, recommendations, RAG, and analytics offered via API
Created Mar 2023
Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
Created Apr 2024
Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀
Created Mar 2024
The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.
Created Jul 2023
GraphRAG-rs is a high-performance, state-of-the-art Rust implementation of GraphRAG (Graph-based Retrieval Augmented Generation) that builds knowledge graphs from documents and enables natural language querying with configurable entity extraction and local LLM integration
Created Sep 2025
Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.
Created May 2026
SUGT - local AI gateway & skills console. Centralize LLM API keys, one OpenAI/Anthropic-compatible proxy for Claude Code, Codex & OpenCode. Homebrew & winget. Fully open source (Apache-2.0).
Created Jul 2026
Alephant is an open-source AI Agent Gateway for routing, tracking, and controlling LLM usage across AI agents, members, and workflows, and for publishing agent capabilities as paid endpoints with x402 and MPP payment rails.
Created May 2026
Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly and quickly.
Created Nov 2024
Free, open-source app that runs AI chat models on your own computer — and lets computers team up over the internet to run models too big for one machine. No account, no crypto. OpenAI- and Anthropic-compatible API.
Created Feb 2026
Pure-Rust GGUF inference engine with quantized CPU, Metal & CUDA kernels, MoE support, OpenAI-compatible server. Benchmarked head-to-head against llama.cpp.
Created Aug 2026
An open source, Native AI Gateway and LLM proxy built in Rust
Created Mar 2026
High-performance AI routing proxy built in Rust with automatic failover, priority-based routing, and support for 15+ providers (Anthropic, OpenAI, Cerebras, Minimax, Kimi, etc.)
Created Nov 2025 Archived