Categories / More categories

LLM Infrastructure

Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.

Repos

123

Stars

73,586

Last 30 days

+5,748

winfunc

A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.

Created Jan 2025

ai anthropic anthropic-claude api chain-of-thought
Total5,352
30 days +9
smg-project

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

Created Nov 2025

anthropic anthropic-api chat claude gemini
Total735
30 days +74
eliasbiondo

🕷️ The fastest open-source AI Gateway and LLM Gateway — one binary with built-in console to route, secure, budget and observe every LLM call across OpenAI, Anthropic, Gemini, Bedrock and more.

Created Oct 2026

a2a ai-gateway anthropic api-gateway bifrost-alternative
Total429
30 days +354
GetBusbar

The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping your native protocols intact: six native LLM protocols in, six native LLM protocols out.

Created May 2026

ai-gateway anthropic bedrock circuit-breaker claude
Total237
30 days +37
lsuen

SUGT - local AI gateway & skills console. Centralize LLM API keys, one OpenAI/Anthropic-compatible proxy for Claude Code, Codex & OpenCode. Homebrew & winget. Fully open source (Apache-2.0).

Created Jul 2026

Total140
30 days +133
soapbucket

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-compatible API, or serve vLLM and llama.cpp on your own GPUs. Keys, budgets, guardrails, semantic cache, WAF.

Created Apr 2026

Total54
30 days +0
9j

High-performance AI routing proxy built in Rust with automatic failover, priority-based routing, and support for 15+ providers (Anthropic, OpenAI, Cerebras, Minimax, Kimi, etc.)

Created Nov 2025 Archived

ai anthropic claude-code claude-code-router openai
Total517
30 days +1