Categories / More categories

LLM Infrastructure

Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.

Repos

124

Stars

73,822

Last 30 days

+5,802

ThinkWatchProject

Self-hosted AI API and MCP gateway for organizations: SSO and RBAC, per-user identity for MCP tool calls, PII redaction and tool-call guards, rate limits, budgets, cost accounting and audit logs.

Created Apr 2026

ai-gateway ai-governance ai-proxy audit-log audit-logging
Total817
30 days +4
eliasbiondo

🕷️ The fastest open-source AI Gateway and LLM Gateway — one binary with built-in console to route, secure, budget and observe every LLM call across OpenAI, Anthropic, Gemini, Bedrock and more.

Created Oct 2026

a2a ai-gateway anthropic api-gateway bifrost-alternative
Total429
30 days +354
taichuy

Build for agents — from the AI Gateway to the entire application stack.

Created Mar 2026

agent-native agent-observability ai-app-builder ai-gateway claude-api
Total296
30 days +15
GetBusbar

The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping your native protocols intact: six native LLM protocols in, six native LLM protocols out.

Created May 2026

ai-gateway anthropic bedrock circuit-breaker claude
Total237
30 days +37
AprilNEA

Turn AI subscriptions into standard API endpoints. OpenAI & Anthropic compatible gateway — for Amp Code, Cursor, Factory CLI, and more.

Created Feb 2026

ai-gateway amp anthropic-api claude copilot
Total141
30 days +3
lsuen

SUGT - local AI gateway & skills console. Centralize LLM API keys, one OpenAI/Anthropic-compatible proxy for Claude Code, Codex & OpenCode. Homebrew & winget. Fully open source (Apache-2.0).

Created Jul 2026

Total140
30 days +133
LiteLLM-Labs

Onboard AI tools in one step, and catch the rest. Relay sets up Claude Desktop, Claude Code, and Codex to run through your LiteLLM AI Gateway with zero dev setup and routes shadow AI (Notion AI, Perplexity, …) there too.

Created Jul 2026

ai-gateway ai-governance ai-monitoring ai-proxy shadow-ai
Total112
30 days +0
azrtydxb

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLLM/SGLang — 0.76 µs per request, no I/O on the request path, cache-affinity routing, RBAC, budgets and a 13-screen UI in the binary.

Created Aug 2026

Total109
30 days +3
Noveum

Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly and quickly.

Created Nov 2024

Total100
30 days +2
soapbucket

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-compatible API, or serve vLLM and llama.cpp on your own GPUs. Keys, budgets, guardrails, semantic cache, WAF.

Created Apr 2026

Total54
30 days +0