Categories / More categories

LLM Infrastructure

Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.

Repos

123

Stars

73,559

Last 30 days

+5,730

yym68686

This is a project that unifies the management of LLM APIs. It can call multiple backend services through a unified API interface, convert them to the OpenAI format uniformly, and support load balancing. Currently supported backend services include: OpenAI, Anthropic, DeepBricks, OpenRouter, Gemini, Vertex, etc.

Created Jul 2024

Total1,273
30 days +10
RockChinQ

LLM 逆向工程接口管理 | 通过标准 OpenAI API 访问 ChatGPT / gpt4free / Bard / Claude / HuggingChat / 通义千问 等 AI 的破解版 || ChatGPT reverse engineering API management | Access all reverse engineered LLM libs by standard OpenAI API format || 免费 ChatGPT Free GPT LLM API | 逆向工程 转 OpenAI API | converts all llm libs to OpenAI API

Created Sep 2023

Total953
30 days +30
smg-project

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

Created Nov 2025

anthropic anthropic-api chat claude gemini
Total734
30 days +76
Ai00-X

The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.

Created Jul 2023

ai aiagents chatgpt chatgpt-api embed
Total624
30 days +2
eliasbiondo

🕷️ The fastest open-source AI Gateway and LLM Gateway — one binary with built-in console to route, secure, budget and observe every LLM call across OpenAI, Anthropic, Gemini, Bedrock and more.

Created Oct 2026

a2a ai-gateway anthropic api-gateway bifrost-alternative
Total429
30 days +365
ghuntley

underclass: an OpenAI-compatible pooling proxy that pins sessions to one account (prompt cache stays warm), cools quota-exhausted subscriptions until their window resets, and fails fast with the earliest Retry-After when the whole pool is dry.

Created Sep 2026

openai openai-proxy-load-balancer
Total303
30 days +42
GetBusbar

The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping your native protocols intact: six native LLM protocols in, six native LLM protocols out.

Created May 2026

ai-gateway anthropic bedrock circuit-breaker claude
Total226
30 days +30
lsuen

SUGT - local AI gateway & skills console. Centralize LLM API keys, one OpenAI/Anthropic-compatible proxy for Claude Code, Codex & OpenCode. Homebrew & winget. Fully open source (Apache-2.0).

Created Jul 2026

Total138
30 days +128
AlephantAI

Alephant is an open-source AI Agent Gateway for routing, tracking, and controlling LLM usage across AI agents, members, and workflows, and for publishing agent capabilities as paid endpoints with x402 and MPP payment rails.

Created May 2026

agent ai ai-agent ai-gateways claude
Total119
30 days +4
mxyhi

Local AI API gateway for OpenAI / Gemini / Anthropic. Runs on your machine, keeps tokens counted (SQLite), offers priority-based load balancing, optional OpenAI Chat↔Responses format conversion, and one-click setup for Claude Code / Codex.

Created Jan 2026

api-key-managenment claude-api claude-code codex openai
Total94
30 days +1
soapbucket

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-compatible API, or serve vLLM and llama.cpp on your own GPUs. Keys, budgets, guardrails, semantic cache, WAF.

Created Apr 2026

Total54
30 days +0
9j

High-performance AI routing proxy built in Rust with automatic failover, priority-based routing, and support for 15+ providers (Anthropic, OpenAI, Cerebras, Minimax, Kimi, etc.)

Created Nov 2025 Archived

ai anthropic claude-code claude-code-router openai
Total519
30 days +1