Categories / More categories

LLM Infrastructure

Running and serving large language models, LLM API gateways and proxies, and retrieval-augmented generation (RAG) systems.

Repos

123

Stars

73,596

Last 30 days

+5,823

winfunc

A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.

Created Jan 2025

ai anthropic anthropic-claude api chain-of-thought
Total5,352
30 days +9
yym68686

This is a project that unifies the management of LLM APIs. It can call multiple backend services through a unified API interface, convert them to the OpenAI format uniformly, and support load balancing. Currently supported backend services include: OpenAI, Anthropic, DeepBricks, OpenRouter, Gemini, Vertex, etc.

Created Jul 2024

Total1,274
30 days +10
RockChinQ

LLM 逆向工程接口管理 | 通过标准 OpenAI API 访问 ChatGPT / gpt4free / Bard / Claude / HuggingChat / 通义千问 等 AI 的破解版 || ChatGPT reverse engineering API management | Access all reverse engineered LLM libs by standard OpenAI API format || 免费 ChatGPT Free GPT LLM API | 逆向工程 转 OpenAI API | converts all llm libs to OpenAI API

Created Sep 2023

Total953
30 days +30
smg-project

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

Created Nov 2025

anthropic anthropic-api chat claude gemini
Total735
30 days +76
LeenHawk

gproxy is a Rust-based multi-channel LLM proxy that exposes OpenAI / Claude / Gemini-style APIs through a unified gateway, with a built-in admin console, user/key management, and request/usage auditing.

Created Feb 2026

Total296
30 days +36
GetBusbar

The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping your native protocols intact: six native LLM protocols in, six native LLM protocols out.

Created May 2026

ai-gateway anthropic bedrock circuit-breaker claude
Total226
30 days +30
AprilNEA

Turn AI subscriptions into standard API endpoints. OpenAI & Anthropic compatible gateway — for Amp Code, Cursor, Factory CLI, and more.

Created Feb 2026

ai-gateway amp anthropic-api claude copilot
Total141
30 days +3
AlephantAI

Alephant is an open-source AI Agent Gateway for routing, tracking, and controlling LLM usage across AI agents, members, and workflows, and for publishing agent capabilities as paid endpoints with x402 and MPP payment rails.

Created May 2026

agent ai ai-agent ai-gateways claude
Total119
30 days +4
ai-switch

Aides大模型本地网关,Ai服务商快捷切换,AI本地路由,AI厂商聚合。

Created Jul 2026

cc-switch claude cocokpit codex cpa
Total79
30 days +6