GitHub topics

openai-api

70% of these repos are in LLM Infrastructure.

Repositories

27

Total stars

7,445

Trending

RustingStudio

Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.

Created Oct 2026

anthropic-api cuda gguf inference llama-cpp
Total294
30 days +286
NIyueeE

DeepSeek网页端API代理,支持OpenAI与Anthropic兼容接口 | OpenAI & Anthropic compatible API proxy for DeepSeek web

Created Mar 2026

api api-proxy claude-api deepseek-api freeapi
Total882
30 days +71
chrishayuk

Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.

Created Mar 2026

huggingface inference llm mechanistic-interpretability metal
Total1,193
30 days +22
mezmo

AURA is a production-tested SRE agent platform you can deploy in minutes. AURA handles the guardrails, APIs, state management, streaming, and failure handling required to put AI to work safely on production infrastructure.

Created Mar 2026

a2a ai-agent-framework ai-agents ai-sre aiops
Total414
30 days +21
Ai00-X

The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.

Created Jul 2023

ai aiagents chatgpt chatgpt-api embed
Total624
30 days +2

Top

chrishayuk

Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.

Created Mar 2026

huggingface inference llm mechanistic-interpretability metal
Total1,193
NIyueeE

DeepSeek网页端API代理,支持OpenAI与Anthropic兼容接口 | OpenAI & Anthropic compatible API proxy for DeepSeek web

Created Mar 2026

api api-proxy claude-api deepseek-api freeapi
Total882
Ai00-X

The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.

Created Jul 2023

ai aiagents chatgpt chatgpt-api embed
Total624
mezmo

AURA is a production-tested SRE agent platform you can deploy in minutes. AURA handles the guardrails, APIs, state management, streaming, and failure handling required to put AI to work safely on production infrastructure.

Created Mar 2026

a2a ai-agent-framework ai-agents ai-sre aiops
Total414
RustingStudio

Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.

Created Oct 2026

anthropic-api cuda gguf inference llama-cpp
Total294
tracel-ai

Democratizing large model inference and training on any device.

Created Dec 2024

burn large-model-inference open-webui openai-api rust
Total267
ferrumox

A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.

Created Mar 2026

Total191
fuyufjh

A simple command-line interface for ChatGPT API written in Rust ⚡️

Created Mar 2023

chatgpt cli gpt gpt-3 gpt-4
Total161
AprilNEA

Turn AI subscriptions into standard API endpoints. OpenAI & Anthropic compatible gateway — for Amp Code, Cursor, Factory CLI, and more.

Created Feb 2026

ai-gateway amp anthropic-api claude copilot
Total141
azrtydxb

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLLM/SGLang — 0.76 µs per request, no I/O on the request path, cache-affinity routing, RBAC, budgets and a 13-screen UI in the binary.

Created Aug 2026

Total109
shenjinti

A free reverse proxy and CLI tool for OpenAI GPT-3.5-turbo.

Created Apr 2024

api chat-gpt free-gpt gpt gpt-35-turbo
Total96
mxyhi

Local AI API gateway for OpenAI / Gemini / Anthropic. Runs on your machine, keeps tokens counted (SQLite), offers priority-based load balancing, optional OpenAI Chat↔Responses format conversion, and one-click setup for Claude Code / Codex.

Created Jan 2026

api-key-managenment claude-api claude-code codex openai
Total94
enapt

Free, open-source app that runs AI chat models on your own computer — and lets computers team up over the internet to run models too big for one machine. No account, no crypto. OpenAI- and Anthropic-compatible API.

Created Feb 2026

ai anthropic-api candle chatgpt-alternative claude-code
Total90
gpillon

An inference engine for NVIDIA RTX card card family, built for a coding agent and its subagents, and a companion that listens, reads and draws: Qwen3.8 27B and 125B Flash-Next MoE, Qwen-Image-2.1, Qwen3-ASR on the CPU. OpenAI-compatible API and Playground in one binary.

Created Sep 2026

Total67
pegainfer-project

OpenAI-compatible speech serving in Rust: streaming TTS and full-duplex voice over GPT-Live. 128 concurrent PersonaPlex sessions on one GPU.

Created Sep 2026

cuda full-duplex gpt-live inference-server llm-serving
Total51

Related

All topics →