GitHub topics

load-balancer

15% of these repos are in LLM Infrastructure.

Repositories

19

Total stars

12,111

Trending

GetBusbar

The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping your native protocols intact: six native LLM protocols in, six native LLM protocols out.

Created May 2026

ai-gateway anthropic bedrock circuit-breaker claude
Total226
30 days +33
intentee

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

Created Apr 2024

Total1,694
30 days +15
pingooio

The fast and secure Load Balancer / API Gateway / Reverse Proxy with built-in service discovery, GeoIP, WAF, bot protection and much more - https://pingoo.io

Created Sep 2025

akamai anti-bot apache2 api api-gateway
Total1,043
30 days +4

Top

intentee

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

Created Apr 2024

Total1,694
pingooio

The fast and secure Load Balancer / API Gateway / Reverse Proxy with built-in service discovery, GeoIP, WAF, bot protection and much more - https://pingoo.io

Created Sep 2025

akamai anti-bot apache2 api api-gateway
Total1,043
GetBusbar

The execution control plane for AI agents. Govern every model request, MCP tool call, A2A delegation, and downstream action—while keeping your native protocols intact: six native LLM protocols in, six native LLM protocols out.

Created May 2026

ai-gateway anthropic bedrock circuit-breaker claude
Total226
lakeops-org

Multi-engine and Agentic AI SQL query router in Rust for all data platforms and lakehouses—with routing, queuing, guardrails, tracking, sqlglot dialect translation and more.

Created Mar 2026

load-balancer queryengine rust
Total161
ats1999

A high-performance API gateway built with Rust, designed for low-latency request processing. Dakia supports dynamic configuration updates, allowing seamless changes to routing rules, authentication settings, rate limits, and other gateway policies without requiring a restart.

Created Aug 2023

api dakia gate-way http load-balancer
Total131
dorianverlaine

⚡ High-performance Rust web server and reverse proxy built on Cloudflare Pingora. 🧾 Caddyfile-compatible config · 🔐 automatic HTTPS · 🚀 HTTP/1.1, HTTP/2, HTTP/3 · ⚖️ load balancing · 📁 static files.

Created Jul 2026

acme async-rust caddyfile http3 letsencrypt
Total115
tako-sh

Build, run, and manage app instances anywhere, without thinking about the boring stuff

Created Feb 2026

application-server deploy development-server load-balancer realtime
Total75
christiandoxa

Prodex is a multi-account, multi-provider Codex wrapper with auto-rotation, Super mode, Smart Context, and token saving for Codex CLI and Claude Code

Created Mar 2026

Total61
soapbucket

Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-compatible API, or serve vLLM and llama.cpp on your own GPUs. Keys, budgets, guardrails, semantic cache, WAF.

Created Apr 2026

Total54

Related

All topics →