Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
Created
Apr 2024
Total stars
1,695
Last 7 days
0
Last 30 days
0
Last 12 months
0
README badge
Show this repo's rank among all Rust repos. Paste the Markdown into the README.
[](https://rustystars.dev/intentee/paddler)