Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

Last push 1 week ago

aillamacppllmllmopsload-balancer

Created

Apr 2024

Total stars

1,695

Last 7 days

0

Last 30 days

0

Last 12 months

0

README badge

Rusty Stars badge for intentee/paddler

Show this repo's rank among all Rust repos. Paste the Markdown into the README.

[![Rusty Stars](https://rustystars.dev/badge/intentee/paddler.svg)](https://rustystars.dev/intentee/paddler)