Categories / LLM Infrastructure
EricLBuehler/candle-vllm
Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.
Created
Oct 2023
Total stars
742
Last 7 days
+3
Last 30 days
+8
Last 12 months
+227
README badge
[](https://rustystars.dev/EricLBuehler/candle-vllm)