Categories / LLM Infrastructure

EricLBuehler

EricLBuehler/candle-vllm

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

Last push 1 month ago

Created

Oct 2023

Total stars

742

Last 7 days

+3

Last 30 days

+8

Last 12 months

+227

README badge

Rusty Stars badge for EricLBuehler/candle-vllm
[![Rusty Stars](https://rustystars.dev/badge/EricLBuehler/candle-vllm.svg)](https://rustystars.dev/EricLBuehler/candle-vllm)