Repositories
56
Total stars
61,227
Trending
Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.
Created Oct 2026
Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.
Created Jul 2022
A model-agnostic GPU runtime for shipping models as verified programs.
Created Aug 2026
Pure Rust Inference Engine
Created May 2026
High-performance Ultralytics YOLO inference in Rust with ONNX Runtime, GPU backends, CLI, and WebGPU/WASM.
Created Nov 2025
Top
Top 50 of 56
Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.
Created Jul 2022
Deep learning in Rust, with shape checked tensors and neural networks
Created Oct 2021
Safe rust wrapper around CUDA toolkit
Created Sep 2022
Pure Rust Inference Engine
Created May 2026
Demo project showing a single Rust codebase running on CPU and directly on GPUs
Created Jul 2025
A machine learning library for Rust.
Created Jun 2019
Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.
Created Oct 2026
NviWatch: A blazingly fast rust based TUI for managing and monitoring NVIDIA GPU processes
Created Sep 2024
SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.
Created Jul 2026
High-performance Ultralytics YOLO inference in Rust with ONNX Runtime, GPU backends, CLI, and WebGPU/WASM.
Created Nov 2025
Safe Rust wrapper for the NVIDIA Management Library
Created Mar 2017
A model-agnostic GPU runtime for shipping models as verified programs.
Created Aug 2026
High-performance quantitative finance in Rust and Python — 130+ stochastic processes, option pricing, calibration, fixed income, risk & copulas, with SIMD/GPU acceleration.
Created May 2023
A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.
Created Mar 2026
Lossless AI compression: 33% less GPU memory, bit for bit.
Created Sep 2026
A TOR v3 vanity url generator designed to run on an NVIDIA GPU.
Created Jul 2020
GPU-accelerated transparent compression S3-compatible storage gateway. Drop-in replacement for AWS S3 endpoints; cuts your S3 bill 50-80% with no app changes (Rust, nvCOMP, zstd).
Created May 2026
OxiCUDA replaces the entire NVIDIA CUDA Toolkit software stack with type-safe, memory-safe Rust code.
Created Apr 2026
Calibration software for the Murchison Widefield Array (MWA) radio telescope
Created Mar 2020
A Deep Learning and preprocessing framework in Rust with support for CPU and GPU.
Created Mar 2023
PSSA: a non-transformer language model written from scratch in Rust. Recurrent state-space layer, episodic memory, self-modifying weights, CUDA training, and a full terminal dashboard.
Created Aug 2026
LLaMa 7b with CUDA acceleration implemented in rust. Minimal GPU memory needed!
Created Apr 2023
A learning-focused, high-performance tensor computation library built from scratch in Rust, featuring automatic differentiation and CPU/CUDA backends.
Created May 2025
A minimal OpenCL, CUDA, Vulkan and host CPU array manipulation engine / framework.
Created Mar 2022
An inference engine for NVIDIA RTX card card family, built for a coding agent and its subagents, and a companion that listens, reads and draws: Qwen3.8 27B and 125B Flash-Next MoE, Qwen-Image-2.1, Qwen3-ASR on the CPU. OpenAI-compatible API and Playground in one binary.
Created Sep 2026