Repositories

35

Total stars

77,530

Trending

pola-rs

Extremely fast Query Engine for DataFrames, written in Rust

Created May 2020

arrow dataframe dataframe-library dataframes out-of-core
Total40,388
30 days +303
vortex-data

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

Created Feb 2024

array arrow compression file multimodal
Total3,331
30 days +61
lakehq

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Created Dec 2023

apache-iceberg apache-spark arrow artificial-intelligence big-data
Total3,503
30 days +46
lakesoul-io

An open lakehouse engine for agents — Postgres-native, vector-native and Python-native. Speaks PostgreSQL, runs on your object storage, written in Rust.

Created Dec 2021

Total3,261
30 days +7

Top

pola-rs

Extremely fast Query Engine for DataFrames, written in Rust

Created May 2020

arrow dataframe dataframe-library dataframes out-of-core
Total40,388
lakehq

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Created Dec 2023

apache-iceberg apache-spark arrow artificial-intelligence big-data
Total3,503
roapi

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

Created Dec 2020

analytics arrow blob-storage cloud-native columnar
Total3,435
vortex-data

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

Created Feb 2024

array arrow compression file multimodal
Total3,331
AkashPriyadarshii

High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, speculative fan-out, 24.0 rows/sec measured.

Created Sep 2026

arrow cli data-cleaning data-engineering dataset-curation
Total115
chmp

Convert sequences of Rust objects to Arrow tables

Created Aug 2021

Total111
SpaceCell

Fast, minimal, and ergonomic zero-copy Apache Arrow tabular data implementation in Rust, with PyO3, Python bindings, NdArray, XArray, shared-memory and low-latency SIMD for data-engineering and columnar workloads. Bridge to Polars and Arrow-rs over FFI effortlessly.

Created Aug 2025

apache apache-arrow arrow arrow-rs bindings
Total103
NOFireAI

A distributed OpenTelemetry-native columnar datastore whose only backend is object storage.

Created Aug 2026

apache-2-0-license arrow columnar-storage datafusion datastore
Total50

Related

All topics →