arrow
Repositories
35
Total stars
77,530
Trending
Apache DataFusion SQL Query Engine
Created Apr 2021
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.
Created Feb 2024
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Created Dec 2023
Apache DataFusion Ballista Distributed Query Engine
Created May 2022
Pushdown cache for DataFusion
Created Dec 2024
An open lakehouse engine for agents — Postgres-native, vector-native and Python-native. Speaks PostgreSQL, runs on your object storage, written in Rust.
Created Dec 2021
Top
Apache DataFusion SQL Query Engine
Created Apr 2021
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Created Dec 2023
Create full-fledged APIs for slowly moving datasets without writing a single line of code.
Created Dec 2020
An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.
Created Feb 2024
An open lakehouse engine for agents — Postgres-native, vector-native and Python-native. Speaks PostgreSQL, runs on your object storage, written in Rust.
Created Dec 2021
Apache DataFusion Ballista Distributed Query Engine
Created May 2022
Rust-based WebAssembly bindings to read and write Apache Parquet data
Created Feb 2022
DataFusion has now been donated to the Apache Arrow project
Created Dec 2016
Pushdown cache for DataFusion
Created Dec 2024
Command line tool for inspecting Parquet files
Created Feb 2021
A fast excel reader for Rust and Python
Created Oct 2022
Unofficial rust implementation of Apache Iceberg with integration for Datafusion
Created Dec 2022
First-class compile‑time Arrow schemas for Rust.
Created Aug 2025
Batteries included CLI, TUI, and server implementations for DataFusion.
Created Mar 2022
Work with bioinformatic files using Arrow, Polars, and/or DuckDB
Created Apr 2023
Library for bringing distributed capabilities to Apache DataFusion
Created Jun 2025
State-first Streaming Applications in Rust
Created Apr 2019
DuckLake took Flight. Welcome to SwanLake.
Created Nov 2025
A data science toolkit for the H3 geospatial grid
Created Sep 2021
Starcraft 2 Protocol Replay Reader
Created Feb 2023
High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, speculative fan-out, 24.0 rows/sec measured.
Created Sep 2026
Data streaming runtime focused on performance, consistency, and extensibility. Write plugins in Rust or WASM and process data with data guarantees.
Created Jun 2026
Fast, minimal, and ergonomic zero-copy Apache Arrow tabular data implementation in Rust, with PyO3, Python bindings, NdArray, XArray, shared-memory and low-latency SIMD for data-engineering and columnar workloads. Bridge to Polars and Arrow-rs over FFI effortlessly.
Created Aug 2025
A set of tools for writing servers that speak PostgreSQL's wire protocol
Created Apr 2021
Incremental view maintenance & query rewriting for materialized views in DataFusion
Created Dec 2024
Exon is an OLAP query engine specifically for biology and life science applications.
Created May 2023
A distributed OpenTelemetry-native columnar datastore whose only backend is object storage.
Created Aug 2026