Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Created Dec 2023
Repositories
32
Total stars
26,402
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Created Dec 2023
Apache Kafka® compatible broker with S3, PostgreSQL, SQLite, Apache Iceberg and Delta Lake
Created Aug 2024
Parseable is an open source, unified infrastructure observability platform built in Rust on a data lake architecture. It tracks logs, metrics, traces, and events across apps, agents, and systems, reducing storage costs by up to 90% through columnar telemetry compression.
Created Aug 2022
Search library built on Parquet. Fast, efficient, and scalable.
Created May 2026
Pushdown cache for DataFusion
Created Dec 2024
Real-time analytics on Postgres tables
Created Sep 2024
View parquet files online
Created Nov 2024
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Created Dec 2023
Create full-fledged APIs for slowly moving datasets without writing a single line of code.
Created Dec 2020
Parseable is an open source, unified infrastructure observability platform built in Rust on a data lake architecture. It tracks logs, metrics, traces, and events across apps, agents, and systems, reducing storage costs by up to 90% through columnar telemetry compression.
Created Aug 2022
Real-time analytics on Postgres tables
Created Sep 2024
Apache Kafka® compatible broker with S3, PostgreSQL, SQLite, Apache Iceberg and Delta Lake
Created Aug 2024
Copy to/from Parquet in S3, Azure Blob Storage, Google Cloud Storage, http(s) stores, local files or standard inout stream from within PostgreSQL
Created Sep 2024
Rust-based WebAssembly bindings to read and write Apache Parquet data
Created Feb 2022
Pushdown cache for DataFusion
Created Dec 2024
Command line SQL interface for relational databases and common data file formats
Created Mar 2024
Command line tool for inspecting Parquet files
Created Feb 2021
Fastest and safest Rust implementation of parquet. `unsafe` free. Integration-tested against pyarrow
Created Mar 2021
A minimal Python library for Apache Arrow, connecting to the Rust Arrow crate
Created Apr 2024
A command line tool to query an ODBC data source and write the result into a parquet file.
Created Aug 2020
View parquet files online
Created Nov 2024
GPU-accelerated transparent compression S3-compatible storage gateway. Drop-in replacement for AWS S3 endpoints; cuts your S3 bill 50-80% with no app changes (Rust, nvCOMP, zstd).
Created May 2026
Search library built on Parquet. Fast, efficient, and scalable.
Created May 2026
Query and transform data with PRQL
Created Oct 2022
🗄️ A simple CLI for converting WARC to Parquet.
Created Jun 2022
High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, speculative fan-out, 24.0 rows/sec measured.
Created Sep 2026
Fast parquet command line tool with many functions, nailed it!
Created Jun 2025
Export PostgreSQL table or query into Parquet file
Created Oct 2022
Fast psql-style database CLI/workbench for PostgreSQL, MySQL, SQLite, ClickHouse, MongoDB, Elasticsearch, and SQL over Parquet/CSV/JSON — with optional AI, SSH/Vault, Python & Django.
Created Jul 2025
Vector search using only Parquet and DataFusion
Created Jan 2026
Collection of AWS Lambdas for creating and managing Delta tables
Created May 2023
Unified observability for logs, metrics and traces — low overhead instrumentation (Rust, Unreal Engine 5), OTLP ingestion, SQL analytics on DataFusion over Parquet, presentation through notebooks, Grafana and Python. Self-hosted.
Created Jan 2024