Technology
Engineer demonstrates database-free analytics dashboards using single Parquet files
A technical experiment shows how browser-based HTTP range requests over object storage can serve interactive charts without dedicated analytical databases.
The short version
- A software engineer built an experimental dashboard running directly in the browser using an 18KB JavaScript reader, HTTP range requests, and Cloudflare R2 object storage without a traditional backend database.
- The demonstration used a 40MB Parquet data cube based on 34 million NYC 311 service request records to deliver fast filtering across 15 years of data.
- The approach relies on precomputed grouping sets, sorted row groups, and metadata footers to allow the browser to fetch only specific required byte ranges.
- The author estimates that storage and write costs for serving 10,000 customers via daily file updates remain low compared to traditional data infrastructure.
Key facts
- An engineer created a browser-based analytics demonstration that queries a single 40MB Parquet file using Hyparquet, an 18KB JavaScript reader, and Cloudflare R2 object storage.[Hacker News]
- The test dataset comprised roughly 34 million NYC 311 service request records spanning approximately 15 years.[Hacker News]
- The architecture eliminates traditional analytical query engines by structuring data into precomputed grouping sets sorted by filter columns.[Hacker News]
- Queries operate by reading the Parquet file's metadata footer to identify and request specific byte ranges, avoiding full file downloads.[Hacker News]
- A Cloudflare Worker was included in the setup to proxy and cache byte ranges at the edge to bypass rate limits on standard storage URLs.[Hacker News]
- The author estimated that rebuilding 10MB cubes once daily for 10,000 customers on R2 would cost approximately $1.50 per month in storage and $1.35 per month in write operations.[Hacker News]
What remains uncertain
- The real-world applicability of the approach is constrained to low-cardinality, predictable dashboards updated on a schedule rather than real-time, high-cardinality applications.[Hacker News]