Parquet

On the JVM, optimizing a hot kernel is not only about writing faster code: it is also about understanding how much the result depends on the machine code HotSpot derives from the scalar loop. Using Parquet bit-unpacking as a concrete case, the piece shows that a SIMD speedup depends on which scalar baseline C2 is handed, when explicit vectorization is actually justified, and why a more specialized scalar routine is not necessarily faster.

A purchases table, one Parquet file per day, every file sorted by customer: you would expect a GROUP BY customer to use that order, yet EXPLAIN often shows it ignored. This explainer follows the ordering from the file scan to the aggregate in Apache DataFusion, shows where it gets lost, how to read it in the plan, and what you can change.

Parquet rarely stores a column’s values directly. Dictionary encoding keeps the distinct values once and represents the column through small bit-packed integer indices. This explainer walks through dictionary encoding, how the indices are bit-packed, and the bit-unpacking step that reconstructs them into an int[] before every dictionary lookup.