Essays

Architectural essays on data systems, infrastructure, and software abstractions. These pieces focus on structural questions, design tensions, and broader shifts in the modern technical stack.

Where Spark Changes Shape: UnsafeRow, the JVM, and the Relocation of Portability

Run df.explain on a Parquet scan and you find ColumnarToRow, an operator whose only job is to change the data’s shape. It is the seam where two architectural eras meet, and a record of how portability in analytical systems has relocated from the JVM runtime to the data format and the query plan.

The Disaggregation of the Lakehouse Stack

How Delta Kernel, Arrow, and pluggable execution are disaggregating the lakehouse stack. The lakehouse stack is not converging on a new dominant engine — it is converging on a layered architecture in which protocol, data representation, and query execution are increasingly isolated behind stable interfaces.

Where Data System Abstractions Break: A Semiotic Reading

Many of the most surprising performance pathologies in modern data systems are semiotic failures — structural divergences between what an interface signifies and what the underlying system does.

Delta Lake MERGE Is Not a Simple Upsert. What Actually Happens at Scale.

At 10 TB, updating 200k rows can mean rewriting thousands of files. Here’s why, and what to do about it.

A Shared Kernel Is a Shared Trust Domain

Containers isolate processes, not trust boundaries. When your platform runs untrusted code, the architectural question is where you place the kernel boundary, and what that costs in memory, latency, and operational complexity.