Delta Lake MERGE Is Not a Simple Upsert. What Actually Happens at Scale.
At 10 TB, updating 200k rows can mean rewriting thousands of files. Here’s why, and what to do about it.
Spark Is Not Just Lazy. Spark Compiles Dataflow.
Why calling Spark ’lazy’ is technically reductive, and how thinking of it as a dataflow compiler changes the way you design pipelines.
A Shared Kernel Is a Shared Trust Domain
Containers isolate processes, not trust boundaries. When your platform runs untrusted code, the architectural question is where you place the kernel boundary, and what that costs in memory, latency, and operational complexity.
Fixing Skewed Nested Joins in Spark with Asymmetric Salting
In large-scale Spark pipelines, skew can occur when a single key carries a disproportionately large nested payload. Asymmetric salting offers a targeted solution: explode, salt, join in parallel, and optionally re-aggregate.
Two Algorithms, One Intuition: Shunting Yard and Pratt Parsing
Parsing arithmetic expressions looks simple… until precedence enters the picture. Two classic algorithms — Dijkstra’s Shunting Yard and Pratt’s Top-Down Operator Precedence — provide radically different answers that reveal the same underlying intuition.
Inside GKE Workload Identity: How Kubernetes Identities Become GCP Service Accounts
GKE Behind the Scenes: Understanding the Interaction Between Kubernetes and GCP Service Accounts Through The Metadata Server.