Spark Is Not Just Lazy. Spark Compiles Dataflow.
Why calling Spark ’lazy’ is technically reductive, and how thinking of it as a dataflow compiler changes the way you design pipelines.
Data-Engineering
Fixing Skewed Nested Joins in Spark with Asymmetric Salting
In large-scale Spark pipelines, skew can occur when a single key carries a disproportionately large nested payload. Asymmetric salting offers a targeted solution: explode, salt, join in parallel, and optionally re-aggregate.