Spark tip: Disable Coalescing Post Shuffle Partitions for compute intensive tasks
TL;DR   Set spark.sql.adaptive.coalescePartitions.enabled to false when performing compute...
Tags: spark, performance, bigdata, machinelearning