Important change is the manual setting of the partitions in parallelize. This has significantly improved cluster performance.