rollback next_limit() again, the performance difference is huge:

Given next_limit = memory_limit
$ time python pyspark/shuffle.py
real    0m20.674s
user    0m6.119s
sys 0m13.993s

Given next_limit = max(memory_limit, used_memory * 1.05)
time python pyspark/shuffle.py
real    0m0.583s
user    0m0.488s
sys 0m0.093s