Limit the computed optimal partition size based on the observed minimum memory
size at the workers.