Only reduce the number of cores once when removing an executor

@markhamstra