Should remove an idle worker