Refactor interval worker destruction