Simplify thread hand-off and remove thread when done

To not allow it lingering around, as a stale future
might cancel it when it's already executing another
task.