Clean up the new messages task properly

So it doesn't embarrassingly backtrace all over anything at shutdown.