Fix panic for sending on closed channel

Make sure we stop our loops before closing the channels.