Don't unref the async handle used as a waker

Otherwise the loop could spin like crazy if there are no active handles