Ensure the waitqueue is empty before removing it