Try to work around rare event when the USB wakeup fires but the host
thinks we are still asleep.