Fix calling wait of a semaphore is always instant.

As title.