Beginners mistake. Set the boolean after releasing the semaphore