Signal the semaphore on failures too.