Fix race condition

Use a condition variable to signal the work is finished

/cc @benjaminmateev