Why not just have the manager call exit when all workers are quiet?  No need to defer to the main thread, afaik.