Handle upsert with multiple semaphores better