a performance step back but simplify the hand off between middle pending and backend pending. this should allow us to treat every job we push into the queue the same. after we've got it threaded we can go back and redesign it to query multiple ids at a time and do batches or whatever means we deem fit
start implementing threadsafe reading from middle