Shift around where the locks in PriorityScheduler happen shortly