I suspect that weakening this assert is just hiding bugs where there is a 'dangling' reference to a PG that should not be scheduled.  I forget exactly what led us down the road of keeping the queue strictly coherent with the PG state but I'm pretty sure there was a reason :)

If the assert is triggering, it is probably because you aren't yet doing unreg/reg surrounding the transtion to/from is_primary() == true.