Add extra asserts to fix intermittent

-Hopefully this makes it easier to pinpoint which section is
intermittent