core: fix bogus condition that broke backstepping with last commit

This broke all cases where indexing was required, and the current frame
wasn't the first frame in a segment.