core: fix backstepping with ordered chapters

There were two problems.

First, frames past the end of the current segment were added to the
index, which messed up backstepping. Check for the endpts before
added a frame to the index.

Second, it wasn't possible to step over segments which change the file.
Changing a file causes decoder reinitialization, which (rightfully)
is treated as discontinuity (and vo_pts_history_seek_ts was changed).
Add some extra code to pretend that a segment-switching seek/reinit
does not introduce discontinuities.

There's still a weird corner case: sometimes, you can frame step forward
on the last frame of a segment without reaching the next segment
immediately. This is because the playloop switches into audio-only mode.
The segment is switched when both audio and video have ended, so the
frame stepping will play random sized chunks of audio until the segment
will be switched. This gives the impression that backstepping doesn't
work perfectly, even though it's the other way around and frame stepping
behaves weird. This is a consequence of wanting to make frame stepping
work with audio, and is not really a bug.