Try not adding an extra jump when editing a file

This will let me just <c-o> to go back a buffer, which may make BufSurf
somewhat useless.