Fix jumping back to the last visited buffer