Change:
BufReader's seek() implementation always dumps the buffer. This is not good for performance, where users may want to skip a variable amount of buffered data -- only BufReader really knows if it's reasonable to move the pointer or not. Additionally, only BufReader can know if the pointer can be reversed or not -- so there's absolutely no cheap way to "unget" data from BufReader, even if you know it's seekable. I'd recommend removing this behavior and moving it to unwrap() or another method (sync?), but it's now baked into a stable API. What are the options now?
(I'm in the process of writing my own BufReader implementation that doesn't do this.)
Currently BufReader saves only the offset within buffered data. As I understand to keep buffer during seek it needs to save "global offset" within inner Reader and update it in all read/seek operations. It needs this "global offset" because seek operation should return new position in underlying Reader which is not currently saved anywhere. Another variant is to make a separate function (e.g. ) which will use only buffered data, will move only relative to current position and return true/false (true if it was able to move current position inside the buffer; false - nothing happened and user needs to make a separate call to "seek" to actually move the pointer).
I'm not trying to avoid seeking the underlying stream, I'm trying to avoid dumping the buffer. In my implementation, small seeks that fit in the existing buffer cause a call to (SeekFrom::Current(0)).
Could you please share your implementation? You use (SeekFrom::Current(0)) to get current file offset. Am I correct?
Not without getting source approval from my employer, sorry. Yes, I implement and then in seek I have snippets like:
Some time ago I had a similiar requirement, I extracted the implementation into a own library: , maybe its useful for you.
I stumbled upon this today. When decoding mp4 movies, the decoder will do small seeks forward to skip in the region of 100 bytes at a time. I used a BufReader with 65k capacity to avoid my underlying reader doing http requests for every seek, only to find that BufReader isn't helping at all. I think BufReader could be more useful if it worked from the buffer also when seek fits.
I would like to see a PR that implements without dumping the buffer if the offset is within the buffered data. I may be missing something but I don't think BufReader would need to store a global offset in order to make this work.
FWIW, this solution requires R:Seek. A function would not have such a requirement.
This should not be closed, the new method is still unstable. It’s been a couple months though: fcp merge
to confirm, this is for the method, right?
Yes, that is the one item with .
Hmm, maybe I needed to reopen the issue before making the command: fcp merge
Team member has proposed to merge this. The next step is review by the rest of the tagged teams: [x] [x] [x] [ ] [x] [x] [x] No concerns currently listed. Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See for info about what commands tagged team members can give me.
:bell: This is now entering its final comment period, as per the . :bell:
The final comment period is now complete.
I agree with and don't think a separate method is needed for this. This can just be the behavior when calling .
I think it could be done something like this:
I don't have any real cases in mind but changing the existing seek semantics could cause subtle bugs for people who are relying on it. E.g. if the user is expecting the data at an in-buffer offset to change in the underlying reader and is using seek to refresh the buffer, this optimization would cause it to yield stale data.
I stumbled over this ticket when implementing a library that parses media files. What makes me uneasy about using (besides being available in nightly only) is the fact that often I do not have the current position. From reading the source code of it seems that the internal buffer is flushed even when doing to get the current position. It would be great to add an optimization path for this case (get current position) to the trait implementation of .
Hi I'm new to rust, I'm surprised by this behaviour. Can I ask why should flush the buffer on seek within the buffer, like ever? I understand this is documented behaviour of a stable api, but I can't think of any reason to write code that would rely on this intentionally, and also can't think of any code that could break unintentionally. I could be wrong but it feels like this was perhaps excessive documentation that now constrains the behaviour of this api to remain inefficient.
Because that's the way to sync the BufReader's idea of where the file pointer is with the underlying reader. The other alternative would be to make do it, but it can't because it doesn't depend on .
It seems like the major issue then is that we are worried people using seekrelative will lose track of where they are in the file/stream and try to seek past the end? The function already returns a result so why is this really an issue? This is my first time stumbling upon a nightly feature so I am just curious.
No, the major issue is one of compatibility guarantees. Right now, if you want to sync the BufReader's position with the underlying Reader and extract it, you and then . So that pattern needs to continue to work.
Can't we just leave alone then and add a separate api or whatever which has the little enhancement of not losing the buffer when you seek within it?
That already happened:
I've been advocating for because the existing method requires where that's not necessary for a BufReader.
do you want to write up a pr? I doubt it would be very difficult. Just refactor s code that already does this out into the method?
I do find it mildly ironic that the that it would always flush the buffer and why.
Life is a little busy right now, so if someone else is excited to do it, go for it. Otherwise I'll see if I can't find some time to put together a PR.
it seems that there was a for this in Mar 2018 that passed with no objections. Does that now mean there can be a stabilization PR?
At least for my use-case of parsing media files I'm still relying on the crate instead of using because there seems to be no way in stdlib BufReader to get the current position without dumping the buffer.
One of the challenges I have been finding with BufReader is that the stream_position convenience method is implemented with which dumps the buffer. Is it possible to have this case specialized so that getting the current position doesn't cause the buffer to be thrown away?
I'm not entirely clear on the status of . It was implemented, this issue was denoted as the tracking issue, there's was a successful fcp, but it's still unstable? What was the fcp for?
someone still needs to open a PR to mark the method as stable.
As someone unfamiliar with the processes of Rust development, is there a chance of getting this into one of the upcoming releases?
The relevant PR passed FCP, has been approved to merge, and this should become stable in 1.53.0 if all goes well.

Discussion:
/// the buffer will not be flushed, allowing for more efficient seeks. /// This method does not return the location of the underlying reader, so the caller /// must track this information themselves if it is required. <del> #[unstable(feature = "bufreader_seek_relative", issue = "31100")] </del> <ins> #[stable(feature = "bufreader_seek_relative", since = "1.53.0")] </ins> pub fn seek_relative(&mut self, offset: i64) -> io::Result<()> { let pos = self.pos as u64; if offset < 0 {