Repository: opencv
Issue: CvCapture_FFMPEG::setProperty(CAP_PROP_POS_*) sets `picture_pts` inconsistently
Labels: bug,category: videoio

This code wasn't touched in a long time. I don't know if there are tests for it.

https://github.com/opencv/opencv/blob/9208dcb07c015e1fda44e40bb07b43c700b4bf46/modules/videoio/src/cap_ffmpeg_impl.hpp#L1957-L1978

I just noticed some inconsistencies in seeking that might be caused by this.

1. the `seek()` call has overloads for `double` (seconds) and `int64_t` (frame number) but in the POS_AVI_RATIO case it's called like it took a timestamp (in timebase)
2. the `picture_pts` value is set wrong _all_ cases

Test video, 23.976 fps, about 10 seconds, specifics not important):

`$ ffmpeg -f lavfi -i testsrc=duration=10:size=320x100:rate=24000/1001:decimals=3 -c:v libx264 -b:v 1M -y testsrc.mov`

Test script:
```python
import cv2 as cv

vid = cv.VideoCapture("testsrc.mov", apiPreference=cv.CAP_FFMPEG)
assert vid.isOpened()

print("seek: frame 0")
vid.set(cv.CAP_PROP_POS_FRAMES, 0)
print("at:", vid.get(cv.CAP_PROP_POS_FRAMES), "frames,", vid.get(cv.CAP_PROP_POS_MSEC), "msec")

for k in range(5):
    (rv, frame) = vid.read()
    assert rv
    print("read")
    #imshow(frame)
    print("at:", vid.get(cv.CAP_PROP_POS_FRAMES), "frames,", vid.get(cv.CAP_PROP_POS_MSEC), "msec")

print("seek: frame 3")
vid.set(cv.CAP_PROP_POS_FRAMES, 3)
#print("seel: msec 130")
#vid.set(cv.CAP_PROP_POS_MSEC, 130)
print("at:", vid.get(cv.CAP_PROP_POS_FRAMES), "frames,", vid.get(cv.CAP_PROP_POS_MSEC), "msec")

for k in range(5):
    (rv, frame) = vid.read()
    assert rv
    #print("read")
    #imshow(frame)
    print("at:", vid.get(cv.CAP_PROP_POS_FRAMES), "frames,", vid.get(cv.CAP_PROP_POS_MSEC), "msec")
```

That emits this:
```
seek: frame 0
at: 0.0 frames, 0.0 msec
at: 1.0 frames, 0.0 msec
at: 2.0 frames, 41.708333333333336 msec
at: 3.0 frames, 83.41666666666667 msec
at: 4.0 frames, 125.12499999999999 msec
at: 5.0 frames, 166.83333333333334 msec
seek: frame 3
at: 3.0 frames, 0.125 msec           ### NOTICE HERE
at: 4.0 frames, 125.12499999999999 msec
at: 5.0 frames, 166.83333333333334 msec
at: 6.0 frames, 208.54166666666666 msec
at: 7.0 frames, 250.24999999999997 msec
at: 8.0 frames, 291.9583333333333 msec
```

As you can see in the implementation of `CvCapture_FFMPEG::setProperty()`, seeking with `CAP_PROP_POS_FRAMES` puts the frame number into `picture_pts`, instead of an actual timestamp. This plain number is then interpreted as a timestamp (using timebase `1001/24000`), which results in `get(CAP_PROP_POS_MSEC)` reporting a number that's clearly off (uses `dts_to_sec(picture_pts) * 1000`, sensible). The following read() fixes that up though because seeking succeeded and `picture_pts` is just a "cached" value.

Seeking with `CAP_PROP_POS_MSEC` is similarly broken.

Some calculations, comparing actual timestamps and frame numbers interpreted as timestamps:
```
TB = 1/24000 # same as vid.get(cv.CAP_PROP_POS_AVI_RATIO)
# one frame of time is an increment of 1001 ticks
dts_to_sec = lambda ts: ts * TB
dts_to_sec(3003) * 1000 # 125.12499999999999
dts_to_sec(3120) * 1000 # 130
dts_to_sec(7007) * 1000 # 291.9583333333333
# seek(POS_FRAMES, 3)
dts_to_sec(3) * 1000 # 0.125
# seek(POS_MSEC, 130)
dts_to_sec(130) * 1000 # 5.416666666666667
```

I should mention the `CAP_PROP_POS_AVI_RATIO` is a complete mess too. `get()` returns the timebase (useful if we could seek using timestamps), set() calculates a timestamp from a 0..1 float and the video's total duration (in timebase), so that doesn't match up at all. If you think that warrants discussion, I could open a separate issue for that.

I haven't given this enough of a look to propose a patch yet. I'd like some opinions on this, some guidance on design decisions that need to be made and documented in the course of fixing this. The AVI_RATIO stuff needs to be brought in line with what the docs state (value range 0 to 1). I'm inclined to introduce `CAP_PROP_TIMEBASE` and `CAP_PROP_POS_TIMESTAMP` just for completeness/passthrough but that wouldn't necessarily map cleanly onto backends besides FFMPEG.

I am planning to propose a separate patch that would let me seek *quickly*, bypassing the frame-accurate positioning that happens in `CvCapture_FFMPEG::seek` (code after `avcodec_flush_buffers` call). I have some use cases that would greatly benefit from quick seeking and then querying the actual achieved position/timestamp.