Repository: k4yt3x/video2x

Issue #1318: Losing 2 frame after upscaling
Created: 2025-02-01T16:58:07Z
Losing 2 Frames After Upscaling with Video2X
Description
When upscaling videos using Video2X, the output consistently loses 2 frames compared to the original video. This issue has been observed across multiple tests with varying video durations and frame counts. For example:

A video with 25 frames outputs 23 frames after upscaling.
A video with 51 frames outputs 49 frames after upscaling.

This mismatch in frame count makes it impossible to evaluate the upscaled video using VMAF.

Steps to Reproduce
Follow these steps to reproduce the issue:


Use the following commands to upscale a video file:

For HEVC input/output:
video2x -i input.hevc -o output.hevc -p realesrgan -s 2 -c libx265 -e preset=slow -e crf=24

For MP4 input/output:
video2x -i input.mp4 -o output.mp4 -p realesrgan -s 2 -c libx264 -e preset=slow -e crf=24




Compare the frame count of the original video and the upscaled video using the following script:
python scripts/video_info_cap.py
here is the full script


Observe the number of frames in the original and upscaled videos.



Expected Behavior
The upscaled video should retain the same number of frames as the original video.

Actual Behavior
The upscaled video consistently loses 2 frames.

Example Output
Original Video:
=== Video File Information ===
File Path: /workspace/vidaio-subnet/services/upscaling/videos/7579959_hd.mp4
========================================
File Format: mov,mp4,m4a,3gp,3g2,mj2
File Size: 0.21 MB
Duration: 1.00 seconds
Bitrate: 1795.94 kbps
Compressed: Yes

=== Stream Information ===
Stream Type: Video
Codec: h264
Codec Long Name: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
Resolution: 2049x1080
Frame Rate: 25.00 fps
Number of Frames: 25
Pixel Format: yuv444p
Bitrate: 1786.65 kbps
Upscaled Video:
=== Video File Information ===
File Path: /workspace/vidaio-subnet/services/upscaling/videos/4k_7579959_hd.mp4
========================================
File Format: mov,mp4,m4a,3gp,3g2,mj2
File Size: 0.57 MB
Duration: 0.92 seconds
Bitrate: 5236.69 kbps
Compressed: Yes

=== Stream Information ===
Stream Type: Video
Codec: h264
Codec Long Name: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
Resolution: 4098x2160
Frame Rate: 25.00 fps
Number of Frames: 23
Pixel Format: yuv444p
Bitrate: 5226.82 kbps

Environment

OS: Ubuntu 24.04
GPU: NVIDIA RTX 6000 Ada Gene...
Video2X Version: 6.4.0


Request
Please investigate why Video2X consistently loses 2 frames during the upscaling process and provide a solution to ensure the output retains the same number of frames as the input.

Duplicate issue #1350: Exported video 2 frames missing
Created: 2025-03-10T12:19:23Z
Something weird is probably happening with the encoder because the exported video is missing 2 frames.
From media info this is the codec information of the original
General
Complete name :	20250307_0124_Loop Video_loop_01jnpzt0dkfkpre012tnzskr5v.mp4
Format :	MPEG-4
Format profile :	Base Media
File size :	5.66 MiB
Duration :	9 s 600 ms
Overall bit rate :	4 949 kb/s
Frame rate :	30.000 FPS
Writing application :	Lavf60.16.100
Video
ID :	1
Format :	AVC
Format/Info :	Advanced Video Codec
Format profile :	High
Format level :	3.2
Format settings, CABAC :	Yes
Format settings, Reference frames :	4 frames
Codec ID :	avc1
Duration :	9 s 600 ms
Bit rate :	4 933 kb/s
Width :	480 pixels
Height :	854 pixels
Display aspect ratio :	0.562
Frame rate mode :	Constant
Frame rate :	30.000 FPS
Color space :	YUV
Chroma subsampling :	4:2:0
Bit depth :	8 bits
Scan type :	Progressive
Bits/(Pixel*Frame) :	0.401
Stream size :	5.64 MiB (100%)
Writing library :	x264 core 164 r3108 31e19f9
Encoding settings :	cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=27 / lookahead_threads=4 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=crf / mbtree=1 / crf=12.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=10000 / vbv_bufsize=20000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box :	avcC
And this is the new realsrgan exported version
General
Complete name :	20250307_0124_Loop Video_loop_01jnpzt0dkfkpre012tnzskr5v.realesrgan.mp4
Format :	MPEG-4
Format profile :	Base Media
File size :	33.6 MiB
Duration :	9 s 534 ms
Overall bit rate :	29.5 Mb/s
Frame rate :	30.000 FPS
Writing application :	Lavf61.7.100
Video
ID :	1
Format :	AVC
Format/Info :	Advanced Video Codec
Format profile :	High
Format level :	5.1
Format settings, CABAC :	Yes
Format settings, Reference frames :	5 frames
Codec ID :	avc1
Duration :	9 s 534 ms
Bit rate :	29.5 Mb/s
Width :	1 920 pixels
Height :	3 416 pixels
Display aspect ratio :	0.562
Frame rate mode :	Constant
Frame rate :	30.000 FPS
Color space :	YUV
Chroma subsampling :	4:2:0
Bit depth :	8 bits
Scan type :	Progressive
Bits/(Pixel*Frame) :	0.150
Stream size :	33.6 MiB (100%)
Writing library :	x264 core 164 r3192 c24e06c
Encoding settings :	cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=8 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=18 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=50 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box :	avcC
Notice that the duration of the realsrgan is slightly shorter.
What is the reason for this ?

Closing comment (2025-03-11T15:06:34Z): This is a duplicate of #1318