Neural B-frame Video Compression with Bi-directional Reference Harmonization
Authors: Yuxi Liu
Dengchao Jin
Shuai Huo
Jiawen Gu
Chao Zhou
Huihui Bai
Ming Lu
Zhan Ma
Venue: NeurIPS 2025 poster ()
Keywords: neural video compression
video coding
B-frame compression
Primary area: deep_learning

Abstract: Neural video compression (NVC) has made significant progress in recent years, while neural B-frame video compression (NBVC) remains underexplored compared to P-frame compression. NBVC can adopt bi-directional reference frames for better compression performance. However, NBVC's hierarchical coding may complicate continuous temporal prediction, especially at some hierarchical levels with a large frame span, which could cause the contribution of the two reference frames to be unbalanced. To optimize reference information utilization, we propose a novel NBVC method, termed Bi-directional Reference Harmonization Video Compression (BRHVC), with the proposed Bi-directional Motion Converge (BMC) and Bi-directional Contextual Fusion (BCF). BMC converges multiple optical flows in motion compression, leading to more accurate motion compensation on a larger scale. Then BCF explicitly models the weights of reference contexts under the guidance of motion compensation accuracy. With more efficient motions and contexts, BRHVC can effectively harmonize bi-directional references. Experimental results indicate that our BRHVC outperforms previous state-of-the-art NVC methods, even surpassing the traditional coding, VTM-RA (under random access configuration), on the HEVC datasets. The source code will be released. The source code is released at https://github.com/kwai/NVC.

Neural B-frame Video Compression with Bi-directional Reference Harmonization
Yuxi Liu 122 Dengchao Jin?
Shuai Huo Jiawen Gu
Chao Zhou²
Huihui Bai
Ming Lu
Zhan Ma
Naanjing University, Nanjing, China KKuaihou Technology, Beijing China 3 Beijing Jiaotong University, Beijing, China
yuxiliuBsmail.nn n.j. edu. cn, [jindengchao, huoshuai, guj gujiawen, zhouchao 2houchao) /kuaisou..iiioooooooo hhbai@bjtu ju. edu. cn, (minglu, mazhan)@nju. edu. cn
Abstract
Neural video compression (NVC) has made significant progress in recent years, while neural B-frame video compression (NBVC) remains underexplored com- pared to P-frame compression. NBVC can adopt bi-directional reference frames for better compression performance. However, NBVC's hierarchical coding may complicate continuous temporal prediction, especially at some hierarchical levels with large frame span, which could cause the contribution of the reference frames to be unbalanced. To optimize reference information utilization, we pro- pose a novel NBVC method, termed Bi-directional Reference Harmonization Video Compression (BRHVC), with the proposed Bi-directional Motion Converge (BMC) and Bi-directional Contextual Fusion (BCF). BMC converges multiple optical flows in motion compression, leading to more accurate motion compen- sation on a larger scale. Then BCF explicitly models the weights of reference contexts under the guidance of motion compensation accuracy. With more ef- ficient motions and contexts, BRHVC can effectively harmonize bi-directional references. Experimental results indicate that our BRHVC outperforms previous wtae-of-the-an NVC methods, even surpassing the traditional coding, VTM-RA (under random access configuration), on the HEVC datasets. The source code is released at /tttps://github
Introduction
Lossy video compression is cornerstone technology in the digital era, enabling efficient storage and transmission of overwhelming video information acToss modern communication systems. support a wide range of compression scenarios. various video coding schemes have been developed [6] 10), with low-delay (LD) P-frame coding and random access (RA) B-frame coding being the most prominent. These two schemes are widely used in global video coding systems.
In LD mode, only the preceding frame is referenced during each coding process. In contrast, RA coding uses bi-directional hierarchical coding structure, which naturally utilizes more reference information and achieves better compression performance than LD. So RA is widely used in scenarios where compression efficiency is more important than latency. The coding processes of LD and RA shown in Figure T within a coding scenario with an Intra Period (IP) of 4 frames, I-frame is
"Tii work done when Yuxi Liu was full-time intern Kuaishou Technology.
Corresponding author.
Prroject leader.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Intra Period
Intra Period
@
I
P
P
P
I
B
B
B
I
(a) Low-delay (LD) P-frame coding.
(b) Random access (RA) B-frame coding.
Two main coding schemes with Intra Period 4 as an example. Each square represents frame, and the frames temporally ordered from left to right. Arrows represent the reference pipeline. The serial numbers @ @ @ indicate the coding order.
Figure I:
encoded independently without referencing any frame, a P-frame references the preceding frame, while a B-frame can reference both forward and backward frames.
Traditional video coding standards such as H.264/AVC (47,, H. H.265/HEVC 42 and H.266/VVC incorporate both LD and RA configurations. With the rise of deep learning, some end-to-end neural video compression (NVC) methods have emerged and gradually surpassed traditional standards under 28
LD configurations. Most NVC research focuses on P-frame coding, such 30 27
[5]
41 33
44 18 43 3 and the DCVC series [20 38 21
35 [7]. Although some neural B-frame video 39 also have made notable advancements, there remains
36
compression (NBVC) methods 8
5T
certain gap between these methods and VTM-RA (the reference software of H.266/VVC under RA configuration). Meanwhile, some issues related to NBVC have not yet been thoroughly investigated.
The hierarchical coding scheme of NBVC exhibits distinctive characteristics uncommon other video processing tasks, and even completely different from neural P-frame video compression. To be specific, the span of each LD coding process is only one frame, thus the representation of motion is relatively easier. However, the frame span of RA is so large the initial hierarchical levels (e.g., the coding of @ in Figure[]]bb that the motions become more unpredictable. In that case, we can not expect all the reference frames to be dependable. We define this phenomenon unbalanced reference contribution (URC). For example, see Figure 2 the information on the number plate can only be referenced in Ig because the movement of the racehorse caused the number plate be obscured in the previous moment in y. So the model needs to pay more attention to the target region of rt.
To optimize reference information utilization under the influence of the URC issue, propose novel NBVC method, termed Bi-directional Reference Harmonization Video Compression (BRHVC), with the proposed Bi-directional Motion Converge (BMC) and Bi-directional Contextual Fusion (BCF). To expand the receptive field of the motion estimation, we downsample the reference frames and compute optical flows across multiple scales. Then, BMC converges compressed representations of the flows and enriches them. This method helps to alleviate the difficulty in estimating motion over large spans. After that, BRHVC conducts more accurate motion compensation contexts. Furthermore, the proposed BCF explicitly models the weights of reference contexts under the guidance of motion compensation accuracy. With more efficient motions and contexts, BRHVC can effectively harmonize bi-directional references. Experimental results indicate that our BRHVC outperforms previous state-of-the-art NVC methods, even surpassing VTM-RA on the HEVC datasets. The main contributions of this paper are summarized as follows:
the reference
To cope with the unbalanced reference contribution between reference frames, we propose a novel NBVC method, termed BRHVC, with the proposed BMC and BCF.
We propose Bi-directional Motion Converge (BMC) to converge multiple optical flows in motion compression, leading more accurate motion compensation on larger scale.
We propose Bi-directional Contextual Fusion (BCF) to explicitly model the weights of reference contexts under the guidance of motion compensation accuracy, which facilitates the harmonization of bi-directional references.
Experiment results show that our BRHVC outperforms previous SOTA NVC methods and surpasses VTM-RA on the HEVC datasets.

---

worse reference
better reference
(a) frame f
(b) frame 2t
(c) frame dbp
Figure 2: The unbalanced contribution issue between reference frames in B-frame coding. The right reference is notably more significant than the left for the compression of the number plate.
150
worse refertnce buttor refermnce Cap of references
121.8
1126
BD-rate (%)
91.2 86
100
0144
70:1
635
.903
50
11.1
6.6
0
32
16
8
4
Figure 3: Quantitative experiment on unbalanced reference contribution. The results of the "gap of references" indicate the average contribution difference between two reference frames across the frame spans of (32, 16, 4, 2). Refer to Section 3 for more explanations.
Related Work
2.1 Neural P-frame Video Compression
Recently, many methods have adopted neural network (NN) to improve video coding, including
using NN to enhance traditional video coding [32
10
16
55
12
26
24 and NN-based end-to-end
video coding. Earlier end-to-end neural P-frame video compression (NPVC) methods still follow For example, Lu et al. (30 31 proposed the first NPVC method, termed DVC. This framework generates and transmits the motion vectors to get the predicted frame. Then the residual between the predicted and current frames is compressed to achieve bitrate savings. However, residuals in the pixel domain lack sufficient information fully leverage the representational capabilities of neural networks. To address it, Li et al. [20) proposed deep contextual video compression framework, termed DCVC, which enables paradigm shift from predictive coding conditional coding. Based on DCVC, Sheng et al. [38 proposed a temporal context mining (TCM) module to learn richer and more accurate contexts at multi-scales. This multi-scale contextual compression paradigm has been employed in many methods. Among them, DCVC-DC (22) and DCVC-FM (3) have achieved state-of-the-ee (SOTA) performance, saving approximately 20% of the bitrate compared the LDB mode of VTM-17.0.
the traditional predictive coding structure B0
27
1 28
29
14
15
41
25
2.2 Neural B-frame Video Compression
Neural B-frame video compression approaches can also be broadly summarized in two paradigms, medictive-based [8
16] and condition-based [1 39] Predictive-based methods compute the predicted frame as the primary reference, with the help of two reference frames. Some methods [8 [ 49 turn to interpolation network with a view avoiding motion compression. These interpolation networks often assign a binary mask to each prediction frame from two reference frames, indicating which pixels are more accurate than the other. Since it shows some degree of adaptation to the URC mentioned above, some other predictive-basse based works (52
52
54 g, [6]
also introduce the binary mask obtain more accurate estimation at the pixel domain. However, as discussed in Section 2.1] condition- methods [1] 39 have high-dimension multi-scall
contextual references in the feature domain, better than the pixel domain of predictive- methods. For example, Yang et al. [51] proposed unified contextual video compression framework for joint P-frame and B-frame coding. Sheng (39] proposed an efficient model, DCVC-B, with customized training strategy, achieving SOTA performance in NBVC.

---

Bi-drretional
02, Par PEu 1P
8y. Ap
Mition Comwerge
Motoon Estimation
AE
AD
MC Enooder
MD Decoder
Motion Compensation
Fg.Fb
Decokl/Bulle
E8-dtrectional
CLas
Contextuaal Fusson
BCF-
BCFDD
C2
GE
C?
yr
9:
Encoder
Decoder
Figure 4: The overall architecture of BRHVC. AE and AD denote arithmetic encoding and arithmetic decoding. MC Encoder and MD Decoder denote Motion Converge Encoder and Motion Diverge Decoder. C and Cp denote {C}, C7, CJ} and {C,, C,), respectively. We omit the entropy model and some outputs from the decoded buffer for brevity.
However, the above condition-bail -based methods do design specific module for the unbalanced contribution issue, merely concatenating the contexts from the two reference frames as the primary reference. Whether can adequately model the weights of the reference information remains an open question. It will be discussed and addressed in this paper.
3 Rethink to Unbalanced Reference Contribution
To investigate how much the URC affects compression performance, we developed an NBVC baseline model (the details will be introduced later) and conducted quantitative experiment using it. For each video sequence, we only compressed the 16th frame øie, setting the two reference frames (.y. to (ão,0332 J. )=( 12, (f 4 and 2. In each compression, we only use a single reference like ( or (ā,,ū)) then calculate their BD-rates [4] with referring { ( ) (the default setting) as the anchor. The lower one taken as "better reference". while the higher one is "worse reference". We then statistically average the BD-rate results over the HEVC datasets [5] as shown Figure 3 can be seen that with the large span, the gap between the better reference and the worse reference is substantial, reaching 20.6% and 11.1% for spans 32 and 16, respectively. As the span decreases, the BD-rate for both references increases dramatically. It because with the small span, the reference information is more relevant to the target, making both of them more important. Therefore, the absence of any reference leads to severe degradation. Considering the significant impact of the URC issue in NBVC, there should be a method that can effectively determine the weights of the two references to better harmonize the reference information. More details of this quantitative experiment are in Appendix A.1
corresponding to spans 32, 16, ($14, (115,517), (
Method
4.1
The Overall Framework
We build our baseline on the backbone of DCVC-B [39 and DCVC-DC 22), while some of the modules have certain differences. More details of our network design are Appendix [A. Our BRHVC introduces the proposed BMC and BCF based on the baseline. As shown in Figure given the current frame F, to be encoded, we use the two decoded reference frames ã 2g and their features Fy, F, as reference. When the IP length 32, the frame span f equals 26-level where level denotes the heirachical level of zt. The overall process of one frame coding is as follows:

---

K7.F3
Coo C. 08 2
Lonky Rel
Dat 84 68
Con G G+i 6B
Labyy Wol
NINIAO 11
Liaii
RONOL
Ey.
weight sharlng
96
SiyyNtt
Multi-sale Flow Canasze
Motion Convarge Encoder
Deaxded BuffledBBufee
AE
1/,t- Va.r
Coar (A, 60
Mudti-scale Flow Diverge
Motion Diverge Decoder
wright sharing
Cum (64, 60
DCan B4 0B
77
W0le el
Wook 690
5 =p0kesx
DC
channel split
Figure 5: The framework Bi-directional Motion Converge. MF Adapter denotes Motion Feature Adapter. ge and qfec are learnable quantization vectors for variable bitrates 22
Motion Estimation and Compression. We use Spynet [37 to generate multi-scale optical flows {vi, " of, { and [w/,6. "I " from two directions. To be specific, we downsample the original image and reference images è 26 twice, then separately compute optical flows by SpyNet at each scale. This approach facilitates the expansion of the receptive field for motion estimation, thereby accommodating larger frame spans. Then we use the proposed BMC to compress and reconstruct these optical flows. Refer to Section 4.2 for more details.
Motion Compensation. Given the reconstructed optical flows (0) o2 23 L and {0) 02 Eeb we use them to warp the reference features at each scale. During this process, Fs, Fb individually pass through share-weighted contextual-feature extraction [22 391 (denoted as CFE, which combines Temporal Context Mining [20, 32) with Group-Based Offset Diversity [20]) with the flows, resulting in features {C}, Cj. CJ} and {Cg, Cf Cg) at multi-scales, where CJ. C. I Hx C7.C% C E 22/x//2xx/2 and C Og E 98 H/4xW/4 It can be expressed as:
- o Ee o.
CFE(F,, 02 o2 02,6).
(I)
Transform and Entropy Coding. NVC methods for coding's ' transform to reduce the dimension of the latent variables. The proposed BCF outputs the multi-scale contexts (C), C? C3) for encoder and (C/L C² C,,,0 for decoder (see more details in Section (433) as refined reference contexts. After the transform, arithmetic coding is utilized to compress latent variables y/ into a bitstream with the probability estimated by an entropy model. Our BRHVC adopts the quadtree partition-based entropy model 22
neural network-basee encoder and decoder
4.2 Bi-directional Motion Converge
As the frame span increases, the motions will become larger and more complex. However, many current motion estimation networks are designed for a frame span one. To capture large motion information much possible, straightforward idea is to downsample the input frames multiple times, so that the network can obtain a larger receptive field. Nevertheless, this approach increases the information content of the optical flows, requiring a customized compression method.
To achieve more efficient optical flow compression, previous works directional motion residual compression (denoted BMRC in this paper), which estimates the bi-
(39 53
employ a bi-
5

---

Herp
BCF-E
BCF-D
PFA
DCCon (4K
DCCan 48g
(48,
DCao 186 90)
DCeen chi,
DC 19s,
DRNA422 96333)
Up14422
DR0%
C3
G
07
(a) BCF module for encoder (BCF-E)
(b) BCF module for decoder (BCF-D)
Figure 6: The network architecture of Bi-directional Contextual Fusion (BCF) for encoder and decoder. PFA denotes Pixel Feature Alignment module.
directional flows vb,, between two reference frames and compresses the residuals (U, 00.5--0.5-vp yy.) and (ve,) 0.5 1b,f) ) to eliminate redundancy. Our baseline model uses BMRC for subsequent comparisons. However, BMRC is not suitable for multi-scale optical flows for two main reasons: first, computing residuals across multiple scales is overly complicated; and second, residuals in the optical flow domain, like those in the pixel domain, not optimal as mentioned in Section 2.1
To effectively compress the multi-scale flows, we propose Bi-directional Motion Converge. The framework of BMC is shown in Figure In optical flow compression, BMC converges multi-scale optical flows into single latent variable and fuses the latent variables from the two directions. After compression, these optical flows diverge through the reverse process and are used for motion compensation in the next part. On this basis, BMC introduces the flow features two directions from the decoded buffer, (F), FF) and Frof f.b as priors. Motion Feature Adapter selects different sub-networks to adapt prior information (F,, based on the frame type (B BB-frame or I-frame), as detailed in Appendix A.2 (Fj, FE provides the motion of two references individually, which is from the hierarchical layer with larger span. In contrast, Free the feature-domain representation reference frames, spanning the
of Ug.b and Pb.f, providing the motion information between flows to be compressed (i.e., Ur ft.f). This approach enriches prior information compared to the residual-basee approach BMRC. Unlike previous works (39 20 38 22 23 which obtain small-scale flows through downsampling, our BMC employs specific optical flows at different scales, thereby increasing the diversity of motion representation.
4.3 Bi-directional Contextual Fusion
the contexts CG, C?} and {C, C. C3) are fused with the latent features of the current frame at different scales in the encoder and decoder. Specifically, features of the current frame and the two references are concatenated along the channel dimension and then fused through subsequent convolution layers. However, this concatenation design may assume that the two reference frames have equal weights, i.c., they considered to be of equal importance. In reality, the contributions of the two reference frames are often different, as discussed Section It is more common in large frame spans, the motion becomes more unpredictable. Even in some cases, a region of the current frame may not benefit from either reference frame, which requires reducing the weights of both references and increasing the weight of the current frame's 's own features.
In previous works 39 51
Based on the above motivation, we propose the Bi-directional Contextual Fusion (BCF) that can adaptively adjust the weights of the reference frames for each region. Its structure is shown in Figure where DConv denotes Depthwise Separable Convolution (9] DB denotes Depth Block 22 UP denotes upsampling using the pixel shuffle [40] convolution kernel. All convolutions there have

---

default kernel size of 3x3. PFA denotes the proposed Pixel Feature Alignment module, and the specific module architectures can be seen Appendix A.2
First of all, BCF analyses the similarity between the current frame and the two motion-compensattd reference frames the guidance Gt- The three pixel-domain images (f sa Lib (x used for
computing the guidance to fuse {C}. C. {C7, Cg). and {Cg. C.). The process is as follows:
C2 C.C. BCFF((C)) C.), (C7, (,, (C). Ci Vis warp "ip (ear))) where Ft. warp(t (..)):
Fz.b 2/ = warp( (2))).
BCFF represents BCF for the encoder (BCF-E). BCF-E generates reference weight information based on the differences from the pixel domain, and through multi-scale weight information propaga- tion, each layer obtains adaptive weight allocation. To avoid the situation where the context features of both reference frames cannot provide accurate reference information, bj provides independent supplementary bias. The weights and the bias are denoted as s'y, sj. and by, their dimensions are the same as those of Cj, Cj where represents the scale. We denote the depth block as DB, @ means element-wisg multiplication. The specific formulation of C,, C?, C2 can be represented as follows:
& + + b)), C = DB &
C² DB2 8 +5
C (8)
C $ C b2).
Besides, after decoding the bitstream obtain je at the decoder, the decoder will use the BCF-D for decoding. The main difference between BCF-E and BCF-D is that the guidance information dr and gl are from different sources. Vt is not available to the decoder, so we utilize ÿe with the PFA module for alignment between pixels and features. The decoding process is as follows:
C.C.,.C. BCF C.), (C7, C.), (C), C. F. FTth
warp marp1)
After harmonizing the reference weights, these contexts are concatenated with the latent features in the encoder and decoder at three scales. It could support more accurate reference information for transform, achieving more bitrate saving and reconstruction performance.
5
Experiments
5.1
Settings
Training. We use Vimeo-90k (50) to train BRHVC from scratch with 7-frame sequences. Then we fine-tune BRHVC on original Vimeo videos with 17-frame sequences following [39 23 We use the multi-stage training strategy in [39. The video frames are randomly cropped into 256x256 patches. We randomly reverse the order of sequences with probability of 50% as data augmentation. AdamW 19 is used as the optimizer with batch size of 8.
Testing. We evaluate the compression performance on HEVC Class B-E (5). UVG (34], and MCL- JCV (46. The resolution of the HEVC Class C-E is in the range of 240p 720p, while HEVC Class B, MCJ-JCV, and UVG have a resolution of 1080p. The first 97 frames of all the datasets are used with Intral Period 32, i.c., coding 93 B/P-frames with 4 I-frames. We convert YUV420 inputs to RGB as the original sequences using 709 for all the methods. To compare the performance of traditional methods, we used the LDB and RA modes of the traditional standards software HM-16.5 42 13 and VTM-17.0 7 45 Since traditional methods do not support compression of RGB format, following the practice 22,, traditional methods compress the sequences through YUV444 format and calculate the PSNR after converting back to RGB using BT.709.. As for NVC methods, we compare the previous SOTA NPVC methods, DCVC-DC 222 and DCVC-FM (3]. addition, we compare the SOTA NBVC method, DCVC-B 39, which has similar structure to our baseline. We use the same I-frame model as DCVC-DC and DCVC-B for our coding. More details about the test configuration can be seen in Appendix A.3
7

---

HIVCC
CLVVC Clear
HPVC Cles
37
36.5
355
34.5
36.5
20.3
36
36
153M
15
335
33.5
104
VIMGLES
32.5
NO4NA
VI3LLIS
32.5
30.3
VINGA
DONGICC
DCYC-151
D335B
3055
305
JLYGB
1808
0.03
0.33
3/M
1000
0220
0.00
1073
018
0.07
0.00
0.11
11.33
1115
1017
008
0.07
0.22
037
HEVC Closs
MEL.JCV
UVG
415
1B4RA
VINSLIM
VIMALED
38.5
DOVG
DCYC
D0X0PM
DCVCB
03458
DCVCB
0.00
001
0.15
003
0005
0.1
0.00
0.0
0.0
0.5
0.65
007
0.00
0.00
042
0.3
000
0.55
008
1pp
Bgo
Figure Rate-distortion curves for HEVC Class B-E, MCJ-JCV and UVG datasets.
Table BD-rate (%) comparision for PSNR. Black bold indicates the best results within all methods, while underline bold indicates the best results within NVC methods.
HEVC Class
HEVC
HEVC
HEVC Class E
HEVC Average
MCL- JCV
All Average
UVG
Class
Class
VTM-LDB
0.0
0.0
0.0
0.0
0.0
0.0
VTM-RA
-33.6
-29.1
-30.2
-30.7
-30.9
-31.0
-33.5
-31.3
HM-LDB
38.5
35.4
32.5
42.3
37.1
41.9
36.3
37.8
HM-RA
0.2
4.2
-0.7
7.1
2.7
5.4
-4.9
1.8
DCVC-DC
-12.4
-12.5
-28.4
-18.2
-17.8
-8,9
-16.8
-16.2
DCVC-FM
-15.2
-19.2
-34.2
-26.5
-23.7
-8.3
-18.9
-00.3
DCVC-B
-18.7
-8.6
-33.5
-32.4
-23.3
-7.8
-17.2
BRHVC
-25.5
-25.2
-44.7
-32.6
-32.0
-16.1
-19.7
-27.3
5.2 Comparison Results
The rate-distortion curves of all methods be seen in Figure and the BD-rate comparison is shown in T Our proposed BRHVC achieves an average bitrate saving of 27.3% compared VTM-LDBB outperforms the previous SOTA NVC methods (i.e., P-frame method DCVC-FM and B-frame method DCVC-B) on all the datasets. In the average performance on HEVC datasets, BRHVC achieves the best bitrate saving of 32.0%, higher than VTM-RA's 30.9%. Particularly in HEVC Class D, BRHVC can save 44.7% bitrate, which is significantly higher than other methods. These experimental results indicate that method achieves significant advancement in the NBVC domain. Additionally, some visual comparison results are shown in Appendix A.4
5.3 Ablation Study and Complexity
To verify the validity our proposed BCF and BMC, we performed ablation experiments as shown in T DCVC-B(re-trained denotes that we re-train the oper-source DCVC-B model from scratch with our settings. Our baseline and DCVC-B performances are close, suggesting that our subsequent improvement mainly comes from the proposed modules rather than the training settings. Due to the slightly different network design, our baseline demonstrates slightly slower speed but with less GPU memory overhead. The results show that BCF can achieve 6.6% bitrate saving, effectively enhancing reference information utilization deal with URC issue. Additionally, the incorporation of BMC strengthens BRHVC's ability represent large span motion, providing more

---

large span reference
035
91
035
mean
mean (b
mean(si.)
328
025
small span reference
2,5
11
02
035
01
mean(sih
mean
(bg
mean
025
0,25
Figure 8: Visualization of BRHVC's reference harmonization effect on different spans. We compress T12 and use dg, I1 (large span) I1 11 (small span) references. We visualize the outputs sy. s bf from BCF-E. mean(-) denotes taking the absolute value and averaging over the channels.
Table 2: Ablation study on different network designs with the corresponding computation cost. The BD-rate results calculated on HEVC datasets. We compare average single-rrame codec times tested in 1080p sequences on one Nvidia RTX 4090 GPU. BL denotes our baseline model.
Model
BD-rate
Parameters
kMACs/pixel
Memory
Enc.
Dec. time
DCVC- DCVC-B(re-trriie ((re(te-rriined)
0.4%
24.4 M
2934.9
13.5 G
646 51 ms
BL
0.0
23.7 M
2831.9
9.6
681 538 ms
BL w/ BMC
-6.4%
24.3 M
3170.9
10.6
780 / 602 ms
BL BCF
-6.6%
28.9 M
3532.8
11.4
896 583 ms
BI. w/ BCF, BMC
-12.3%
29.4 M
3887.8
12.4
983 670 ms
accurate reference BCF, which further improves the performance 12.3% bitrate saving with 24% additional decoding time. It can be observed that the additional computational overhead incurred is relatively acceptable compared the significant improvements by our BRHVC.
5.4 Visulization
Figure & shows the visualization of our BRHVC harmonize the references of different spans. In this example, the number plate is the focal target of our compression. The motions are more complex over large spans (i.e., &g. Zi6 as references), so the network needs to pay more attention the useful one (i.e., 216). Thus, (s)a (computed by BCF-E) demonstrates higher weight than mean(s)) this basis, mean(1)) provides supplementary details to compensate for the loss of information. If the span is small, the difference between mean(b),) ) and mean(bis) will decrease because their information is both useful in most regions, and the supplementary mean(b))) will also decrease accordingly. It shows that proposed modules are working expected.

---

Conclusion and Limitation
In this paper, we introduce the phenomenon of unbalanced reference contribution in P-frame coding and analyze it quantitatively. To cope with it, we propose novel NBVC method, termed BRHVC, with proposed BMC and BCF. BMC converges multiple optical flows in motion compression, leading to more accurate motion compensation for both references. BCF explicitly models the weights of reference contexts under the guidance of motion compensation accuracy, which facilitates the harmonization of bi-directional references. Experiment results show that our BRHVC outperforms previous SOTA NVC methods and surpasses VTM-RA on the HEVC datasets. However, our BRHVC does not perform sufficiently well on high frame rate datasets, such MCL-JCV and UVG, which may require further investigation in future work.
Acknowledgements
The authors would like express their sincere gratitude to the Kuaishou Research and Development Department (R&D), and the Interdisciplinary Research Center for Future Intelligent Chips (Chip-X) and Yachen Foundation for their invaluable support. This work supported in part by National Key Research and Development Project of China (Grant No. 2022YFF0902402), Kuaishou Research and Development Department (R&D), Natural Science Foundation of Jiangsu Province (Grant No. BK20241226), Beijing Natural Science Foundation (L223022), and National Natural Science Foundation of China (Grant No. 62401251, 62431011, 62331003).
10

---

References
[1] Eirikur Agustsson, David Minnen, Nick Johnston, Johannes Balle, Sung Jin Hwang, and George Toderici. Scale-space flow for end-to-end optimized video compression. CVPR, pages 8503-8512, 2020.
[2] David Alexandre, Hsueh-Ming Hang, and Wen-Hsiao Peng. Hierarchical B-frame video coding using two-layer CANF without motion coding. CVPR, pages 10249-10258, 2023.
[3] Yifan Bian, Chuanbo Tang, Li Li, and Dong Liu. Augmented deep contexts for spatially embedded vidco coding. In CVPR, pages 2094-2104, 2025.
[4] Gisle Bjontegaard. Calcuation of average PSNR differences between RD-curves. Technical Report VCEG-M33, VCEG, 2001.
[5] F Bossen. Common HM test conditions and software reference configurations. Technical Report JCTVC- L1100, Joint Collaborative Team Video Coding (JCT-VC) of I SG, 2013.
[6] Benjamin Bross, Jianle Chen, Jens-Rainer Ohm, Gary Sullivan, and Ye-Kui Wang. Developments in international video coding standardization after AVC, with an overview of versatile video coding (VVC). Proceedings of IEEE, 109(9): 1463-1166111 1493, 2021.
[7] Benjamin Bross, Ye-Kui Wang. Yan Ye, Shan Liu, Jianle Chen, Gary J Sullivan, and Jens-Raincr Ohm. Overview of the versatile vidco coding (VVC) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, (10))337736-3
2021.
[8] Mu-Jung Chen, Yi-Hsin Chen, and Wen-Hsiao Peng. B-CANE: Adaptive B-Frame coding with condi- tional augmented normalizing flows. IEEE Transactions on Circuits and Systems for Video Technology, 34(4) 2921, 2024.
[9] François Chollet. Xception: Deep learning with depthwise separable convolutions. CVPR, pages 2258- 2017.
[10] Dandan Ding, Zhan Ma, Chen. Qingshuang Chen, Liu, Fengging Zhu. Advances in video compression system using deep neural network: a review and case studies. Proceedings of the IEEE, 109(9):1444-152) 2021.
[11] Ahdelaziz Djelouah, Joaquim Campos, Simone Schaub-Meyer, and Christopher Schroers. Neural inter- frame compression for video coding. In ICCV, pages 6421-6429, 2019.
[12] Cunhui Dong, Haichuan Ma, Zhuoyuan Li, Li Li, and Dong Liu. Temporal wavelet transform-based low-complexity perceptual quality enhancement of compressed video. IEEE Transactions on Circuits and Systems for Video Technology, 34(5):4040-4053, 2023.
[13] HM. https://vegit.ttt hhi fraunhofer.defiee//IN..
[14] Zhihao Hu, Zhenghao Chen, Dong Xu, Guo Lu, Wanli Ouyang, and Shuhang Gu. Improving deep video compression by resolution-adaptic flow coding. In ECCV, pages 193-209. Springer, 2020.
[15] Zhihao Hu, Guo Lu, and Dong Xu, FVC: new framework towards deep video compression in feature space. In CVPR, pages 1502 1511, 2021.
[16] Shuai Huo, Dong Liu, Haotian Zhang, Li Li, Siwei Ma, Feng Wu, and Wen Gao. Towards hybrid- optimization video coding. ACM Computing Surveys, 56(9):136,, 2024.
[17] Zhaoyang Jia, Bin Li, Jiahao Li, Wenxuan Xie, Linfeng Qi, Houqiang Li, and Yan Lu. Towards practical real-time neural video compression. arXiv preprint arXiv:2502. 20762, 2025.
[18] Wei Jiang, Junru Li, Kai Zhang, and Li Zhang, ECVC: Exploiting non-local conrelations in multiple frames for contextual video compression. arXiv preprint arXiv:2410.09706, 2024.
[19] Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.60 101.69.9..980..11 2014.
[20] Jiahao Li, Bin Li, and Yan Lu. Deep contextual video compression. NewrIPS, 34:18114 18125, 2021. 2021.
[21] Jiahao Li, Bin Li, and Yan Lu. Hybrid seatial-tempora entropy modelling for neural video compression. In ACM Multimedia, pages 1503-1511, 2022.
[22] Jiahao Li, Bin Li. and Yan Lu. Neural video compression with diverse contexts. In CVPR, pages 2023.
11

---

[23] Jiahao Li, Bin Li. and Yan Lu. Neural video compression with feature modulation. In CVPR, pages 26099-26108, 2024.
[24] Zhuoyuan Li, Jiacheng Li, Yao Li, Li, Dong Liu, and Feng Wu. In-loop filtering via trained look-up tables. In VCIP. pages 1-5. IEEE, 2024.
[25] Zhuoyuan Li, Junqi Liao, Chuanbo Tang. Haotian Zhang, Yuqi Li, Yifan Bian, Xihua Sheng, Xinmin Feng. Yao Li, Changsheng Gao, al. USTC-TD: A test dataset and benchmark for image and video coding in 2020s. IEEE Trunsactions on Multimedia, DOI: 10.1 109/TMM. 2025. 3608643. 2025.
[26] Zhuoyuan Li, Zikun Yuan, Li Li. Dong Liu, Xiaohu Tang, and Feng Wu. Object segmentation-assisee inter prediction for versatile video coding. IEEE Transactions on Broadcasting. 70(4):1236 1253, 2024.
[27] Jianping Lin, Dong Liu, Houqiang Li, and Feng Wu. M-LVC: multiple frames prediction for learned video compression. CVPR, pages 3544-3554, 2020.
[28] Haojie Liu, Ming Lu, Zhan Ma, Fan Wang, Zhihuang Xie, Xun Cao, and Yao Wang. Neural video coding using multiscale motion compensation and spatiotemporal context model. IEEE Transuctions on Circuits and Syxtems for Video Technology, 31())318223313 2021.
[29] Guo Lu, Chunlei Cai, Xiaoyun Zhang, Li Chen, Wanli Ouyang, Dong Xu, and Zhiyong Gao. Content adaptive and emror propagation aware deep video compression. In ECCV, pages 456-472. Springer, 2020.
[30] Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, and Zhiyong Gao. DVC: An end-to-end deep video compression framework. In CVPR, pages 1000--1015, 2019.
[31] Guo Lu, Xiaoyun Zhang, Wanli Ouyang, Li Chen, Zhiyong Gao, and Dong Xu. An end-to-end learning framework for vidco compression. IEEE transactions on pattern analysis and machine intelligence, 43(10):3292-3308 2021.
[32] Siwei Ma, Xinfeng Zhang, Chuanmin Jia, Zhenghui Zhao, Shiqi Wang, and Shanshe Wang. Image and video compression with neural networks: A review. IEEF Transactions on Circuits and Systems for Video Technology, 306)):6 2020.
[33] Fabian Mentzer, George Toderici, David Minnen, Sung-Jin Hwang, Sergi Caelles, Mario Lucic, and Eirikur Agustsson. VCT: A video compression transformer. arXiv preprint 27X1v:2206.07307, 2022.
[34] Alexandre Mercat, Marko Vittane,, and Jamo Vanne. Uvg dataset: 50/120fps 4K sequences for vidco codec analysis and development. In Proceedings of the 11th ACM Multimedia Systems Conference, page 297-302, 2020.
[35] Linfeng Qi, Zhaoyang Jia, Jiahao Li, Bin Li, Houqiang Li, and Yan Lu. Long-term temporal context gathering for neural video compression. In ECCV, pages 305-322. Springer, 2024.
[36] Linfeng Qi. Jiahao Li, Bin Li, Houqiang Li, and Yan Lu. Motion information propagation for neural vidco compression. CVPR, pages 61 11-6100, 2023.
[37] Anurag Ranjan and Michael J Black. Optical flow estimation using spatial pyramid network. In CVPR, pages 4161-4170, 2017.
[38] Xihua Sheng, Jiahao Li, Bin Li, Li, Dong Liu, and Yan Lu. Temporal context mining learned video compression. IEEE Transactions on Multimedia, 1-722, 2022.
[39] Xihua Sheng, Li Li, Dong Liu, and Shiqi Wang. Bi-directional deep contextual vidco compression. JEEE Transactions on Multimedia, DOI: 10. (1000//MM..225.55530000 2025.
[40] Wenzhe Shi, Jose Caballero, Ferene Huszár, Johannes Totz, Andrew Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, pages 1874-1883, 2016.
[41] Yibo Shi, Yunying Ge, Jing Wang, and Jue Mao. Alphave: High- peeerrerffrmacce and efficient learned vidco compression. In ECCV, pages 616-66 631.6316666 Springer, 2022.
[42] Gary J Sullivan, Jens Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology, 16412) 1668, 2012.
[43] Chuanbo Tang. Zhuoyuan Li, Yifan Bian, Li Li, and Dong Liu. Neural video compression with context modulation. In CVPR, pages 12553-12563, 2025.
12

---

[44] Chuanbo Tang, Xihua Sheng, Zhuoyuan Li. Haotian Zhang, Li Li, and Dong Liu. Offline and online optical flow enhancement for deep video compression. In AAAI, volume 38, pages 5118-5126, 2024.
[45]
VTM. http:h/...aabbbbee/////////sotre VTM/.
[46] Haiqiang Wang, Weihao Gan, Sudeng Hu, Joe Yuchieh Lin, Lina Jin, Longguang Song, Ping Wang, Ioannis Katsavounidis, Anne Aaron, and C-C Jay Kuo. MCL-JCV: A JND-based HL264//VC video quality assessment dataset. In ICIP, pages 1509-1513. IEEE, 2016.
[47] Thomas Wiegand, Gary J Sullivan, Gisle Bjontegaard, and Ajay Luthra. Overview of the H.264/AVC, video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 13(7):560-576, 2003.
[48] Chao- Yuan Wu, Nayan Singhal, and Philipp Krähenbühl. Video compression through image interpolation. In ECCV, 2018
[49]
Chenming Xu, Meiqin Liu, Chao Yao, Weisi Lin, and Yao Zhao. IBVC: Interpolation-drriee B-frame video compression. Pattern Recognition, 153:110465, 2024.
[50] Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with task-oriented flow. International Journal of Computer Vision, 127:1 106-1105-, 2019.
[51] Jiayu Yang. Wei Jiang, Yonggi Zhai, Chunhui Yang. and Ronggang Wang. UCVC: unified contextual video compression framework with joint P-frame and B-frame coding. In DCC, pages 382-391. IEEE, 2024.
[52] Ren Yang, Fabian Mentzer, Luc Van Gool, and Radu Timofte. Learning for video compression with hierarchical quality and recurrent enhancement. CVPR, pages 6628-6637, 2020.
[53] M Akin Yilmaz and A Murat Tekalp. End-to-end rate-distortion optimization for bi-directional learned video compression. ICIP, pages 1311-1315. IEEF, 2020.
[54] M Akın Yılmaz and A Murat Tekalp. End-to-end rate-distortion optimized learned hierarchical bi- directional video compression. IEEE Transactions on Image Processing, 31:974-983, 2021.
[55] Yanchen Zuo, Changsheng Gao, Dong Liu, Li, Yueyi Zhang, and Xiaoyan Sun. Learned rate-distortion cost prediction for ultrafast screen content intra coding. IEFE Transactions on Circuits and Systems for Video Technology, 34(3):1976-1 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main contributions of the paper, including the new method proposed and the experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations of the work in Section 6 Limitation".
"Conclusion and
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiiatinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: Our paper does not contain mathematical theories. Otherwise, we provide experimental validation for the issue of unbalanced reference contribution and analyze its effects across different spans, as shown in Figure
Guidelines:
The answer NA means that the paper does not include theoretical results,
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides detailed descriptions of the experimental setup, including the algorithms used, the datasets employed, and the specific parameters and configurations.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
We will release the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
public/guides
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
Codeeobmiiiie SanSSuubissss weabbbbmsssioo ssisssiicccyyyi for more details. 7/nips cc/ wtc/publicc/ /esoc/uuuiie
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides a comprehensive description of the experimental setup in Section 5.1 and Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification: The assessment metries used in the paper explicitly report the statistical basis.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper details the computational resources utilized for each experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research presented in the paper strictly adheres to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification:
The paper implements safety filters and usage guidelines for released models and uses content moderation for datasets to prevent misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The paper properly credits the creators of all used assets, specifies the version and URL where applicable, and includes the license information for each asset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper provides comprehensive documentation for the new assets, including details about training, license, and limitations.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Quantitative Experiment for the URC issue
Experimental Setting. For each video sequence, we only compressed the 16th frame 216. setting the two reference frames (f..,t)) ] to (fg. J. ( J. (2122.220 11 (+115)17)) corresponding to spans of 32, 16, 8, 4 and 2. We configured the baseline model three different settings: both reference frames are forward frames (denoted as (x,
or M for simplicity), both refer- ence frames are backward frames (denoted as (zz (1 (b) or M6,s), and the two reference frames a forward frame and a backward frame (denoted B-frame compression), respectively. All the reference frames are compressed by the same I-frame model [22]. each compression. the bascline model outputs four compressed results at four different bitrates, in order to calculate the BD-rate in one frame. Then we calculate the BD-rate of M and Mp... respectively, with y, as the anchor. The lower one taken as "better reference", while the higher one is "worse reference". We then statistically average the BD-rate results over the HEVC datasets as shown in Figure 33 and Table 3
(x (.t) ) or My.. the default setting in
The formulation as follows:
Better(Iz) jb f) = min| Drate (M,,)); BDrate MA (M,,)))
(5)
Worse(z, - f) - max (BaraDDe (M,,)) BDrate My (Mo,)).
where b- the span, B Drate (M2) denotes calculating the BD-rate of M2 using MI as the anchor. min[:; and ma denotes taking the lower value and the higher value, respectively.
Experimental Analysis. The detailed results of the quantitative experiment in Section is shown as Table As mentioned in Section 3] with the large frame span, the gap between the better and the worse reference is substantial, reaching 20.6% and 11 for spans of 32 and 16, respectively. As the span decreases, the BD-rate for both references increases dramatically. It is because with the small span, the reference information is more relevant the target, making both of them Therefore, the absence of any reference leads to severe degradation. However, it should be noted that while the ratio of the gap (between "Better" and "Worse") to "Worse" exhibits an approximately monotonic decreasing trend with span, the value of the gap itself does not. This is also because, as the span decreases, the BD-rate of both "Worse" and "Better" increases, possibly causing the gap to increase correspondingly. Additionally, it can be observed that the gap is more pronounced on Class C Cndd D. This may be due to the presence of many citense-motion scenes in these datasets, which increase frame differences and thus the gap.
important.
Table 3: Quantitative experiment on the URC issue with detailed results. The metric is BD-rate (%).
Span of the Reference Frames
Dataset
Reference
32
16
8
Better
39.2
49.3
63.5
86.
112.6
Average
Worse
59.8
60.4
70.1
94.2
121.8
Better Worse
32.2
39.1
53.3
76.4
114.1
Class B
40.2
45.7
57.1
86.6
123.4
Better Worse
33.6
46.7
53,4
63.7
83.8
Class C
69.3
62.8
61.8
69.1
92.8
Better Worse
39.4
50.9
72.1
113.2
148.3
Class D
68.7
67.1
80.0
121.4
160.8
Better Worse
58.3
68.2
83.1
96.1
101.1
Class E
68.2
73.0
89.5
103.2
105.9
A.2 Network Designs
Encoder and Decoder. As shown in Figure we use the same architecture of the encoder and decoder as DCVC-DC [22]. The main difference between our architecture and DCVC-DC's that we use the bi-directi reference contexts (C), C2, CP) and (Cr), Cy. C"2) as contextual features, while the features of DCVC-DC are from single preceding direction.
21

---

RInE 120
RBN 100
Cooo (1 (90
Con 1300
0.2
y4
qste
AD
Fhane
RBN 120
ESPOI22 689
tPPk 9r 2
Decoder
C7
C2
Figure 9: The network architecture the encoder and decoder. qp and qs; are quantization variables for variable bitrates. "RB(Cin.Com is Residual Block with input channel Can and ouput channel Cout. Refer to [22] for more details.
imput
input
Ar-k
114
Imput
DB
DCoLvv
DCCen is
Ujr 128 sG) 2
DCum C3 96) .2
UP
1x ICumn (Fruz Craz
1x XCrn Cor
I Laaly Rell x
Lavily Rel
Cora :f)
PaelUinsautleeg)
Lesky ReL.
IX CCam Ce
Lealy
Uper
DUto
Coutr Cayi)
Lealey Rel.
1CorC (G,, Caae)
PEA
DRFEX
output
9's
output
output
(a) Tou
(b)
(c) DCCov(Cinn Cout))
(d) PFA
Figure 10: The architecture of the sub-modules in BCF.
MF Adopter-11
If xy is I-frame and is I-frame
MF Addpta--
If is I-frame and xy B-frame
input
DR22,22
input
output
Fg
2
DEE 6Q
oulput
equals
MF AFFapttte-BBB
xy B-frame and Ny is framee
MF Adapter-B
xy B-frame and x 1-freme
input
input
Fg
138,22 2
DRU 60
output
F/
2.2
DWE/1 69
output
Fg
138,2
2,2
Figure 1:: The architecture of Motion Feature Adapter (MF Adapter) in BMC. All the "DB1222" share the same network weights.
22

---

Sub-Modules in BCF. The detailed module designs of BCF is as shown in Figure T0 Not that there may be some parameter differences based on the basic architectures. For example. means DConv layer with input channel C. and ouput channel Codt, and "s" denotes the sample stride (defaults to 1). "Is" means the downsample stride is "s", while "fs" means upsample with stride "s".
MF Adapter in BMC. Motion Feature Adapter selects different sub-networks adapt prior informa- tion (Fy, F based on the frame type (B-frame or I-frame). For example, if the reference frame B-frame, the I-frame model does not need to compute the optical flow and therefore can not output Fj intermediate result. In that case,
not introduce Fj. If both frames are I-frames, the input is solely processed to generate the output (see "MF Adapter-II" in Figure II
A.3
Test Configuration
We compare traditional methods HM-16.5 and VTM-17.0 with LDB and RA configuration in Sec- tion 5.1 To evaluate the compression performance of traditional methods, follow the configuration of [22] for fair comparison. We convert YUV420 inputs to RGB as the original sequences using BT.709 for all the methods. Since traditional methods do support compression of RGB format, traditional methods compress the sequences through YUV444 format and calculate the PSNR after converting back to RGB using BT.709. To configure the best traditional codecs, we use the 10 bit depth for YUV444 colorspace. These methods share some command- arguments. While for the (config_flle, VTM-RA sets to encoder_raue cow.uderrrrddmuccess...o sttrsss_mmmfff..... VTM-LDB sets to encoder lowedeeowwwwllyyynne vim.cfg, HM-LDB sets it to encoder Jowdelay_wwalyy__aee saaii__maiaiaaee siciii__extcfff.iee and HM-RA sets it to encoder eeccoderrrandoneee randomaccesssss.... petprrccceessmmaeeeeeeefffff. The command line is as follows:
c (config ]
InpuuBBiiDepth=100
-OuppuBBidDephh=10
-OuppuBBiieephhh=100
InputChroommFormat4444
Frameeaat==(rumme_ruee
DecoodnngRefrsshTTyp-22
FrameesToBcEEncoeed=977
SSuurce
-Sourreeeeighht((heiii
-IntraPeriod=32
-QPP(qp)
-evel=66.2
BissrcannFllc
A.4 Visual Quality Comparison
The visual quality comparison is shown in Figure 12 can be seen that our method has better reconstruction of detailed textures with lower bitrates, especially the parts accompanied by intense motion and obscuration.
A.5 Effectiveness of multi-scale optical flows
The optical flows at each scale contribute to noticeable performance improvement. To investigate the role at each scale, we designed the following experiment: we iteratively replace the optical flows scale with the downsaapleddupsaappee version from another scale and then fine-tune the network until convergence. The results are in Table 4 where "(.) f22 denotes 2x upsample, all shorthand for the corresponding scales: specifically, "v"" denotes vi v2.b and likewise for the others. For example, Method MI
"(-) 42" denotes 2x downsample. "v', vrl,,?, 03 01. 8² ja"
replaces the largest-size flows vt, 0² with the intermediate-scat flows v 8² its result measures the
23

---

Graud Truth BipFSNI
VTM-RA
DCVC-FM
DCVC-B 0,0063933.09
BRHVC (our))
0,005833107
0.007269094
0.005433434
Ground Thath BipTSNR
VTM-RA
DCVC-FM
DCVC-B
BRHVC (ours)
0,0083330.67
0.00777/3.21
0.00772/29.94
0,0075331.28
Groud Iruth BypPSNR
VTM-RA 0.012633066
DCVC-FM 0.0171/27.25
DCVC-R
BRHVC (mur))
0,01633001
0.0139/31004
Figure 12: Visual quality comparison.
Table 4: Contributions of the optical flows different scales.
Method
Encoder Flows
Decoder Flows
BD-rate
BRHVC
0
M1
2 f- (v²) f2
i +- (62) f2
6.7%
M2
2² t- (v) 42
1 + [6]] 42
4.2%
M3
v3 + (v²)
63 t (02) L2
3.0%
performance change caused by the absence of u 61. Likewise, M2 and M3 denote replacing the intermediate- and smallest-size optical flows, respectively.
The results show that the flow at every scale provides noticeable gain, demonstrating that leveraging additional flows indeed enables more effective motion compensation. Moreover, optical flows at different scales exhibit varying contributions. As the spatial resolution decreases, the contribution of the optical flow at that scale diminishes due to the reduced amount of information it carries.
A.6 Visualization of BMC
As shown in Figure 13 the camera locks onto the moving racehorse, and the motion of the background should be consistent. Due to the limited receptive field the flow generation network, it is prone to losing track of the moving target when the span is large, resulting in disordered estimated motion in the background (e.g., vi. in BRHVC and 14,0 the baseline). However, BMC can enhance compressed flows (e.g., (2,0) with large-scale flows (e.g., vi and v²,o), thereby achieving more accurate motion compensation for large spans.
A.7 Model Comparison
Details of our baseline. At the onset of this work, given that DCVC-B had not yet released its source code, we referred to its design to reimplement the Baseline. Subsequently, once the DCVC-B framework became open-source, we re-trained it using our training settings to obtain DCVC-B (re-trained). As result, there are differences in the implementation details between the two. The main difference is as follows:
24

---

span reference
small spen referemace
3,2/2
2,/2
orgiinll Trane
Corward relerence Lrame
motion visualization
original
Iorward receounce Irame
motoo visualization
12.
Di-dtrectionl Motion Converge (UVVCC)
Els Ui-directonal Modion Converge (UDVIC)
BRHVC
Baseltne
Motion Compression
Comprissin
diveramolee
desmssmpppee
AC
Figure 13: Comparison of motion compression between BRHVC and the baseline. We only present the forward motion and omit the backward motion. The smiling face denotes regular and consistent background motion, while the sad face denotes irregular and disordered background motion.
The Baseline and BRHVC first dimensionize the references to match the dimensions of latent features and then feed them into Encoder/Decoder, while DCVC-B directly inputs them. Taking the largest size as an example, the process of integrating reference information into the Encoder is as follows:
DCVC-B: y! Coow'' (cat(x: CJ,, CJ)),
Baseline: C RB
RBB (C). Cb) then y Conv- Ct)),
BRHVC: C,, BCF C..... then y Conv' (cat(z, C.)),
where v denotes the latent feature integrated with reference information, Conv' denotes convolu- tion layer in the Encoder, catl.) denotes concatenation along the channel dimension, RB denotes stacking of some ResBlocks. It should be noted that the tensor C, has the same dimensions as Cj and Cn. which reduces the computational overhead of the Baseline and BRHVC in the Encoder/Decoder compared DCVC-B. As result, the Baseline has fewer kMACs and memory overhead compared to DCVC-B, but its decoding speed slightly slower. This may indicate that there still room for optimization in our code, which we will attempt to improve future work. Moreover, since there is not much difference in the overall mechanism, the performance of the two is also quite close.
Table 5: Comparison with models of the comparable complexity or encoding/dccoding speed.
Model
BD-rate
Parameters
kMACs/pixel
Memory
Enc.
Dec. time
DCVC-B(re-trained)
0.4%
24.4 M
2934.9
13.5 G
646 5 ms
Baseline (BL)
0.0
23.7 M
2831.9
9.6 G
681 538 ms
BL-1
-3.9%
28.4 M
4060.9
10.4
879 670 ms
BL-2
-4.1%
31.7 M
4833.9
11.1
983 671 ms
BRHVC
-12.3%
29.4 M
3887.8
12.4
983 / 670 ms
Comparison with additional models. We construct two additional models for comparison to verify whether simply stacking modules can yield comparable gains. To match the complexity of BRHVC,
25

---

BL-1 stack 4 ResBlocks per scale (three scales in total) for reference fusion in the encoder and decoder. To match the encoding/decoding speed of BRHVC, BL-2 stack and ResBlocks in the encoder and decoder, respectively. In contrast, the baseline employs only ResBlock per scale at both the encoder and decoder.
Table 5 shows that BL-1 can save 3.9% and 4.1% bitrates than our baseline, far below BRHVC's 12.3% bitrate saving with a relatively small GPU memory cost. This indicates that simply stacking network modules can hardly further improve performance, while the task-specific network design of BRHVC plays significant role.
A.8 MS-SSIM performance
We adopt MS-SSIM, the most commonly used alternative metric besides PSNR, for evaluation. Following the DCVC-B setup, fine-tune MS-SSIM model for only epochs, starting from the MSE-optimized model. The results are shown in T
Table 6: BD-rate comparison for MS-SSIM. DCVC-B and BRHVC are both optimized by MS-SSIM in fine-tuning.
Method
НЕVС
HEVC DEVC
НЕVС
Average
VTM-LD
0
0
0
0
VTM-RA
-35.70%
-30.30%
-31.51%
-30.53%
-31.0%
DCVC-B
-43.87%
-43.1%%
-58.50%
-47.78%
-42.4%
BRHVC (ours)
-45.58%
-45.48%
-59.97%
-45.83%
-44.0%
A.9 Experiments with different Intra Periods
We conducted experiments with different settings in Table Since the configuration files provided by HM-RA and VTM-RA have maximum IP of 32, we performed experiments with IP settings of 16, and 32. For the sake of expediency, we use the first 33 frames on HEVC for testing. Moreover, since the bitrate proportion of I-frames varies significantly across different IP settings, we also provide the BDD-raee computed only over B-frames, denoted as BD-rate w/o I-frame.
Table 7: BD-rate variations across different IP settings the first 33 frames.
Method
Anchor
BD-rate
BD-rate w/o I-frame
BRHVC (IP8)
DCVC-B (IP8)
-2.0%
-15.3%
BRHVC (IP16)
DCVC-B (IP16)
-6.2%
-17.8%
BRHVC (IP32)
DCVC-B (IP32)
-8.8%
-21.6%
The results show that, as IP increases, the frame span also increases in some hierarchical levels, thereby increasing the severity of the URC problem. Under this condition, the improvement of BRHVC over DCVC-B also increases (from -15.3% to -21.6%). This indicates that our method successfully handles URC while DCVC-B does not, thereby aligning with our motivation.
26


Reviews:
[{"review_id": "VCufwWdD5l", "reviewer": "Reviewer_BaaM", "summary": "In this paper, the authors propose a new video coding model, Bi-directional Reference Harmonization Video Compression (BRHVC). The model is based on the conditional coding framework; Unlike most existing work, which focuses on low-delay settings, BRHVC targets hierarchical B-frame coding. While existing learned B-frame coding models offer limited gains over low-delay methods, the authors tackle these limitations by (1) utilising multi-scale optical flow to warp features at different scales and jointly encoding these flows into the latent representation, which improves performance on sequences with large motions, and (2) explicitly modelling the weighting of different reference contexts, enabling better use of bi-directional information.", "questions": "- How exactly were the experiments performed when removing the BCF/BMC modules?\n- The authors should also validate the use of different motion vectors at each scale.\n- It may help to clarify the two main B-frame coding strategies - low-delay B and hierarchical B. The proposed model only address the latter category.", "limitations": "The authors have discussed some limitations of the proposed methods. Please refer to the weaknesses listed above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Overall, although the proposed model demonstrates improvement over the baselines, its novelty is modest, and the new modules have not yet been validated comprehensively.\n\nStrengths\n- The proposed model focuses on hierarchical B-frame coding, addressing a major limitation of current neural video codecs.\n- The motivation is sound: balancing the use of multiple references is indeed important for video coding.\n\nWeaknesses\n- While the proposed techniques are verified by an ablation study showing BD-rate gains for each component, the experiments are not comprehensive. In particular, adding these modules increases both computational complexity and model size. It is unclear whether the added components are the best choice - could similar gains be achieved by simply stacking more layers at the same positions with comparable complexity?\n- The memory footprint (a limitation of conditional coding models) and complexity (e.g., MACs per pixel) are not provided. While encoding/decoding speeds are reported, memory usage and MACs are also critical for practical application, since the decoding speed can vary significantly across platforms.\n- Although the experiments show that there are different importance of references, the authors did not demonstrate how existing models fail to leverage bi-directional context. This omission weakens the motivation for the paper.", "was_revised": true, "final_justification": "In this paper, the authors propose a new B-frame coding model that exploits both past and future reference frames, especially when those references contribute unequally. Experiments show consistent performance gains over prior methods. \n\nHowever, while the paper introduces the notion of “unbalanced reference contribution” as its key motivation, it does not demonstrate how existing models are limited by this phenomenon. In principle, with sufficient data and capacity, end-to-end optimised compression models could learn to balance reference contributions, so handcrafting modules for handling this phenomenon does not seem to be necessary; showing that they fail to do so would substantially strengthen the work’s insight.\n\nBut overall, the authors have addressed most of my other concerns, including additional experiments that support the design, thus I have adjusted my score accordingly.", "rebuttal": "We sincerely thank you for your detailed and instructive reviews, and for recognizing our contributions to B-frame coding. Below are some responses that we hope will address your concerns.\n\n**Weakness 1,2: Additional metrics and comparison under equivalent complexity.**\n\nWe conducted more comprehensive comparison on additional metrics such as kMACs and Max Memory. Furthermore, to verify whether simply stacking modules can yield sufficient gains, we construct a more complex model (denoted as *Baseline w/ Resblocks*) for comparison. The results are as follows:\n\n**Table 4:** Ablation study on different network designs (based on Table 2 in the main text).\n\n| Method | BD-rate | Parameters | kMACs/pixel | Max Memory | Encoding time | Decoding time |\n| --- | --- | --- | --- | --- | --- | --- |\n| DCVC-B (re-trained) | 0.4% | 24.4M | 2934.9 | 13.5G | 646 ms | 511 ms |\n| Baseline | 0   | 23.7M | 2831.9 | 9.6G | 681 ms | 538 ms |\n| Baseline w/ BMC | -6.4% | 24.3M | 3170.9 | 10.6G | 780 ms | 602ms |\n| Baseline w/ BCF | -6.6% | 28.9M | 3532.8 | 11.4G | 896 ms | 583 ms |\n| Baseline w/ BCF,BMC (BRHVC) | -12.3% | 29.4M | 3887.8 | 12.4G | 983 ms | 670 ms |\n| Baseline w/ Resblocks | -4.1% | 31.7M | 4833.9 | 11.1G | 983 ms | 671 ms |\n\nConsidering that inference speed is influenced by many factors and may not always correlate directly with kMACs or memory usage, as noted in [15] and other studies, we suggest using actual coding time as a more reliable metric in this case. Experimental results show that merely increasing model complexity to match BRHVC’s encoding/decoding speed yields only a **4.1 %** improvement—far below BRHVC’s **12.3 %**. It indicates that **the improvement of BRHVC stems from the motivation behind our design, rather than merely from increased complexity.** Moreover, Table 6 in Reviewer iau7's Weakness 2 indicates that **the increase in complexity resulting from our method design is acceptable.**\n\nHere is a detailed description of *Baseline w/ Resblocks*. To match the encoding/decoding speed of BRHVC, we stack 9 ResBlocks per scale (three scales in total) for reference fusion in the encoder and 4 ResBlocks per scale in the decoder. In contrast, the baseline employs only 1 ResBlock per scale at both the encoder and decoder. Please refer to Reviewer L376’s Weakness 1 for additional details of the Resblocks (denoted as $RB$).\n\nFurther ablation experiments on BMC can be found in Question 2.\n\n**Weekness 3: How existing models fail to use reference information.**\n\nIn the main text, we have provided explanations of our design principles based on existing models and demonstrated the visualization effects. To further validate the effectiveness, we conducted additional comparative experiments. Below are the details.\n\nFirst, existing methods did not have specialized designs for the URC issue. As stated in lines 170–174, existing methods [33,34] do not explicitly model the weights of the two reference frames; instead, they perform reference fusion directly via concatenate&convolution. The weights of references are learned in training and fixed after training, which can not adapt to the value of references. In contrast, as shown in Equation 3, our BRHVC explicitly models the weights with $s_f$, $s_b$, and $b_t$.\n\nSecond, the visualization in Figure 8 shows that our design performs as expected. BRHVC focuses its attention on more valuable regions, and this mechanism enables it to handle the URC issue more effectively.\n\nThird, experiments demonstrate that BRHVC is capable of handling the URC issue more effectively. The experiments in Table 4 show that BRHVC achieves a significant overall performance improvement. Moreover, the experiments in Table 8 (from Reviewer iau7's Weakness 6) show that BRHVC performs better than DCVC-B when the URC problem is more severe.\n\n**Question 1: Experiment of removing BCF/BMC.**\n\nThe newly added Table 4 presents the experimental results for these modules, where *Baseline w/ BCF,BMC* and *Baseline* corresponds to our *BRHVC* and *BRHVC w/o BCF,BMC*, respectively. For details on the baseline design, please refer to Reviewer L376's Weakness 1.\n\n**Question 2: Validate optical flows at different scales.**\n\nThe optical flows at each scale contribute to a noticeable performance improvement. To investigate the role at each scale, we designed the following experiment: we iteratively replace the optical flows at one scale with the downsampled/upsampled version from another scale and then fine-tune the network until convergence. The results are in Table 5, where \"$(\\cdot)\\uparrow_2$\" denotes 2× upsample, \"$(\\cdot)\\downarrow_2$\" denotes 2× downsample. \"$v^1, v^2, v^3,\\hat{v}^1,\\hat{v}^2,\\hat{v}^3$\" are all shorthand for the corresponding scales: specifically, \"$v^1$\" denotes $v^1_{t,f}, v^1_{t,b}$, and likewise for the others. For example, Method M1 replaces the largest-size flows $v^1,\\hat{v}^1$ with the intermediate-scale flows $v^2,\\hat{v}^2$, its result measures the performance change caused by the absence of $v^1,\\hat{v}^1$. Likewise, M2 and M3 denote replacing the intermediate- and smallest-size optical flows, respectively.\n\n**Table 5:** Contributions of the optical flows at different scales.\n\n| Method | Encoder Flows | Decoder Flows | BD-rate |\n| --- | --- | --- | --- |\n| BRHVC | --  | --  | 0   |\n| M1  | $v^1\\leftarrow (v^2)\\uparrow_2$ | $\\hat{v}^1 \\leftarrow (\\hat{v}^2)\\uparrow_2$ | 6.7% |\n| M2  | $v^2\\leftarrow (v^1)\\downarrow_2$ | $\\hat{v}^2 \\leftarrow (\\hat{v}^1)\\downarrow_2$ | 4.2% |\n| M3  | $v^3\\leftarrow (v^2)\\downarrow_2$ | $\\hat{v}^3 \\leftarrow (\\hat{v}^2)\\downarrow_2$ | 3.0% |\n\nExperimental results demonstrate that **the flow at every scale provides a noticeable gain**, demonstrating that leveraging additional flows indeed enables more effective motion compensation. Moreover, optical flows at different scales exhibit varying contributions. As the spatial resolution decreases, the contribution of the optical flow at that scale diminishes due to the reduced amount of information it carries.\n\n**Question 3:  Carify the two main B-frame coding strategies.**\n\nThank you for your feedback. We will specifically distinguish the type of coding in the manuscript."}, {"review_id": "CHparkTp1E", "reviewer": "Reviewer_5ESm", "summary": "This paper proposes BRHVC, a novel neural B-frame video compression method that addresses the inaccurate motion compensation and unbalanced reference contribution issue in bi-directional coding. The paper introduces two key modules(BMC and BCF) to enhance motion compensation and adaptively harmonizes reference information. Despite underperforming VTM-RA on several test datasets, the paper provides visualized analysis of module design and effect, and includes runtime efficiency evaluation, offering a well-rounded and transparent study.", "questions": "1. The paper states that the Motion Compensation module adopts the Contextual-Feature Extraction (CFE) method as used in references [20] and [33]. However, this specific concept does not appear explicitly in the main texts of those references. Could the authors clarify what is meant by this term?\n\n2. Could the authors provide a theoretical analysis of why the proposed system performs less effectively on high-resolution datasets, such as UVG and MCL-JCV?\n\n3. How is the proposed method compared to recent multimodal-based video compression methods, such as M3-CVC?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1. Multiscale optical flow extraction is employed in this paper, which is a novel approach that effectively mitigates inaccuracies in motion estimation. Its effectiveness is further supported by clear visualizations provided in the appendix.\n\n2. The BCF module is well-designed to leverage multiscale motion-compensated features through a weighted summation scheme to improve contextual reconstruction. The approach demonstrates a certain level of originality.\n\n3. The authors perform a quantitative evaluation of runtime efficiency, including encoding and decoding time, which contributes to assessing the system’s practical applicability.\n\n\nWeakness:\n\n1. The multiscale optical flow extraction process effectively requires three separate calls to SpyNet for each frame, which may incur redundant computation. As shown in Figure 13 of the appendix, when the motion between frames is small, extracting optical flow at the original resolution and directly downsample can yield comparable results, suggesting the current approach may be computationally excessive in such cases.\n\n2. The paper lacks detailed reporting on the overall computational complexity (e.g., in terms of kMACs/pixel), as well as comparisons with similar systems. In addition, it does not provide a breakdown of the computational cost and latency associated with each individual module.\n\n3. The proposed system underperforms the VTM-RA baseline on 4 out of 6 benchmark datasets, indicating that its overall compression performance still requires further improvement.", "was_revised": true, "final_justification": "Thanks for the authors' feedback, which mainly addresses my concerns. I will keep my score.", "rebuttal": "We sincerely thank you for your positive feedback on our work. Below are our responses to your concerns, and we hope these efforts can better clarify our work.\n\n**Weakness 1: Computationally excessive of BMC.**\n\nIt is indeed the case that invoking SpyNet three times may seem excessive, especially for small motions, as you pointed out. However, as shown in Reviewer BaaM's Question 2, on average, the optical flow at each scale has made an indispensable contribution. Thus, we think retaining three separate flows remains the best compromise for now. In future work, we intend to migrate to a single-pass architecture that delivers all three flows in one inference. Thank you for your comment, it has reinforced this plan. Nevertheless, BRHVC currently still achieves good performance under the same complexity (see *Baseline w/ Resblocks* in Table 4).\n\n**Weakness 2: More detailed computational complexity.**\n\nRefer to Table 4, we provide more detailed metrics, including kMACs/pixel and max memory usage. Specifically, BCF adds approximately 700 kMACs/pixel and 50 ms to decoding latency, while BMC adds about 340 kMACs/pixel and 80 ms. Although BRHVC increases complexity, as can be seen from Table 4, our method is more effective for methods of equivalent complexity (i.e., *Baseline w/ Resblocks*).\n\n**Weakness 3: Performance comparison with VTM-RA.**\n\nAlthough BRHVC underperforms VTM-RA on some datasets, it indeed achieves a substantial improvement over previous Nerual Video Compression (NVC) methods, outperforming them on all datasets (see Table 1). Additionally, the gap with VTM-RA is mainly observed on UVG and MCL-JCV, which will be analyzed in Question 2.\n\n**Question 1: More details on Contextual-Feature Extraction (CFE).**\n\nThe CFE module combines Temporal Context Mining [20, 32] with the post-processing Group-Based Offset Diversity mechanism [20]. For conciseness, we merge the two modules into one, denoted as CFE. The role of CFE is to extract reference information from $F_f,F_b$ and perform motion compensation on them with optical flows. To convey this point more clearly, we will refine the wording in this part. Thank you for your feedback.\n\n**Question 2: Performance on UVG and MCL-JCV.**\n\nWe speculate that the main reason for our inferior performance compared to VTM-RA on MCL-JCV and UVG is that these datasets are high-resolution (1920×1080), whereas the Vimeo-90K training set commonly used for Neural Video Compression (NVC) lacks such high-resolution data. Besides BRHVC, this shortcoming results in generally poor performance of NVCs methods on these datasets.\n\nTo verify this, we downsampled the two datasets and re-evaluated them against VTM-RA. We resized the images to 480×270 and, for efficiency, selected the first 33 frames for testing. As shown in Table 9, **the performances of NVC methods overall show significant improvements in low-resolution settings, with BRHVC even outperforming VTM-RA noticeably.**\n\n**Table 9:** BD-rate comparison on downsampled MCL-JCV and UVG.\n\n| Method | MCL-JCV (downsample) | UVG (downsample) |\n| --- | --- | --- |\n| HM-RA | 35.4% | 42.5% |\n| VTM-RA | 0   | 0   |\n| DCVC-DC | 5.6% | 6.8% |\n| DCVC-B | 13.9% | 0.9% |\n| BRHVC | -5.3% | -10.5% |\n\nMost NVC methods train on cropped 256×256 patches, which aids training stability and conserves computational resources but limits the model's ability to handle larger spatial contexts (e.g., 1080P images). We encourage the NVC community to further explore this issue, which may lead to further improvements in the performance of NVC. Despite these challenges, our method has made considerable progress, bringing NVC closer to VTM-RA than ever before.\n\n**Question 3: Comparison with multimodal-based method.**\n\nRegrettably, we believe it may not be feasible to make a fully fair comparison between BRHVC and these approaches. First, there are differences in research focus: their methods target subjective metrics (e.g., LPIPS) at extremely low bitrate, while our work aims for objective metrics (i.e., PSNR) and video fidelity at medium to high bitrate. Second, the lack of open-source code and PSNR results for M3-CVC further complicates direct comparison. Nevertheless, we will open-source our model in the future, enabling other researchers to compare it with other video coding methods across different fields."}, {"review_id": "m8oZZCI1g0", "reviewer": "Reviewer_iau7", "summary": "This paper introduces BRHVC, a novel neural B-frame video compression method designed to tackle the challenge of unbalanced reference contributions in hierarchical coding. It achieves this through two proposed modules: Bi-directional Motion Converge (BMC) for improved motion handling and Bi-directional Contextual Fusion (BCF) for adaptively weighting bi-directional references. BRHVC demonstrates state-of-the-art performance, outperforming previous neural methods and surpassing VTM-RA on HEVC datasets.", "questions": "- Could you please clarify the exact differences between the \"Baseline\" model used in the ablation study and the \"DCVC-B (re-trained)\"? What accounts for the small performance difference shown in Table 2?\n- The experiments focus on an Intra Period (IP) of 32. How does BRHVC's performance, and the severity of the Unbalanced Reference Contribution problem, vary with different IP settings? Would BRHVC maintain its advantage at shorter or longer IP values?\n- The performance gains are less significant on high frame rate datasets like MCL-JCV and UVG. What are the potential reasons or hypotheses for this observation? Are there specific characteristics of these datasets that limit BRHVC's effectiveness compared to HEVC datasets?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "** Pros\n - The paper effectively identifies and proposes solutions for the Unbalanced Reference Contribution problem inherent in hierarchical B-frame coding.\n - It achieves SOTA performance, notably surpassing the high-performance VTM-RA codec on standard datasets.\n - It introduces innovative BMC and BCF modules that enhance motion compensation and context fusion for better reference utilization.\n\n** Cons\n - The proposed framework integrates BMC and BCF modules, where BMC refines motion estimation and compensation, and BCF uses the resulting context features for fusion. This suggests a sequential flow of information. One may consider a mechanism for tighter integration, such as joint optimization or feedback loops, where the effectiveness of the context fusion (BCF output) could inform or adapt the motion processing (BMC). Exploring a more deeply intertwined design between motion and context modules might potentially enable further performance improvements by allowing them to learn to cooperate more effectively end-to-end.\n\n - As shown in the ablation study (Table 2), BRHVC introduces a substantial increase in encoding time (about 50% over baseline) and decoding time (about 30% over baseline) compared to the baseline model. While the authors state this overhead is \"relatively acceptable\" given the BD-rate improvement, such a significant increase in computational cost could be a major barrier for real-world deployment, especially in scenarios with strict real-time constraints or limited hardware resources (like mobile devices for decoding). The trade-off between performance and complexity might not be universally \"acceptable\" depending on the specific application.\n\n - The ablation study effectively demonstrates the contribution of BCF alone and the additional contribution of adding BMC after BCF. However, it lacks the result of using BMC alone (e.g., Baseline w/ BMC). Presenting this result would provide clearer insight into the independent contribution of the BMC module and its efficiency, as well as how its interaction with BCF leads to the final performance gain. \n\n - The paper primarily uses PSNR-based BD-rate to demonstrate performance gains. While PSNR is a standard objective metric in video compression, it does not always correlate perfectly with perceived visual quality by humans. Metrics like SSIM, MS-SSIM, or VMAF are often considered more representative of subjective user experience. The absence of results using these alternative metrics makes it difficult to fully assess the practical benefits of BRHVC in terms of visual perception.\n\n - The paper acknowledges that BRHVC's performance gains are less pronounced on high frame rate datasets such as MCL-JCV and UVG compared to HEVC datasets. While identifying this limitation is valuable, the paper does not provide a detailed analysis or hypothesis as to why this performance gap exists. Is the Unbalanced Reference Contribution problem less severe in high frame rate videos due to smaller frame spans? Are there other dominant challenges in high frame rate compression that BRHVC's current design doesn't adequately address? A deeper analysis of the causes would be crucial for guiding future research on improving NBVC for high frame rate content.\n\n- All primary experiments are conducted using an Intra Period (IP) of 32. The hierarchical coding structure of B-frames, and consequently the characteristics of reference spans and the severity of the URC problem, can change significantly with different IP lengths. While IP=32 is a common setting for RA configurations, evaluating BRHVC's performance across a range of IP values (e.g., shorter IPs like 8 or 16, or even longer ones) would provide a more comprehensive understanding of its robustness and the generalizability of its benefits across various hierarchical coding structures.", "was_revised": true, "final_justification": "The paper tackles an important problem in hierarchical B-frame coding, with well-designed BMC and BCF modules delivering strong SOTA results. The rebuttal addressed most earlier concerns with additional experiments and analyses. While the computational overhead is notable, the justification is reasonable for many scenarios. Given its technical merit, clear gains, and potential for further optimization, I raise my rating to borderline accept.", "rebuttal": "We sincerely thank your time, effort, and professional expertise. We have carefully considered your concerns and provided detailed answers, hoping these will make our work more solid.\n\n**Weakness 1: Enhance collaboration between the two modules.**\n\nThis is a great idea. Perhaps we can implement a mechanism similar to a 2-pass or coarse-to-fine approach. It would allow the two modules to interact and harmonize with each other, thereby enhancing performance. However, it might introduce some complexity. In any case, thank you for your suggestion. We will try to incorporate it in our future work.\n\n**Weakness 2: Is the computational cost \"acceptable\"?**\n\nWe believe the overhead is **indeed \"acceptable\"** given the performance gains achieved. To verify it, we use the most critical metrics—percentage overhead of decoding time over the baseline and BD-rate gain based on its baseline as the criterion.\n\n**Table 6:** Gain–overhead comparison across some methods.\n\n|     | Venue | Baseline | Gain | Overhead |\n| --- | --- | --- | --- | --- |\n| DCVC-DC | CVPR'23 | DCVC-HEM | 24.9% | 17% |\n| [s1] | CVPR'25 | DCVC-DC | 8.3% | 32% |\n| [s2] | CVPR'25 | DCVC-DC | 5.3% | 45% |\n| BRHVC (ours) | --  | DCVC-B | 10.1% | 31% |\n\n**References**\n\n[s1] Augmented deep contexts for spatially embedded video coding.\n\n[s2] Neural video compression with context modulation.\n\nThe results are shown in Table 6. Although the anchor of gains may be not strictly aligned, it still provides partial evidence. It can be seen that our method’s trade-off between gain and overhead falls within an acceptable range. Several additional factors should be considered:\n\nFirst, coding efficiency exhibits diminishing returns: DCVC-DC (2023) achieves an attractive gain-overhead ratio, whereas the two 2025 works [s1,s2] require substantially higher costs for further improvements.\n\nSecond, RA coding inherently processes more information than LD coding, so any enhancement built upon it is more prone to increased overhead. However, unlike LD coding, RA theoretically allows parallel encoding of multiple frames at the same level, which can further reduce the overall complexity when deployed in practice.\n\nThird, RA is not designed for low-latency requirements. Much like the historical development of LD, it may be more appropriate for RA to first explore performance-boosting mechanisms and then seek complexity reduction.\n\nIn conclusion, we believe the overhead is indeed \"acceptable\" given the performance gains achieved.\n\n**Weakness 3: Performance of Baseline w/ BMC.**\n\nAs shown in Table 4, using BMC alone (denoted as *Baseline w/ BMC*) can save 6.4% bitrate. Moreover, the complexity of each module has been explicitly listed.\n\n**Weakness 4 : Other image-quality metrics.**\n\nWe adopt MS-SSIM, the most commonly used alternative metric besides PSNR, for evaluation. Following the DCVC-B setup, we fine-tune our MS-SSIM model for only 2 epochs, starting from the MSE-optimized model. **The results in Table 7 show that BRHVC significantly outperforms VTM-RA in terms of MS-SSIM.** Due to the time constraints of the rebuttal period, we can not thoroughly fine-tune the model, which may limit further performance gains relative to DCVC-B.\n\n**Table 7:** BD-rate comparison for MS-SSIM. DCVC-B and BRHVC are both optimized by MS-SSIM in fine-tuning.\n\n| Method | HEVC B | HEVC C | HEVC D | HEVC E | MCL-JCV | UVG | Average |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| VTM-LD | 0   | 0   | 0   | 0   | 0   | 0   | 0   |\n| VTM-RA | -35.70% | -30.30% | -31.51% | -30.53% | -28.62% | -29.59% | -31.0% |\n| DCVC-B | -43.87% | -43.18% | -58.50% | -47.78% | -37.35% | -23.91% | -42.4% |\n| BRHVC (ours) | -45.58% | -45.48% | -59.97% | -45.83% | -39.32% | -27.38% | -44.0% |\n\n**Weakness 5, Quesion 3: Performance on UVG and MCL-JCV.**\n\nThis Weakness is similar to Reviewer 5ESm's Question 2. The primary reason is the absence of high-resolution training data in our dataset. Please see it for more details.\n\n**Weakness 6, Question 2: Experiments with different IPs.**\n\nWe conducted experiments with different IP settings in Table 8. Since the configuration files provided by HM-RA and VTM-RA have a maximum IP of 32, we performed experiments with IP settings of 8, 16, and 32. For the sake of expediency, we use the first 33 frames on HEVC for testing. Moreover, since the bitrate proportion of I-frames varies significantly across different IP settings, we also provide the BD-rate computed only over B-frames, denoted as *BD-rate w/o I-frame*.\n\n**Table 8:** BD-rate variations across different IP settings on the first 33 frames.\n\n| Method | Anchor | BD-rate | BD-rate w/o I-frame |\n| --- | --- | --- | --- |\n| BRHVC (IP8) | DCVC-B (IP8) | -2.0% | -15.3% |\n| BRHVC (IP16) | DCVC-B (IP16) | -6.2% | -17.8% |\n| BRHVC (IP32) | DCVC-B (IP32) | -8.8% | -21.6% |\n\nThe results show that, as IP increases, the frame span also increases in some hierarchical levels, thereby increasing the severity of the URC problem. Under this condition, the improvement of BRHVC over DCVC-B also increases (from -15.3% to -21.6%). This indicates that our method successfully handles URC while DCVC-B does not, thereby aligning with our motivation.\n\n**Question 1: The difference between Baseline and DCVC-B (re-trained).**\n\nThis Question is similar to Reviewer L376's Weakness 1. Please see it for more details."}, {"review_id": "K6VWnTZv6P", "reviewer": "Reviewer_L376", "summary": "The authors analyse the effect of the Unbalanced Reference Contribution (URC) issue with a quantitative experiment in Section 3. Based on the motivation of coping with URC issue, they build a novel framework termed BRHVC, with proposed BMC and BCF. Experimental results show that their BRHVC outperforms previous SOTA nerual video compression methods, which is a remarkable progress.", "questions": "See Weaknesses.", "limitations": "As mentioned in paper, BRHVC does not perform sufficiently well on high frame rate datasets, which may require further investigation in future work.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The proposition and analysis of the URC issue are very interesting. The authors design a clever toy experiment to illustrate the significant impact of URC on bi-directional compression.\n2. The motivation behind the design of BRHVC is reasonable. The BMC can provide more accurate optical flows in a large scale, while the BCF offers reference weights to better integrate reference information.\n3. The progress of BRHVC is remarkable compared with the baseline. And the overall performance outperforms previous SOTA nerual video compression methods, which is comparable to VTM-RA. \n4. The paper is well-written with good readability and is easy to understand.\n\nWeaknesses:\n1. The description of the baseline is too brief. It seems that the baseline employs a similar architecture to DCVC-B [33], and its performance is also quite comparable. So why not directly use DCVC-B as the baseline? What are the main differences between the baseline and DCVC-B?\n2. There is a lack of ablation study. Although, as mentioned above, the improvement of BRHVC over the baseline is reasonable, there is still a need for a model with similar complexity to be compared with BRHVC.", "was_revised": true, "final_justification": "Taking into account the paper's contributions and the additional evidence provided during the rebuttal, I have decided to raise my score to 5. The authors identify and analyze the URC issue in bi-directional compression and, building on this insight, propose BRHVC, which substantially outperforms previous Neural Video Compression (NVC) methods. In the rebuttal, they supplied further ablation studies and complexity comparisons, demonstrating that the design is well-motivated and the computational overhead is acceptable.", "rebuttal": "We sincerely appreciate your affirmation and recognition of our work. Here are our detailed responses to your concerns, and we hope they will clarify our work.\n\n**Weakness 1: The difference between Baseline and DCVC-B (re-trained).**\n\nAt the onset of this work, given that DCVC-B had not yet released its source code, we referred to its design to reimplement the *Baseline*. Subsequently, once the DCVC-B framework became open-source, we re-trained it using our training settings to obtain *DCVC-B (re-trained)*. As a result, there are differences in the implementation details between the two. The main difference is as follows:\n\nThe Baseline and BRHVC first dimensionizes the references to match the dimensions of latent features and then feeds them into Encoder/Decoder, while DCVC-B directly input them. Taking the largest size as an example, the process of integrating reference information into the Encoder is as follows:\n\nDCVC-B: $y^1_r=Conv^1(cat[x,C^1_f,C^1_b])$,\n\nBaseline: $C^1_t=RB^1(C^1_f, C^1_b)$ then $y^1_r=Conv^1(cat[x,C^1_t])$,\n\nBRHVC: $C^1_t,...=BCF_E(C^1_f, C^1_b,...)$ then $y^1_r=Conv^1(cat[x,C^1_t])$,\n\nwhere $y^1_r$ denotes the latent feature integrated with reference information, $Conv^1$ denotes a convolution layer in the Encoder, $cat[\\cdot]$ denotes concatenation along the channel dimension, $RB^1$ denotes stacking of some ResBlocks.\n\nIt should be noted that tensor $C_t$ has the same dimensions as $C_f$ and $C_b$, which reduces the computational overhead of the Baseline and BRHVC in the Encoder/Decoder compared to DCVC-B.\n\nAs a result, the Baseline has fewer kMACs and memory overhead compared to DCVC-B, but its decoding speed is slightly slower. This may indicate that there is still room for optimization in our code, which we will attempt to improve in future work. Moreover, since there is not much difference in the overall mechanism, the performance of the two is also quite close.\n\n**Weakness 2: Comparison under equivalent complexity.**\n\nThis question is similar to Reviewer BaaM's Question 1. As shown in Table 4, we constructed a more complex model (denoted as *Baseline w/ Resblocks*) for comparison. Experimental results show that merely increasing model complexity to match BRHVC’s encoding/decoding speed yields only a 4.1 % improvement—far below BRHVC’s 12.3 %. It confirms that BRHVC indeed enhances the performance effectively."}]

Author rebuttal:

Meta-review: This paper introduces BRHVC, a novel framework that addresses the Unbalanced Reference Contribution problem, a significant but underexplored challenge in neural B-frame video compression. Through its innovative Bi-directional Motion Converge (BMC) and Contextual Fusion (BCF) modules, the method achieves SOTA performance, even surpassing the traditional VTM-RA codec on standard datasets. The authors' engagement during the rebuttal was exemplary; they provided extensive new experiments and offered thorough, compelling responses to the reviewers' questions.

However, despite the paper's clear contributions, the initial reviews raised several critical concerns, focusing on the model's computational overhead, comparisons against equivalently complex models, performance under different coding configurations, and the performance drop on specific high-resolution/high-frame-rate datasets. The authors' rebuttal proved decisive: they provided detailed complexity analyses (kMACs/memory), supplied missing ablation studies, demonstrated performance across different IP settings, and reasonably explained the model's limitations on high-resolution video with new downsampling experiments, successfully resolving all core issues.

This productive discussion was directly reflected in the final reviewer stances; all reviewers were satisfied with the authors' responses, and those who were initially "borderline accept" explicitly raised their ratings, forging a strong and unanimous consensus for acceptance. Therefore, given the work's solid technical contributions and the authors' comprehensive resolution of all critical points raised during the review process, my recommendation is to Accept.
Decision: Accept (poster)