3D-LMVIC: Learning-based Multi-View Image Compression with 3D Gaussian Geometric Priors
Authors: Yujun Huang
Bin Chen
Niu Lian
Xin Wang
Baoyi An
Tao Dai
Shu-Tao Xia
Venue: ICML 2025 poster (2025)
Keywords: Multi-View Image Compression; 3D Gaussian Splatting; Deep Learning
Primary area: applications->computer_vision

Abstract: Existing multi-view image compression methods often rely on 2D projection-based similarities between views to estimate disparities. While effective for small disparities, such as those in stereo images, these methods struggle with the more complex disparities encountered in wide-baseline multi-camera systems, commonly found in virtual reality and autonomous driving applications. To address this limitation, we propose 3D-LMVIC, a novel learning-based multi-view image compression framework that leverages 3D Gaussian Splatting to derive geometric priors for accurate disparity estimation. Furthermore, we introduce a depth map compression model to minimize geometric redundancy across views, along with a multi-view sequence ordering strategy based on a defined distance measure between views to enhance correlations between adjacent views. Experimental results demonstrate that 3D-LMVIC achieves superior performance compared to both traditional and learning-based methods. Additionally, it significantly improves disparity estimation accuracy over existing two-view approaches.

3D-LMVIC:
Learning-based Multi- View Image Compression with 3D Gaussian Geometric Priors
Yujun Huang" 1
Bin Chen
Niu Lian?
Xin Wang'
Baoyi An Tao Dai Shu-TToo Xia
Abstract
Existing multi-view image compression methods often rely on 2D projection- based similarities be- tween views to estimate disparities. While effec- tive for small disparities, such as those in stereo images, these methods struggle with the more complex disparities encountered in wide-baseline multi-camera systems, commonly found in virtual reality and autonomous driving applications. To address this limitation, we propose 3D-LMVIC, novel learning-based multi-view image com- pression framework that leverages 3D Gaussian Splatting to derive geometric priors for accurate disparity estimation. Furthermore, introduce depth map compression model to minimize ge- ometric redundancy across views, along with a multi-view sequence ordering strategy based on a defined distance measure between views to en- hance correlations between adjacent views. Ex- perimental results demonstrate that 3D-LMVIC achieves superior performance compared to both traditional and learning- based methods. Addition- ally, it significantly improves disparity estimation accuracy over existing two-view approaches.
(a)
(b)
Figure 1. Illustrations of camera systems. (a) A stereo camera
configuration. (b) A wide-baseline multi-camera configuration.
tonomous driving (Chen et al., 2017), and medical imaging (Hosseinian & Arefi, 2015). In particular, applications like VR and AR, which rely on high high-quality multi-view visual content to create immersive experiences, generate a massive volume of data that poses significant challenges for storage and transmission. This makes the development of efficient compression techniques crucial for managing the increasing data demands these fields.
As illustrated in Figure 1, unlike the commonly studied stereo camera systems, 3D applications often rely wide- baseline multi-camera systems to capture global scene infor- mation (Xu et al., 2020; Yan et al., 2024). In such scenarios, the spatial positions and viewing angles cameras differ significantly compared to stereo setups, resulting in large disparities between images captured from different views. Existing disparity estimation methods typically rely find- ing similar local regions in the image domain estimate disparities. However, this approach faces significant chal- lenges when dealing with complex and large disparities.
1. Introductioin
The rapid advancement of 3D applications has led to an explosion of multi-view image data across various fields, including virtual reality (VR) (Anthes et al., 2016), aug- mented reality (AR) (Schmalstieg & Hollerer, 2016), visual simultaneous localization and mapping (vSLAM) (Mokssit et al., 2023), 3D scene understanding (Dai et al., 2017),
Current multi-view coding standards, such as H. H.264-based MVC (Vetro al., 201 and H.265-based MV-HEVC (Han- nuksela al., 2015), have been developed compress multi-view media by extending their respective base stan- dards and exploiting redundancies across multiple views. These standards employ disparity estimation to calculate positional differences of objects between views, aiding in the prediction of pixel values. However, these methods rely on manually designed modules, limiting the system's 's ability to fully leverage end-to-end optimization.
"quual contribution 'Shenzhen International Graduate School, Tsinghua University, Shenzhen, China "School Computer Sci- ence and Technology. Harbin Institute of Technology, Shen- zhen, China 'Huawei Technologies Company Ltd., Shenzhen, China 'Department of Software Engineering, Shenzhen Univer- sity, Shenzhen, China. Correspondence to: Bin Chen <chen- bin2021 hit.edu.cn.>.
Proceedings of the 4dnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
Learning-basee single image compression has seen remark- able advancements (Ballé al., 2017; 2018; Minnen al., 2018), inspiring extensions of these methods to multi-view image coding (Deng et al., 2021; Lei et al., 2022; Zhang et al., 2023; Liu et al., 2024). A central challenge in these ex- tensions lies in the accurate estimation of disparities across different views. For example, Deng et al. (2021; 2023) employ a simple 3x3 homography matrix for disparity es- timation, which, while efficient, struggles with complex scene disparities. Alternatively, Ayzik & Avidan (2020); Huang et al. (2023) utilize patch matching method to align the reference view with the target view. This approach is effective for horizontal or vertical view shifts but falls short when addressing non-rigid deformations caused by view rotations. Similarly, Zhai et al. (2022) assume that dis- parity occurs only along the horizontal axis in their stereo matching method, which suffices for stereo images but is inadequate for more complex view transformations where disparity is not limited to the horizontal axis. Some methods leverage cross- aesse-attnntoon mechanisms for implicit alignment (Wödlinger et al., 2022; Zhang et al., 2023; Liu et al., 2024). For instance, Zhang et al. (2023) enhance the target view's representation by multiplying its query with the reference view's key and value, effectively incorporating reference view features into the target view. However, these methods primarily establish correlations between two views by 2D projection similarities, without considering the 3D spatial relationships between the views and the captured objects.
We propose learning-basee multi-view image com- pression framework with 3D Gaussian geometric priors (3D-LMVIC), which utilizes 3D Gaussian geometric priors for precise disparity estimation between views, thereby enhancing multi-view image compression ef- ficiency. Additionally, we design a mask based on these priors to identify overlapping regions between views, effectively guiding the model to retain useful cross-view information.
We also present a depth map compression model aimed reducing geometric redundancy across views. Ad- ditionally, we define and prove distance measure between views, upon which multi-view sequence or- dering method is proposed to improve the correlation between adjacent views.
Experimental results show that our framework sur- passes both traditional and learning-based multi-view image coding methods in compression efficiency. Moreover, our disparity estimation method demon- strates greater visual accuracy compared to existing two-view disparity estimation methods.
2. Related Works
Single Image Coding. Traditional image codecs, such as JPEG (Wallace, 1992), BPG (Bellard, 2014), and VVC (Bross et al., 2021), employ manually designed modules like DCT, block-based coding, and quadtree plus binary tree partitioning to balance compression and visual quality. These methods, however, do not achieve end-to-end joint optimization, limiting their performance.
Building on prior investigation, we propose a novel learning- based multi-view image compression framework with 3D Gaussian geometric priors (3D-LMVIC), which employs 3D-GS as a geometric prior to guide disparity estimation between views. Specifically., 3D-GS generates a depth map for each view, providing precise spatial information at the pixel level. This enables accurate correspondence between views, allowing the compression model to effectively fuse features from reference views. Due to positional and an- gular disparities between views. images generally do not fully overlap, and merging non-overlapping regions may introduce noise. To address this, we design a mask based on the 3D Gaussian geometric prior to identify overlapping re- gions, ensuring more accurate feature fusion. Additionally, since depth maps are required during decoding, we propose depth map compression model to efficiently reduce geo- metric redundancy across views, incorporating a cross-view depth prediction module to capture inter-view geometric correlations. Finally, recognizing the importance of field of view (FoV) overlap in redundancy reduction, we introduce a multi-view sequence ordering method to address the issuc of low overlap between adjacent views in unordered sequences. This method defines and proves a distance measure between view pairs to guide the ordering of view sequences.
In recent years, learning-based image compression methods have integrated autoencoders with differentiable entropy models to enable end-to-end optimization of rate-distortion loss. Early works, such as Ballé et al. (2017: 2018), in- troduced generalized divisive normalization (GDN) (Ballé et al., 2016) and proposed factorized and hyperprior en- tropy models. Subsequent research (Minnen al., 2018; He et al., 2021; Jiang et al., 2023) incorporated autoregressive structures into entropy models, resulting in more accurate probability predictions. These advancements have laid the foundation for learning-based multi-view image coding,
Multi-view Image Coding. Traditional multi-view image codecs, such as MVC (Vetro et al., 201 1) and MV-HEVC (Hannuksela et al., 2015), extend H.264 and H.265, respec- tively, by incorporating inter-view correlation modeling to eliminate redundant information between different views. However, these modules are manually designed, potentially limiting their ability to fully exploit cross-view information.
Learning- based multi-view image coding primarily focuses on stereo image coding (Deng et al., 2021; Lei et al., 2022:
2

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
3D World Point
(%e Yo. Xe)
|CH
Image
Image
Image
Image XN
3D-GS
Depth d
Depth d
Depth
Depth dx
(% I. Yoil
(ou yul
Camera Center
Camera Cester
Training
Prediction
Reference Relationship Coding
View n-1
View
(a) Overview of the 3D-LMVIC pipeline.
(b) Depth-based disparity estimation process.
Figure 2. Overall Pipeline.
Wödlinger et al., 2022; Zhai et al., 2022; Deng et al., 2023; Liu et al., 2024) and distributed image coding (Ayzik & Avi- dan, 2020; Huang et al., 2023; Zhang et al., 2023). These methods either rely on finding explicit pixel coordinate cor- respondences between views or use attention-base implicit correspondence modeling to capture inter-view correlations. However, they model inter-view correlations based solely on two-dimensional view images, which may not fully re- flect the correspondences in the original cires-dimensional space.
model parameters are used consistently across all views for both image compression and depth map compression.
The remainder of this section structured as follows: Sec- tion 3.1 elaborates on the method for depth map estimation for a given view using the 3D-GS and the estimation of inter- view disparities. Section 3.2 covers the compression model for both images and depth maps, well the multi-view sequence ordering method.
3.1. 3D-GS Based Depth and Disparity Estimation
Gaussian Splatting. 3D Gaussian Splatting (Kerbl al., 2023; Hamdi al., 2024) introduces differentiable point-based rendering technique that represents 3D points Gaussian functions (mean, variance, opacity, color) and projects these 3D Gaussians onto a view to form image. This differentiablee point-based rendering function allows for the backward update of the attributes of the 3D Gaussians, ensuring that their geometrical and textural properties match the original 3D scene. This approach inspired us to utilize 3D Gaussian Splatting to obtain geometric priors of the original 3D scene, aiding in the task of multi-view image compression.
3.1.1. DEPTH ESTIMATION
For image Hn E Wx with spatial dimensions W and H, we aim to derive depth map d,, E TR= repre- senting the z-axis coordinates of each pixel's corresponding 3D world point in the camera coordinate system. This depth map facilitates the estimation of disparities between differ- ent views.
In the context of the 3D-GS framework, consider set of M ordered 3D points projected along ray from the camera through a pixel. The rendered pixel color e can be expressed as:
3. Proposed Method
M
1-1 II (1 - az).
Figure 2(a) shows the overall pipeline of 3D-LMVIC. Given set of multi-view image sequences X (x1, 222 23. ,,,) ). 3D-GS is trained to estimate depth map dt, for each image x Both v,, and d,, are compressed, with the coding reference relationships indicated by black solid arrows in the figure. Prior to compressing the image Tn. it is necessary to compress In d,-1, and d,,. The disparity relationship between the (n 1) and n-th views inferred from the reconstructed depth maps d and d Subsequently, based on the estimated disparity relationship, well the extracted features of the (n 1)-th view, æ,, is compressed. When compressing the depth map d,,, the model employs the predicted depth map derived from d as a reference. The same neural network architecture and
==
Rasc with T,
(I)
Here, C and a, represent the color and opacity (density) of the point, respectively, derived from the point's 3D Gaussian properties. The factor T, denotes the transmittance along the ray, indicating the fraction of light reaching the camera without being occluded.
In (1), T, serves a weight for the contribution of each point's color to the pixel's final color, diminishing from to O as increases due to cumulative absorption. To estimate the depth of a pixel d, we adopt a median depth estimation approach. Specifically, the depth is determined as the depth of the first point where T, drops below 0.5:
3

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
1. The projected pixel must reside within the valid image region in the (n 1)-th view.
d Zer, where i" min (il T 0.5).
2. The corresponding 3D world point must lie in the positive
It is worth noting that the original 3D-GS (Kerbl al., 2023) employs a weighted averaging approach, using Tja; as the weight for each 3D Gaussian along the ray to compute depth. In contrast, alignment experiments Section 4.3 demon- strate that the median depth estimation approach achieves better alignment performance.
2 half--space of the (n 1) view's 's coordinate system.
3. No occlusion must exist along the line of sight, i.c., - from (3) must be less than the estimated depth along the ray in the (n 1)-th view.
This can be formulated as:
3.1.2. DISPARITY ESTIMATION
1
if < A [..... + W and
0 < A A H and
Next, we aim to estimate the disparity A, 2 between views based the estimated depth map. This disparity represents the pixel-wise shift of each 3D world point's projection across different views. Disparity estima- tion captures the geometric relationships between views, facilitating the modeling of inter-view correlations.
0 D<d [ Warp(d A 1t,
otherwise,
(4)
where d RwxH represents the tensor containing the depth values d for each pixel, and Warp(.,)) denotes the warping operation based on the given disparity. Ap- pendix A outlines the algorithmic process for disparity and mask estimation.
Figure 2(b) illustrates the depth-based disparity estimation. To estimate the disparity, pixel (Z,s Bn) in the nith view is back-projected into 3D space using the depth dn to obtain the world coordinates (Iw, yw, 2-). This 3D world point is then projected into the (n - 1) view to obtain the corresponding pixel coordinates (In-1s Bn-1). The trans- formations involved are as follows:
3.2. Compression Framework for Images and Depth Maps
3.2.1. IMAGE COMPRESSION MODEL
As shown in Figure 3, the disparity extractor DISE utilizes reconstructed depth maps d and d, to extract multi- scale disparities and feature masks. The reference feature ex- tractor RFB E generates multi-scale reference features from the reconstructed image -n- and its intermediate recon- struction features {f i 1,2.3). Subsequently, the image encoder I E and decoder ID incorporate the refer- ence features, aligned using the extracted disparities, into the backbone network. This process is formalized as:
Fiw
Zn
Vw
V
aug
K
dn
V/
Zw
1
1
(3)
2n-1 1
d'-1
yn
= K deaug
Vn-1
B/
Zw
Ya IF(x,, DISF DISFF(d--,,.))) RF RFF (fa
where K E 33333 denotes the camera intrinsic matrix, and Vn, Vn-1 4x4 are the extrinsic matrices corresponding to the n-th and (n 1)-th views, respectively. The camera parameters are calibrated using SfM (Schonberger & Frahm, 2016). d' represents the depth of the 3D world point in the camera coordinate system of the (n - 1) view. aug denotes the operation of augmenting a vector by adding an additional dimension with a value of as its final ele- ment. Conversely, deaug refers to the operation of reducing vector by removing its last dimension. The resulting dis- parity 5, = (In-1 Tu: Vn-1 yn) for each pixel is then compiled into the disparity map A..
9n Qlyn);
f. = ID(O,, DISE d.),, RF ()))
For entropy coding, utilize the hyperprior entropy model (Ballé al., 2018) and the quadtree partition- based entropy model (QPEM) (Li et al., 2023). The hyperprior entropy model transforms yn into hyperprior representation Z,. The quantized hyperprior representation & is then used to accurately model the probability distribution of Φ,. The conditional probability distribution Po., l,, is defined as:
Finally, we define a mask En.m E R WX to determine whether the disparity estimation is meaningful, i.e., whether the information from the reference pixel is relevant or merely noise. The mask's s criteria are as follows:
Po. 1a. (InlEn) NN o
(6)
Disparity extractor. As illustrated in Figure 3, we firstly employ the disparity estimation (DPE) module to derive the

---

3D-LMVIC: Learning-based Multi- View Image Compression with 3D Gaussian Geometric Priors
Image Encoder IE
h
f
Disparity Extractor DISE
ICT Mardle
IC1 Moddle
IC1 Moddle
Cattr
y,
Zg
x
Coco
Oxnitet and Hyperpior Ebange Meck
DFE Module
A A
Comoat
ConriL
CommoLix
Disparity and Reference Feature Extrador
AE
AE
ECT Module
ICL -000O
IC1 Moxile
LC1 Modale
AD
AD
Con--CDD
CDEN
Corae Coms-CGDN -GDN
x
TGD
Decater
Qure
hh'
IGLON
y,
2,
Image Decoder
Reference Feature Extractor RFE
Figure 3. The architecture of the proposed image compression model. 'LR' represents the Leaky ReLU activation function, 'Q' denotes the quantization operation, and 'AE'" AD' refer to the arithmetic encoder/decoller respectively.
3.2.2. DEPTH MAP COMPRESSION MODEL
Leely Nok
Lably Tor.17
The compression and decompression of the depth map d. leverage d as reference. Initially, d processed by the depth prediction extractor DEPE, which gener- ates multi-scale depth prediction features and corresponding feature masks. Subsequently, the depth encoder DE and decoder DD integrate these extracted features and masks into the backbone network. This process is formalized as:
Waap
Cone
Come
Cone
θ f
A
Conf
Residual Block
Figure 4. Illustration of the proposed image context transfer mod-
yd. DE(dn, (d.).,
dd. Q(Va,), d. DD(D.,, DEPE((d.)
disparity map A and the corresponding mask 2n,m- using d and d,,, following the method outlined in Section 3.1. Subsequently, A, undergoes series of downsampling operations to produce multi-scale disparity maps (A', i= 1,2,3). which will facilitate multi scale feature alignment. The mask Fn.m further processed by the disparity mask 1,2, 4).
The entropy coding scheme incorporates both the hyperprior entropy model and the QPEM. The latent representation ya, is transformed into a hyperprior representation =d,, using the hyperprior entropy model. Similar to the image compression model, the quantized hyperprior representation éd, is used model the probability distribution of 9d. Additional details about the depth map compression model are provided in Appendix B.
extractor extract feature masks {f".... nm
Reference feature extractor. The reference feature ex- tractor takes -n (fr-1 i = 1,,,3), and A in- puts to extract multi-scale reference features (h i= 1, 4), shown in Figure 3.
3.2.3. MULTI- NULTI-VIEW SEQUENCE ORDERING
Image context transfer module. To incorporate the ref- erence feature (h.
Given the significant impact of FoV overlap between adja- cent views on inter-view correlations, we propose multi- view sequence ordering method to alleviate the issue of insufficient overlap unordered sequences. We define distance metrie evaluate inter-view overlap and employ greedy algorithm to find improved sequence.
= 1, 2.33 obtained from the (n 1--11 (1-th view into the image backbone encoder and coder, enhancing feature representation, we introduce the image context transfer (ICT) module. As depicted in Fig- ure the module enhances the input feature f", from the backbone network by leveraging the aligned reference fea- ture h via A By applying feature masks, the module filters relevant information and refines the features, ulti- mately producing the output feature f through a residual enhancement process.
In (3), if Vn-1V,-1 = L, then (Ixx Dn) = (In-14 Vn-1). 1). This indicates that each pixel in the n-th view lies within the valid image area of the (n - 1)-th view, indicating high overlap. Thus, for any two views and j, we measure
5

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
Temps&Temples
Mip-NeRF 360
Deep Bleading
36
55-
18NN
PSNNI
31
MARC
32-
HA5K
10sse
LEMO:
SASE:
LOMIC
54517
30
10000
29
LOME
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
12
0.1
0.2
0.3
0.4
0..
0.6
0.7
byp
hpp
hyp
Tanks&TTeman Temples
Mip-NeRF 360
Deep Blending
0.90
0.99
0.98
MS-581M
0.95
MSI
MS
0.06
0.97
HBBC
J04ENITI
BESIC D033C
SONAMVIC
IESC
BBC
0.95
0.96
M441VV
0.2
0.4
0.5
0.8
0.94
0.2
0.4
1.0
12
0.1
0.3
0.1
0.5
0.6
bpp
hpp
Figure 5. Rate-distortion curves the proposed method compared with baselines.
overlap by the proximity V.V-1 to the identity matrix:
4. Experiments
4.1. Experimental Setup
Dv(í,))= -VAV, Z|I.
(8)
Datasets. We evaluate our model on three multi-view image datasets: Tanks& Temples (Knapitsch et al., 2017), Mip- NeRF 360 (Barron et al., 2022), and Deep Blending (Hed- man et al., 2018). Further details on the datasets are pro- vided in Appendix D.
Appendix proves that j)) is distance metric for both the 2-norm and Frobenius norm. The Frobenius norm is utilized in our experiments. After determining pairwise distances, greedy algorithm is employed, starting from an initial sequence with only one view, iteratively selecting the view closest to the last view in the sequence.
Benchmarks. We compare our approach against several baselines, including traditional multi-view codec: MV- HEVC (Hannuksela et al., 2015); learning-based multi-view image codecs: two variants of HESIC (Deng et al., 202 1), MASIC (Deng et al., 2023), SASIC (Wödlinger et al., 2022), two variants of LDMIC (Zhang et al., 2023), and two vari- ants of BiSIC (Liu et al., 2024); as well as the 3D-GS com- pression method: HAC (Chen et al., 2024). Further details on the baseline configurations are provided in Appendix D.
3.2.4. TRAINING LOSS
For each training step, randomly selected subsequence of length 4 from multi-view sequence serves as the training sample. The training loss comprises the distortion losses for both the reconstructed image and depth map, as well as the estimated compression rates for the encoded image and depth map:
Metrics. Image reconstruction quality is measured using peak signal-to-noise ratio (PSNR) and multi-scale structural similarity index IMS-SSMM (Wang et al., 2003). Bitrate expressed in bits per pixel (bpp). In addition plotting RD curves, the Bjøntegaard Delta bitrate (BDBR) is calculated to quantify the average bitrate savings across varying re- construction qualities. Lower BDBR values indicate better performance.
+43
L
Σ Wn-st1 Aung D(I,, 2n.) + MSE((n. dr)) n=s
+R(0.) + R(2.) + R(Da,) + (R(2((2)))
where -)-))) denotes the distortion, MSE(..)) represents the mean squared error (MSE), and R(-) indicates the es- timated compression rates. The hyperparameters Xing and Adop control the contributions of the image and depth map distortion losses, respectively. The weights {w: = 1, 2, 3, 4} adjust the influence of each view on the overall training loss.
Implementation Details. The model was trained using five different configurations of (Aung. Adep): ((566, (512, 128), (1024, 128), (2048, 128), (4096, 128)) when the image distortion loss is MSE, and ((8, 64), (16, 128), (32, 128), (64, 128), (128, 128)) when using MS-SSIM. The weights for four consecutive
6

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
Table BDBR comparison different methods relative to MV-HEVC.
Tanks& Temples
Mip-NeRF 360
Deep Blending
Methods
PSNR
MS-SSIM
PSNR
MS-SSIM
PSNR
MS-SSIM
HAC
636.81%
350.72%
374.20%
294.42%
673.57%
418.85%
HESIC
12.66%
-26.29%
28.41%
-6.8%%
85.38%
3.91%
HESIC+
-4.85%
-30.42%
9.48%
-5.11%
32.5%
-19.14%
MASIC
-1..5%%
-34.19%
3.26%
-9.11%
43.6%
-9.33%
SASIC
3.39%
-18.59%
2.40%
-3.70%
24.64%
-9.48%
LDMIC-Fast
-8.56%
-27.76%
1.72%
-6.21%
24.25%
-23.31%
LDMIC
-16.27%
-44.33%
-13.12%
-25.39%
16.88%
-41.94%
BiSIC-Fast
26.559%
-42.93%
-20.61%
-23.23%
-8.24%
-4180%
BiSIC
-30.89%
-49.96%
229.7%%
-30.75%
-15.46%
-48.47%
3D-LMVIC
-47.48%
-63.69% %
-34.69%
-00.55%
-27.31%
-54.15%
Table Average alignment quality (PSNR, MS-SSIM) of different alignment methods on the Train scene of the Tanks& Temples dataset.
Methods
Metrics
HT
PM
SPyNet
PWC-Net
FlowFormert+
3D-GS
COLMAP
MVSFormert+
Proposed
PSNR
15.16
17.94
16.12
17.59
18.08
17.36
14.32
15.31
18.14
MS-SSIM
0.3435
0.7633
0.6289
0.7707
0.7863
0.7410
0.7446
0.5544
0.8053
views were set to (0.5, 1.2, 0.5, 0.9) as referenced from Li et al. (2023). The model was trained for 300 epochs with an initial learning rate 10 which was progressively decayed by factor of 0.5 every 60 epochs.
mation (HT) (Deng et al., 2021) and Patch Matching (PM)
(Huang et al., 2023).
2. Optical flow estimation methods, such as SPyNet (Ranjan & Black, 2017). PWC-Net (Sun al., 2018), and Flow- Former++ (Shi et al., 2023).
4.2. Experimental Results
3. Depth map estimation methods, including original 3D- GS (Kerbl et al., 2023), COLMAP (Schonberger & Frahm, 2016; Schönberger et al., 2016) and MVSFormert+ (Chen- jie Cao & Fu, 2024).
Coding performance. Figure presents the distortiin
curves of the compared methods, while Table summarizes the BDBR of each codec relative MV-HEVC. Across the three datasets, the proposed 3D-LMVIC consistently outperforms the baselines in both PSNR and MS-SSIM, demonstrating its effectiveness in reducing inter-view re- dundancy. For instance, on the Tanks& Temples dataset, 3D- LMVIC achieves BDBR reduction of 16.59% for PSNR and 13.73% for MS-SSIM compared to BISIC. The BDBR of HAC is relatively higher, likely due to the inclusion of 3D scene information in addition 2D image representations. Appendix F provides examples of visual comparisons. Ap- pendix G presents an analysis of computational complexity. Appendix H includes supplementary experiments on coding performance.
Aligmment quality was assessed by computing PSNR and MS-SSIM between the aligned reference view images and the target view images. Table 2 summarizes the average alignment quality and runtime for each method. The pro- posed method outperformed the baselines in both PSNR and MS-SSIM, indicating its effectiveness in capturing complex disparities between views. Figure provides visual com- parisons, demonstrating that the proposed method achieves closer alignment with the target view images. Appendix D further investigates the relationship between the mask de- fined in (4) and the ghosting artifacts introduced during alignment.
4.3. Alignment Experiments
4.4. Ablation Study
To evaluate the effectiveness of the proposed 3D Gaus- sian geometric priors-based alignment method, we con- ducted alignment experiments on the Train scene from the Tanks& T Temples dataset. The baselines for comparison in- clude:
Codee components. To assess the contribution of codec components, we performed ablation experiments the Tanks&Temple dataset. The rate-distortion curves are shown in Figure Specifically, we evaluated the following baselines: (1) Sepurate: encoding and decoding without cross-view information; (2) Concatenation: direct feature concatenation from reference view without alignment; (3)
1. Alignment methods commonly used in learning beanedgg-based multi-vien image codecs, such as Homography Transfor-

---

3D-LMVIC: Learning-basee Multi- View Image Compression with 3D Gaussian Geometric Priors
Reference View
HT 12. 12.82/0.6065 6065
PM 10.05/0.4078
SPyNet 10.25/0.199
Target View
PWC-Net
Flow +
Proposed 13.94/0.7217
9.65/0.2840
13. 13.94/0.7216
Figure 6. Visual comparison of different alignment methods on an adjacent view pair in the Train scene of the & Tanks&Temples dataset.
Alignment quality is reported PSNR/MS-SSIM.
Tanks& Temples
Tanks& Temples
0.99
35
0.98
MS SS-S
PSNR
Propised
Pruposed
33
0.97
Seperate
Concatrnation
Concatenatoon
WAD
Mack
W-13 Dop Prod
WKO Prod
Sert
Soot
Random
0.96
Rasdin
0.2
0.1
0.3
0.5
bpp
bpp
Figure 7. Rate-distortion curves of different ablation baselines the Tanks& Temples dataset.
W/O Mask: removal of both image and depth mask: (4) W/O Pred: excluding depth prediction in the depth map com- pression model. These baselines resulted in bitrate increases of 41.07% (44.24%), 42.75% (47.52%) 7.19% (8.47%) and 8.03% (8.02%) for PSNR (MS-SSIM), respectively, compared to the proposed method. The experimental results validate the effectiveness of the proposed components.
5. Conclusion
Pred:
In this paper, we present 3D-LMVIC, a novel learning- based multi-view image coding framework incorporating 3D Gaussian geometric priors. This framework exploits these geometric priors to estimate complex disparities and masks between views for effectively utilizing reference view information in the compression process. Additionally, propose depth map compression model designed to com- pactly and accurately represent the geometry of each view, incorporating a cross-view depth prediction module to cap- ture inter-view geometric correlations. Moreover, we intro- duce multi-vier sequence ordering method for unordered sequences, enhancing the overlap between adjacent views by defining inter-view distance measure guide the sequence ordering. Experimental results confirm that 3D- LMVIC surpasses existing learning-base based coding schemes in compression efficiency while achieving accurate disparity estimation.
Multi-view sequence ordering. As illustrated in Figure 7, we evaluated two baselines to assess the effectiveness of the proposed multi-view sequence ordering method: (1)
Sort: sequences ordered using the proposed method: (2)
Random: sequences are randomly ordered. The Random baseline led to a 42.4% (50.64%) increase in bitrate for PSNR (MS-SSIM) compared to Sort. Furthermore, Sort exhibited only a 3.76% (2.97%) bitrate increase for PSNR (MS-SSIM) compared the manually sorted sequences in the Lanks&Temples dataset. These results demonstrate the effectiveness of the proposed ordering method for unsorted multi-view sequences, achieving performance close to that of manual sorting.
8

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
Acknowledgments
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P. P., and Hedman, P. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. CVPR, 2022.
This work supported in part by the National Natural Sci- ence Foundation of China under grant 62171248, 62301 189, the project of Peng Cheng Laboratory (PCL2023A08), Guangdong Provincial Key Laboratory of Novel Secu- rity Intelligence Technologies (2022B121 121 and Shenzhen Science and Technology Program under Grant KIZI20024490330057720000,, GXWD202208111295600
Bellard, F. Bpg image format. https://bellass org /ttp//p//////////// 2tpg//20100 2014.
Bross, B., Wang, Y.-.., Ye, Y., Liu, S., Chen, J., Sullivan, G. J., and Ohm, J.-R. Overview of the versatile video coding (vvc) standard and its applications. IEEE Trans- actions on Circuits and Systems for Video Technology. 31 (10):336-3764, 2021.
(100000228,
Impact Statement
Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp. 1907-1915, 2017.
This work introduces 3D-LMVIC, novel learning- based framework for multi-view image compression that lever- ages 3D Gaussian geometric priors to enable more accurate disparity estimation and efficient inter-view redundancy re- duction. The proposed method demonstrates significant improvements over both traditional and learning-bass base- lines in compression efficiency and alignment quality across diverse 3D scene datasets. This advancement is especially valuable for applications requiring scalable and high-quality multi-view data processing, such as immersive virtual and augmented reality, autonomous driving, and 3D) reconstruc- tion. By incorporating geometric priors into the learning pipeline, this work contributes bridging the gap between geometric scene understanding and data-driven compression. We believe method offers promising direction for fur- ther research in geometry-awan compression, though care should be taken to evaluate generalization to in-the-wild multi-view scenes with dynamic content.
Chen, Y., Wu, Q., Cai, J., Harandi, M., and Lin, W. Hac: Hash-gris assisted context for 3d gaussian splatting com- pression. In European Conference on Computer Vision, 2024.
Chenjic Cao, X. R. and Fu, Y. Mysformer++: Revealing the devil in transformer's 's details for multi-view stereo. In International Conference on Learning Representations (ICLR), 2024.
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M. Scannet: Richly-annotated 3d recon- structions of indoor scenes. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5828-5839, 2017.
References
Deng, X., Yang, W., Yang, R., Xu, M., Liu, E.,, Feng, Q., and Timofte, R. Deep homography for efficient stereo image compression. In 2021 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pp. 1492- 1501, 2021. doi: 10. 10.1009/CVPR44443722222 20211300722211011.44
Anthes, C., Gareía-Hernández R. J., Wiedemann, M., and Kranzlmüller, D. State of the art of virtual reality tech- nology. In 2016 IEEE aerospace conference, pp. 1-19. IEEE, 2016.
Deng, X., Deng, Y., Yang, R., Yang, W., Timofte, R., and Xu, M. Masic: Deep mask stereo image compression. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
Ayzik, S. and Avidan, S. Deep image compression using decoder side information. In European Conference on Computer Vision, pp. 699-714, 2020.
Ballé, J., Laparra, V., and Simoncelli, E. P. Density model- ing of images using generalized normalization transfor- mation. In International Conference on Learning Repre- sentations, 2016.
Hamdi, A., Melas- Kyriazi, L., Mai, J., Qian, G., Liu, R., Vondrick, C., Ghanem, B., and Vedaldi, A. Ges: Gen- eralized exponential splatting for efficient radiance field rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19812-19822, June 2024.
Ballé, J., Laparra, V., and Simoncelli, E. P. End-to-end opti- mized image compression. In International Conference Learning Representations, 2017.
Hannuksela, M. M., Yan, Y., Huang, X., and Li, H. Overview of the multiview high efficiency video coding (mv-heve) standard. In 2015 IEFE International Confer- ence on Image Processing (ICIP), pp. 2154-2158, 2015. doi: 10.1 10.1100//11220055735111
Ballé, J., Minnen, D., Singh, S., Hwang, S. J.. and Johnston, N. Variational image compression with a scale hyperprior. In International Conference on Learning Representations, 2018.

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
He, D., Zheng, Y., Sun, B., Wang. Y., and Qin, H. Checker- board context model for efficient learned image compres- sion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14771- 14780, 2021.
Liu, X., Chen, B., Liu, Z., Wang, Y., and Xia, S.-T. An exploration with entropy constrained 3d gaussians for 2d video compression. In The Thirteenth International Conference on Learning Representations.
Liu, Z., Zhang, X., Shao, J., Lin, Z., and Zhang. J. Bidirec- tional stereo image compression with cross-dimensional entropy model. In European Conference Computer Vision, 2024.
Hedman, P., Philip. J., Price, T., Frahm, J.-M., Dret- takis, G., and Brostow, G. Deep blending for free- viewpoint image-basee rendering. ACM Trans. Graph., 37(6), dec 2018. ISSN 0730-0301. doi: 10.1145/ 3272127.3275084. URL attps://dci..rre 1145/3272121 011553322122733250682
Minnen, D., Ballé, J., and Toderici, G. Joint autoregressive and hierarchical priors for learned image compression. In Proceedings of the 32nd International Conference on Neu- ral Information P Processing Systems, pp. 10794-10803, 2018.
Hosseinian, S. and Arefi, H. 3d reconstruction from multi- view medical x-ray images- and evaluation of ex- isting methods. The international archives of the pho- togrammetry, remote sensing and spatial information sci- ences, 19-66, 2015.
Mokssit, S., Licea, D. B., Guermah, B., and Ghogho, M. Deep learning techniques for visual slam: A survey. IEEE Access, 11:20026-200500, 2023.
Huang, Y., Chen, B., Zhang, J., Han, Q. and Xia, S.-T. Compressive sensing based asymmetric semantic image compression for resource-constrained iot system. In Pro- ceedings of the 59th ACM/IEEE Design Automation Con- ference, pp. 877-882, 2022.
Qin, S., Chen, B., Huang, Y., An, B., Dai, T., and Xia, S.-T. Perceptual image compression with cooperative cross- modal side information. arXiv e-prints, pp. arXiv-2311 2023.
Huang, Y., Chen, B., Qin, S., Li, J., Wang, Y., Dai, T., and Xia, S.-T. Learned distributed image compression with multi-scale patch matching feature domain. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 37, pp. 4322 4329, 2023.
Qin, S., Wang. J., Zhou, Y., Chen, B., Luo, T., An, B., Dai, T., Xia, S., and Wang, Y. Mambave: Learned visual compression with selective state spaces. arXiv preprint 2rXiv:2405.1544 2024a.
Qin, S., Zhou, Y.-M., Wang, J.-P., Chen, B., An, B.-Y., Dai, T., and Xia, S.-T. Progressive learning with visual prompt tuning for variable-rate image compression. In 2024 IEEE International Conference on Image Processing (ICIP), pp. 1767-1773. IEEE, 2024b.
Jiang, W., Yang. J., Zhai, Y., Ning, P., Gao, F., and Wang, R. Mlie: Multi-reference entropy model for learned image compression. In Proceedings of the 31 ACM Interna- tional Conference on Multimedia, pp. 7618-7622 2023.
Kerbl, B., Kopanas, G., Leimkuehler, T., and Drettakis, G. 3d gaussian splatting for real-time radiance field render- ing. ACM Transactions on Graphics (TOG), 42(4):1-14, 2023.
Ranjan, A. and Black, M. J. Optical flow estimation using spatial pyramid network. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
Knapitsch, A., Park, J., Zhou, Q.-Y., and Koltun, V. Tanks and temples: Benchmarking large-scale scene reconstruc- tion. ACM Transactions on Graphics, 36(4), 2017.
Schmalstieg, and Hollerer, Augmented reality: princi- ples and practice. Addison- Addison-Wesley Professional, 2016.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML. 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Schonberger, J. L. and Frahm, J.-M. Structure-from-nnee revisited. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pp. 2016.
Lei, J., Liu, X., Peng, B., Jin, D., Li, W., and Gu, J. Deep stereo image compression via bi-directional coding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 19678, 2022.
Schönberger, J. L., Zheng, E., Pollefeys, M., and Frahm, J.-M. Pixelwise view selection for unstructured multi- view stereo. In European Conference on Computer Vision (ECCV), 2016.
Li, J., Li, B., and Lu, Y. Neural video compression with diverse contexts. In 2023 JEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pp. 22616 22626, 2023. doi: 10.1 109/CVPR52729.2022
Shi, X., Huang, Z. Li. Zhang, M., Cheung, K. C., See, S., Qin, H., Dai, J.,, and Li, H. Flowformer++: Masked cost volume autoencoding for pretraining optical flow estimation. In Proceedings of the IEEE/CVF Conference
10

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
on Computer Vision and Pultern Recognition (CVPR), pp. 1599-1610, June 2023.
Sun, D., Yang, X., Liu, M.-Y., and Kautz, J. Pwe-net: Cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
Vetro, A., Wiegand, T., and Sullivan, G. J. Overview of the stereo and multiview video coding extensions of the h. 264/mpeg-4 ave standard. Proceedings of the IEEE, 99 (4):626-6422 2011.
Wallace, G. K. The jpeg still picture compression standard. IEEE transactions consumer electronics, 38(1)))))))) xxvvii- xxxiv, 1992.
Wang, Z., Simoncelli, E. P., and Bovik, A. C. Multiscale structural similarity for image quality assessment. In The Shrity-Seventh Asilomar Conference on Signals, Sys- tems & Computers, 2003, volume 2, pp. 1398-1402, Ieee, 2003.
Wödlinger, M., Kotera, J., Xu, J., and Sablatnig, R. Sasic: Stereo image compression with latent shifts and stereo attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 661-670, 2022.
Xia, Y., Huang, Y., Chen, B., Wang, G.. Wang, H., and Wang, Y. Fea-net: Accelerating stereo image compres- sion through cascade aligmment side information. Puttern Recognition, 168: 168:111799, 2025. ISSN 0031- 3203. doi: https//doi.aag///o.ooooipgeeegg2 URL
http:://www.ecieeeeedrreeec... /cienee/arrtca///////00112223331
Xu, L., Su, Z., Han, L., Yu, Liu, Y., and Fang. L. Unstruc- turedfusion: Realtime 4d geometry and texture recon- struction using commercial rgbd cameras. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 42(10):2508 2522, 2020. doi: 10.1109/TPAM.2019, 2915229.
Yan, C., Qu, D., Xu, D., Zhao, B., Wang, Z., Wang. D., and Li, X. Gs-slam: Dense visual slam with 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 19595-19604, June 2024.
Zhai, Y., Tang, L., Ma, Y., Peng, R., and Wang, R. Disparity- based stereo image compression with aligned cross-view priors. Proceedings of the 30th ACM International Conference on Multimedia, pp. 2351 2360, 2022.
Zhang. X., Shao, J., and Zhang, J. Ldmic: bease based distributed multi-view view image coding. In International Conference on Learning Representations 2023.

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
A. Disparity and Mask Estimation Algorithm
Algorithm Disparity and Mask Estimation
Depth estimation function GSDE, intrinsic matrix K, extrinsic matrices V, and - Output: Disparity map Ans n. mask nn.m
Input:
d, f- (SDE(K, Vi))
d - GSDE(K, Vin-))
A, d n-1 &- EisparityEstimattinn (n,, K, Vn, Vn-1)
Fn.m f MaskEstimation( A, 1 d1
Depth Encoder DE
gep g
DXC1 Modil
Coay Cony- CDDN
Cone GDDV
DXC1 Modile
DVDP Module
y2
Z
GDNNCOor
GDIN
d
Cooo-
Coatast and Hay.eeyaaaaoooo pattor Estmpp Model
Oucer
Fixtorbod Frttogy Mork
Depth Prediction Feature Extractor
Degtt Prediction Extractor
AF
AF
Depth Mask Extractor
DCI Modile
Dooner HIDN
DC Medale
HCDN
DKC sapeaa
AD
Cour
IRACW
S
d
Donre
0
4g
2
Depth Decoder DD
Depth Prediction Extractor
Figure 8. The architecture of the proposed depth map compression model. 'LR' represents the Leaky ReLU activation function, 'Q' denotes the quantization operation, and 'E'''A'' refer to the arithmetic encoder/decoder, respectively.
B. Supplementary Information for the Depth Map Compression Model
As illustrated in Figure 8, during the compression and decompression of d,, d is initially processed by the depth prediction extractor, which extracts multi-scale depth prediction features and associated feature masks. These extracted features and masks are then integrated into the depth backbone encoder and decoder via the depth context integration (DCI) module. Detailed explanations of the depth prediction extractor and the DCI module are provided in the subsequent content.
Depth prediction extractor. As illustrated Figure 8, we first utilize the proposed cross-viis view depth prediction (CVDP) module to predict the depth map dr.,p R. H and the associated mask dh..m R x for the 7-th view, based on d Specifically, for each pixel (In-1, 1n in the (n 1 1)-th view and its corresponding reconstructed depth d the CVDP module determines corresponding pixel coordinates (I,, yn) and the depth prediction d', in the n-th view using the method described in (3). The depth at the nearest grid point ([æ,, 0.57, Lyx 0.5]) then d'.
dn In 0.5], Lyn 0.5]] d',
(10)
This cross-view depth prediction is applied to each pixel in the (n 1)-th view construct dn.p. If multiple pixel coordinates map to the same grid point, the depth prediction for that point is set to the minimum of these predicted depths. Additionally, if a grid point has no corresponding pixel coordinates, its depth prediction value is set to 0. The mask d.,,m indicates whether each grid point has at least one corresponding pixel coordinate, with values set to 1 where a correspondence exists and otherwise.
Subsequently, dn.p is fed into the depth prediction feature extractor to produce multi-scale depth prediction features, denoted as (g',p 2, 3, Concurrently, the mask d is processed by the depth mask extractor to derive the associated multi-scale feature masks (gi.m i= 1,2,3,4 4).
12

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
Depth Context Integration Module. Each DCI module integrates the input features O from the backbone network with In. through channel-wise concatenation, followed by element-wise multiplication with In.m produce the output feature g.
gn - + Dn.p) gm.
(1I)
where @ denotes Channel-wise concatenation and @ denotes dlement-wise multiplication.
C. Proof of ji) as a Distance Measure for 2-Norm and Frobenius Norm
C.1. Proof for 2-Norm
C.I.1. DEFINITION
Definition C.1. For B) and D), where A,C
C' E gaxm and B, E Raxi, we define (u, 0)2 ||ACE + BD2 For any scalar oru = (a.A, aB). Additionally, + o= (A+ C, D).
C.1.2. LEMMA
Lemma C.2. For any u = B) and v = (C, D) as defined in Definition C.I, the following inequality holds:
(u, f)2 I (u,,u)ul u)2(U,
Proof. For any real number 1, we have:
(u + tu, + Lu)2 = [(A + 10) (14440000041000
$ ||AAT + BBB +| 1 +
+ +DD"12
u)2 + t(u, u)2 + ( 1)2
u), 2)
The right-hand side of the last equation can be viewed as a quadratic expression in t and greater than or equal to (u + bu, u + 2v)2, which is non-negative. Therefore, the discriminant of this quadratic must non-positive:
(2(u, u))² 4(u, u)2(2, u)2 0
Thus, we obtain:
(u,r)2 1 (u, (u,u))((, u)2(u,
C.1.3. THEOREM
Theorem C.3. Dv(i.)) = [IV.V=1 1- 1|] is a distance metric.
Proof. We need to prove that (i j) satisfies non-negativity, symmetry, and the triangle inequality.
Non-negativity:
Since (i is norm, it is non-negative. Additionally, the extrinsic matrices for different views distinct, V, * Vú for i + j) = ( if and only if V.V,-1 = 0, which holds only when V; V,, i.e., i = j.
R4 4 where R3 E IR3x3 is rotation matrix and
Symmetry: The extrinsic matrix V, can be represented V;
0 1)
33x1 is translation vector. We have:
Dy(i.j) IVVV -112
'A rotation matrix an orthogonal matrix, meaning its inverse is equal to its transpose.
13

---

3D-LMVIC: Learning- Multi- Image Compression with 3D Gaussian Geometric Priors
Ra
4
12
-
1
0
1
R 0
-1
R, +1)
+1,111
0
(RaR) -I -R, +1, T
[ -
R2 +ti
0
0
= 2| R - RR + R - -
t, tf + +
= || R. (21 - -R, RR + +
+40 +40
-1121 R - RR,R - -td -ff R.RR
iDv(.
The fourth equality follows from the fact that for any matrix A, ||A| IAAT The sixth equality is due to the orthogonality of R3 and Ry. and the invariance of the 2-norm under orthogonal transformations. The final equality holds because interchanging the indices and j in the expression the right-hand side of the fifth equality leads to the same expression as Dy(j.t) i), which matches the right-hand side of the seventh equality.
Triangle inequality: For views i, j. and h; define Aij RT. R. B., = - RT + R. 4, and similarly for A5.k, B5,k, Ac,i: BR.i- Let Uz.k = (A Br,x) and un,i = (Ak.s, Bx,)). Starting from the fourth equation in the sym- metry proof, we proceed as follows:
R - - RR +t T
R - -R,
Dv(1,5)= =
0
0
0
11" ( ||
(R,RT - R) -17
111 +
|[RR - R -R) + + +,)²² R:112
| - -1 ((([[[_I(=(((RR))))))))---
|Ar B. 11
[(Aj + AA + A + (Bau + (B,.. + BB,
|A + BT + A + BR, BE. + Aj, + Bj BF A + Bi
+ I (1IAAAAT, KM + +ByrBJ + |A, + BR B 2/ Ay + By, By, 11))
= (DvliN)) + +Dukk + +2(uj,k ().))))
s (DvLi,))= - Dv(k, 1)2 + +2 -Du,)),))
(us,k Ug (2,))((un,, un.)2
(Dvli,A) + ") k)Dx(k
= Dv(i,k) + i)
The second inequality follows from Lemma C.2.
C.2. Proof for Frobenius Norm
C.2.1. DEFINITION
Definition C.4. For u (A, B) and (C, D) as defined in Definition C. we define (u, tr (ACT + BDT).
14

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
C.2.2. LEMMA
Lemma C.5. For u and as defined in Definition C.4, the following inequality holds:
(u, v) S (u, (),
Proof. The method of proof is analogous to that used in Lemma C.2. By leveraging the properties of the trace and following a similar reasoning process, the result is derived.
C.2.3. THEOREM
Theorem C.6. Pv( IVVVV" I[ is distance metric.
Proof.
We need to prove that j)) satisfies non-negativity, symmetry, and the triangle inequality.
Non-negativity:
The proof follows a similar approach to that of Theorem C.3, so we omit the details here.
Symmetry:
D)(i, = |VVV-1 T|E
(
R4 4,
R
- R
)
-1
-1/
0
0
1
F
R
-I -R, +t,) +
0
F
tr (R:Bg - I - R ty +1i
(R.A) -I RR
(R.R.
0
0
tr (2T - R - R + -
(tr(2T)
- (1)
= ())
The fourth equality holds because, for any matrix A, we have |||r tr(AAT)) The sixth equality result of the linearity of the trace operator. The seventh equality follows from the cyclic property of the trace, for instance, tr( (R,RR R,),)) = RR tr)
=
Triangle Inequality: For views i, j. and k. we follow the same definitions of Ai.j. Bij. Aj.,, BiA AKi. BR Uj.k, and u as in the proof of the triangle inequality in Theorem C.3. Starting from the fourth equality in the proof of symmetry, we have:
0 RR.RT -I -RR 7
Dv(i,j)= =t tr ( [R.RT- -I
tR
+1)
0
0
tr - --
+
tr (R. R| - (--
R.)
-tr tr(( - (R) - RR)"" + (-R5t + R tx) (1
= =tr + Bi. B }
= =tr((A;A + Ak.)(Ag. + ACT.T + (By, + BR.)) + 7)
15

---

3D-LMVIC: Learning-based Multi- Image Compression with 3D Gaussian Geometric Priors
(A + BaaB A + BR + A By + As. + BR Br, = =(v(A,,A + Bz, ti (A + Br, + 2tr (Aj,AE, + B,,
(D) + D)(k, 2(UjR: uc))))
(DvCi + +Du(k + +2
Wigh (1) uR))
(Dvli.)) DD(,, 2Dy(,k)Dy(kk1)) +
The third equality holds because the trace is invariant under similarity transformations.
D. Experimental Details
Datasets. Our evaluation is conducted on three multi-view image datasets: Tanks&Temples. Mip-NeRF 360, and Deep Blending. Tanks& Temples consists of 21 diverse indoor and outdoor scenes, ranging from sculptures and large vehicles to complex large-scale environments, with intricate geometry and varied lighting conditions. Mip-NeRF 360 includes scenes outdoor and indoor indoor-capplnn captured in unbounded settings, allowing for 360-degree camera rotations and capturing content varying distances. From the Deep Blending dataset, we selected representative scenes that span indoor, outdoor, vegetation-rich, and nighttime environments. For all datasets, 90% of the images in each scene were allocated for training, with the remaining 10% used for testing.
Benchmarks. We assess the coding performance of MV-HEVC using the HTM-166.3 software?. The learning- based multi-view image codecs used as baselines, along with our proposed method, are trained under the same conditions on a shared training set and evaluated on a common test set. For the 3D Gaussian Splatting compression method (HAC), we train the 3D Gaussian representations on each scene's test data and measure the reconstruction quality of the rendered images. The bpp is determined by dividing the size of the compressed 3D Gaussian file by the total number of pixels in the test images.
Implementation Details. We utilize the Adam optimizer for training with batch size of 2. To facilitate data augmentation and optimize memory usage, each image is randomly cropped to 256 x 256. Correspondingly, the principal point in the intrinsic matrix is adjusted to reflect the new crop. The intrinsic matrix is given by:
fz
0
Cr
K =
0
fg
Cy 1
0
where far and fy represent the focal lengths along the and y axes, respectively, and eu and co are the principal point coordinates. If the top-left corner of the crop is located at (y)) in the original image, the updated intrinsic matrix K" becomes:
Jx
0 C2 Ps.
K'
fy
Cy Py
0
Ablation study details. To implement Separate, we set the reference view images, predicted depth maps, and masks to full-zero tensors, with Adep set to zero. In Concatenation, alignment operations in the ICT modules are removed. For W/O Mask, we eliminate all relaere multiplications in the ICT and DCI modules. In W/O Dep. Pred, the predicted depth maps are replaced with full-zero tensors. For both Sort and Random, sequences in the training and test sets are reordered accordingly.
16

---

3D-LMVIC: Learning-basee Multi-View Image Compression with 3D Gaussian Geometric Priors
Reference View
Target View
Proposed
Mask
Figure 9. Visual examples of proposed alignment method and the mask from (4).
Table Complexity of learning-based image codecs evaluated on images resolution as 978x546 the & Temples dataset.
Codecs
MACs Enc.
MACs Dec.
Params Enc.
Params Dec.
Time Enc.
Time Dec.
Memory
HESIC+
48.16G
134.31G
17.18M
15.1M
4.35s
10.73s
2248M
MASIC
65.62G
511.34G
32.03M
30.73M
4.38s
10.78s
5202M
SASIC
91.80G
438.09G
3.57M
4.44M
0.06s
0.09s
4498M
LDMIC-Fast
37,49G
94,43G
7.73M
11.15M
0.11s
0.09s
1168M
LDMIC
30.91G
87.84G
7.73M
11.15M
4.24s
10.63s
1096M
BiSIC-Fast
1880G DEec.+Dec.)
85.9M (Enc.+Dec.)
3552M
BiSIC
1770G (Dec.+Dee.)
78.21M (Enc.+Dec.)
3006M
3D-LMVIC
479.43G
436.16G
41.92M
36.87M
0.19s
0.18s
3164M
E. Supplementary Alignment Experiments
Figure 9 shows visual examples of proposed alignment method along with the corresponding masks. Notably, ghosting artifacts due to occlusion, such as those involving the iron bars and the edge of the train shell, are effectively identified by the mask, aiding the codec in filtering out irrelevant information when merging features from the reference view.
F. Visualization
In Figure 10, present examples from the Tanks& Temples and 3D-LMVIC. The results demonstrate that 3D-LMVIC preserves more texture details and achieves higher reconstruction quality for elements like branches, humans, and text, while consuming fewer bits.
Temks&Temples dataset visually compare the performance of LDMIC, BiSIC,
G. Complexity Analysis
Table 3 summarizes the Multiply-Acoumin Operations (MACs), model parameters, coding speed, and memory usage of eight learning- based image codecs. These evaluations were conducted on a platform with an Intel(R) Xeon(R) Gold 6330 CPU @ 2.00GHz and GPU containing 10,752 parallel processing cores. The neural network components were executed on the GPU, while entropy coding was performed on the CPU.
Due to the absence of separate encoder and decoder implementations in the open-source code of BiSIC, we measured only its overall computational complexity. The proposed 3D-LMVIC demonstrates computational complexity within an acceptable range, comparable to the SOTA BiSIC and slightly better than BiSIC-Fast. Specifically, 3D-LMVIC achieved encoding and decoding times of 19s and 18s, respectively, ranking among the faster methods.
17

---

3D-LMVIC: Learning-based Multi-View Image Compression with 3D Gaussian Geometric Priors
0.9482/36.88M) 88700.908
0.7685/36. 8/0.98887
0.6050/37.73/00999
1.1425/36.06/099917
0.8934/35.3400.9913
0.6545/37.440.9924
TERN
L.1475/34.550099936
0.9431/33.5200.9977
0.7596/36.72/0.9946
0.8110/37.8 08110/37..50009955 LDMIC
0.6574/36.75/0.09 BISIC
0.6063/38.
1.666/33.89/0.000 89/0.9950
3D-LMVIC Figure 10. Visual Comparison of LDMIC, BISIC, and 3D-LMVIC on the Tanks& Temples Dataset. Compression performance is reported bpD/PSNR/MS-SSIM
Ground truth
Table BDBR of 3D-LMVIC relative HEVC.
Lanks&Temples
Mip-NeRF 360
Deep Blending
Methods
PSNR
MS-SSIM
PSNR
MS-SSIM
PSNR
MS-SSIM
3D-LMVIC
-20.69%
-40.75%
-14.48%
-22.06%
-17.29%
-43.06%
While the MACs of the 3D-LMVIC encoder are relatively high, they remain lower than those of BiSIC, which employs symmetric encoder-decoder structure. For BiSIC, estimate that the MACs for its encoder and decoder each account for approximately half of the total MACs. Additionally, the inclusion of depth map codec in 3D-LMVIC contributes to the higher MACs and model parameter count.
H. Supplementary Coding Performance
We present a supplementary comparison of the coding performance between the proposed 3D-LMVIC and the HEVC video coding standard. The multi-view sequences treated single video and compressed using HEVC with the lowdelay JP configuration and YUV444 input format. HEVC's coding efficiency is evaluated using the HM-18.0 software". Table 4 reports the BDBR of 3D-LMVIC relative to HEVC. On the three datasets, 3D-LMVIC consistently surpasses HEVC in both PSNR and MS-SSIM, demonstrating its effectiveness in reducing inter-view redundancy in multi-view sequences.
18


Reviews:
[{"review_id": "igwh20tDdl", "reviewer": "Reviewer_w8oP", "summary": "This paper targets on the multi-view image compression task. The main contribution includes a Gaussian Splatting-based disparity estimator for wide-baseline images, a depth map compression model to minimize geometric redundancy, and a multi-view sequence ordering strategy to enhance correlations between adjacent views. Comprehensive experimental results on three datasets demonstrate the superior performance of proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments! We are grateful for your positive feedback, especially regarding the quality of writing, theoretical justifications, and experimental results. We address your remaining concerns as follows:\n\n### R1[Use of median over weighted average for depth estimation]\n---\nSince 3D Gaussians are trained under the supervision of real RGB images, some \"noisy\" Gaussians may emerge that do not affect rendering quality but can bias weighted average depth estimation [1]. In contrast, **median depth estimation inherently provides a denoising effect**: instead of aggregating depth over all Gaussians along a ray, we select the depth of the Gaussian most likely to represent the true 3D world point. This allows us to ignore many background Gaussians that are occluded by foreground Gaussians and invisible in the rendered image, yet would otherwise negatively influence weighted depth estimation.\n\n[1] Chung, Jaeyoung, Jeongtaek Oh, and Kyoung Mu Lee. \"Depth-regularized optimization for 3d gaussian splatting in few-shot images.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n### R2[Runtime of the 3D Gaussian optimization]\n---\nFor the *Train* scene of the Tanks&Temples dataset, which contains 301 images, we conducted tests on a platform equipped with an Intel(R) Xeon(R) Gold 6330 CPU @ 2.00GHz and an NVIDIA RTX A6000 GPU. Training the 3D Gaussian representation for 30,000 iterations took approximately 13 minutes and 35 seconds, or an average of 2.71 seconds per image. **We also tested a reduced training setup with 7,000 iterations, which resulted in a total training time of 2 minutes and 22 seconds, or 0.47 seconds per image.**\n\nAdditionally, we assessed the alignment performance of depth estimation using the 3D Gaussian representation trained for 7,000 iterations under the same experimental conditions as detailed in Table 2 of the main text. The results are summarized in the table below:\n\n| Metrics | FlowFormer++ | MVSFormer++ | Proposed (7000 iterations) | Proposed (30000 iterations) |\n|:-------:|:-------:|:-------:|:-------:|:-------:|\n| PSNR↑ | 18.08 | 15.31 | 17.99 | 18.14 |\n| MS-SSIM↑ | 0.7863 | 0.5544 | 0.7918 | 0.8053 |\n\n**Despite the significant reduction in training iterations, it is evident that the proposed alignment method still achieves high alignment accuracy.**", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "I have checked the proofs in Section C of the supplementary metarial.", "experimental_designs_or_analyses": "I have checked the experimental designs in the main text.", "relation_to_broader_scientific_literature": "This paper is based on the common framework of multi-view image compression problem, but introduces innovation on disparity estimation, depth compression and correlation enhancement.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "- The paper is well written and structured.\n- The essential theories are well proven.\n- The proposed method significantly outperforms existing works.", "other_comments_or_suggestions": "- L156 indicates that the median depth estimation is adopted instead of the original weighted average depth estimation. What's the intuition behind this?", "questions_for_authors": "- Table 3 in the supplementary material reports the compression time. But the runtime of the 3D Gaussian optimization seems not to be included. Can you provide some details about this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XYu8juNOsG", "reviewer": "Reviewer_xX4X", "summary": "The paper proposes a learning-based multi-view image compression framework, 3D-LMVIC, which utilizes the 3D Gaussian geometric prior for disparity estimation. Through experiments, its advantages in compression efficiency and disparity estimation accuracy have been verified.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments! We are grateful for your positive feedback, especially regarding the algorithm's running speed and compression performance. We address your remaining concerns as follows:\n\n### R1[Test on Cityspace, KITTI, and Instereo2K]\n---\n**1. Limitations of Stereo Images for Learning Geometry in 3D Gaussian Representation**  \nWe found that for stereo image datasets with only two viewpoints, the 3D Gaussian representation struggles to learn accurate geometric information. **In contrast, multi-view data under wide-baseline setups provides rich scene information sampled from widely varying viewpoints, which helps the 3D Gaussian representation learn accurate geometry.** In stereo image datasets, the cameras are often positioned close to each other with similar orientations, making it difficult for the 3D Gaussian representation to correct geometric errors through large viewpoint variations and learn the correct geometric structure. The table below presents the alignment performance of various disparity estimation methods on the Cityscapes and KITTI Stereo datasets under the same experimental settings as the alignment experiments described in Table 2 of the main text.\n| Datasets | Metrics | HT | PM | SPyNet| PWC-Net | FlowFormer++ | Proposed |\n|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|\n| Cityscapes | PSNR↑ | 24.43 | 27.40 | 28.63 | 29.16 | 27.26 | 14.64 |\n|  | MS-SSIM↑ | 0.7906 | 0.9546 | 0.9598 | 0.9616 | 0.8864 | 0.3934 |\n| KITTI Stereo | PSNR↑ | 14.05 | 18.33 | 18.11 | 18.92 | - | 7.77 |\n|  | MS-SSIM↑ | 0.5855 | 0.8691 | 0.8768 | 0.8952 | - | 0.1512 |\n\nExperiments have shown that the 3D Gaussian representation cannot learn accurate geometry based on stereo image data with only two viewpoints. The disparity estimation modules in current multi-view image codecs, such as Patch Matching (PM), already perform well on stereo image datasets.\n\n**2. Motivation and Practical Applications**  \nWe aim to design a multi-view image compression framework tailored for wide-baseline setups to achieve accurate disparity estimation and effectively eliminate inter-view redundancy. Unlike KITTI and InStereo2K datasets, which contain stereo images with small, mostly horizontal disparities captured by closely positioned cameras, wide-baseline setups in datasets like Mip-NeRF 360 and TnT feature irregular view relationships and less consistent disparities. These characteristics make existing disparity estimation methods, such as homography transformation and patch matching, less effective, as shown in Table 2 on the TnT dataset. **Wide-baseline setups are also critical for practical applications, where scenes often consist of dozens to hundreds of images, creating significant challenges for storage and transmission.**\n\n### R2[Questions regarding HAC]\n---\n**1. Reason for including HAC in the comparison**:  \nAlthough HAC is a 3DGS-based compression method, it inherently encodes multi-view information. Therefore, we believe it is meaningful to compare its performance with dedicated multi-view image compression methods under the same multi-view compression setting.\n\n**2. Explanation for HAC’s poor performance**:  \nFor evaluation, we use one-third of the images from each scene as test samples. For example, in the *Train* scene of the Tanks&Temples dataset (301 images), we used 101 images for testing.  \nIncreasing the number of test images is likely to benefit HAC's performance. We will consider this in future work.\n\nThe table below presents a comparison between HAC and our proposed 3D-LMVIC method on the Tanks&Temples dataset:\n\n| Methods   | bpp    | PSNR  | MS-SSIM |\n|:---------:|:------:|:-----:|:-------:|\n| HAC       | 1.7915 | 30.32 | 0.9697  |\n|           | 1.3472 | 30.00 | 0.9674  |\n|           | 1.1316 | 29.68 | 0.9651  |\n| 3D-LMVIC  | 0.6056 | 38.43 | 0.9926  |\n|           | 0.2901 | 34.95 | 0.9840  |\n|           | 0.1242 | 31.33 | 0.9648  |\n\nAs we can see, HAC's PSNR and MS-SSIM in terms of rendering quality reach a plateau around 30dB and 0.97. Therefore, based on the experimental results, it seems that HAC might perform better with more test images, such as when the number of test images reaches 1,000. On the other hand, HAC does have a rendering quality bottleneck. \n\n**3. Comparison between HAC and dual-branch VAE architectures (e.g., HESIC and BiSIC)**:  \nHAC compresses the entire 3D scene, not just multi-view images. While this enables novel view synthesis and richer scene understanding, it also leads to larger model sizes.  \nIn contrast, methods like HESIC and BiSIC (dual-branch VAE-based) focus more on efficient representation and compression for the given views.", "claims_and_evidence": "Please see Other Strengths And Weaknesses.", "methods_and_evaluation_criteria": "Please see Other Strengths And Weaknesses.", "theoretical_claims": "Please see Other Strengths And Weaknesses.", "experimental_designs_or_analyses": "Please see Other Strengths And Weaknesses.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strength\n\n(1)The Gaussian Splatting method is used to replace the traditional depth estimation method, which significantly improves the performance of the model.\n\n(2)The encoding and decoding speed of this method is extremely fast, which facilitates the practical deployment of the model.\n\n(3)This method achieves advanced performance compared with the existing multi-view methods.\n\n\nWeakness\n\n(1)3D-LMVIC is a framework for multi-view image compression, and the dataset selected by the author is the one used for evaluating the 3DGS model. Methods such as LDMIC and HESIC are mainly evaluated based on datasets like Cityspace, KITTI, and Instereo2K. Although these datasets have a limited number of views, which restricts the performance of 3DGS depth estimation, I still recommend that the author conduct evaluations on these datasets to verify the generalization ability of the method.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "I'm curious as to why the author included HAC in the comparison. As far as I know, HAC is a pure 3DGS compression framework. What is the reason for the extremely poor performance of HAC in Table 1? Is it because the model has a large size, and the number of evaluated images during rendering is small, which restricts the model's performance? Or is it that HAC itself has poor rendering capabilities? Are there any technical bottlenecks currently in directly using the 3DGS model to render multi-view images? What advantages does it have compared with this traditional dual-branch VAE architecture (such as HESIC and BiSIC)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rUnWUnw1tp", "reviewer": "Reviewer_EHuz", "summary": "In this paper, 3D-LMVIC is proposed as a novel learning-based multi-view image compression framework, which relies on 3D Gaussian\nSplatting to derive geometric priors for accurate disparity estimation. In details, for each image, a depth map is derived from a trained 3D Gaussian. Then the disparity between views is estimated by leveraging the estimated depth map. Finally, minimizing the training loss to optimize the proposed image and depth compression model.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments! We are grateful for your positive feedback, especially regarding the proposed methods and the ablation experiments. We address your remaining concerns as follows:\n\n### R1[Clarify alignment verification and target view selection]\n---\nGiven a reference view and a target view, we first align the reference view to the target view. Specifically, each pixel in the target image is projected to the corresponding position in the reference image based on the estimated disparity. The color at each projected location is then obtained through bilinear interpolation from the surrounding pixels in the reference image.  \nWe then compute the similarity between the aligned reference view and the target view using PSNR and MS-SSIM. A higher similarity score indicates better alignment accuracy.\n\nIn our alignment experiments, we assume that the ground-truth target view is known and use it as supervision to evaluate alignment quality. Specifically, for a multi-view sequence $v_1, v_2, ..., v_n$, we take $v_i$ as the target view and $v_{i-1}$ as the reference view.\n\n### R2[The training time of the 3D Gaussian]\n---\nFor the *Train* scene of the Tanks&Temples dataset, which contains 301 images, we conducted tests on a platform equipped with an Intel(R) Xeon(R) Gold 6330 CPU @ 2.00GHz and an NVIDIA RTX A6000 GPU. Training the 3D Gaussian representation for 30,000 iterations took approximately 13 minutes and 35 seconds, or an average of 2.71 seconds per image. **We also tested a reduced training setup with 7,000 iterations, which resulted in a total training time of 2 minutes and 22 seconds, or 0.47 seconds per image.**\n\nAdditionally, we assessed the alignment performance of depth estimation using the 3D Gaussian representation trained for 7,000 iterations under the same experimental conditions as detailed in Table 2 of the main text. The results are summarized in the table below:\n\n| Metrics | FlowFormer++ | MVSFormer++ | Proposed (7000 iterations) | Proposed (30000 iterations) |\n|:-------:|:-------:|:-------:|:-------:|:-------:|\n| PSNR↑ | 18.08 | 15.31 | 17.99 | 18.14 |\n| MS-SSIM↑ | 0.7863 | 0.5544 | 0.7918 | 0.8053 |\n\n**Despite the significant reduction in training iterations, it is evident that the proposed alignment method still achieves high alignment accuracy.**\n\n### R3[Test the depth estimation method on a large-scale scene]\n\nWe further evaluate alignment performance on two large-scale scenes (*Temple*, *Church*) and one medium-scale scene (*Truck*) from the Tanks&Temples dataset under identical experimental conditions (Table 2, main text).  \n- **Scenes**:  \n  - *Temple*: 302 images (outdoor temple)  \n  - *Church*: 600 images (indoor church)  \n  - *Truck*: 251 images (mid-scale object)  \n\nResults are summarized below:\n\n| Scenes      | Metrics    | HT     | SPyNet | FlowFormer++ | Proposed |\n|:-----------:|:----------:|:------:|:------:|:------------:|:--------:|\n| Temple      | PSNR↑      | 14.39  | 15.07  | 16.29        | **16.85**|\n|             | MS-SSIM↑   | 0.5135 | 0.5819 | **0.7778**    | 0.7652   |\n| Church      | PSNR↑      | 17.88  | 19.70  | 20.56        | **21.38**|\n|             | MS-SSIM↑   | 0.6604 | 0.7554 | 0.8406       | **0.8753**|\n| Truck       | PSNR↑      | 14.52  | 16.97  | 17.75        | **18.74**|\n|             | MS-SSIM↑   | 0.4756 | 0.7108 | 0.7370       | **0.8233**|\n\n**Key Observations**:  \n1. **Consistent Superiority**: Our method achieves **best PSNR** across all scenes and **best MS-SSIM** in 2/3 cases, demonstrating robust performance scalability.  \n2. **Large-Scale Competitiveness**: While FlowFormer++ shows marginal MS-SSIM advantage in *Temple* (+1.6%), our method still leads in PSNR (+3.4%) and dominates in *Church* (PSNR: +4.0%, MS-SSIM: +4.1%).  \n3. **Mid-Scale Strength**: The **significant gains** in *Truck* (PSNR: +5.6%, MS-SSIM: +11.7% over FlowFormer++) validate the method's effectiveness for complex object-level scenes.  \n\n**Interpretation**: The 3D Gaussian-based approach maintains strong competitiveness in large-scale settings, with performance variations likely attributable to scene-specific characteristics (e.g., texture uniformity in *Temple*). Ongoing work focuses on further optimizing large-scale adaptability without sacrificing mid-scale advantages.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes. I checked the proof of $D_V(i, j)$ as a distance metric.", "experimental_designs_or_analyses": "Yes. For the alignment experiments, the authors do not clearly illustrate how to verify alignment given a pair of reference and target views, and how to get the target view for the propose method.", "relation_to_broader_scientific_literature": "N/A.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n1. Using 3D Gaussian Splatting to get geometric prior is reasonable. The author also using another try to get the estimated depth map in eq. (2) and verify the effectiveness of this change in ablation study.\n2. The authors define a metric for measuring the overlap between two views. This will be a useful metric in multi view reconstruction.\n\nWeaknesses:\n1. It may be time consuming to obtain a trained 3D Gaussian to estimate depth map. It would be better that the authors can discuss the training time of the 3D Gaussian.\n2. For a large scale scene, the insufficient performance of trained 3D Gaussian may degrade the performance of this proposed method. It would be better if the authors can discuss the impact of the quality of estimated depth maps.", "other_comments_or_suggestions": "Please refer to Other Strengths And Weaknesses.", "questions_for_authors": "Please refer to Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7krtmNjn7H", "reviewer": "Reviewer_MNQ5", "summary": "The paper presents 3D-LMVIC, a learning-based multi-view image compression framework that leverages 3D Gaussian Splatting (3D-GS) as a geometric prior for accurate disparity estimation. Unlike traditional methods that rely on 2D projection-based similarities, this approach improves disparity estimation in wide-baseline multi-camera systems by using depth maps derived from 3D-GS. To further enhance compression efficiency, it introduces a depth map compression model to reduce geometric redundancy and a multi-view sequence ordering strategy to maximize inter-view correlations. Experimental results on Tanks&Temples, Mip-NeRF 360, and Deep Blending datasets show that 3D-LMVIC outperforms both traditional and learning-based methods in rate-distortion efficiency and disparity estimation accuracy. The framework effectively utilizes 3D spatial relationships to improve multi-view image compression, making it highly suitable for applications in VR, AR, and 3D vision.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks to you for the valuable comments. We are grateful for your positive feedback, especially regarding the model performance. We address your remaining concerns as follows:\n\n### R1[The training time of the 3D Gaussian]\n---\nFor the *Train* scene of the Tanks&Temples dataset, which contains 301 images, we conducted experiments on a platform equipped with an Intel(R) Xeon(R) Gold 6330 CPU @ 2.00GHz and an NVIDIA RTX A6000 GPU. Training the 3D Gaussian representation for 30,000 iterations took approximately 13 minutes and 35 seconds, averaging 2.71 seconds per image. Additionally, the average encoding time per image is 0.19 seconds. Therefore, the total time to compress a single image—including both training and encoding—is approximately 2.90 seconds.\n\n### R2[Efficient 3D Gaussian training approach]\n---\nWe tested a reduced training setup with 7,000 iterations, which resulted in a total training time of 2 minutes and 22 seconds, or 0.47 seconds per image. Including the image encoding time, the average total time to compress each image is 0.66 seconds.\n\nAdditionally, we assessed the alignment performance of depth estimation using the 3D Gaussian representation trained for 7,000 iterations under the same experimental conditions as detailed in Table 2 of the main text. The results are summarized in the table below:\n\n| Metrics | FlowFormer++ | MVSFormer++ | Proposed (7000 iterations) | Proposed (30000 iterations) |\n|:-------:|:-------:|:-------:|:-------:|:-------:|\n| PSNR↑ | 18.08 | 15.31 | 17.99 | 18.14 |\n| MS-SSIM↑ | 0.7863 | 0.5544 | 0.7918 | 0.8053 |\n\n**Despite the significant reduction in training iterations, it is evident that the proposed alignment method still achieves high alignment accuracy.**\n\n### R3[Explain the role of the image context transfer module]  \n---  \nThe image context transfer module is designed to incorporate features from the reference view into the current view to enhance the overall feature representation. Its functionality is similar to the joint context transfer (JCT) module proposed in LDMIC [1]. While LDMIC adopts a cross-attention mechanism to fuse reference features, our approach performs pixel-wise alignment for feature integration.\n\nSpecifically, due to the disparity between the two views, the module first aligns the reference view features to the current view using the estimated disparity. The aligned features are then concatenated with the current view's features.  \nSubsequently, a feature mask is applied to the combined features to filter out non-overlapping regions and retain only the relevant contextual information. Finally, the resulting features are passed through a residual block to produce the enhanced feature representation.\n\n[1] Zhang, Xinjie, Jiawei Shao, and Jun Zhang. \"LDMIC: Learning-based Distributed Multi-view Image Coding.\" The Eleventh International Conference on Learning Representations.\n\n### R4[Clarify the greedy algorithm for multi-view sequence sorting]\n---\nThank you for pointing out this issue. We provide a more detailed explanation of the algorithm below.  \nSpecifically, given a multi-view sequence $v_1, v_2, ..., v_n$, the algorithm starts from the first view $v_1$ and iteratively selects the view that is closest to the current one. For example, if $v_i$ is the closest to $v_1$, it is placed immediately after $v_1$. The process then continues by finding the closest view to $v_i$ among the remaining views and placing it next. This procedure is repeated until all views are ordered into the sequence.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "I have reviewed the proof in Appendix C regarding the definition of inter-view distance as a valid distance measure for the 2-Norm and Frobenius Norm. I did not find any issues in the proof.", "experimental_designs_or_analyses": "The authors conducted comprehensive experiments, including comparisons with SOTA baselines, an analysis of the alignment accuracy of the proposed method, ablation studies, visualization experiments, and complexity analysis. The experimental section is relatively thorough and complete. One concern is whether the encoding time in the complexity analysis includes the time required for training the 3D Gaussians. Please analyze the proportion of time this part takes.", "relation_to_broader_scientific_literature": "This paper demonstrates that 3D Gaussians can be used to construct accurate geometric structures, which can not only benefit multi-view image compression but also other tasks requiring 3D modeling.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n1. This paper demonstrates that 3D Gaussians can generate accurate geometric structures, outperforming some methods that rely on local similarity matching between two-view projections. This approach is not only applicable to multi-view image compression but also has potential for other 3D modeling tasks.  \n2. Future 3D applications will require denser multi-view data, which significantly exceeds the data volume of single-view images. The proposed method has the potential to reduce storage and transmission costs for such data.  \n3. The experiments are comprehensive and thoroughly validate the effectiveness of the proposed method.  \n4. The inter-view distance properties are theoretically proven.  \n\nWeaknesses:\n1. The analysis of encoding time does not seem to consider the time required for training 3D Gaussians.  \n2. Training 3D Gaussians is generally time-consuming; is there a more efficient training approach?", "other_comments_or_suggestions": "No.", "questions_for_authors": "1. Could you further explain the role of the image context transfer module in feature enhancement?\n2. The explanation of the greedy algorithm for sorting multi-view sequences is not sufficient and can easily cause confusion.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors propose a method for compressing multi-view images in a wide-baseline setting. The primary contribution is in the use of 3D Gaussian Splatting to aid in the estimation of disparity between different views. The reviewers agreed in the merit of using 3DGS over conventional disparity estimation (matching) and noted the empirical performance of the proposed method over existing works.
Decision: Accept (poster)