Learning Disentangled Representations for Perceptual Point Cloud Quality Assessment via Mutual Information Minimization
Authors: Ziyu Shan
Yujie Zhang
Yipeng Liu
Yiling Xu
Venue: NeurIPS 2024 poster (2024)
Keywords: Point Cloud Quality Assessment
Low-Level Vision
Primary area: machine_vision

Abstract: No-Reference Point Cloud Quality Assessment (NR-PCQA) aims to objectively assess the human perceptual quality of point clouds without relying on pristine-quality point clouds for reference. It is becoming increasingly significant with the rapid advancement of immersive media applications such as virtual reality (VR) and augmented reality (AR). However, current NR-PCQA models attempt to indiscriminately learn point cloud content and distortion representations within a single network, overlooking their distinct contributions to quality information. To address this issue, we propose DisPA, a novel disentangled representation learning framework for NR-PCQA. The framework trains a dual-branch disentanglement network to minimize mutual information (MI) between representations of point cloud content and distortion. Specifically, to fully disentangle representations, the two branches adopt different philosophies: the content-aware encoder is pretrained by a masked auto-encoding strategy, which can allow the encoder to capture semantic information from rendered images of distorted point clouds; the distortion-aware encoder takes a mini-patch map as input, which forces the encoder to focus on low-level distortion patterns. Furthermore, we utilize an MI estimator to estimate the tight upper bound of the actual MI and further minimize it to achieve explicit representation disentanglement. Extensive experimental results demonstrate that DisPA outperforms state-of-the-art methods on multiple PCQA datasets.

Learning Disentangled Representations for Perceptual Point Cloud Quality Assessment via Mutual Information Minimization
Ziyu Shan', Yujie Zhang", Yipeng Liu, Yiling Xu Cooperative Medianet Innovation Center, Shanghai Jiao Tong University {shanziyu, yujie19981026, liuyipeng, yl. y1.xu]0sjtu. edu. cn
Abstract
No-Reference Point Cloud Quality Assessment (NR-PCQA) aims to objectively assess the human perceptual quality of point clouds without relying on pristine- quality point clouds for reference. It is becoming increasingly significant with the rapid advancement of immersive media applications such as virtual reality (VR) and augmented reality (AR). However, current NR-PCQA models attempt to indiscriminately learn point cloud content and distortion representations within a single network, overlooking their distinct contributions to quality information. To address this issue, we propose DisPA, novel disentangled representation learning framework for NR-PCQA. The framework trains dual-branci disentanglement network to minimize mutual information (MI) between representations of point cloud content and distortion. Specifically, fully disentangle representations, the two branches adopt different philosophies: the content-aware encoder is pretrained by masked auto-encoding strategy. which allow the encoder capture semantic information from rendered images of distorted point clouds; the distortion- aware encoder takes mini-patch map input, which forces the encoder to focus on low-level distortion patterns. Furthermore, we utilize an MI estimator to estimate the tight upper bound of the actual MI and further minimize representation disentanglement, Extensive experimental results demonstrate that DisPA outperforms state-of-the-ar methods multiple PCQA datasets.
achieve explicit
Introduction
With recent advances 3D capture devices, point clouds have become a prominent media format to represent 3D visual content in various immersive applications, such autonomous driving and virtual reality [4, 43]. These extensive applications stem from the rich information provided by point clouds (e.g., geometric coordinates, color). Nevertheless, before reaching the user-client, point clouds inevitably undergo various distortions at multiple stages, including acquisition, compression, transmission and rendering, leading to undesired perceptual quality degradation. Accordingly, is necessary to develop an effective metric that introduces human perception into the research of point cloud quality assessment (PCQA), especially in the common no-reference (NR) situation where pristine reference point clouds are unavailable.
In recent years, many deep learning-based NR-PCQA methods [21, 47, 51, 32, 3] have shown remarkable performance on multiple benchmarks, which can be applied directly to 3D point cloud data or 2D rendered images. Most of these methods [47, 22, 51, 3] tend to learn unified representation for quality prediction, ignoring the fact that perceptual quality is determined by both point cloud
"Ziyu Shan Yujie Zhang contribute equally to this work. Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Ground Truth
(b) PQA-Net
(c) GPA-Net
Quality Score
Distortion Type
Color Noise
Downsamplig
Gaussian Noise
score span
score
score span
2
4
3
2 3 Distortion Intensity
Distortion Intensity
Distortion Intensity
Figure I: Statistics of SITU-PCQA (part) [46] and predicted quality scores of NR-PCQA models (PQA-Net [21] and GPA-Net [32]). Quality scores of different distortion types are lines of different colors. Red circles are to highlight the score span of different contents with the same distortion.
content information and distortion pattern. Although some other models [21, 33] alternately learn content and distortion representations through different training objectives, they are still based on a single-branch network and thus may lead highly entangled features in the representation space.
From the perspective of visual rules, insufficient disentanglement between representations of point cloud content and distortion disobeys the perception mechanisms of the human vision system (HVS), further limiting performance improvement. In fact, many studies [35, 16] highlight the distinct visual processing of high-level (e.g., semantics) and low-level information (e.g., distortions) in different areas of the brain. Concretely, the left and right hemispheres of the brain are specialized in processing high-level and low-level information, respectively. These findings suggest relatively disentangled processing mechanism in our brain, challenging existing methods that seek to learn these conflicting representations using single network indiscriminately.
The difficulty of disentangled feature learning is relatively great for NR-PCQA due to data imbalance. Specifically, although a wide range of distortion types and intensities in current PCQA datasets can enable the learning of robust low-level distortion representations, it is non-trivial to learn the representations of point cloud content that lies in a considerable high dimensional space because these PCQA datasets are extremely limited in terms of content (e.g., up to 104 contents in LS-PCQA lead to overfitting of NR-PCQA models regarding point cloud content,
[23]). This data limitation that is, when the content changes, the prediction score changes in the undesired manner, even with the same distortion pattern. As illustrated in Figure 1 (b) and (c), the NR-PCQA models PQA-Net [21] and GPA-Net [32] correctly predict the trend of quality degradation with increasing distortion intensity, but their predicted score spans deviate lot from the ground truth in Figure (a), where the content varies but the distortion pattern remains intact. Based on these observations, we expect a new disentangled representation learning framework that can obey the separate information processing mechanism of HVS, and alleviate the difficulty of content and distortion representation learning introduced by data imbalance.
In this paper, we propose a new Disentangled representation learning framework tailored for NR- PCQA, named DisPA. Motivated by the HVS perception mechanism, DisPA employs dual-branch structure to learn representations of point cloud content and distortion (called content-aware and distortion-aware branches). DisPA has three steps to achieve disentanglement: 1) To address the problem introduced by data imbalance, we pretrain a content-aware encoder based on masked autoencoding strategy. Specifically, in this pretraining process, the distorted point cloud is rendered into multi-view images whose patches will be partially masked. The partially masked images are then fed into the content-aware encoder reconstruct the rendered images of the corresponding reference point cloud. 2) To facilitate learning of distortion-aware representations, we decompose the distorted multi-view images into a mini-patch map through grid mini-patch sampling [40], which can prominently present local distortions and forces the distortion-awe aware encoder to ignore the global content. 3) Inspired by the utilization of mutual information (MI) in disentangled representation learning [8], we propose an MI-based regularization to explicitly disentangle the latent representations. Compared to simple linear correlation coefficients (e.g., cosine similarity), mutual information can capture the nonlinear statistical dependence between representations [15]. To achieve this, we utilize an MI estimator to estimate tight upper bound of the MI and further minimize to achieve straightforward disentanglement. We summarize the main contributions as follows:

---

We propose a novel disentangled representation learning method for NR-PCQA called DisPA, which obeys the particular HVS perception mechanism. To the best of our knowledge, DisPA is the first framework to explore representation disentanglement in PCQA.
We propose the key MI-based regularization that can explicitly disentangle the representa- tions of point cloud content and distortion through MI minimization.
We conduct comprehensive experiments on three datasets (SJTU-PCQA [46], WPC [20], LS-PCQA [23]), and achieve superior performance over the state-of-the-art methods on all of these datasets.
2 Related Work
No-Reference Point Cloud Quality Assessment. NR-PCQA aims to evaluate the perceptual quality of distorted point clouds without available references. According to the modalities, the NR-PCQA methods can categorized into three types: projection-based. point-based and multi-modal methods. For the projection-based methods, various learning-basee based networks [21, 47, 52, 33, 34] adopt multi- view projection for feature extraction, while Zhang el al. [53] integrates the projected images into a video conveniently utilize video quality assessment methods to evaluate the perceptual quality. Xie et al. [45] first computes four types of projected images (i.e., texture, normal, depth and roughness) and fuses their latent features using a graph- basph network. For the point-based methods, Zhang et al. extracts carefully designed hand-crafted features, while Liu et al.(23) transforms point clouds into voxels and utilizes 3D sparse convolution to learn the quality representations. Some 3D native methods [37, 32, 39] divide point clouds into local patches and utilize hierarchical networks structurally like PointNet++ [29] to learn the representations. For the mudti-modal methods, Zhang [l.[51] utilizes individual 2D and 3D encoders to separately extract features, and fuse them using symmetric attention module. Other works [38, 3, 22] leverage various cross-modal interaction mechanisms to enhance the fusion between 2D and 3D modalities. Compared to previous methods that learn quality representations indiscriminately, our work solves quality representation disentanglement from more essential perspective of mutual information, which reveals the intrinsic correlations between point cloud content and distortion pattern.
Representation Learning for Image/Video Quality Assessment. As for image quality assessment (IQA), CONTRIQUE [25] learns distortion-related information on images with synthetic and realistic distortions based on contrastive learning. Re-IQA [31] trains two separate encoders to learn high- level content and low-level image quality features through an improved contrastive paradigm. QPT [54] also learns quality-aware representations through contrastive learning, where the patches from the same image are treated as positive samples, while the negative sample are categorized into content- wise and distortion-wise samples to contribute distinctly to the contrastive loss. QPTv2 [44] is based on masked image modeling (MIM), which learns both quality-aware and aesthetics- representations through performing the MIM that considers degradation patterns.
As for VQA, CSPT [5] learns useful feature representation by using distorted video samples not only to formulate content-aware distorted instance contrasting but also to constitute an extra self-supervision signal for the distortion prediction task. DisCoVQA [41] models both temporal distortions and content- related temporal quality attention via transformer-baat based architectures. Ada-DQA [19] considers video distribution diversity and employ diverse pretrained models to benefit quality representation. DOVER [42] divides and conquers cesthetic-relatee and technical-related ddistortion-relatte in videos, introduces inductive biases for each perspective, including specific inputs, regularization strategies, and pretraining. However, there is no current work to utilize mutual information (MI) to achieve representation disentanglement, which has not been explored in IQA/VQA.
perspectives
Mutual Information Estimation. Mutual information (MI) has been widely used as regularizers or objectives to constrain independence between variables [2, 7, 13, 14]. Hjelm er al. [14] performs unsupervised representation learning by maximizing MI between the input and output of deep neural network. Kim et [l.[ learns disentangled representations by encouraging the distribution of representations to be factorial and hence independent across the dimensions. Moreover, MI minimization has been drawing increasing attention in disentangled representation learning [6. 15, 55). Chen el al.[8] introduces contrastive log-ratio upper bound for mutual information estimation, and extends the estimator to variational version for general scenarios when only samples of the
3

---

joint distribution are obtainable. Dunion et al.[11] minimizes the conditional mutual information between representations improve generalization abilities under correlation shifts and enhances training performance in scenarios with correlated features. However, to our knowledge, there has been no previous work focusing on learning disentangled representations or exploring MI estimation for visual quality assessment.
3 Mutual Information Estimation and Minimization
Given the content-aware and distortion- aware representations (x,y), goal is to estimate the MI between x and and further minimize it. In this section, we explain the mathematical background of how to leverage neural network estimate the MI between and y.
The MI between x and can be defined as:
I(x:y) = / p(x, y) log p(x,y) y) dxdy
p(x)p(y)
(I)
Ep(x,r) log p(x,y)
p(x)p(y)]
where y) is the joint distribution, p(x) and ply) are the marginal distributions.
Unfortunately, the exact computation of MI between high-dimensional representations is actually intractable. Therefore, inspired by [6, 8, 15], we focus on estimating the MI upper bound and further minimize it. The tight upper bound of mutual information (MI) means an upper boundary that is always higher the actual value of MI. A tight upper bound means the bound is close to the actual value of MI and equal to MI under certain conditions. An MI upper bound estimator T(x; y) can be formulated as (proof in Appendix A):
y(x:y) = Ep(x,)
x)] Bptc) plyg
Since the conditional distribution pxy unavailable in our case. we approximate using variational distribution (x = 2x (x, where the conditional distribution is inferred by
another light neural network 2s with parameters ø. Then the variational form I, (x:y) can be formulated as (in a discretized form):
N log do (y./x.) - log go (y,]):)]
(x:x)) N2
(3)
where ((x, y.) N samples pairs drawn from the joint distribution p(x, y). To make Í. (x; tight MI upper bound, 2p is trained to accurately approximate p(y(x) by minimizing the KL divergence between p(y ix) and qs (x(x):
min KL xx)r)))l4
min E (lggy Ep(x,)) [log go (y(x)]
(4)
be minimized
Obviously, the first term in Equation 4 has no relation with c, thus Equation 4 equals minimization of second term. Therefore, the can a tight MI upper bound if we minimize the following negative log-likelihood of the inferred conditional distribution:
N
CMI
log (yslxx))
Σ = N
log (x.,
(5)
Now, given the representations x and y, we can train the MI estimator ī. y) predict the MI between and by minimizing Lxn- Afterwards, we minimize I, (x: y) for explicit disentanglement, detailed implementations will be explained in Section 4.4.

---

(a) Architecture Rendered Images
(b) Content-aware Pretraining Partially Masked Images
Content- Aware Encoder
Distorted PC
Distorted PC
Content- Aware Encoder
MI
Quality Score
Estimator
Decoder
|JRegvaaizztion
Reference PC
Distortion-
Aware Encoder
Mini-Patch Map
Rendered Images
Figure 2: Architecture of proposed DisPA (a). Our DisPA consists of two encoders F and G for learning content-aware and distortion-aware representations, and an MI estimator M. The content- aware encoder F is pretrained using masked autoencoding (b). "Ø" denotes concatenation.
4 Proposed Framework
4.1 Overall Architecture
The aforementioned analysis in Section reveals that HVS processes high-level and low-level features in a relatively separate manner. To obey this mechanism, as illustrated Figure 2 (a), the architecture of DisPA is divided into two branches to learn content-aware and distortion- aware representations, respectively. Given a distorted point cloud P, we first render it into multi-view images I. The multi-view images are fed into frozen pretrained vision transformer (VIT) [10] F with parameters e to generate the content-aware representation x. Next, the multi-view images are decomposed into mini-pater patch map M through grid mini-pate sampling. The mini-patch map is encoded by the [24]) with parameters O, to obtain representation y. After obtaining x and y. also them to train the MI estimator M and obtain the estimated MI (x; y) following the process in Section 3. Finally, concatenate and y (denoted as [+, :D and regress it by fully-connected layers H. with parameters O₁ to predict quality score q. The whole process can be described follows:
distortion- aware encoder G (a Swin Transformer)
(6a)
(x;x) = ([FF;
(6b)
4.2 Content-Aware Pretraining via Masked Autoencoding
As analyzed in Section 1, the learning difficulty of content representation is more intractable than distortion due to the limited dataset scale in terms of point cloud content. To address this problem, we pretrain the content-aware encoder F via the proposed masked autoencoding strategy. As illustrated in Figure (b), given a distorted point cloud and its corresponding reference point cloud Peess our goal is to render P and Pee into multi-view images (I(") E R"x x (T(m) E R x Wx33 J==1 I==1
N,
and pretrain F by using partially masked (n) to reconstruct (()) where N, is the number views.
Multi- View Rendering. Instead of directly performing masked autoencoding in 3D space, we render point clouds into 2D images to achieve pixel-to-pixel correspondence between the rendered images of P and Piet. which facilitates the computation of pixel-wise reconstruction loss between the predicted patches and the ground truth patches. To perform the rendering, we translate P (or Pret) to the origin and geometrically normalize it to the unit sphere to achieve consistent spatial scale. Then, to fully capture the quality information of 3D point clouds, we apply random rotations before rendering P, Pre into { (m)) n= and ( (f(n) N. J.N.
J==1-
Patchifying and Masking, After obtaining the rendered image 7(0), we partition it into non- overlapping 16 16 patches following [10]. Then we randomly sample subset of patches and mask the remaining ones, where the masking ratio relaxed to 50% instead of the high ratio in [12] (e.g..
5

---

75% and even higher) because some point cloud samples in PCQA datasets exhibit severe distortions, necessitating more patches to extract effective content-aware information. addition, the relatively low masking ratio can mitigate the influence of the background of [(m).
Selected Mini-Patches
Encoding and Reconstruction. The un- masked patches of [(m) are fed into the ViT F for initial embedding and subsequent encoding to obtain the representation x. To compensate for the scarcity of point cloud content in PCQA datasets, initialize the encoder using the pa- rameters optimized on ImageNet- [9]. Next,
to reconstruct T(x) feed x into decoder and reshape it into 2D pixels to generate the recon- structed f(x) By reconstructing masked refer- ence patches from unmasked distorted patches, the encoder F is forced focus semantic information than distortion patterns. The content- aware representation can be learned by the reconstruction loss Lree:
Figure 3: Illustration of mini-patch map generation.
Ng
Lee
2
ref
4.3 Distortion-Aware Mini-patch Map Generation
To learn effective distortion-aware representation y, we decompose the multi-view images w [, illustrated in Figure 3, the distortion pattern is well preserved and even more obvious on the mini- patch map while the content information blurred. More concretely, for each multi-view image [(m), we first split it into uniform L I. grids, the of grids ((n) can described as:
mini-patch map through grid mini-patch sampling. following [40, 42, 52]. As
((a)
-[90,0
(x)
Di.j
(n)
(n) IL.LJ:
(n) Pi 7(0) i x H. (i+ (i+1) H j x W
G+1)xW
L
L.
(8)
(n) where Rij E x x3 denotes the grid in the i-th row and j-th column of 7(x), Then we sample (n) and splice all the selected mini-patches to get the mini-patch map M. Note that blank mini-patches (i.e., image background) are ignored, and the map is ensured to
the mini-patches from each 9i
M E R H x x3 by filling in the unemployed mini-patches. After the mini tini-patch map generation, we
feed it into the distortion-an aware aware encoder G to generate the corresponding representation y.
4.4 Disentangled Representation Learning
MI-hased Regularization. After obtaining content-aware and distortion- representations and y. we further disentangle them by minimizing the MI upper bound in I, y) Equation 3. As revealed in Equation and 5, the key to accurately estimate tight 2, (x; y) is to minimize the negative log-likelihood of the variational network (x, Here we implement the 2p using MLPs, and model the variational distribution as an isotropic Gaussian parameterized by a mean value n [p1( .... [x))] and a diagonal covariance matrix Σ = o where os = (x1),, o (zD)), D is the feature dimension of x and y. Then the variational distribution can be inferred as:
qo(r)x) (x
II d=1
D
1
exp
(ya HS (xa))² 20 (xa))
D2 (xa)
where Ne and o² are obtained via the last two MLP layers. & is optimized by minimizing LMr in Equation 5, the negative lig-likelihood of (x y). It is noted that the parameters of ó are optimized independently with the main networks O, and O seeing Algorithm 1.
Loss Function. After obtaining the (x; we incorporate into our total training objective as regularizer disentangle the content-aware and distortion-aware representations, the total training

---

Algorithm Disentangled Representation Learning Pipeline
Input: A batch of rendered images [T | 16=1; mini-patch map {M, networks F. 9, H with parameters , g, θn; MI estimator M with variational network Qs: optimizer; A1 Output: Updated parameters O O, and ø'
// Parameters 6' is frozen
1: Encode rendered images to generate content-aware representation x < =F((1,
2: Encode mini-patch map to generate distortion-aware representation 4- 9(MM); 2g)
3: for m = + NM do
// Update the MI estimator by training 2
4: Compute negative log-likelihood LMI t Σb=1 2b=1 (x.
5: Update & by minimizing Lvi Ø' optimizer (6) VoLMi)
6: Compute the estimated MI (x; t 1
R² Σ Σ 2ー=1 log Qar (x), log Q (x,,
7: Predict the quality scores 9b f- H( Xx, hp)
8: Compute the total loss C Σ 1 (6t 212² Arnk + )21, 9: Update the parameters ( 0% optimizer((0,, mn (ee,
e C, Ve, C))
loss function L can be formulated as:
> @b 96)2 + A1 x2{((x;))
(10)
where B is the batch size and Lrank is differential ranking loss following [51, 33]. The λι and A2 are weighting factors to balance each loss term. To better recognize quality differences for the point clouds with close MOSs, the differential ranking loss [51] Crank used model the ranking relationship between & and q:
B B
Drank = B
Σ
max (0, (0, |g: - 9í1 -el-ec (gi, q;) - 4;)),
i=1 1=
(11)
{
q: ス 9j -1, qi < 9j
<((igg)) -1
Algorithm summarizes the overall pipeline of the disentangled representation learning framework (one iteration), where NM is the steps of updating for variational networks per epoch, and JF is initialized with the pretrained parameters 9, after masked autoencoding. The parameters the main network Gg O and the variational networks are updated alternately.
5
Experiments
5.1 Datasets and Evaluation Metrics
Our experiments are performed on three popular PCQA datasets, including LSS-PQQA [23), SITU-PCQA [46], and WPC [20]. The content-aware pretraining is based on LS-PCQA, which contains 24,024 distorted point clouds, and each reference point cloud is impaired with 33 types distortions (e.g., V-PCC, G-PCC) under levels. The disentangled representation learning conducted on all three datasets separately using labeled data, where SITU-PCQA includes reference point clouds and 378 distorted samples impaired with 7 types of distortions (e.g., color noise, downsampling) under levels, while WPC contains 20 reference point clouds and 740 distorted samples disturbed by 5 types of distortions (e.g., compression, gaussian noise).
Datasets.
Evaluation Metries. Three widely adopted evaluation metrics are employed to quantify the level
of agreement between predicted quality scores and ground truth (i.e., Mean Opinion Score, MOS): Spearman rank order correlation coefficient (SROCC), Pearson linear correlation coefficient (PLCC), and root mean square error (RMSE). To ensure consistency between the value ranges of the predicted scores and subjective values, nonlinear Logistic-4 regression is used to align their ranges.

---

Table 1: Quantitative comparison of the state-of-the-an methods and proposed DisPA on LS-PCQA [23], SITU-PCQA [46], WPC [20]. The best results are shown in bold, and second results are underlined. "P" "T" stands for the method is based on the point cloud/image modality, respectively. "1" / """"""" indicates that larger/smaller is better.
Ref
Modal
Method
SROCC 1
PLCC't 1
RMSE
SROOC
SITU-POQA PLCC
RMSE
SROCC
PLCC T
WPC
RMSE
P
MSE-p2po [26]
0.325
0.528
0.158
0.783
0.845
0.122
0.564
0.557
0.188
P
HiX-p2po (26]
0.291
0,488
0.163
0.681
0.748
0.156
0.106
0.166
0.222
P
MSE-p2pl [36]
0.311
0.498
0.160
0.703
0.779
0,149
0.445
0.491
0.199
P
HD>p2pl [36]
0.291
04788
0.617
0.661
0.177
0.344
0.380
0.211
FR
P
PSNR-yuv [36]
0.548
0.547
0.155
0.704
0.715
0.165
0.563
0.579
0.186
P
PointSSIM
0.178
0.183
0.747
0.157
0,453
0.481
0.200
P
POQM 127)
0.439
0.510
0.152
0.864
0.853
0.112
0.750
0.754
0.150
P
GraphSSIM (48]
0.320
0.28
0.178
0.856
0.874
0,114
0.679
0.693
0.165
MS GraphSUM [49]
I
POA-Net [21]
0.588
0.592
0.202
0.659
0.687
0.172
0,547
0.579
0.189
IT-PCQA [47]
0.326
0.347
0.224
0.539
0.221
P
GPA-Ne: 1321
0.592
0.619
0.186
0.878
0.886
0.122
0.758
0.769
0.162
NR
P
ReSSCNN [23]
0.594
0.624
0.172
0.834
0.863
0.153
0.735
0.752
0.177
P+1
MM-PCQA [51]
0.761
I CoPA (33]
0.621
0.636
0.161
0.897
0.913
0.092
0.779
0.785
0.144
I
DisPA (ours)
0.625
0.631
0.160
0.908
0.919
0.089
0.788
0.790
0.138
5.2 Implementation Details
Our experiments are performed using PyTorch [28] on NVIDIA 3090 GPUs. All point clouds are rendered into 6-view projected images with a spatial resolution of 512 x 512 by PyTorch3D [30]. The encoders FF and G are ViT-B [10] and Swin-T [24], respectively.
Content-Aware Pretraining. The pretraining is performed for 200 epochs, the initial learning rate is 3e-4, and the batch size is 64 by default. Adam optimizer [18] is employed with weight decay of 0,0001. Each point cloud is randomly rotated 6 times before being rendered into 6-view images to fully take advantage of quality information of point clouds.
Disentangle Representation Learning. The steps of updating of MI estimator is set to 10. We use the Adam optimizer with weight decay of 0.0001 and batch size of 16. The hidden dimension of fully-connected layers is set to 64. The learning rate is initialized with 0.003 and decayed by 0.95 exponentially per epoch. For LS-PCQA, the model trained for 20 epochs, while 150 epochs for SITU-PCQA and WPC. The
A X2 is set to 1 and 0.01 according to the loss scales.
Data Split. Considering the limited dataset scale, in the training stage, 5-fold cross validation is used for SJTU-PCQA and WPC to reduce content bias. Take SITU-PCQA for example, in each fold, the dataset split into train-test with ratio 7:2 according to the reference point clouds, where the performance testing set with minimal training loss is recorded and then averaged across five folds to get the final result. A similar procedure is repeated for WPC where the train-test ratio is 4:1. As for the large-scale LS LS-PCQA, it is split into rrain-val-tess with ratio around 8:1:1 (no content overlap exists). The result on the testing set with the best validation performance recorded. Note that the pretraining is only conducted the training set of LS-PCQA.
5.3 Comparison with State-of-the-art Methods
15 state-of-the-art PCQA methods are selected for comparison, including 9 FR-PCQA and NR- PCQA methods. For comprehensive comparison, we conduct the experiment in four aspects: 1) We compare quantitative comparison of prediction accuracy, following the cross-validation configuration in Section 5.2. 2) We perform the statistical analysis in Figure for DisPA. 3) We present qualitative examples to demonstrate the superiority of our model in terms of avoiding overfitting when point content varies. 4) We report the results of datsset evaluation for the NR-PCQA methods to verify the generalizability of our model.
Quantitative Comparison. The prediction accuracy of selected methods are presented Table 1, which demonstrates the competitive performance of proposed DisPA across all three datasets, and outperforms all the FR-PCQA methods on SITU-PCQA and WPC. Compared with the NR-PCQA methods, DisPA outperforms CoPA [33] by about 1.3% in terms of SROCC on LS LS-PCQA, and
8

---

(a)
(D)
(c)
(d)
GT: PQA-Net: 7.22
PQA-Net: 5.66 5.01
3.47 PQA-Net: 7.44
3.46
PQA-Net:
CoPA: 6.95
Ours: 6.89
CoPA: Ours: 5.28 (f)
CoPA: 2.17 (g)
Ours: 3.35
CoPA: 7.75 (h)
Ours: 3.69
(e)
GT: 9.60 PQA-Net: 8.24 CoPA: 8.76
GT: 18.24 PQA-Net: 20.25
GT: 23.08
PQAA-Nt:: 35.67
GT: 17.45
PQA-Net: 36,70 Ours: 19.63
Ours: 9.74
CoPA: 18.99
Ours: 18.40
CoPA; 14.60
Ours: 25.88
CoPA: 26.54
Figure 5: Qualitative Evaluation of NR-PCQA methods (PQA-Net [21], CoPA [33] and DisPA) on SJTU-PCQA [46] and WPC [20]. Figure (b)-(d) share the same distortion pattern (i.e., color noise), same for (f)(h) (i.e., downsampling). "GT" denotes ground truth.
1.3% on SJTU-PCQA. Note that CoPA has also been pretrained on LS-PCQA. Furthermore, DisPA significantly reduces RMSE by 4.2% compared to CoPA.
(a) Ground Truth
(b) D&PA
Statistical Analysis. We perform the statis- tical analysis on SITU-PCQA in Figure 4 for DisPA. Compared with the statistics of PQA- Net and GPA-Net in Figure I, our DisPA not only predicts quality scores more accurately, but also obviously predicts closer score spans when point cloud content varies, even when the distor- tion intensity is at the highest level. The statisti- cal analysis demonstrates that the content-aware pretraining strategy can effectively address the problem of superior difficulty learning rep- resentations for point cloud content caused by
Qualty
scorespar
Distartian Intersity
Distortion Intensity
Figure 4: Statistical Analysis of SITU-PCQA (part)
and predicted quality scores of DisPA.
data imbalance.
Qualitative Evaluation. In Figure 5, we present examples of SJTU-PCQA and WPC with predicted scores of PQA-Net [21] and CoPA [33], where Figure 5 (a)(b), (e)(f) share the same content, (b)-(d), (f)-(h) share the the same distortion. Note that each score is predicted on the testing set of 5-fold validation. We can see that the predicted score of our DisPA is obviously closer to the ground truth (i.e., Mean Opinion Score, MOS) and dose deviate from the MOS when content varies, which further validates the effectiveness of content-aware pretraining and representation disentanglement.
Cross-Dataset Validation. To test the generalization capability of NR-PCQA methods when encountering various data distribution, we perform cross-dataset on LS-PCQA [23], SITU-PCQA [46] and WPC [20]. In Table we mainly train the compared models on the complete LS-P and test the trained model on the complete SITU-PCQA and WPC, and the result with minimal training loss is recorded. This procedure repeated for mutual cross-dataset validation between SITU-PCQA and WPC. From Table 2, we can see that the performance of the cross-dataset validation relatively low due to the tremendous variation data distribution. However, our method still present competitive performances, demonstrating the superior generalizability of DisPA.

---

Table 2: Cross-dataset validation on LS-PCQA Table 3: Ablation study of DisPA on SITU-PCQA [23]. SITU-PCQA [46] and WPC [20] (com- [46]. The best results are in bold. plete set). The best results (PLCC) are in bold, and the second results are underlined.
Index
Pretraining
Disentanglemeut
Loss
SROCC
PLCC
0.908
0.919
0.876
0.897
Train
Test
PQA-Nec
GPA-Net
MM-PCQQA
CoPt
D6PPA
0.849
0.365
1.5
STTU
0.556
0.581
0.644
0.683
Cos, Sumularity Con. only
0,884
0.910
1.5
WPC
0.266
0.433
0.454
0.516
0.505
0.811
WPC S7TU
STTU
0.235
0.553
0.612
0.643
0.657
Dis. only
0.763
0.778
WPC
0.220
0.438
0.200
0.53
0.535
who Crok
0.892
0.910
PQA-Nec
GPA-Net
Content (ours)
Distortion (ours)
Cluterng
Clssering
14
Origyrtion
Desortion
boundary
CBLCK
Contone
INACE
Currert
DOAG1
Figure 6: Visualization of t-EE embeddings of representations of PQA-Net, GPA-Net, content-aware and distortion-awe branches of our DisPA.
5.4 Ablation Study
We conduct ablation study of DisPA on SITU-PCQA [46] Table 3. From Table we have following observations: 1) Seeing ① and @ the pretraining strategy effectively improves the performance of DisPA. 2) Seeing D. 3 and ④ the philosophy of representation disentanglement brings significant improvements to our model, because using simple cosine similarity in ④ for disentanglement can achieve fair performance. However, using MI for disentanglement can better constrain the dependence between representations. 3) Seeing C. 5 and (. using single branch infer quality scores causes poor performance, since PCQA is a combination judgement based on the interaction of distortion estimation and content recognition. 4) Seeing ① and ①. the performance is close, demonstrating the robustness of our model using different training loss functions.
Furthermore, shown in Figure 6, we conduct t-SNE visualization to compare the representation embeddings of Pet-Net, GPA-Net, content-aware and distortion- branches of our DisPA on the testing set of SJTU-PCQA. PQA-Net and GPA-Net are selected for comparison because these two methods both use distortion type prediction to learn distortion-awann representations. The scattered points are color and shape marked according to distortion type and content. The distortion- aware features are visualized in 3rd sub-image, where we can see that the learned distortion-awe aware representation shows clear and separate clustering for different distortion types, indicating strong correlation with degradations. The content-awan features present non-clustering for distortion types but a clear boundary to split the content.
Conclusion
In this paper, we propose disentangled representation learning framework (DisPA) for No-Reference Point Cloud Quality Assessment (NR-PCQA) based on mutual information (MI) minimization. As for the MI minimization, we use a variational network to infer the upper bound of the MI and further minimize to achieve explicit representation disentanglement. learning difficulty of content-awan representations, we propose novel content-aware pretraining strategy to enable the encoder to capture effective semantic information from distorted point clouds. Furthermore, to learn effective distortion-awann into mini-patch maps, which can preserve original distortion pattern and make the encoder ignore the global content. We demonstrate the high performance of DisPA on three popular PCQA benchmarks and validate the generalizability compared with multiple NR-PCQA models.
addition, to tackle the nontrivial
representations, we decompose the rendered images
Acknowledgments. This paper is supported in part by National Natural Science Foundation of China (62371290) U20A20185), the Fundamental Research Funds for the Central Universities of China, and 111 project (BP0719010)
10

---

References
[1] Evangelos Alexiou and Touradj Ebrahimi. Towards point cloud structural similarity metric. In ICMEW, pages 1-6, 2020.
[2] Mohamed Ishmacl Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural cstimation. In International conference on machine learning, pages 531-540 PMLR, 2018.
[3] Xiongli Chai, Feng Shao, Baoyang Mu, Hangwei Chen, Qiuping Jiang, and Yo-Sung Ho. Plain- peqqa: No-reference point cloud quality assessment by analysis plain visual and geometrical components. JEEE Transactions on Circuits and Systems for Video Technology, 2024.
[4] Guangyan Chen, Meiling Wang, Yi Yang. Kai Yu, Li Yuan, and Yufeng Yue, Pointgpt: Auto-regressively generative pre-training from point clouds. Advances in Neural Information Processing Systems, 36, 2024.
[5] Pengfei Chen, Leida Li, Jinjian Wu, Weisheng Dong, and Guangming Shi. Contrastive self supervied pre-training for video quality assessment. IEFE transactions on image processing, 31:458-471, 2021.
[6] Ricky TQ Chen, Xuechen Li. Roger Grosse, David K Duvenaud. Isolating sources of disentanglement in variational autoencoders. Advances in neural information processing systems, 31, 2018.
[7] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29, 2016.
[8] Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: contrastive log-ratio upper bound of mutual information. In International conference on machine learning, pages 1779-1788, PMLR, 2020.
[9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248 -555. leee, 2009.
[10] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint M0Ki::0000.11922 2020.
[11] Mhairi Dunion, Trevor Mclnroe, Kevin Sebastian Luck, Josiah Hanna, and Stefano Albrecht. Conditional mutual information for disentangled representations in reinforcement learning. Advances neural information processing Syxtems, 36, 2023.
[12] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 16000-16009, 2022.
[13] Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. International Conference on Learning Representations. 3, 2017.
[14] R Devon Hjelm, Alex Fedorov, Samuel I. MarchlMar,,,,,,,,, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. I Learning deep representations by mutual information estimation and maximization. arXiv preprint 1808 806.0600 2018
[15] Xuege Hou, Yali Li, and Shengjin Wang. Disentangled representation for age-invariant face recognition: mutual information minimization perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3692-3701, 3701, 2021.
[16] Ryosuke Kawakami, Yoshiaki Shinohara, Yuichiro Kato, Hiroyuki Sugiyama, Ryuichi Shigemoto, and Isao Ito. Asymmetrical allocation of nmda receptor e2 subunits in hippocampal circuitry. Science, 300K5621).990-994 2003.
[17] Hyunjik Kim and Andriy Mnih. Disentangling by factorising. International conference on machine learning, pages 2649-2658. PMLR, 2018.
[18] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 141.,6991,69 2014.

---

[19] Hongbo Liu, Mingda Wu, Kun Yuan, Ming Sun. Yansong Tang, Chuanchuan Zheng, Xing Wen, and Xiu Li. Ada-dqa: Adaptive diverse quality- feature acquisition for video quality assessment. In Proceedings of the 31st ACM International Conference on Multimedia, pages 6695-6704, 2023.
[20] Qi Liu, Honglei Su, Zhengfang Duanmu, Wentao Liu, and Zhou Wang. Perceptual quality assessment of colored 3d point clouds. IEEE Transactions on Visualization and Computer Graphics, 2022.
[21] Qi Liu, Hui Yuan, Honglei Su, Hao Liu, Yu Wang, Huan Yang, and Junhui Hou. Pqa-net: Deep no reference point cloud quality assessment via multi-view projection. IEEE Transactions on Circuits and Systems for Video Technology, 31(12):4645-4660 2021.
[22] Yating Liu, Ziyu Shan, Yujie Zhang, and Yiling Xu. Mcq- Multi-modal fusion transformer for no-reference point cloud quality assessment. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal P Processing (ICASSP), pages 7965-7969. IEEE, 2024.
[23] Yipeng Liu, Qi Yang, Yiling Xu, and Le Yang. Point cloud quality assessment: Dataset construction and beaning-based no-reference metric. ACM Transactions on Multimedia Computing, Communications and Applications, 19(2s):1-26, 2023.
[24] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang. Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012-10022, 2021.
[25] Pavan C Madhusudana, Neil Birkbeck. Yilin Wang, Balu Adsumilli, and Alan C Bovik. Image quality assessment using contrastive lcarning. IEEE Transactions Image Processing, 31:4149-411 4161441111220 2022.
[26] R Mekuria, Z Li, C Tulvan, and Chou, Evaluation criteria for point cloud compression. ISO/EC MPEG, (16332), 2016.
[27] Gabriel Meynet, Yana Nchmé, Julie Digne, and Guillaume Lavoué, Poqm: A full-reference quality metric for colored 3d point clouds. In QoMEX, pages 1-6, 2020.
[28] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[29] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnct+++: Deep hierarchical feature learning on point sets a metric space, Advances in neural information processing systems, 30, 2017.
[30] Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Ye Yen Lo. Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007. 08501, 2020.
[31] Avinab Saha, Sandeep Mishra, and Alan C Bovik. Re-iqa: Unsupervised learning for image quality assessment in the wild. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5846-505 58455585,.20 2023.
[32] Ziyu Shan, Qi Yang, Rui Ye, Yujie Zhang, Yiling Xu, Xiaozhong Xu, and Shan Liu. Gpa-net: No-reference point cloud quality assessment with multi-task graph convolutional network. IEEE Transactions on Visualization and Computer Graphics, 2023.
[33] Ziyu Shan, Yujie Zhang, Qi Yang. Haichen Yang, Yiling Xu, Jenq-Neng Hwang, Xianzhong Xu, and Shan Liu. Contrastive pre-training with multi-view fusion for no-reference point cloud quality assessment. IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024.
[34] Ziyu Shan, Yujie Zhang, Qi Yang, Haichen Yang, Yiling Xu, and Shan Liu. Pame: Self-supervised masked autoencoder for no-reference point cloud quality assessment. arXiv preprini arXiv:2403. 10061, 2024.
[35] Yoshiaki Shinohara, Hajime Hirase, Masahiko Watanabe, Makoto Itakura, Masami Takahashi, and Ryuichi Shigemoto. Left-right asymmetry of the hippocampal synapses with differential subunit allocation of glutamate receptors. Proceedings of the National Academy of Sciences, (9449)):9998-999999 2008.
[36] Dong Tian, Hideaki Ochimizu, Chen Feng, Robert Cohen, and Anthony Vetro. Geometric distortion metrics for point cloud compression. In /EEE ICIP, pages 3460-3464, 2017.
[37] Marouane Tliba, Aladine Chetouani, Giuseppe Valenzise, and Fréderic Dufaux. Pega-graphpoint: efficient deep- based graph metric for point cloud quality assessment. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, IEEE, 2023.
12

---

[38] Jilong Wang. Wei Gao, and Ge Li. Applying collaborative adversarial learning to blind point cloud quality measurement. IEEE Transactions on Instrumentation and Measurement, 2023.
[39] Songtao Wang. Xiaoqi Wang. Hao Cao, and Jian Xiong. Non-local geometry and color gradient aggregation graph model for no-reference point cloud quality assessment. In Proceedings of the 31 ACM International Conference on Multimedia, pages
2023.
[40] Haoning Wu, Chaofeng Chen, Jingwen Hou, Liang Liao, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Fast-vqa: Efficient end-to-end video quality assessment with fragment sampling. In European conference on computer vision, pages 538-554. Springer, 2022.
[41] Haoning Wu, Chaofeng Chen, Liang Liao, Jingwen Hou, Wenxiu Sun, Qiong Yan, and Weisi Lin. Discovqa: Temporal distortion-comtent transformers for video quality assessment, IEEE Transactions on Circuits and Systems for Video Technology, 33(0))4840-458 2023.
[42] Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang. Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20154, 2023.
[43] Tianhe Wu, Shuwei Shi, Haoming Cai, Mingdeng Cao, Jing Xiao, Yinqiang Zheng, and Yujuu Yang. Assessor360: Multi-sequence network for blind omnidirectional image quality assessment. Advances in Neural Information Processing Systems, 36, 2024.
[44] Qizhi Xie, Kun Yuan, Yunpeng Qu, Mingda Wu, Ming Sun, Chao Zhou, and Jihong Zhu. Opt v2: Masked image modeling advances visual scoring, arXiv preprint Sxki:2207.165444 2024.
[45] Wuyuan Xie, Kaimin Wang, Yakun Ju, and Miaohui Wang. pmbqa: Projection based blind point cloud quality assessment via multimodal learning. In Proceedings of the 31 ACM International Conference on Multimedia, pages 3250-3258, 2023.
[46] Qi Yang, Hao Chen, Zhan Ma, Yiling Xu, Rongjun Tang, and Jun Sun. Predicting the perceptual quality of point cloud: A 3d-to-2d projection-basee based exploration. IEEE Transactions on Multimedia, 23:3877-3891, 2020.
[47] Qi Yang. Yipeng Liu, Siheng Chen, Yiling Xu, and Jun Sun. No-reference point cloud quality assessment via domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21 219-222110 188, 2022.
[48] Qi Yang, Zhan Ma. Yiling Xu, Zhu Li. and Jun Sun. Inferring point cloud quality via graph similarity. IEFE transactions on pattern analysis and machine intelligence, 44(6):3015-3029, 2020.
[49] Yujie Zhang, Qi Yang, and Yiling Xu. Ms-graphsing Inferring point cloud quality via multiscalc graph similarity. In Proceedings of the 29th ACM International Conference on Multimedia, pages 1230-1238, 2021.
[50] Zicheng Zhang, Wei Sun, Xiongkuo Min, Tao Wang. Wei Lu, and Guangtao Zhai No-reference quality assessment for 3d colored point cloud and mesh models. IEEE Transactions on Circuits and Systems for Video Technology, 32( 32111177188- 2022.
[51] Zicheng Zhang. Wei Sun, Xiongkuo Min, Quan Zhou, Jun He, Qiyuan Wang, and Guangtao Zhai Mm-poqa: Multi-modal learning for no-reference point cloud quality assessment, arXiv preprint arXiv:2209.00244 2022.
[52] Zicheng Zhang, Wei Sun, Haoning Wu, Yingjie Zhou, Chunyi Li, Zijian Chen, Xiongkuo Min, Guang- tao Zhai, and Weisi Lin. Gms-3dqa: Projection- grid mini-patch sampling for 3d model quality assessment. ACM Transactions on Multimedia Computing, Communications and Applications, 1016):1-19, 2024.
[53] Zicheng Zhang, Wei Sun, Yucheng Zhu, Xiongkuo Min, Wei Wu, Ying Chen, and Guangtao Zhai. Evaluating point cloud from moving camera videos: A no-reference metric. IEEF Transactions on Multimedia, 2023.
[54] Kai Zhao, Kun Yuan, Ming Sun, Mading Li, and Xing Wen. Quality-aware pre-trained models for blind image quality assessment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pages 22302-220 22313, 2023.
[55] Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, and Jiebo Luo. Leaming bias invariant representation by cross sample mutual information minimization. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 15002 15012, 2021.
13

---

A Proof of Theorems
Proof of Equation 2. To show that (x:; ) an upper bound of ( y), we calculate their difference:
y(x:y) - I(x:y) = Bp(x. ply x)] - pp(x) Ep(r) (lgg p(y|x)
Delinition [(x: Equatiinn
E pix,y) [ogp(y|x] x) log p(y)]
Detinition I (xc; y) Equation
E y)x,y) [logg(y))
(12)
Since log p(y) has relation with x, (xxx)))) p(y)] Ep(y) [log p(y)). Then the equation can be formulated as:
y(: I(x; pyys) [lyg((y) Re(x) [log p(y x)]]
(13)
Recalling that the marginal distribution can obtained by integrating the joint distribution over the values of the other random variables:
ply) = y(xx - )
(14)
Note that log(-) is concave function, by Jensen's Inequality, we have
log log (E))))) p(y()))) Pogplyyggy
(15)
Definition of murginal distribution in Equation 14
Applying this inequality to Equation 13, we can conclude that I(x;y) is always greater than T(x; y). Therefore, [xx y) is upper bound of [(x; y). I(x:y) yx occurs only when p(y (x) holds the same for any x, which means x and y are two totally independent representations.
B Limitations and Future Work
We designed no-reference point cloud quality assessment (NR-PCQA) framework, whose experi- mental performances have been validated. However, our current design has two main limitations:
Limitation of pretraining datasets. Ideally, the content-aware encoder J can be pretrained on a much larger dataset, despite the already large scale of LS-PCQA [23]. In our future work, we may pretrain it on distorted natural images or create larger dataset of point clouds with various distortions.
Estimation of mutual information (MI). Although using the current MI minimization strategy can achieve satisfactory results, we may make efforts in the future to work more efficient MI estimation method for high-dimensional representations.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The contributions clearly included in the abstract and the Introduction (listed in the end).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please see B, which includes the limitation of our work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification: Please see A, which proves that the utilized MI upper bound is always greater than the actual MI, and when they are equal. The proof includes each detailed theoretical support.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our implementation part has included all the necessary information to repro- duce the main experimental results. And the model architecture has been clearly introduced in the methodology part.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The used data is all public open. In addition, we have uploaded our code the in a zip file as a supplementary document.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
public/gus httpe//gggideee/eeeubiiiii wwriiisssoooicyiooco for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //iiss... wwwpcccpppc/.aee..c.ccccce.cetle pwmblc/gguides....//e /ttpc/gggidee/eeeuuuaiiiiiiiiiii /CodeSubmisiiond sww.bbiiissioooioccccocoo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The experimental settings and training/testing details have been included in Implemantion Details part.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
The results of our paper are not accompanied by error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please Implementation part, where we have introduced the used compute resources.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https http//nnurrpp.pppe wttpiipp...pppulpeeeeee cc/public/ Eetthtttisiseeeeee csGuidelines
Answer: [Yes]
Justification: Our work strictly conform the code the ethics, presenting no socictal impact, potential harm or non-anonymity behaviors.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Honestly, our work presents no potential negative socictal impacts, because the used data is all public open and the task is not potentially harmful.
Guidelines:
18

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work obviously poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: CC-BY 4.0 is included in our work.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cow datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have uploaded our assets (e.g., the code) in anonymized URL link and zip file.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work dose not involve crowdsourcing nor research with huamn subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work dose not involve crowdsourcing nor research with huamn subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "vuQzb2eGsL", "reviewer": "Reviewer_2ATd", "summary": "This article's motivation is interesting. It combines content-aware and distortion-aware characteristics to train a 3D quality assessment network. Additionally, it employs a MAE-based method to train a content-aware encoder, uses patches to focus the network on learning distortion, and applies an MI module to integrate both features.", "questions": "This section corresponds to the Weaknesses:\n\n1. How does the network address the gap between the input images in the first and second stages?\n2. Why is a clean image used as a constraint? This results in learning the restoration characteristics, which is not very helpful for quality assessment.\n3. The motivation for MI needs further explanation.\n4. Visualize or analyze the distortion-aware features.", "limitations": "No potential negative societal impact. Please address the questions/suggestions above.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The authors first analyze the shortcomings of existing networks, specifically that data imbalance leads to overfitting in current methods, resulting in poor processing of other content images with the same degradation. Therefore, the authors aim to use an MAE-based method to learn content-related features.\n2. The proposed key MI-based regularization is effective.\n3. The presented methods can obtain impressive results on multiple datasets.", "weaknesses": "1. Regarding the masked part, there is a gap between the first and second stages because the input in the first stage is a partially masked image, while the input in the second stage is the complete image.\n2. I am also a bit confused about why the constraint in the first stage is a clean/reference image. This would give the masked encoder the characteristic of restoration, while the core of quality assessment is to evaluate the quality of the image. If the image features are restored, will it affect the accuracy of the image quality assessment?\n3. The motivation for the MI part should provide more details and explanations, which leaves me somewhat puzzled.\n4. I am not quite sure what the distortion-aware encoder has learned. Is it truly related to degradation features? This might require some verification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer #1 (2ATd) for the insightful comments. Our responses are as follows:\n\n**`R1-Weakness 1`**: Thank you for your in-depth comments. The gap between the partially masked image and the complete image **is addressed by fine-tuning**, as well as the main training objective (see Equation 10 in our paper). Our content-aware pretraining is based on masked auto-encoding (MAE), where the **MAE is also conducted on partially masked images and fine-tuning is conducted on complete images.**  The large-scale pretraining can enable the encoder to learn useful semantic representations, while the gap between masked and complete images can be addressed by adequately training the pretrained encoder with small-scale complete images. \n\nIn our case, our MAE-based pretraining can make the encoder learn to recognize different point cloud contents (on LS-PCQA with a large scale of different point clouds), and the gap can be addressed fine-tuning the encoder on specific PCQA datasets. Furthermore, the pretraining can effectively enhance the prediction accuracy (see Table 3 in our paper) by a large margin, which indicates the gap has been sufficiently addressed after fine-tuning.\n\n**`R1-Weakness 2`**: Thanks for your comments. As you said, using a clean image as a constraint resembles image restoration. However, Since our goal is to learn disentangled representations, **we use a clean image as a constraint to force the content-aware encoder to ignore the distortions and capture point cloud content information from distorted projected images**. In contrast, if we use a distorted image as a constraint, the encoder tries to restore the images with distortion and will certainly learn joint features of distortion and content, which disobeys our goal of representation disentanglement.\n\nFurthermore, we use t-SNE visualization to verify that the learned content-aware representations are strongly correlated to point cloud content but not to distortions.  The visualization is in the `attached file Figure A`. The embeddings of different point cloud contents are separated by a clear boundary.\n\n**`R1-Weakness 3`**: Sorry for the lack of detailed explanation. **The mutual information (MI) part is designed to explicitly disentangle the representations of the two branches**, since the MI directly describes the correlation between two variables. In terms of detailed motivation of MI estimation and minimization (Section 3 in our paper), we have several key motivations: \n\n* Instead of computing the intractable exact value of MI (denoted as $\\mathcal I$), we compute its upper bound $\\hat{\\mathcal I}$ because **minimizing the upper bound can also minimize the actual MI** as long as $\\hat{\\mathcal I}$ is a tight upper bound (i.e., close to the exact value of MI). Proof of the tight upper bound is in appendix A. \n* Since **we have no prior knowledge of the conditional distribution** $p(y|x)$ (reasons in R2-Question 1), **we can only approximate it with a variational distribution** $\\hat{\\mathcal I_v}$. Specifically, we use an MLP $\\mathcal Q_\\phi$ to infer a distribution $q_\\phi(y|x)$ that is close to $p(y|x)$, and the inferred distributions of samples in a mini-batch are summarized to predict $\\hat{\\mathcal{I}}_v$ (see Equation 3 in our paper). \n* As mentioned above, minimizing the upper bound can only work when the upper bound is tight. **To make $\\hat{\\mathcal I_v}$ close to the tight upper bound $\\hat{\\mathcal I}$, we minimize the KL divergency between $p(y|x)$ and $q_\\phi(y|x)$**. After simplification, the KL divergency can formulated as the $\\mathcal L_\\text{MI}$​ (Equation 5 in our paper).\n* Finaly, for each epoch during the training, we first train the MLP $\\mathcal Q_\\phi$ (i.e., minimizing the KL divergency between$p(y|x)$ and $q_\\phi(y|x)$). Then we use the trained $\\mathcal Q_\\phi$ to estimate the MI to regularize the encoders, as shown in Equation 10 in our paper, where the estimated $\\hat I_v$ is used as a regularization term to achieve explicit disentanglement.\n\n**`R1-Weakness 4`**: To verify that the distortion-aware features are related to degradations, we conduct the t-SNE visualization of learned representations of the distortion-aware branch. The results are in `attached file Figure A`. We also present the visualization results of PQA-Net and GPA-Net for comparison, because these two methods both use distortion type prediction to learn distortion-aware representations. From `attached file Figure A` we can see that **the learned distortion-aware representation shows clear and separate clustering for different distortion types, indicating a strong correlation with degradations.** In addition, the content-aware representations present worse clustering for distortion types.\n\nIn addition, many papers [1,2,3] have verified that the mini-patch map can force the network to learn features that are related to degradations. \n\n[1] Fast-vqa: Efficient end-to-end video quality assessment with fragment sampling\n\n[2] Exploring video quality assessment on user generated contents from aesthetic and technical perspectives\n\n[3] Gms-3dqa: Projection-based grid mini-patch sampling for 3d model quality assessment", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jm1Pdx8ZpB", "reviewer": "Reviewer_oeD7", "summary": "This paper proposes a novel disentangled representation learning framework called DisPA to decouple the representation learning process of point cloud content and distortion. To sufficiently disentangle these two representations, the DisPA uses two branches to learn them and adopt different training philosophies separately. For the content-aware branch, DisPA pretrains one encoder using a proposed masked auto-encoding strategy, which partially masks the images projected from distorted point clouds and reconstructs the corresponding patches of the images projected from pristine point clouds; For the distortion-aware branch, the DisPA integrates the mini patches of the rendered multi-view images into a mini-patch map, which can focus on local distortions and ignore the global point cloud content. Furthermore, to disentangle the learned representations, the DisPA uses a trainable mutual information estimator to estimate the mutual information (actually the tight upper bound) between these two branches and further minimize it alternatively along with the training of the main network (i.e., the two encoders and regression layers). Finally, the experimental results demonstrate the superior performance of DisPA in terms of both prediction accuracy and generalizability.", "questions": "1. Why the conditional distribution p(y|x) is unavailable in your case? This needs more explanations, because the mathematical analysis is somewhat abstract so more details are needed to make it more understandable.\n2. What is the advantanges of the differential ranking loss function?\n3. Can this work be trained end-to-end for IQA or video quality assessment (VQA) without modification of specific modules (just replacing the input/output)?\n4. Can the masked auto-encoding strategy use natural images for pretraining?", "limitations": "Yes, the authors have adequately addressed the limitations and discussed the potential negative societal impact in the appendix D Limitations and Future Work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ This is the first paper exploring disentangled representation learning for PCQA. The disentanglement is reasonable and even necessary because the point cloud content and distortion are differently perceived by humans.\n\n+ The proposed methodology is well-motivated. The masked auto-encoding strategy can intuitively make the encoder capture the point cloud content information.\n\n+ The paper is written well. The motivation of why disentangled representation learning is essential for PCQA has been introduced clearly from the observation of human vision systems.\n\n+ The mathematical derivations are detailed and easy to follow, including the approximation of MI in Section 3 and further proof in Appendix A.", "weaknesses": "1. From my viewpoint, the DisPA can be totally used for IQA, or even more suitable, since this work does not process 3D native point cloud data but just projects point clouds into images and uses 2D networks. Have the authors tried to introduce unique attribute information that is closely related to point clouds?\n2. The whole DisPA is based on projections of point clouds, so the number of viewpoints is very important to the quality score prediction. However, the authors did not discuss the impact of the number of viewpoints.\n3. How much time does it take to pretrain the content-aware encoder? The authors did not discuss the pretraining details in the implementation details part.\n4. Why does not the LS-PCQA follow the K-fold data splitting? The authors should explain this for the loss of a unified experiment setting.\n5. What is the actual architecture of the MI estimator? And what is the relation between the MI estimator and the lightweight neural network Q_phi? The idea of alternative training of the MI estimator and the main network is easy to understand, but the components of the MI estimator need more clarification. Is it just simple MLPs?\n6. The mini-patch map generation has been used in many papers [1,2,3]. However, considering this is a minor contribution and the mini-patch map is actually effective for learning disentangled representations, this limited contribution is acceptable.\n7. A minor weakness: There are some small flaws (gray bounding box) with the presentation of Figure 5 when zooming in. Please ensure all figures can be presented clearly without error.\n\n[1] Wu, Haoning, et al. \"Exploring video quality assessment on user generated contents from aesthetic and technical perspectives.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[2] Wu, Haoning, et al. \"Fast-vqa: Efficient end-to-end video quality assessment with fragment sampling.\" European conference on computer vision. Cham: Springer Nature Switzerland, 2022.\n\n[3] Zhang, Zicheng, et al. \"Gms-3dqa: Projection-based grid mini-patch sampling for 3d model quality assessment.\" ACM Transactions on Multimedia Computing, Communications and Applications 20.6 (2024): 1-19.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer #2 (oeD7) for the insightful comments. Our responses are as follows:\n\n**`R2-Weakness 1`**: Thanks for your constructive comment. As you said, the DisPA can be used for IQA without changing the architecture and training pipeline. We use projected images instead of 3D native point cloud for the following reasons: **(1) Large scale pretraining.** In the scenario of PCQA, we focus on dense point clouds with huge data volume. Rendering point clouds into multi-view images can effectively reduce the data volume to facilitate large scale content-aware pretraining (i.e., the masked autoencoding strategy) with large batch size . **(2) Pixel-to-pixel correspondence.** After projection, the pixel-to-pixel correspondence can be established between the projected images of distorted and reference point clouds, which facilitates the computation of reconstruction loss for the content-aware pretraining. **(3) Generation of mini-patch map.** Projecting point clouds into images can enable the generation of mini-patch map for the distortion-aware branch to enhance the presentation of low-level distortion patterns.\n\nFurthermore, we have introduced some unique characteristics of point clouds to achieve better performance. For example, we **randomly rotate point clouds before projection** to fully investigate the quality information of point clouds. In addition, the **mini-patch map integrates the patches from multi-view images** to take advantage of the multi-view characteristic of point clouds.\n\n**`R2-Weakness 2`**: Thanks for your insightful comment and we have tested the performance of DisPA trained on SJTU-PCQA and WPC with different numbers of viewpoints (i.e., 2, 4, 6 and 12 views). The configuration of mini-patch map generation is accordingly modified. The viewpoints are all evenly distributed in the 3D space. \n\n| Performance | SJTU-PCQA |         |         |          | WPC     |         |         |          |\n| :---------- | :-------- | :------ | :------ | :------- | ------- | ------- | ------- | -------- |\n|             | 2 views   | 4 views | 6 views | 12 views | 2 views | 4 views | 6 views | 12 views |\n| SROCC       | 0.657     | 0.886   | 0.908   | 0.910    | 0.554   | 0.705   | 0.788   | 0.786    |\n| PLCC        | 0.636     | 0.893   | 0.919   | 0.919    | 0.577   | 0.711   | 0.790   | 0.793    |\n\nFrom the above table we can see that with the number of viewpoints increasing, the model performance first increases and then keep stabilized. More specifically, the performance with 12 viewpoints is slightly better than 6 viewpoints, and the performances of 2 and 4 viewpoints are relatively unsatisfactory. Considering the efficiency, we select 6 viewpoints in our paper. We will add these results in the revised paper.\n\n**`R2-Weakness 3`**: Thank you for your question. It takes about 20 hours for the content-aware pretraining on a single NVIDIA 3090 GPU because of the large dataset scale of LS-PCQA and the random rotations before projection during pretraining. We will add these details in our implementation part. \n\n**`R2-Weakness 4`**: We select K-fold because of the small dataset scale of SJTU-PCQA and WPC. However, LS-PCQA has **a large scale** so we select the train-val-test splitting. To further validate the stable performance on LS-PCQA, we test the the performance of several NR-PCQA models on LS-PCQA following unified 5-fold cross validation:\n\n| Performance | PQA-Net | IT-PCQA | GPA-Net | ResSCNN | MM-PCQA | DisPA(ours) |\n| ----------- | ------- | ------- | ------- | ------- | ------- | ----------- |\n| SROCC       | 0.583   | 0.337   | 0.587   | 0.593   | 0.587   | 0.623       |\n| PLCC        | 0.590   | 0.348   | 0.606   | 0.625   | 0.603   | 0.635       |\n| RMSE        | 0.199   | 0.226   | 0.192   | 0.170   | 0.191   | 0.161       |\n\nFrom the above table we can find the performances of NR-PCQA are relatively stable on LS-PCQA (compared with Table 1 of our main paper).\n\n**`R2-Weakness 5`**: As you said, the MI estimator is based on MLPs, as well as the $Q_\\phi$. The $Q_\\phi$ infers the variational distribution and the MI estimator computes the MI following Equation (3) in the main paper. Therefore, the **lightweight MLP $Q_\\phi$ is the key component of MI estimator**, and MI estimator integrates the outputs of $Q_\\phi$ and computes the MI.\n\n**`R2-Weakness 6`**: Thanks for your insightful comment. Although grid mini-patch sampling has been employed in the papers you listed, this is our sub-contribution and quite useful in our scenario when learning distortion-aware representations. Furthermore, our mini-patch sampling is different because we utilize the mutli-view characteristic of point clouds and integrates multi-view patches into the mini-patch map.\n\n**`R2-Weakness 7`**: Thanks for your kind comments. This is due to the issues of pdf browsers, we will fix this problem of image presentation.\n\n**`R2-Question 1`**: Thank you for your insightful question. In our case, the conditional distribution $p(\\mathbf y |\\mathbf x)$  denotes the distribution of distortion-aware representation given the content-aware representation. This is unavailable because the distribution of distortion patterns is unknown, since there is no prior knowledge of distortion types and intensities for DisPA during inference. Furthermore, there are cases of unseen distortion types for cross-dataset evaluations.\n\n**`R2-Question 2`**: The differential ranking loss can better assist the model to distinguish the quality difference when the point clouds have close quality labels.\n\n**`R2-Question 3`**: The DisPA can be trained end-to-end for IQA and VQA. However, the module of mini-patch map generation should be accordingly modified. For example, the IQA can be supposed as a special case of PCQA with only one viewpoint.\n\n**`R2-Question 4`**: The masked-encoding can definitely used on natural images. Acutually, the content-aware encoder is initialized with parameters optimized on ImageNet-1K with natural images.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hhwO4DBKSn", "reviewer": "Reviewer_hUdG", "summary": "This paper proposes a novel no-reference quality assessment model tailored for point-cloud data. A disentangled representation learning strategy is leveraged to account for both content-aware information and distortion-aware information. Comprehensive experiments are conducted and the effectiveness of this proposed model is well verified.", "questions": "Please see the Weaknesses.", "limitations": "Please see the Weaknesses.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed framework is novel and the paper is easy to understand.", "weaknesses": "1. The idea of framing the quality assessment into content-/semantic-aware and distortion-aware aspects is actually quite common in IQA/VQA on 2D visual data. It’d be better to review some related QA metrics for 2D data, and analyze the key difference on the implementation of this idea between point-cloud data and 2D data quality assessment.\n2. Lack of comparison on computational complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer #3 (hUdG) for the constructive comments. Our responses are as follows:\n\n**`R3-Weakness 1`**: Thank you for your insightful comments. We first review the related IQA/VQA papers and then analyze the key difference with our DisPA. The paper review will be added to our main paper.\n\nAs for IQA, **CONTRIQUE** [1] learns distortion-related information on images with synthetic and realistic distortions based on contrastive learning. **Re-IQA** [2] trains two separate encoders to learn high-level content and low-level image quality features through an improved contrastive paradigm. **QPT** [3] also learns quality-aware representations through contrastive learning, where the patches from the same image are treated as positive samples, while the negative sample are categorized into content-wise and distortion-wise samples to contribute distinctly to the contrastive loss. **QPTv2** [4] is based on masked image modeling (MIM), which learns both quality-aware and aesthetics-aware representations through performing the MIM that considers degradation patterns.\n\nAs for VQA, **CSPT** [5] learns useful feature representation by using distorted video samples not only to formulate content-aware distorted instance contrasting but also to constitute an extra self-supervision signal for the distortion prediction task. **DisCoVQA** [6] models both temporal distortions and content-related temporal quality attention via transformer-based architectures. **Ada-DQA** [7] considers video distribution diversity and employ diverse pretrained models to benefit quality representation. **DOVER** [8]  divides and conquers aesthetic-related and technical-related (distortion-related) perspectives in videos, introduces inductive biases for each perspective, including specific inputs, regularization strategies, and pretraining. \n\nOur DisPA's key difference with these papers lies in the utilization of unique characteristics of point clouds. For example, to fully investigate the quality information of point clouds, we **randomly rotate point clouds before projection** . In addition, the **mini-patch map integrates the patches from multi-view images** to take advantage of the multi-view characteristic of point clouds. Furthermore, it is worthwhile to note that our paper is first to use mutual information (MI) to achieve representation disentanglement, which has not been explored in IQA/VQA.\n\n**`R3-Weakness 2`**: Thank you for your comments. As your request, we compare the computational complexity of NR-PCQA models as follows:\n\n| Method       | Parameters (M) | Inference Time (s) |\n| ------------ | -------------- | ------------------ |\n| IT-PCQA      | 0.61           | 2.87               |\n| ResSCNN      | 1.23           | 1.92               |\n| PQA-Net      | 0.22           | 4.86               |\n| MM-PCQA      | 52.96          | 2.64               |\n| DisPA (ours) | 140.77         | 1.89               |\n\nFrom the above table, we can see that compared to the IT-PCQA, ResSCNN and PQA-Net whose architecture is specifically designed, our DisPA has large parameters because Vision Transformer (ViT) and Swin Transformer have been used, like MM-PCQA using ResNet-50. Instead of pretraining self-designed light-weight networks, we use popular encoders to make the pretrained encoders easier to be transferred to other tasks.\n\nHowever, although our DisPA has a larger model size, the inference time is the fastest, even including the rendering process, which demonstrates the efficiency of our DisPA.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LZRPbCcs9z", "reviewer": "Reviewer_nBew", "summary": "This paper explores the method for No-Reference Point Cloud Quality Assessment. The key idea is to involve the disentangled representation learning to minimize mutual information between representations of point cloud content and distortion. The authors conduct experimental performance comparisons on three public databases and compare their proposed method with 15 existing models. The proposed method achieves optimal or suboptimal results in most of the metrics. The ablation study demonstrated the necessity\nof each design part.", "questions": "See Weaknesses.", "limitations": "See Weaknesses.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is clearly written and provides a detailed formulation of the approach.", "weaknesses": "The weak point of the paper is its presentation. Many terms are introduced without explaining them properly, e.g. \"the tight upper bound\" or \"the masked autoencoding strategy”. Furthermore, the Fig 2 is not well-designed with content aware branch. The Fig2 splits the proposed architecture and the content-aware pretraining and masked autoencoding strategy, which does not help the reader to understand architecture with figure. Lastly, it would be highly beneficial to make the code publicly available to enhance collaborative efforts and facilitate the sharing of this work. Apart from this, there are quite a few typos and grammar mistakes that should be corrected, such as “the can be”, “masked auto-encoding / autoencoding", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**`R4-Weakness 1`**:  Thank you for your precious comments about presentation. We will provide more detailed explanations and make the paper easier to understand. The explanations of some terms are as follow:\n\n* The *tight upper bound* of mutual information (MI) means an upper boundary that is always higher the actual value of MI. A tight upper bound means the bound is close to the actual value of MI and equal to MI under certain conditions.\n\n* The *masked autoencoding (MAE)* strategy is a popular visual self-supervised learning scheme, which maskes some patches of the input image and training a model to reconstruct the masked patches. This method helps in learning robust representations of unlabeled images.\n* The *grid mini-patch sampling* means dividing an image into smaller, non-overlapping patches (mini-patches) arranged in a grid-like structure. \n* The *variational distribution* is a key concept in variational inference, a technique used in Bayesian statistics for approximating complex probability distributions. It is essentially a tractable distribution used to approximate the true posterior distribution of the model's latent variables\n\n**`R4-Weakness 2`**: Thank you for your constructive comments about the Fig 2, we have modified the Fig 2 following your suggestion. We split the architecture and the content-aware pretraining (i.e., the proposed masked autoencoding strategy) into two figures, and make some detailed modifications , as shown in `attached file Figure B` and `attached file Figure C`. This can make readers better understand the overall architecture and the pretraining strategy.\n\n**`R4-Weakness 3`**: Thank you for your comments about the code. The code is attached in the supplementary material (in the .zip file), and we will upload the code to Github to make the code publicly available.\n\n**`R4-Weakness 4`**: Thank you for comments about typos and grammar mistakes. We will carefully correct these detailed mistakes and ensure there will be no such mistakes in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers and ACs:

We sincerely appreciate all the reviews. They give insightful and high-quality comments on our paper. We would like to emphasize that our work proposes a novel and effective disentangled representation learning framework for point cloud quality assessment called DisPA. We appreciate your recognition of our efforts in attempting to disentangle the content-aware and distortion-aware representations via mutual information minimization. 

We have carefully read your comments and made responses to all the reviewers, where we have supplemented our work with additional experiments and visualization results to incorporate the insightful suggestions of the reviewers. **The related figures are attached in the pdf file.** Thank you all for the valuable suggestions.

Thanks,

Paper 6653 Authors.

Meta-review: Following the author-reviewer discussion stage, three reviewers expressed a positive opinion of the paper. While reviewer nBew raised concerns regarding the presentation, no critical technical issues were mentioned. The AC reviewed the authors' response to reviewer nBew and believes that these concerns can be addressed in the preparation of the camera-ready version. Therefore, the AC recommends acceptance.
Decision: Accept (poster)