LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-25 17:12:55

Abstract: Although large multi-modality models (LMMs) have seen extensive exploration and application in various quality assessment studies, their integration into Point Cloud Quality Assessment (PCQA) remains unexplored. Given LMMs' exceptional performance and robustness in low-level vision and quality assessment tasks, this study aims to investigate the feasibility of imparting PCQA knowledge to LMMs through text supervision. To achieve this, we transform quality labels into textual descriptions during the fine-tuning phase, enabling LMMs to derive quality rating logits from 2D projections of point clouds. To compensate for the loss of perception in the 3D domain, structural features are extracted as well. These quality logits and structural features are then combined and regressed into quality scores. Our experimental results affirm the effectiveness of our approach, showcasing a novel integration of LMMs into PCQA that enhances model understanding and assessment accuracy. We hope our contributions can inspire subsequent investigations into the fusion of LMMs with PCQA, fostering advancements in 3D visual quality analysis and beyond.

Main paper: 
 introduction: 
 Point clouds are increasingly used across diverse real-world scenarios, including virtual/augmented reality [15, 21, 31] , autonomous vehicles [9] , and video post-production [26] . This surge is attributed to their adeptness in three-dimensional representation. Consequently, significant research efforts have been channeled towards enhancing high-level areas like point cloud classification [5, 13, 17, 37, 39, 44, 51] , detection [9] , and segmentation [7, 24] . Meanwhile, as a key component for ensuring the point cloud quality, point cloud quality assessment (PCQA) has seen comparable advancements as well during the last decade. PCQA's objective is to appraise point clouds' visual quality, a pivotal factor for refining simplification and compression strategies in practical applications [10] , and to elevate the Quality of Experience (QoE) for end-users. The complexity of PCQA is further compounded when assessing point clouds depicting realistic objects or individuals, due to their intricate geometric structures and the dense aggregation of points, often augmented with color attributes, posing additional challenges. Generally, PCQA methods are divided into three categories, depending on their reliance on reference point clouds: Full-Reference PCQA (FR-PCQA), Reduced-Reference PCQA (RR-PCQA), and No-Reference PCQA (NR-PCQA) respective. The availability of pristine reference point clouds is often limited in real-world applications, emphasizing the demand for NR-PCQA approaches, which is why our research primarily focuses on NR-PCQA.There are already some cutting-edge studies that have begun applying Large Multi-Modality Models (LMMs) to low-level vision and quality assessment fields [16, [40] [41] [42] 52] , achieving notable success. LMMs demonstrate highly competitive performance and robustness in these tasks. However, the main focus of these studies has remained on two-dimensional (2D) media such as images and videos, while no research has explored the possibility of applying LMMs to three-dimensional (3D) media like point clouds. It is known that both 2D and 3D media exhibit similar distortions, i.e., blur and noise. Given the robust quality perception of LMMs in 2D, we can hold the hypothesis that LMM also has significant quality perception abilities in 3D point clouds. Hence, investigating the application of LMMs for point cloud quality assessment is not only valuable but also meaningful, reflecting their established visual perception strengths. Therefore, in this study, we carry out a novel method named LMM-PCQA to provide an interesting solution for handling the PCQA problem with the assistance of LMM.First, we treat the point clouds as sequences of projections to enable LMM to perceive the point cloud visual quality. Afterward, we try to teach LMM about the quality alignment between the predefined 5-level qualitative adjectives (i.e., excellent, good, fair, poor, bad) and point cloud projections. Specifically, we employ the existing PCQA databases to provide the necessary knowledge, where the quality labels are transformed into corresponding qualitative adjectives. Then we specially design a prompt structure to produce the question-answer pairs, which are composed of the question 'Rate the quality of the point cloud from the projections[img1],[img2],[img3],[img4],[img5],[img6]' and the answer 'The quality of the point cloud is excellent/good/fair/poor/bad'. These question-answer pairs can be utilized to teach LMM PCQA knowledge during the fine-tuning stage. After the instruction tuning, we can expect the trained LMM to give the predicted [SCORE_TOKEN] with the same prompt structure (the qualitative adjectives position is left blank for LMM response), which is shown in Fig. 2 . The predicted [SCORE_TOKEN] can be recognized as a probability map to the qualitative adjectives, and we convert it into 5-level probabilities as the LMM evaluation results.Secondly, to address the potential insensitivity to geometric distortions (e.g., compression, downsampling) when only analyzing projections, we propose the extraction of multi-scale structural features. This approach enhances the LMM-PCQA's holistic comprehension of point cloud visual quality. The point clouds are converted into quality-aware structural domains, a technique validated for effective quality feature extraction in prior research [3, 58, 65] . We modify the scale parameters in the k-nearest neighbors (k-NN) algorithm to offer a multi-scale perspective, aligning with the human vision system's perception mechanism. Subsequently, key statistical parameters are used to quantify structural distortions within these domains. Finally, we combine the LMM's evaluative results and the structural features, utilizing support vector regression (SVR) to derive the quality values. The experimental outcomes affirm that our LMM-PCQA model is on par with, or surpasses, current leading PCQA methods.By carrying out LMM-PCQA, the contributions of this paper can be summarized as follows:• We are the first to employ LMM for PCQA tasks. We design a novel prompt structure to enable the LMM to perceive the point cloud visual quality. The existing PCQA databases are converted into question-answer pairs, which are then used to inject PCQA knowledge into LMM.  
related works 2.1 lmm for quality assessment: 
 Recent studies have begun to explore the utilization of LMMs for visual quality assessment, marking a significant shift in the field. For instance, X-IQE [6] employs LMMs to assess the quality of text-toimage generation methods, leveraging a chain of thoughts strategy to understand and evaluate visual outputs. Following this, the Q-Bench study [40] presents a binary softmax approach that enables LMMs to compute quantifiable quality scores. This is achieved by utilizing softmax pooling on the logits corresponding to the tokens good and poor, facilitating a more nuanced assessment of visual quality. Further refining this methodology, Q-Instruct [41] demonstrates the potential of fine-tuning LMMs using a text-based questionanswering format, targeting specific low-level visual queries to improve their accuracy in visual quality evaluation. Drawing inspiration from these developments, the Q-Align [42] framework mimics human evaluation mechanisms and post-assessment recalibrations in visual quality scoring, thereby attaining exceptional performance metrics. This progression in research underscores the evolving capabilities of LMMs in the nuanced domain of visual quality assessment, setting a new benchmark for future explorations. These works have fully illustrated the potential of LMMs' quality assessment abilities on 2D media such as images and videos. However, no effort has been put into adapting LMMs to the 3D quality assessment field.In the initial phase of PCQA exploration, the MPEG group introduces key Full-Reference PCQA (FR-PCQA) methods such as p2point [27] and p2plane [34] , aimed at evaluating point cloud quality. They later present a point-based PSNR-yuv technique to handle colored point clouds [35] . However, these early attempts face some critical challenges in accurately capturing complex distortions through point-level differences. To overcome these challenges mentioned above, more sophisticated FR-PCQA metrics emerge, including PCQM [28] , GraphSIM [48] and PointSSIM [2] , which integrate structural features to significantly improve performance.Geometry PointsStructural Domains-How would you rate the quality of the point cloud from the projections?

...-The quality of this point cloud is [SCORE_TOKEN] . Inspired by recent advances in no-reference image and video quality assessment (NR-I/VQA) [14, 23, 53, 54, 56, 61] , and to cater to a broader range of real-world applications, various NR-PCQA methods have been carried out during the last decade. Chetouani 𝑒𝑡 𝑎𝑙 . [8] apply classical Convolutional Neural Network (CNN) models for quality regression with the assistance of patch-wise handcrafted features. PQA-net [22] adopts multi-view projection for feature extraction, while Zhang 𝑒𝑡 𝑎𝑙 . [58] predict quality-aware parameters by analyzing distributions of geometry and color attributes. Furthermore, Liu 𝑒𝑡 𝑎𝑙 . [25] utilize an end-to-end sparse CNN for direct quality prediction, and Zhou 𝑒𝑡 𝑎𝑙 . [64] focus on structure-guided resampling for extracting relevant features. Some researchers attempt to convert point clouds into videos to employ video quality assessment (VQA) techniques for evaluating perceptual quality [11, 59] . To mitigate computational complexity, GMS-3DQA approach [62] integrates multi-projections into a singular quality map to facilitate feature extraction. Yang 𝑒𝑡 𝑎𝑙 . [47] extend quality assessment methodologies from natural images to point cloud rendering images via domain adaptation techniques. Additionally, Wang 𝑒𝑡 𝑎𝑙 . [38] employs a sophisticated non-local geometry and color gradient aggregation graph model for accurate quality estimation. Furthermore, to provide a more comprehensive learning framework for point clouds from both color and geometry perspectives, MM-PCQA [60] and pmBQA [45] suggest leveraging a multi-modal learning approach to improve PCQA outcomes. In the evolution of PCQA, the potential of text modality for quality representation remains largely untapped. Investigating the approach of leveraging text supervision to facilitate model quality assessment learning emerges as an innovative strategy. 
proposed method: 
 The framework of the proposed method is briefly illustrated in Fig. 2 , which includes the LMM evaluation module, the structural feature extraction module, and the quality regression module. As illustrated in Fig. 2 , six orthogonal viewpoints are utilized, each mapping to one of the cube's six faces for generating the projections.For a point cloud P, the rendering process is derived as:EQUATIONwhere I represents the set of the 6 rendered projections and 𝜓 (•) denotes the Open3D-based [63] projections capturing process.LMMs ? LMM has shown competitive performance in 2D image/video quality assessment [40] [41] [42] , therefore it is feasible to apply LMM for PCQA tasks by taking the point cloud as a sequence of projections. Then it is vital to solve the core problem of PCQA Knowledge Injection. Following the common approaches of training LMMs [18, 49] , it is natural to come up with the solution of instructing the LMM with question-answer pairs regarding PCQA issues. Thus we carry out the specific prompt structure as follows:-How would you rate the quality of the point cloud from the projections?

...-The quality of this point cloud is [QA(mos)].where 

... stands for the image set of projections and QA(s) is the qualitative adjective of the point cloud which can be obtained from the mean opinion score (MOS) corresponding to the point cloud. Afterward, we can use the designed question-answer pairs to teach LMM PCQA knowledge. LMM are akin to humans, which have a better understanding and perception of qualitative adjectives. Therefore, converting MOS into corresponding qualitative adjectives for its learning is more intuitive than having it learn directly from numbers. Specifically, the transformation of MOS can be achieved by evenly splitting the range from the highest score (M) to the lowest score (m) into five unique intervals, with scores in each interval designated as corresponding quality levels:EQUATIONwhere {𝑤 𝑖 | 5 𝑖=1 } = {bad, poor, fair, good, excellent} are the standard text rating levels as defined by ITU [1] .Obtaining evaluation results via LMM inference. After training, we can get the evaluation results with the same prompt structure and get the response [SCORE_TOKEN] via LMM inference. The [SCORE_TOKEN] can be recognized as a log probability map to the qualitative adjectives. Then we can compute the final probabilities to the 5-level qualitative adjectives from the corresponding log probabilities via softmax as the LMM evaluation results:EQUATIONwhere 𝛼 𝑖 indicates the log probability of 𝑖-th qualitative adjectives and 𝐹 𝐿 represents the LMM evaluation results which consist of 5 probabilities after softmax.The inadequacy of single-modality information from projections for point cloud quality evaluation has been established [45, 60] . Therefore, our approach enhances the accuracy of LMM evaluation results (projections only) by integrating geometric structural features, aiming for a more detailed and accurate assessment. EQUATION)where 𝑁 represents the total count of points within the point cloud, the term P Nb denotes the collection of neighborhoods, while KNN(•) signifies the function of the k-nearest neighbors algorithm.The distance between points 𝑝 and 𝑞 is calculated using the Euclidean distance, expressed as 𝐷𝑖𝑠𝑡 (𝑝, 𝑞). Given the neighborhood set 𝑃 𝑁𝑏𝑖 of point 𝑝 𝑖 , we can define the covariance matrix 𝐶 𝑖 for each point 𝑝 𝑖 , which is characterized by its 3D geometric coordinates:EQUATIONwhere the term 𝐾 denotes size of neighborhood set 𝑃 𝑁𝑏𝑖 , 𝑝 𝑛 𝑗 is the 𝑗-th neighboring point in 𝑃 𝑁𝑏𝑖 , p is the centroid of this neighborhood, 𝑝 𝑛 𝑗 and p are vectors with dimensions R 3×1 , while 𝐶 𝑖 is a matrix with dimensions R 3×3 . Consequently, the eigenvectors for the covariance matrix 𝐶 𝑖 can be derived as follows:EQUATIONwhere (𝜆 1 , 𝜆 2 , 𝜆 3 ) stand for the eigenvalues and (𝑣1, 𝑣2, 𝑣3) represent the respective eigenvectors, with 𝜆1 > 𝜆 2 > 𝜆 3 . Consequently, we derive three eigenvalues for each point 𝑝 𝑖 within the point cloud P.Then we can compute the linearity and planarity structural domains of the point cloud as:EQUATIONwhere 𝐿𝑖𝑛(𝑝 𝑖 ) and 𝑃𝑙𝑎(𝑝 𝑖 ) represent the linearity and planarity values for point 𝑝 𝑖 . The chosen structural domains (linearity, planarity) have been demonstrated to exhibit a strong correlation with geometric visual losses, such as compression and downsampling, and have been extensively utilized in numerous PCQA tasks [3, 58, 65] .The multi-scale nature of point cloud visual perception has been noted in the literature [57] . To account for this, we compute structural domains across various scales by varying the scale parameter 𝑘 in the KNN(•) process. This approach allows us to derive the multi-scale structural domains as follows:EQUATIONwhere D 𝑘=𝑘 𝑚𝑠 denotes the multi-scale structural domains, with 𝑘 𝑚𝑠 represents the set of scale parameters, and S(•) refers to the process of calculating structural domains as previously described.In our study, we establish the default set of scale parameters as {10, 20}, signifying that the linearity and planarity domains are computed using the 10-nearest-neighbor and 20-nearest-neighbor configurations, respectively.To quantify the quality representation from the structural domains, we employ some basic statistical parameters estimation process:EQUATIONwhere 𝑎𝑣𝑔(•), 𝑠𝑡𝑑 (•), and 𝑒𝑛𝑡 (•) represent the average function, standard deviation function, and entropy function respectively, and 𝐹 𝑆 indicates the set of the final extracted structural features.To clearly demonstrate the efficacy of the proposed features, we merge the LMM evaluation results with the structural features, and then incorporate them into the visual quality score using support vector regression (SVR):EQUATIONwhere 𝑄 indicates the quality values, SVR(•) represents the SVR regression process, and ⊕ denotes the concatenation process. 
experiment 4.1 validation databases: 
 To assess the efficacy of the proposed method, we employ the SJTU-PCQA database [46] , the Waterloo point cloud assessment database (WPC) [19] , and the WPC2.0 database [20] EQUATIONwhere H 𝑣 and H 𝑡 represent the abstracted tokens for the visual and text input respectively, O stands for the output. For all PCQA databases, the batch size is maintained at 64. The learning rate is fixed at 2 × 10 -5 , with the training process extending over 2 epochs for each variant. We utilize the common GPT [33] loss mechanism, specifically the cross-entropy between the predicted logits and actual labels. The evaluation of performance metrics is conducted using the final weights obtained post-training. Four NVIDIA A100 80G GPUs are employed for the training phase, while a single RTX3090 24G GPU is used to measure inference latency. In the inference stage, only the input texts preceding the [SCORE_TOKEN] are inputted into the LMM, leading to the final element of O representing the targeted probability map.Strategy. Following the methodologies in [4, 11, 60] , we utilize a k-fold cross-validation approach in our experiments to ensure a dependable performance evaluation of our proposed method. The SJTU-PCQA, WPC, and WPC2.0 databases consist of 9, 20, and 16 point cloud groups, respectively, leading us to adopt 9-fold, 5-fold, and 4-fold cross-validation for these databases to achieve an approximate 8:2 train-test split. The average of the performance metrics is considered the definitive result. It is crucial to note that the training and testing sets are mutually exclusive to prevent content overlap. For FR-PCQA methods, which do not require training, we evaluate them using the same test sets and report the average performance.17 state-of-the-art quality assessment methods are selected for comparison, which consists of 8 FR-PCQA and 9 NR-PCQA methods:• The FR-PCQA methods include MSE-p2point (MSE-p2po) [27] , Hausdorff-p2point (HD-p2po) [27] , MSE-p2plane (MSE-p2pl) [34] , Hausdorff-p2plane (HD-p2pl) [34] , PSNR-yuv [35] , PCQM [28] , GraphSIM [48] , and PointSSIM [2] . • The NR-PCQA methods include BRISQUE [29] , NIQE [30] ,IL-NIQE [55] , IT-PCQA [47] , ResSCNN [25] , PQA-net [22] , 3D-NSS [58] , GMS-3DQA [62] and MM-PCQA [60] .Note that BRISQUE, NIQE, IL-NIQE are image-based quality assessment metrics and are validated on the same projections. Four mainstream evaluation criteria in the quality assessment field are utilized to compare the correlation between the predicted scores and MOSs, which include Spearman Rank Correlation Coefficient (SRCC), Kendall's Rank Correlation Coefficient (KRCC), Pearson Linear Correlation Coefficient (PLCC), Root Mean Squared Error (RMSE). An excellent quality assessment model should obtain values of SRCC, KRCC, PLCC close to 1 and RMSE to 0.The performance comparison between the proposed LMM-PCQA and other PCQA competitors on the SJTU-PCQA, WPC, and WPC2.0 databases is illustrated in Table 1 , from which we can draw several conclusions: 1) The LMM-PCQA demonstrates superior performance across all three PCQA databases, even outperforming FR-PCQA methods. For instance, the proposed LMM-PCQA surpasses the second-best NR-PCQA method by about 0.027 (against GMS-3DQA), 0.041 (against MM-PCQA), and 0.034 (against GMS-3DQA) on the SJTU-PCQA, WPC, and WPC2.0 databases from the SRCC values. This highlights LMM's capability to effectively assimilate PCQA knowledge and apply it actively. The consistency in LMM-PCQA's performance across various databases underscores its potential to set new baselines in the PCQA field. 2) All PCQA competitors generally perform better on the SJTU-PCQA database but face significant performance declines on the WPC and WPC2.0 databases. In contrast, LMM-PCQA exhibits the smallest performance drops, with decreases of approximately 0.06 and 0.08 in SRCC values when transitioning from the SJTU-PCQA to the WPC and WPC2.0 databases, respectively. This performance stability underscores LMM-PCQA's robustness and superior ability to handle diverse content effectively, showcasing its adaptability and consistency in quality assessment across different point cloud databases. To fully investigate the contributions and validate the rationality behind the proposed dual streams of features, we decide to undertake an ablation study in this section. The results, as detailed in Table 2 , clearly demonstrate that the integration of both feature streams leads to superior performance compared to employing a single feature stream. Upon a detailed examination, it is apparent that LMM evaluation results markedly surpass the structural features in terms of performance. This disparity primarily stems from the fact that some distorted point clouds within the SJTU-PCQA, WPC, and WPC2.0 databases suffer only from color distortions. Structural features, derived from geometric information, inherently lack the capability to recognize these color distortions, resulting in their relatively lower performance in total. However, incorporating structural features enhances the comprehensive understanding of point cloud quality, contributing to the refinement and elevation of the assessment precision in LMM evaluation results. This combined method highlights how important it is to look at features from different angles to fully understand and capture the subtle details of point cloud quality. 4.6.2 Influence of the number of projections. The proposed LMM-PCQA utilizes 6 projections as default. In this section, we further change the number of used projections to test the corresponding performance influence. Specifically, we randomly select 1-6 projections from the cube-like 6 projections setting as the input projections of LMM-PCQA. The performance tendency is illustrated in Fig. 4 , from which we can draw several interesting conclusions: 1) As the number of projections increases, the performance of LMM-PCQA also improves correspondingly, indicating that increasing the number of projections can encompass more effective quality information, thereby enhancing the final performance. 2) Specifically, when the number of projections increases from 2 to 5, the improvement is significantly more pronounced. This suggests that at this stage, the quality information is not yet redundant, and the benefit of increasing the number of projections is relatively large. However, when the number of projections increases from 5 to 6, the performance improvement is relatively low, indicating that the quality information has become somewhat saturated, and further increases in the number of projections yield diminishing returns. 3 .From the table, we can find that the multi-scale structural features with k=10,20 perform better than the single-scale features whether the LMM evaluation results are involved or not, which confirms the effectiveness of the proposed multi-scale mechanism. This can be attributed to that humans tend to perceive the visual quality of point clouds from a multi-scale perspective.To the generalization ability of the proposed LMM-PCQA, we conduct the cross-database validation in this section. Considering that the SJTU-PCQA, WPC, and WPC2.0 databases contain 378, 740, and 400 distorted point clouds respectively, we pre-train LMM-PCQA on the WPC database (largest in scale) and validate the performance on the SJTU-PCQA and WPC2.0 databases (smaller in scale). The competitive NR-PCQA methods (PQA-net, 3d-NSS, GMS-3DQA, and MM-PCQA) are included for comparison. The experimental performance is shown in Table 4 , from which we can make several observations: 1) The proposed LMM-PCQA achieves the best cross-database validation performance against all competitors, which confirms the strong generalization ability of LMM-PCQA. 2) Most methods obtain higher WPC→WPC2.0 performance than WPC→SJTU-PCQA performance. This might be because the WPC2.0 database contains only compression distortions, which undergo part of a similar distortion generation process of the WPC database. Therefore, the quality representation learned from the WPC database is more effective on the WPC2.0 database.To verify the effectiveness of the proposed LMM-PCQA on different kinds of distortions, we carry out the distortion-specific evaluation experiment in this section. The performance comparison is shown in Table 5 (The 'nan' values for MSE-p2po, HD-p2po, MSE-p2pl, and HD-p2pl arise because these methods only analyze the geometric differences between the reference and distorted point clouds, thereby failing to account for color distortions.) and Table 6 , from which we can obtain several findings: 1) The proposed LMM-PCQA achieves the best performance on 4 of 7 distortion types and all distortion types on the SJTU-PCQA database and the WPC database respectively, which suggests that LMM-PCQA is effective at dealing with various kinds of distortions. 2) Although LMM-PCQA does not achieve the top performance on DS+CN, GGN, and CN+GGN distortions within the SJTU-PCQA database, the performance gap to the best is minimal, with a difference of no more than 0.02 in terms of SRCC values. This suggests that LMM-PCQA remains in the top tier.3) Upon examining Table 6 more closely, it is evident that all PCQA methods exhibit substantial declines in performance with the 'DS' distortion. The underlying reason for this trend is the simplicity of the point cloud reference models used in the WPC database, leading to a reduced sensitivity to downsampling distortion. 
conclusion: 
 In conclusion, this paper pioneers the integration of LMMs with PCQA, unveiling the untapped potential of LMMs in this domain. Our research successfully demonstrates the feasibility of adapting LMMs for PCQA through text supervision, enhancing their capability to evaluate 3D visual quality from 2D projections. We also propose to capture multi-scale structural features to offer a more holistic view of the point cloud quality. By combining LMM evaluation results and structural features, our approach significantly improves the accuracy of PCQA. We carry out thorough experiments to demonstrate the effectiveness of the proposed LMM-PCQA, as well as its robustness and ability to generalize across various distortion types and diverse point cloud content. The encouraging results not only validate our methodology but also lay the groundwork for future research. We hope our work will serve as a stepping stone for further exploration into the synergy between LMMs and PCQA, driving forward innovations in 3D visual quality analysis. 
 
 Appendix:  


Decision: This paper was reviewed by four experts in the field. The recommendations were mixed initially, but after authors’ feedback and discussion, the reviewers reached a consensus of acceptance, including Accept, Weak Accept, Borderline Accept, Borderline Accept. ACs also agreed this is an interesting work with solid experimental support. Based on this, the decision is to recommend the paper for acceptance to ACM Multimedia 2024.

Still, we recommended the authors to carefully read all reviewers’ final feedback, and revise the manuscript as needed. We congratulate the authors on the acceptance of their paper!

Reviews: review: paper_summary: paper_summary: In this paper, a point cloud quality assessment metric is proposed that relies on quality ratings from a large multi-modality model (LMM) on multiple 2D projections of the 3D point cloud, and multi-scale structural features computed on the 3D point cloud itself. The authors claim that this is the first attempt to use LMMs for the quality evaluation of 3D models and, specifically, for point clouds.
strength_weakness: strength_weakness: + timely topic
+ well-written
+ sufficient analysis

The paper is generally good. The topic is nicely motivated and the paper is well-structured and mostly clear (some details are missing; see below). The performance evaluations are sufficient, although some extra results could be added (see below).
limitations: limitations: - some clarifications and further details are needed

In section 3.1.1, it is written:
"We then adopt the conventional cube-like viewpoints configuration, widely used in the standard point cloud compression scheme MPEG VPCC [12]."
In VPCC, there is no cube-like projection process, as the one indicated in Figure 2. Instead, the projection process is patch-based and a bit more complex. The provided visual example resembles more approaches such as the ones discussed in [1,2] (from the references below) and [46] (from the paper's references). Please rephrase as needed. 

In section 3.1.1, please provide some further information regarding the projection process. Specifically, do the authors consider the distance of the virtual camera from the model or apply any model scaling (i.e., could be potentially used to fit the projected model into certain 2D boundaries) in the "Open 3D-based projections" of the point clouds? What if the projected region of the model is too small compared to the background region, or too large that shows only a fraction of the model? How is the LMM model affected by such variations/configurations? On the same note, I think that it would make the related work more complete if the authors would refer to some projection-based point cloud quality assessment methods in section 2.2, such as [1,2] (from the references below) and [46] (from the paper's references).

In section 3.2.3, I understand that the average, standard deviation, and entropy functions are applied to structural feature values obtained for all points of a point cloud. Is this correct? Maybe the authors could make explicit in the text that all points are used for the computation of the structural domain (in 3.2.1) and the statistical parameters estimation (in 3.2.3).

In section 4.2.2, it is written:
".. that the training and testing sets are mutually exclusive to prevent content overlap."
Please clarify in the text whether distorted versions of the same content are used for both training and testing. I assume not, but this is not explicit.

In section 4.6.2, it is written: 
"Specifically, we randomly select 1-6 projections from the cube-like 6 projections setting as the input projections of LMM-PCQA."
I understand that in the results shown in Figure 4 for 1 projection, the authors randomly chose 1 out of the 6 projections per stimulus, for both training and testing sets. Is this correct? If yes, what is the reason for this choice? The authors could also comment on the effect of this choice on the obtained results.

In section 4.7, the same experiment could be performed using SJTU and WPC2.0 for training (and testing on the remaining two). 

In section 4.8, the authors chose SJTU and WPC and not WPC2.0 in this experiment. Is there a reason for this? Please add a sentence explaining that, or add corresponding results for WPC2.0.

[1] https://doi.org/10.1117/12.2322741
[2] https://doi.org/10.1109/QoMEX.2019.8743277
paper_summary: paper_summary: In this paper, the authors claimed that they investigate the feasibility of imparting Point Cloud Quality Assessment (PCQA) knowledge to large multi-modality models (LMMs) through text supervision and proposed a PCQA pipeline named LMM-PCQA. In addition, they extracted multi-scale structural features of point cloud to helps LMM gain a more comprehensive understanding of point cloud visual quality. And experimental results affirm the effectiveness of LMM-PCQA.
strength_weakness: strength_weakness: The article provides a clear introduction to the methodology, and the corresponding experimental results substantiate the contributions mentioned in the paper.
limitations: limitations: 1. The main contribution of this paper lies in the introduction of LMMs for performing PCQA. The idea is straightforward, and the proposed pipeline in the paper simply combines structural features with the inference results of the LMMs in a loosely coupled manner, which lacks novelty.

2. The paper using the structural features of point cloud to assist the decision-making of LMMs, and the corresponding ablation experiments demonstrate that utilizing only LMMs does not achieve the best performance. These raises the question of whether the limitation lies in the inherent constraints of the large language model itself or in the methodology employed.

3. It is advised that the author considers tightly integrating the structural features within the inference process of the LMMs, rather than simply replacing traditional modules with a LMMs.
paper_summary: paper_summary: This work investigates the feasibility of imparting Point Cloud Quality Assessment (PCQA) knowledge to LMMs through text supervision. They implement this by firstly fine-tuning LLM from 2D rating logins projected from point clouds. After that, by combining LMM evaluation results and proposed multi-scale structural features, LMM-PCQA demonstrates exceptional performance across various PCQA databases.
strength_weakness: strength_weakness: * The idea of this paper is interesting, exploring the quality representation potential of LMM on 3D point clouds. 
* Experiments in this paper are well designed.
limitations: limitations: * The writing of the paper could be sometimes misleading. Some key information, such as mean opinion score (MOS) in S3.1, a qualitative measurement of point cloud is missing, how is it calculated? Furthermore, the format of text makes the essay hard to read, such as bold text, italics. For example, from L122-139, the paragraph is kinda sloppy, and with specific font in Line 135, not sure if it is deliberate to do so and why.

* The performance increase is likely small. One example is in discussion from L628 to L637, the stability analysis of algorithm across various dataset is likely less convincing. As we can see from Table 1, LMM-PCQA exhibits performance drops with decreases of approximately 0.06 and 0.08 in SRCC values when transitioning from the SJTU-PCQA to the WPC and WPC2.0 databases, while GMS-3DQA decreases 0.08 and 0.09, MM-PCQA 0.07 and 0.09 respectively. Furthermore, it is interesting to analyse why FR-Type methods (like PointSSIM, GraphSIM) shows worse robustness to handle diverse content.
paper_summary: paper_summary: The paper introduces *LMM-PCQA*, which utilizes both large multi-modal models (LLMs) and multi-scale geometry feature extraction to tackle the challenge of point cloud quality assessment (PCQA). The experimental results show the effectiveness of LMMs on this task and validate the rationality of employing the proposed modules. In general, the method is quite interesting and experiment is solid.
strength_weakness: strength_weakness: >1. The idea of employing LMM in PCQA task is the first trial in this field. The framework is well designed, which helps expand the quality assessment applications of LMMs. Aligning quality levels of the point clouds with text descriptions is interesting and novel.
>2. The multi-scale geometry feature extraction is proposed to make up the disadvantages of LMMs, which is quite suitable in this framework.

The paper is well-written and can provide inspiration for further studies.
limitations: limitations: >1. Since training LMMs is quite expensive and consumes large GPU resources, I want to know whether the authors will release the LMM weights. It is very important to the community.
>2. More details are needed in the Sec. 3.1.1. How do the authors define the camera parameters such field of view and viewing distances. Moreover, in Figure 4,  please further explain the curve tendency regarding number of the projections.
>3. It is very interesting to classify the quality levels rather than directly regression with LMMs. Is there any insight or theory behind this trial? Please explain.