Large Multi-modality Model Assisted AI-Generated Image Quality Assessment

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 09:30:46

Abstract: Traditional  deep neural network (DNN)-based image quality assessment (IQA) models leverage convolutional neural networks (CNN) or Transformer to learn the quality-aware feature representation, achieving commendable performance on natural scene images. However, when applied to AI-Generated images (AGIs), these DNN-based IQA models exhibit subpar performance. This situation is largely due to the semantic inaccuracies inherent in certain AGIs caused by uncontrollable nature of the generation process. Thus, the capability to discern semantic content becomes crucial for assessing the quality of AGIs. Traditional  DNN-based IQA models, constrained by limited parameter complexity and training data, struggle to capture complex fine-grained semantic features, making it challenging to grasp the existence and coherence of semantic content of the entire image. To address the shortfall in semantic content perception of current IQA models, we introduce a large ***M***ulti-modality model ***A***ssisted ***A***I-***G***enerated ***I***mage ***Q***uality ***A***ssessment (***MA-AGIQA***) model, which utilizes semantically informed guidance to sense semantic information and extract semantic vectors through carefully designed text prompts. Moreover, it employs a mixture of experts (MoE) structure to dynamically integrate the semantic information with the quality-aware features extracted by traditional  DNN-based IQA models. Comprehensive experiments conducted on two AI-generated content datasets, AIGCQA-20k and AGIQA-3k show that MA-AGIQA achieves state-of-the-art performance, and demonstrate its superior generalization capabilities on assessing the quality of AGIs. The code will be available.

Main paper: 
 introduction: 
 The rapid advancement of artificial intelligence (AI) has led to a proliferation of AI-generated images (AGIs) on the Internet. However, current AI-driven image generation systems often produce multiple images, necessitating manual selection by users to identify the best ones. This labor-intensive process is not only time-consuming but also a significant barrier to fully automating image processing pipelines. Visual quality, as an important factor to select attractive AGIs, has gained lots of attention in recent years [17, 20] . In this paper, we focus on how to evaluate the visual quality of AGIs, which on the one hand can be used to filter high-quality images from generation systems and on the other hand, can sever as reward function to optimize image generation models [2] , propelling progress in the field of AI-based image generation techniques.While a substantial number of deep neural network (DNN)-based image quality assessment (IQA) models, such as HyperIQA [39] , MANIQA [49] , DBCNN [55] , etc., have been developed, these models were specifically designed for and trained on natural scene images. When applied directly to AGIs, these models often exhibit poor performance. This is due to the fact that quality assessment of natural images primarily targets issues such as blur, noise, and other forms of degradation caused by photography equipment or techniques, which are not applicable to AGIs as they do not undergo such degradation during the generation process. Therefore, overemphasizing factors like blur or noise during the evaluation of AGIs is inappropriate.As shown in Figure 1 , AI-generated images, derived from advanced image generative models such as generative adversarial networks (GANs) [23] , diffision [10] and related variant [4, 6, 11, 29, [32] [33] [34] 48] , often exhibit issues not commonly found in naturally captured images. Visual quality of AGIs depends not only on basic visual features such as noise, blur [18, 38, 58] , etc., but also on more intricate semantic perception [17] , such as existence of reasonable semantic content, scene plausibility, and the coherence among objects [19, 43, 44, 46, 57] . Although re-training existing IQA models on AGIs datasets leads to improved outcomes, it fails to achieve optimal performance. One reason is that traditional DNN models, especially early convolutional neural networks (CNNs), despite their notable achievements in tasks like image recognition and classification [9, 37, 41] , still struggle to grasp the fine-grained semantic content of images [56] . What's more, traditional DNN-based IQA models fail to capture the intrinsic characteristics essential for assessing image quality and thus exhibit poor generalization abilities. Hence, we argue that the quality assessment models of AGIs are still in their infancy and need further exploration.To address the issue of semantic awareness, we resort to large multi-modality models (LMMs). Because LMMs is typically pretrained on large-scale datasets and has already learned a rich set of joint visual and language knowledge, it can effectively capture the fine-grained semantic features relevant to input prompts. However, LMMs perform excellently in high-level visual understanding tasks [1, 16] , yet they do not perform well on tasks that are relatively simple for humans, such as identifying structural and textural distortions, color differences, and geometric transformations [47] . In contrast, traditional deep learning networks excel at perceiving low-dimensional features and can fit better to the data distribution of specific task [12] . Therefore, the idea of combining LMMs with traditional deep learning networks is a natural progression.In this paper, we introduce a large Multi-modality model Assisted AI-Generated I mage Quality Assessment (MA-AGIQA) framework, which enhances the capacity of traditional DNN-based IQA models to understand semantic content by incorporating LMM. Our approach initially repurposes a DNN, MANIQA [49] , as an extractor for quality-aware features and establishes it as the training backbone for the MA-AGIQA framework. Subsequently, we guide a LMM, mPLUG-Owl2 [50] , to focus on fine-grained semantic information through meticulously crafted prompts. We then extract and store the last-layer hidden vector from mPLUG-Owl2, merging it  
related work: 
 Traditional IQA models. In the field of No-Reference Image Quality Assessment (NR-IQA) [53] , traditional models primarily fall into two categories: handcrafted feature-based and DNN-based. Models based on handcrafted features, such as BRISQUE [24] , ILNIQE [54] , and NIQE [25] , primarily utilize natural scene statistics (NSS) [24, 25] derived from natural images. These models are adept at detecting domain variations introduced by synthetic distortions, including spatial [24, 25, 54] , gradient [24] , discrete cosine transform (DCT) [36] , and wavelet-based distortions [26] . However, despite their effectiveness on datasets with type-specific distortions, these handcrafted feature-based approaches exhibit limited capabilities in modeling real-world distortions.With the advent of deep learning, CNNs have revolutionized many tasks in computer vision. [13] is pioneer in applying deep convolutional neural networks to NR-IQA. Its methodology employs CNNs to directly learn representations of image quality from raw image patches, bypassing the need for handcrafted features or a reference image. Following this, DBCNN [55] introduces a deep bilinear CNN for blind image quality assessment (BIQA) [53] , innovatively merging two CNN streams to address both synthetic and authentic image distortions separately. Furthermore, Hyper-IQA [39] , a self-adaptive hyper network, evaluates the quality of authentically distorted images through a novel three-stage process: content understanding, perception rule learning, and quality prediction.The success of Vision Transformers (ViT) [5] in various computer vision tasks has led to significant advancements. In the realm of IQA, IQT [52] leverages the combination of reference and distorted image features, extracted by CNNs, as inputs for a Transformerbased quality prediction task. MUSIQ [14] utilizes a Transformer to encode distortion image features across three scales, addressing the challenge of varying input image sizes during training and testing. TReS introduces relative ranking and self-consistency loss to capitalize on the abundant self-supervisory information available, aiming to decrease the network's sensitivity. What's more, MANIQA [49] explored multi-dimensional feature interaction, utilizing spatial and channel structural information to calculate a non-local representation of the image, enhancing the model's ability to assess image quality comprehensively.LMMs for IQA. Recent methodologies employing LMMs for IQA either utilize LMMs in isolation or combine them with DNNs as feature extractors to enhance performance. [30] introduces an innovative image-prompt fusion module, along with a specially designed quality assessment token, aiming to learn comprehensive representations for AGIs, providing insights from image-prompt alignment. However, the evaluation of AGIs in practical scenarios often does not involve prompts and image-prompt alignment is more significant for assessing the capabilities of generative models rather than images quality. CLIPIQA [42] signifies a breakthrough in assessing image quality and perception by harnessing the strengths of CLIP [31] models. This method bridges the divide between measurable image quality attributes and subjective perceptions of quality without necessitating extensive labeling efforts. Nonetheless, their [30, 42] dependence on visual-text similarity for quality score prediction often constrains their performance, rendering it marginally less effective compared to methods that exclusively focus on visual analysis. What's more, Q-Bench [43] innovates with a softmax strategy, allowing LMMs to deduce quantifiable quality scores. This is achieved by extracting results from softmax pooling on logits corresponding to five quality-related tokens. And Q-Align [45] employs strategic alignment techniques to foster accuracy. Expanding further, [47] delves into enhancing the assessment of AGIs by focusing on optimizing individual text prompts to leverage the intrinsic capabilities of LMMs, aiming to provide a more nuanced understanding and evaluation of image quality of AGIs. However, these methods, while notable, fall short of achieving satisfying efficacy, leaving considerable room for improvement. 
method: 
 As depicted in Figure 3 , framework of MA-AGIQA is structured into three sections. Section 3.1 introduces our adoption of a DNN, specifically MANIQA [49] , tailored for the AGIs quality assessment task, serving as our primary training backbone. In Section 3.2, we incorporate the LMM mPLUG-Owl2 [50] as a feature extractor. This component is crucial for acquiring fine-grained semantic features via carefully crafted text prompts. Lastly, Section 3.3 addresses the variability in focal points across different images. To adaptively integrate the feature vectors during training, we utilizes a MoE structure for feature fusion. This approach ensures that the most salient features are emphasized. Further details are elaborated below.To leverage the capability of DNNs to adapt to the data distribution of specific tasks, we employ MANIQA [49] as a quality-aware feature extractor. MANIQA enhances the evaluation of image quality by applying attention mechanisms across both the channel and spatial dimensions, thereby increasing the interaction among various regions of the image, both globally and locally. This approach generates projections 𝑤𝑒𝑖𝑔ℎ𝑡 (W ) and 𝑠𝑐𝑜𝑟𝑒 (S ) for a given image, and the final rating of the whole image is determined through the sum of multiplication of S by W , which can be illustrated as Equation ( 1) :EQUATIONwhere S and W are one dimensional vectors. However, directly applying MANIQA to the quality assessment of AGIs presents challenges, as illustrated in Figure 4 . Image (a) displays a complex, symmetrical pattern, devoid of meaningful semantic content. Image (b) features incoherent areas, such as two grey holes in the sky that are inconsistent with the common sense. The blurriness and fuzziness of the man's face in image (c) along the edges significantly impair human perception. Conversely, image (d), despite its severe graininess, retains its semantic integrity, To address this issue, modifications were made so that the generated S and W no longer produce a rating. Instead, they yield a quality-aware feature f 1 , setting the stage for the subsequent fusion with features extracted by LMM. f 1 is generated as:EQUATION)During the training phase, the parameters of modified MANIQA are continuously updated. This refinement process ensures that MANIQA can extract features more relevant to the quality of AGIs. Furthermore, the training process facilitates a more seamless integration between MANIQA and LMM, leading to superior outcomes.LMMs are capable of understanding and analyzing the semantic content of images and their relationship with human cognition. They assess whether different parts of an image form a cohesive whole and evaluate whether the elements within the picture are semantically coherent [7, 21, 28] . mPLUG-Owl2 [50] employs a modality-adaptive language decoder to handle different modalities within distinct modules, which mitigates the issue of modality interference. Given the importance of effectively guiding the model through textual prompts to elicit the desired output, we have selected mPLUG-Owl2 as our feature extractor.We consider the application of mPLUG-Owl2 in the following aspects of semantic content:• Existence of Semantic Content. The importance of semantic content in an image lies in its ability to convey a clear and meaningful message to the viewer. An image lacking in semantic content may be difficult to understand, fail to effectively convey its intended message, reducing audience engagement and satisfaction. • Coherence of Semantic Content. The coherence of semantic content in an image relates to whether the generated image can provide a coherent, logically sound visual experience for human viewers. When the various parts of an image are semantically consistent, it is better able to convey a clear story, emotion, or message. In contrast, any inconsistency in the primary focus of images will greatly detract from their quality and convey a significantly negative impression.Consequently, we try to propose the rational design of prompts leading LMMs to obtain those image semantic content. mPLUG-Owl2 possess the ability to understand fine-grained semantic contents, but without carefully designed input prompts, some prompts, such as "Please evaluate if the image quality is compromised due to violations of common human sense or logic?" although it expresses the desire for the model to assess whether the semantic content of the image contradicts human perception, would lead to unsatisfactory results. To better utilize mPLUG-Owl2 for the task of evaluating AGIs, we meticulously designed prompts to guide the LMM. Specifically, we designed two prompts, denoted as 𝑝𝑟𝑜𝑚𝑝𝑡 𝑎 and 𝑝𝑟𝑜𝑚𝑝𝑡 𝑏 respctively,• "Evaluate the input image to determine if its quality is compromised due to a lack of meaningful semantic content. " • "Evaluate if the image quality is compromised due to violations of coherence. "corresponding to the existence of semantic content and the coherence of semantic content in images, respectively. Test results, as shown in Figure 5 using the mPLUG-Owl2 official demo 1 , have proven these questions to be effective. However, the textual output from mPLUG-Owl2 is not immediately conducive to being utilized by MANIQA to impart semantic insights. To bridge this gap, it's essential to obtain the information provided by mPLUG-Owl2 into a format that MANIQA can easily leverage. SO we extract features from the final layer of mPLUG-Owl2's hidden layers, achieving an accessible embedded representation of the LMM's output. This output is a tensor with dimensions of [token_length, hidden_size], where "token_length" represents the number of output tokens, and "hidden_size" denotes the dimensionality of the hidden layer representations associated with each token. For mPLUG-Owl2, the hidden_size is set to 4096. Subsequently, we conduct an averaging operation across the token dimension, yielding a vector with dimensions 1x4096. This vector 1 https://modelscope.cn/studios/iic/mPLUG-Owl2/summary Evaluate if the image quality is compromised due to violations of coherence.The image features a blurry cityscape with a ghostly appearance, but the blurriness and the presence of ghostly elements make it difficult to discern the actual details of the cityscape. The image is not clear, and the ghostly appearance might be distracting or confusing for the viewer.Evaluate the input image to determine if its quality is compromised due to a lack of meaningful semantic content.The input image is a close-up of a patterned surface, possibly a tiled floor or a decorative piece, The pattern is intricate and features a combination of blue, green, and yellow colors. However, the image lacks meaningful semantic content, as it does not depict any recognizable objects, people, or scenes.The focus is solely on the pattern, which might be visually appealing, but does not provide any context or information. then serves as the basis for further feature fusion procedures. The process can be represented as Equation ( 3) :EQUATIONwhere m 𝑘 𝑖 represents a hidden vector of token 𝑘 corresponding to 𝑝𝑟𝑜𝑚𝑝𝑡 𝑖 , and M denotes mPLUG-Owl2.It is important to note that throughout the entire training and testing process, the parameters of mPLUG-Owl2 are fixed. Because mPLUG-Owl2 is typically pre-trained on large-scale datasets and has already learned a rich set of joint visual and language knowledge, it can effectively capture the fine-grained semantic information relevant to input prompts, even with fixed parameters. Additionally, fine-tuning LMMs in every training iteration would significantly increase training time. Using it solely as a feature extractor significantly reduces computational costs, making the training process more efficient. So, we pre-obtain and save the semantic content features of each image in advance.Given the complex influence of color, composition, details, semantic content, and other factors on image quality, simply concatenating the extracted features may not always yield the best results. To dynamically fuse a variety of complementary features, we propose the adaptive fusion module (AFM) for organic feature integration. This process can be divided into two main parts. The first part involves transforming the extracted features into a unified vector space of the same dimension, allowing for vector fusion operations. Specifically, for features extracted by MANIQA, this transformation block applies a fully connected (Fc) layer, transforming them to the same dimension as the original features (1x784) to provide a richer combination. For features derived from mPLUG-Owl2, it uses a Fc layer to project them onto a 1x784 dimension, followed by a relu activation layer and a dropout layer to enhance the network's expressive power and generalization. The second part employs a MoE to dynamically fuse the three features. The MoE's gating network takes the transformed three features as input and outputs dynamic weights 𝜶 , corresponding to the three features' contributions to image quality. Structurally, this gating network comprises a Fc layer and a sigmoid layer. The final image quality representation vector g can be obtained through a weighted sum of the three feature vectors. Following the denotation which sign the three features as 𝑓 1 , 𝑓 𝑎 , 𝑓 𝑏 , this process can be represented as:EQUATIONwhere F 𝑡𝑟𝑎𝑛𝑠 𝑖 is the transformation block of feature 𝑖, and F 𝑔𝑎𝑡𝑒 is the gating network's mapping function, 𝑓 𝑖 is the original extracted feature and 𝑓 ′ 𝑖 is the transformed feature. R 𝑑 is the dimension space of 𝑓 ′ 𝑖 . Finally, we obtain the final image quality score output through a simple regression layer, consisting of a Fc layer. 
experiments 4.1 dataset and evaluation metrics: 
 Dataset. Our model is evaluated on two AI-Generated image datasets, including AIGCQA-20k [17] and AGIQA-3k [20] . Specifically, AIGCQA-20k contains 20k images, but at the time of writing, only 14k images have been published. Our experiments are conducted on these 14k images. The MOS for AIGCQA-20k images are distributed between 0-5, with higher scores indicating better image quality. Images in AIGCQA-20k are generated by 15 models, including DALLE2 [32] , DALLE3 [32] , Dream [6], IF [4] , LCM Pixart [22] , LCM SD1.5 [22] , LCM SDXL [22] , Midjourney [11] , Pixart 𝛼 [3] , Playground [29] , SD1.4 [34] , SD1.5 [34] , SDXL [35] and SSD1B [8] . AGIQA-3k includes 2982 images, with MOS also distributed between 0-5, where higher values represent better quality. Images in AGIQA-3k are derived from six models, including GLIDE [27] , Stable Diffusion V-1.5 [34] , Stable Diffusion XL-2.2 [35] , Midjourney [11] , AttnGAN [48] , and DALLE2 [32] . During training, we split the entire dataset into 70% for training, 10% for validation, and 20% for testing. To ensure the same set of images in each subset when testing across different models, we set the same random seed during the split to control variables and ensure reproducibility.Evaluation Metric. Spearman's Rank-Order Correlation Coefficient (SRCC), Pearson's Linear Correlation Coefficient (PLCC), the Kullback-Leibler Correlation Coefficient (KLCC), and the Root Mean Square Error (RMSE) are selected as metrics to measure monotonicity and accuracy. SRCC, PLCC, and KLCC range from -1.0 to 1.0, with larger values indicating better results. In our experiments, we employ the sum of SRCC and PLCC as the criterion for selecting the optimal validation case, and emphasize SRCC for comparing model performance.Our method is implemented based on PyTorch, and all experiments are conducted on 4 NVIDIA 3090 GPUs. For all datasets, we opt for handcrafted feature-based BRISQUE [24] , NIQE [25] and IL-NIQE [54] , deep learning (DL)-based HyperIQA [39] , MANIQA [49] , MUSIQ [14] , DBCNN [55] , StairIQA [40] , BAID [51] , and LMMbased CLIPIQA [42] , CLIPIQA+ [42] and Q-Align [45] . During the training process of deep learning models, we use the Adam optimizer [15] with a weight decay of 1e-5, and the initial learning rate is 1e-5. The batch size is 8 during training, validation, and testing. All DL-based models are trained for 30 epochs using MSE loss and validated after each training process. The checkpoint with the highest sum of SRCC and PLCC during validation is used for testing. Handcrafted feature-based and LMM based models are used directly without training.Table 1 lists the results of MA-AGIQA and 12 other models on the AGIQA-3k and AIGCQA-20k dataset. It has been observed that LMM-based models significantly outperform those that rely on handcrafted features. This superior performance is attributed to LMMs being trained on extensive datasets, which provides them with a robust understanding of images and enhances their generalizability. However, trained DL-based models generally perform far better than the LMM-based models because DL-based models tend to fit the data distribution of specific tasks better, thereby resulting in improved performance. Among these twelve models, the ViT-based MANIQA outperforms the other eleven models, and our method still significantly surpasses it on the same training and testing split with large margins (+3.72% of SRCC, +1.73% of PLCC and +5.43% of KRCC in AGIQA-3k & +1.61% of SRCC, +2.02% of PLCC and +2.90% of KRCC in AIGCQA-20k). This demonstrates the superiority of integrating features extracted by LMM into traditional DNN, significantly improving the accuracy and consistency of prediction results.To evaluate the generalization capability of our MA-AGIQA model, we conducted cross-dataset evaluations. Table 2 shows that MA-AGIQA significantly outperforms the other two models, Hyper-IQA and StairIQA, which performed best on single datasets, with large margins. This superior performance can largely be attributed to the robust generalization capability of the LMM and the benefits of the MoE architecture, which excels in dynamically fusing features.Table 1 : Comparisons with SOTA (State-Of-The-Art) methods on AGIQA-3k and AIGCQA-20K-Image datasets. The up arrow "↑" means that a larger value indicates better performance. The best and second best performances are bolded and underlined, respectively. MA-AGIQA outperforms existing SOTA methods on both datasets by large margins. Note: to ensure fair comparisons, we trained and tested all deep learning based models and ours with the same dataset splitting method. Necessity of Fine-grained Semantic Features. To assess the benefits of integrating features extracted by mPLUG-Owl2 [50] into MANIQA [49] , we carried out comprehensive ablation studies on each component and their various combinations, as detailed in Tables 3 and 4 . Our findings indicate that using either the features extracted by the LMM alone or solely relying on a traditional network does not yield the best outcomes. In contrast, integrating one fine-grained semantic feature with the original MANIQA network can enhance the network's performance. However, the optimal results were achieved by combining two features extracted by the LMM with MANIQA, which led to significant improvements on the AGIQA-3k dataset (increases of 1.57%, 0.83%, and 2.56% in SRCC, PLCC, and KRCC, respectively) and on the AIGCQA-20k dataset (enhancements of 2.72%, 1.94%, and 4.35%). The marked enhancements achieved by incorporating two finegrained semantic features suggest that LMM is adept at capturing 5 demonstrates that incorporating the MoE structure, rather than simply concatenating three vectors, does indeed improve network performance, albeit marginally. Specifically, on the AGIQA-3k dataset, we observed increases of 0.20%, 0.17%, and 0.16% in SRCC, PLCC, and KRCC, respectively. For the AIGCQA-20k dataset, the improvements were 0.67%, 0.95%, and 1.37%. The gains, although seemingly modest, highlight the potential of MoE structure in complex systems where integrating diverse expertise can yield better decision-making and predictive outcomes. To vividly demonstrate the efficacy of the MA-AGIQA framework, we selected 300 images from the AIGCQA-20k and AGIQA-3k datasets where MANIQA had the poorest performance. These images primarily exhibit issues in semantic content. We computed the absolute values of the differences between the model scores and the image ground truth, and illustrated these differences in Figure 6 , using 0. Figure 7 presents a collection of images where the assessments from the MANIQA model were mostly off the mark. Scores assigned by MANIQA alongside those given by the proposed MA-AGIQA model and the ground truth are listed, which reveal that the MA-AGIQA model markedly enhances alignment with the ground truth in contrast to MANIQA. For instance, in the first image of the top row, MANIQA's score is 3.50, which diverging substantially from the ground truth score of 1.50. However, MA-AGIQA's score is 2.98, demonstrating a much closer approximation to the ground truth. This pattern is consistent across the images shown, with MA-AGIQA consistently producing scores that are closer to the ground truth, reflecting a more accurate assessment of image quality. 
conclusion: 
 To mitigate the shortcomings of traditional DNNs in capturing semantic content in AGIs, this study explored the integration of LMMs with traditional DNNs and introduced the MA-AGIQA network. Leveraging mPLUG-Owl2 [50] , our network efficiently extracts semantic features to enhance MANIQA [49] for quality assessment. The MA-AGIQA network's ability to dynamically integrate finegrained semantic features with quality-aware features enables it to effectively handle the varied quality aspects of AGIs. Experiment results across two prominent AGIs datasets confirm our model's superior performance. Through thorough ablation studies, the indispensable role of each component within our framework has been validated. This research aspires to catalyze further exploration into the fusion of LMMs within AI-generated content quality assessment and envisions broader application potentials for such methodology. 
coherence semantics: 
 No semantic meaningsNote: The holes in the sky and blurry upon the building contradict common sense.Note: Sharks should have eyes but it not always the case in AI generated images.Note: Images generated with complex, symmetrical pattern but lack meaningful semantic content.Note: From the perspective of noise, the details in this image are grainy. However, as a whole, the image still clearly presents the semantics of an orange kitten, giving people a hazy and artistic feeling Note: The image has a lot of blurriness, and the most crucial facial area has become completely unrecognizable, which severely affects the viewing experience. The blurriness in this image makes the overall impression quite disharmonious.Figure 1 : The quality of AI-generated images is greatly influenced by the semantic content, i.e., the coherence and existence of semantic content. The coherence of a picture's semantic content is crucial for providing a logically sound visual experience for human viewers, and images lacking semantic content fail to effectively communicate their intended design, reducing viewer engagement and satisfaction. This article primarily focuses on "Contrary to common sense" (CCS), "Noise image" (NI) and "No semantic meanings" (NSM). We categorize "CCS" and "NI" under the coherence of semantics, while "NSM" falls under the existence of semantics. 
 
 Appendix:  


Decision: The final ratings are 2 weak accept and 1 accept. The reviewers acknowledged novelty and good performance. Initial questions were addressed through the rebuttal.

Reviews: review: paper_summary: paper_summary: Compared to general (real) image quality assessment, AI-generated image quality assessment is more complex due to unexpected components (contents) in images. The authors show that existing methods of IQA on AGI datasets have weaknesses and analyze them. They apply LMM models to address this problem and achieve high performance compared to existing methods.
strength_weakness: strength_weakness: [S1] The proposed method is novel with logical grounds. They analyze why the traditional DNN-based models cannot overcome the performance that the models can achieve in real-image quality assessment. Based on the analysis, they conclude that using LMM models is beneficial to IQA on AGI datasets.

[S2] The paper is well-written and easy to understand.
limitations: limitations: [W1] The traditional DNN-based methods might have weaknesses when it comes to AGI, and therefore, introducing LMM seems like a good idea. However, the analysis of these weaknesses is not convincing. The evidence presented relies solely on Fig. 4, which does not adequately represent the overall trends in the AGI dataset. The authors need to demonstrate that this is a universally observable trend. For instance, they could show how many mis-predicted samples among N sampled images are distributed across a, b, c, and d in Fig. 4.

[W2] (minor) Using LMM in conjunction with traditional methods is likely to require more computational resources, and this should be reported when comparing performance. 

[W3] (minor) Evaluating the quality of generated images is a crucial part of developing generative AI models. While it is common to evaluate the generated image dataset as a whole, recent techniques have been developed to assess individual generated images[1,2,3]. It would be beneficial to discuss how IQA on AGIs dataset fundamentally operates differently from these methods.

[1] Improved precision and recall metric for assessing generative models, NeurIPS 2019   
[2] Rarity score : A new metric to evaluate the uncommonness of synthesized images, ICLR 2023   
[3] Anomaly Score: Evaluating Generative Models and Individual Generated Images based on Complexity and Vulnerability, CVPR 2024
paper_summary: paper_summary: This paper proposes a quality assessment method for AI-generated images, which combines a visual language model with an IQA model. Experimental results show that the method achieves good performance.
strength_weakness: strength_weakness: The method is designed with prompts for judging semantic information, thus solving the problem that the model has difficulty in extracting semantic information of AI-generated images. Specifically, coarse-grained semantic features are extracted using a visual language model and fine-grained distorted features are extracted using a traditional IQA model. This approach combines the advantages of both to achieve excellent results.
limitations: limitations: 1. Cross-database experiments should be compared with AGIQA method instead of traditional IQA model.
2. There are fewer experiments in the paper. Performance comparison of traditional IQA benchmark databases can be added.
paper_summary: paper_summary: This paper addresses the challenge of assessing the quality of AI-generated images (AGIs), which traditional deep neural network (DNN) models struggle with due to their inability to handle semantic inaccuracies typical of AGIs. To overcome these limitations, the authors introduce a novel model named MA-AGIQA that integrates large multi-modality models (LMMs) with traditional DNNs to enhance the assessment of semantic content within images. The paper's achievements are demonstrated through comprehensive experiments, where MA-AGIQA outperforms existing state-of-the-art models by a large margin, showcasing improved generalization capabilities and a deeper understanding of the quality-related aspects of AGIs.
strength_weakness: strength_weakness: 1.Identifying the limitations of traditional DNN-based IQA models in assessing AGIs, particularly their inability to recognize and interpret semantic content.
2.Introducing an innovative approach, the MA-AGIQA model that incorporates LMMs for extracting fine-grained semantic features and dynamically integrates these with traditional DNN features to improve quality assessment.
3.Demonstrating through extensive experiments on two AI-generated images datasets that MA-AGIQA achieves state-of-the-art performance, significantly improving the evaluation of image quality in AGIs compared to existing models.
4.Opening future research directions that integrate LMMs for semantic analysis in image quality and encouraging further exploration into how these models can be optimized and adapted for other types of multimedia content beyond AI-generated images.
5. The paper's focus on improving image quality assessment for AI-generated images is highly relevant to the ACM MM community, including aspects of visual media analysis, semantic content interpretation, and the integration of multimodal data for enhanced computational understanding.
limitations: limitations: There are some minor problems, which need to be solved before publication. If the following problems are well-addressed, this reviewer believes that the essential contribution of this paper is important for IQA.

1.Ensure that appropriate references are added at relevant sections of the paper, such as in the 'Related Work' where TReS is discussed.
2.Carefully review the text for any grammatical and spelling errors to maintain the professionalism and readability of the paper.