A Chinese Multimodal Social Video Dataset for Controversy Detection

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-09 03:40:19

Abstract: Social video platforms have emerged as significant channels for information dissemination, facilitating lively public discussions that often give rise to controversies. However, existing approaches to controversy detection primarily focus on textual features, which raises three key concerns: it underutilizes the potential of visual information available on social media platforms; it is ineffective when faced with incomplete or absent textual information; and the existing datasets fail to adequately address the need for comprehensive multimodal resources on social media platforms. To address these challenges, we construct a large-scale Multimodal Controversial Dataset (MMCD) in Chinese. Additionally, we propose a novel framework named Multi-view Controversy Detection (MVCD) to effectively model controversies from multiple perspectives. Through extensive experiments using state-of-the-art models on the MMCD, we demonstrate MVCD's effectiveness and potential impact.

Main paper: 
 introduction: 
 With the prevalence of social video platforms, videos have become an important information-sharing channel. Videos uploaded on social media platforms quickly accumulate thousands of views within seconds, facilitating worldwide user engagement in opinion shaping [40] . However, the openness of these social platforms also gives rise to fervent discussions and the exchange of divergent opinions. Consequently, the proliferation of numerous videos necessitates implementing risk management and control measures. Our work focuses on controversy detection, which serves as the basis for exploring various advanced applications such as risk indication and brand reputation management. Previous research on controversy detection has primarily focused on the textual modality, neglecting the necessity of incorporating multimodal in situations where textual information is limited. In light of the ubiquity of social video platforms, we specifically concentrate on multimodal controversy detection. This involves the utilization of various modalities, including video, text, and metadata, to discern the controversy inherent in content.Controversy exists as a form of public discourse, attracting an increasing number of opposing viewpoints and resulting in escalating divergence or polarization [10, 16, 36, 45] . As controversies stem from the beliefs and values of participants, the exchange of opinions goes beyond mere "facts" and evokes intense emotions [28, 37] . Certain individuals may find their rights infringed upon due to controversial videos, primarily manifested through verbal accusations from others or conflicts and dissatisfaction arising from them. While being under the spotlight, social media environment can potentially cause psychological harm to these individuals, and in extreme cases, even lead to radical actions. In particular, when addressing controversial videos that discuss policies related to specific interests or incite intense discussions on social media platforms, there exists an inherent potential for the emergence of broader public controversies in the future [12, 37] . Hence, controversy detection on social video platforms plays a crucial role in providing an indicator for assessing the contentious nature of videos. It is necessary to curb the spread of controversies and mitigate public opinion risks. Additionally, controversy detection can generate recommendations that promote a "healthier diet" on social media [21] .Inspired by previous research work [10, 16, 30, 36, 45] , we define controversial videos on social media platforms by considering three aspects. The first aspect concerns whether the video content itself is prone to controversy, such as whether it contains sensationalism, violent information, and so on. The second aspect examines a conflict between the video content and the users' comments. Controversial videos are identified if the comments exhibit opposing viewpoints to the video, personal attacks against the video creator, criticism of depicted phenomena, or criticism/questioning of individuals/objects featured in the video. The third aspect focuses on the controversy within comments, specifically looking for clear opposing viewpoints expressed through support and opposition. Figure 1 provides an example of a controversial video and a noncontroversial one. Figure 1 (a) depicts a video showing a woman pushing a baby in a stroller, displaying an apparent lack of experience in child care. The comments associated with the video express suspicion and criticism towards the individual portrayed, evoking intense emotions and indicating a certain level of controversy. On the other hand, Figure 1 (b) showcases a comedic video featuring a Yiwu Mountain parrot, wherein the comments applaud the parrot's performance, conveying a more positive sentiment without any controversial elements.Existing approaches for detecting controversy on social media have primarily focused on leveraging semantic and structural features of target posts and their comments. However, there are three critical concerns. First, they overlook the potential of utilizing visual features available on social media platforms. Second, the current models for controversy detection often underperform when confronted with incomplete or missing text. Lastly, existing datasets offer a limited number of instances and lack simultaneous information on video, text, and user profiles [2, 31, 54] .To bridge this gap in multimodal controversy detection datasets, we introduce Multimodal Controversy Detection Dataset (MMCD), a large-scale dataset in Chinese that encompasses video content and rich social context. The MMCD dataset offers abundant features, providing an opportunity to evaluate various approaches for controversy detection and facilitate a deeper understanding of controversy dissemination and potential interventions. To comprehensively analyze the characteristics of controversial videos, we conduct an exploratory analysis of MMCD from multiple perspectives, offering valuable insights into effective detection strategies.Furthermore, to address the challenges associated with multimodal controversy detection, we propose Multi-view Controversy Detection (MVCD) framework and conduct extensive experiments to compare with existing methods on MMCD. Experimental results validate the superiority of our proposed framework. Additionally, ablation experiment results validate the effectiveness of each individual module and modality within the framework. Moreover, early predictions indicate that our proposed framework is capable of handling scenarios with limited availability of comments.Our main contributions are summarized as follows:• We have developed and released MMCD, a Multimodal Controversy Dataset in Chinese, providing a valuable resource for studying controversies. This dataset is derived from social video platforms and includes a wide range of video content accompanied by extensive social context. • We have conducted a comprehensive analysis of the constructed dataset, providing insights and findings relevant to further research. • We have devised a multimodal controversy detection framework, Multi-view Controversy Detection (MVCD), which effectively models multimodal video content and captures the interaction between social contexts, enhancing the accuracy of controversy detection.• Extensive experiments using state-of-the-art methods have been conducted on the MMCD, showcasing the effectiveness of our proposed approach and shedding light on the inherent challenges associated with multimodal controversy detection. To facilitate further research, we have made our work publicly available, including the codebase1 . 
related work 2.1 controversy detection datasets: 
 Currently, controversial detection datasets focus primarily on the textual modality. Table 1 provides a comprehensive overview of available datasets, covering aspects such as feature, category, language, accessibility, source, and time span information. These datasets are largely derived from three primary sources: web pages, with a particular focus on Wikipedia [17, 31] ; news websites, such as The Guardian2 , EMOL3 , and Toutiao4 , which have contributed valuable datasets in this domain [2, 30, 39] ; and social media platforms including Twitter, Weibo, Reddit, and others, which have also served as substantial sources of controversy detection datasets [9, 11, 13, 48, 54] . However, we identify a noteworthy gap in the availability of datasets specifically tailored for social video platforms. These play a crucial role in information dissemination and are frequent generators of controversies [16, 21, 40] . Consequently, we propose the collection of multimodal data as an effort to bridge this gap and provide necessary resources for detecting controversies in social videos.Early methods for controversy detection primarily relied on statisticbased approaches, which involved analyzing user edit history [47] , revision time [26] , and context information [18, 49, 53] . Furthermore, some researchers incorporated textural features, such as controversial vocabulary [14] , sentiment [27, 32, 38] , writing style [27] , and combination statistical features [22, 41] . Recently, the focus gradually shifted towards end-to-end approaches without explicitly relying on specific features [43, 48] . Notably, utilizing Graph Neural Network (GNN) to capture structural relationships gained popularity [4, 39] . These approaches mainly include modeling users' relationship [5, 11] , modeling the relationship between topics and comments [3, 30, 54] , and examining controversies through introducing entities and polarity [39] , and so on. In addition, researchers have also explored early comments to predict controversy [23] .More recently, with advancements in Pretrained Language Models, they have gradually been employed in controversy detection and other related tasks [7, 9, 46] . However, the aforementioned methods for controversy detection are mostly limited to textual modality. To the best of our knowledge, there have been no approaches developed for multimodal controversy detection thus far.  
the mmcd dataset: 
 To fill the existing void of publicly available datasets, we introduce the Multimodal Controversy Dataset (MMCD). This dataset comprises over 10,000 Chinese videos, each accompanied by a wealth of social context information. Our intention in creating this dataset is to offer researchers an invaluable resource for studying multimodal controversy detection. By providing MMCD, we enable the development and evaluation of innovative approaches in this area of research.We collected raw videos from Douyin5 , a popular Chinese social video platform, known for its vast user base of millions of active participants. To obtain a comprehensive set of controversial videos and establish reliable ground truth labels for controversy, we manually formed a set of 139 keywords (listed in supplementary material) based on the popular news rankings on Weibo. Leveraging these query keywords, we searched for videos and crawled relevant content.Our data collection primarily involves crawling for video content, metadata, publishers' profiles, and comments context. The video content category encompasses fundamental attributes including videos' IDs, publication timestamps, descriptions, URLs, and lengths; metadata includes metrics including the number of likes, shares, and comments; publishers' profile contains relevant information about the publisher, providing insights into their characteristics. Additionally, the comment category provides details regarding user comments associated with the videos. Notably, the comments are sorted by the Douyin platform, mostly considering factors such as the comments' popularity and timestamp, and we selected the Top 40 from them. To ensure data quality, we implemented a filtering process to exclude aberrant unplayable videos. As a result, we collected a total of 18,623 Chinese videos released between Dec. 2017 and Dec. 2023.We implemented a meticulous manual annotation process. During the selection of annotators, we prioritize maximizing demographic diversity and including individuals from various cultural backgrounds. Our group of 25 annotators consists of 2 Ph.D. students, 3 graduate students, and 20 undergraduate students from 5 departments at our university. Among the annotators, 11 identified as women and 14 as men, with ages ranging from 18 to 30.Annotating multimodal data presents additional challenges compared to annotating textual data, primarily due to the requirement for annotators to watch lengthy videos. To address this challenge and ensure consistent annotation quality across all videos, annotators were provided with explicit instructions through a comprehensive guideline prepared by us. These guidelines instruct annotators to evaluate the level of controversy in three aspects: (1) the controversy within the video itself, (2) the controversy between the video and its comments, and (3) the controversy among the comments. Based on these assessments, annotators classified each video as either controversial or non-controversial. We developed a website for the annotation process. To ensure reliability, each video was annotated by three annotators. The annotation process resulted in substantial consensus among the annotators, with a Kappa value of 0.78 indicating significant agreement. Following the annotation process, we obtained a dataset consisting of 5,643 controversial videos and 11,164 non-controversial videos.To gain insights into the distinctive characteristics of controversial and non-controversial videos, we conducted an exploratory analysis of the collected dataset from three perspectives: data distribution, indicators statistics, and sentiment analysis. These analyses aim to provide valuable insights into the underlying behaviors and patterns associated with these two types of videos, thereby contributing to controversy detection.Data Distribution. The MMCD Dataset is categorized into 14 domains using DBpedia [1] and manual judgment (details in supplementary materials). Figure 2 dataset. Among these domains, the "Life" exhibits the least degree of controversy in its data distribution. This can be attributed to the close connection between "Life" and everyday experiences, which enables individuals to have a deeper understanding of this domain.On the other hand, the "Traffic" domain exhibits the highest degree of controversy in its data distribution. This can be primarily attributed to the complex nature of traffic conditions, making it difficult to determine responsibility for accidents. Moreover, conflicts arising from conflicting interests among different parties further contribute to the elevated controversy surrounding this domain. Indicators Statistics. We extensively analyzed various video content indicators, as presented in Table 2 . These indicators include the average length of videos, the number of forwards, likes and comments, as well as the number of videos & likes associated with publishers. Based on the statistical findings, we observed the following patterns. First, regarding video length, it was observed that controversial videos tend to be longer on average. This suggests that controversial videos likely require more substantial content to stimulate user discussions. Second, it was noted that controversial videos garnered a higher number of forwards, likes, and comments compared to non-controversial videos. This implies that controversial videos have a greater propensity to attract attention and engagement from viewers. Additionally, publishers of controversial videos tend to have more videos, likes, and followers, which implies that individuals who disseminate controversial content tend to be more active.Figure 3 depicts the distribution of video counts and corresponding likes by publisher. An intriguing observation arises from the graph, where the distribution of likes and video counts for publishers of controversial videos exhibits a higher degree of scattering, suggesting diverse behavior among them. This observation may suggest two potential explanations: 1) publishers with a low video count but a high number of likes may represent specialized marketing accounts that attract viewers with sensational content; 2) publishers with a high video count but a low number of likes could potentially be engaged in bot-like behavior, actively spreading controversial videos.Sentiment Analysis. We conducted sentiment analysis on the textual data present in the crawled dataset, including video descriptions, comments, and ASR text. To perform a fine-grained analysis, we utilized the CNsenti tool [51] , which covers seven emotion categories: good, happy, sad, angry, fearful, disgusted, and shocked. Figure 4 illustrates the average scores of these fine-grained emotions. Upon comparison, we observed that controversial videos exhibited higher scores for emotions like "angry" and "disgusted", while non-controversial videos received higher scores for the emotions of "good", "happy", and "shocked". 
method: 
 We propose the Multi-view Controversy Detection (MVCD) framework, which integrates various modules for detecting controversial videos. Figure 5 MVCD aims to detect whether a given video and its associated content is controversial. Formally, let T denote an input sample consisting of six elements: video keyframes V, description D, comments C, publisher profiles P, ASR text R, and ground truth label 𝑦. The length of these elements are representes as 𝑛 𝑣 , 𝑛 𝑑 , 𝑛 𝑐 , 𝑛 𝑝 , and 𝑛 𝑟 , respectively. Our objective is to develop a multi-modal controversy detection model F:EQUATIONwhere ŷ denotes the binary classification prediction result for a given sample obtained by model F, and Θ represents the set of all parameters associated with the model.To effectively capture the diverse characteristics present in various modalities, we leverage a pretrained Chinese model CN-CLIP [52] , which is trained on large-scale image-text pairs. Video. To extract frame-level features, we first utilize the ffmpeg tool 6 to extract keyframes from each video. These keyframes are then passed through the image encoder. The resulting encoded video representation is denoted asX 𝑣 = [x 𝑣 1 , ..., x 𝑣 𝑛 𝑣 ], where x 𝑣 𝑖 ∈ R 𝑑 𝑣 represents the feature vector extracted from the 𝑖-th keyframe, 𝑛 𝑣 signifies the total number of keyframes in the video, and 𝑑 𝑣 denotes the dimension of the image encoded by Vision Transformer (ViT) [19] in CN-CLIP.Text. We employ RoBERTa [33] for text feature extraction in CN-CLIP. The features extracted from the text elements (description, publisher's profile, ASR text) are represented as x 𝑑 , x 𝑝 , x 𝑟 ∈ R 𝑑 𝑡 , where 𝑑 𝑡 represents the dimension of the encoded text sequences. Specifically, for publisher profiles, we concatenate attributes such 6 https://git.ffmpeg.org/ffmpeg.git as nickname, personal introduction, verification information, the number of videos and likes. As for comments, The features are represented as X 𝑐 = [x 𝑐 1 , ..., x 𝑐 𝑛 𝑐 ], where x 𝑐 𝑖 ∈ R 𝑑 𝑡 represents the feature vector of the 𝑖-th comment within X 𝑐 , and 𝑛 𝑐 denotes the total number of comments.Features extracted from video keyframes, descriptions, publishers' profiles, and ASR text are passed through individual Fully Connected layers (FC) to align multimodal features. These aligned features are then concatenated and input into a single Transformer Layer (TL) to capture temporal information. The calculations involved in this process are as follows:EQUATIONEQUATIONwhere W 𝑣 , W 𝑑 , W 𝑝 , and W 𝑟 denote the weight parameters, while b 𝑣 , b 𝑑 , b 𝑝 , and b 𝑟 represent the corresponding bias parameters.Considering the diverse impacts of different modalities on the perception of varied audiences, it is imperative to tackle the challenges arising from inconsistent and insufficient attention given to these modalities. To overcome these challenges, we utilize the Mixture of Expert (MoE) architecture [44] to enhance the overall modeling performance. The MoE layer consists of a set of 𝑚 expert networks denoted as E(•), along with a gating network referred to as G(•). The output feature H 𝑜 from the previous step is fed into the gating network G(•) and expert network E(•) to gain the output:EQUATIONEQUATIONwhere W 𝑚𝑜𝑒 denotes learnable parameters, KeepTopK [44] is a function to select Top 𝑘 highest gate values given input feature H 𝑜 , and 𝑞 denotes the ordinal position of the expert network.To effectively capture the controversy present between videos and comments, we employ the Graph Convolutional Network (GCN) [25] to capture the semantic and structural relationships that exist between these two modalities. The comments feature X 𝑐 is aligned with X 𝑣 to establish a unified feature:EQUATIONwhere W 𝑐 denotes the weight parameter, and b 𝑐 represents the bias parameter. We then construct a Video-Context graph denoted as G = (V, E) for each video, where the set of nodes V consist of features of video or comment, and an edge exists between a video and its corresponding comments when the comment is associated with that specific video. The initial representations of the nodes can be defined as:EQUATIONDuring the message-passing process, each node updates its representation based on the aggregated information obtained from its neighboring nodes and its features. This allows the learned representation to encompass valuable insights from both the content and structure of the graph. Specifically, for a given node v 𝑖 ∈ V (0) , the update rule can be expressed as:EQUATIONwhere v(𝑙 )𝑖 represents the hidden state of node 𝑣 𝑖 in the 𝑙-th layer of GCN, 𝜎 denotes the Rectified Linear Unit (ReLU) activation function, N 𝑖 denotes the neighbors of node 𝑣 𝑖 (including the node itself), g(•) is the aggregation function, and b (𝑙 ) represents the bias term.At the layer level, we use the embedding vectors V (0) as input to a two-layer GCN, resulting in a condensed representation denoted as V (2) . Incoming messages from the neighbor set N 𝑖 are aggregated by g(•), which is implemented as a linear function. Thus, for the 𝑙-th layer, the propagation rule is given by:EQUATIONwhere V (𝑙 ) contains all node vectors in the 𝑙-th layer, Â is the normalized adjacency matrix, W (𝑙 ) is the weight matrix. At last, we employ MaxPooling to extract the most significant features for subsequent calculations:EQUATIONGiven the intrinsic inconsistencies observed in controversial comments, which often include discrepancies in content and sentiment, we propose utilizing an affective matrix and semantic attention matrix [35] to capture and model these inconsistencies. Context Affective Computing. To capture and analyze the affective inconsistencies in comments, we construct an affective matrix A ∈ 𝑅 𝑛 𝑐 * 𝑛 𝑐 based on a set of comments C = {𝑐 1 , 𝑐 2 , ..., 𝑐 𝑛 𝑐 }. Each element 𝑎 𝑖,𝑗 in A is computed by:EQUATIONwhere 𝑢 (𝑐 𝑖 ) denotes the affective score of comment 𝑐 𝑖 calculated using an external sentiment dictionary SenticNet [8] , and | • | represents the absolute value calculation. By employing this approach, the weight of the corresponding edge increases in proportion to the magnitude of sentiment reversal between every two comments, allowing significant attention to be directed towards comments that exhibit opposing sentiments.Context Semantic Computing. We compute the semantic attention matrix T to measure the semantic inconsistency between comments. Specifically, for each pair of comment features (x 𝑐 𝑖 , x 𝑐 𝑗 ), 𝑖, 𝑗 ∈ (1, 2, ..., 𝑛 𝑐 ), the attention score 𝑡 𝑖,𝑗 is calculated as:EQUATIONwhere W 𝑖,𝑗 is a trainable parameter matrix, and (•) 𝑇 signifies matrix transposition.Fusion & Enhanced. We incorporate the learned affective features A and semantic attention features T for a more expressive representation:EQUATIONwhere 𝛼 ∈ R is the hyperparameter. In order to comprehensively analyze the comment featuresEQUATIONTo simultaneously consider these two inconsistencies, we employ the inner product operation to obtain the output:EQUATIONwhere W 𝑎 is the weight matrix. Subsequently, the MaxPooling function is applied to derive the final output Z 3 :EQUATIONTo obtain the final integration outputs from the three modules, we concatenate Z 1 , Z 2 , Z 3 , and pass them through a constructed classifier to obtain the final outputs. The classifier consists of a stacked architecture with two fully connected layers comprising layer normalization, ReLU, and dropout. The final probability distributions are calculated by:EQUATIONEQUATIONwhereW 𝑜 , W ′ 𝑜 , b 𝑜 and b ′𝑜 are model parameters, and LN means Layer Normalization function. The probability matrix p comprises 𝑝 0 and 𝑝 1 , representing the predicted probability for the label being 0 (non-controversy) and 1 (controversy), respectively. Ultimately, the predicted label ŷ is defined as:EQUATIONTo train the whole framework, we combine three loss functions. First, we use the cross-entropy loss function to measure dissimilarity between predicted probabilities and ground truth labels. Given that 𝑦 ∈ {0, 1} denotes the ground truth label, the loss function is calculated as: Additionally, we employ a balancing loss for the MoE layer to ensure fair load and importance among experts. The calculation is as follows:EQUATIONEQUATIONwhere CV denotes the coefficient of variation, P(•) is the smooth function described by [44] , and the hyperparameters 𝑤 𝑖𝑚𝑝 and 𝑤 𝑙𝑑 are used to balance expert importance and load. Furthermore, we add a regularization loss to improve the quality of learned semantic information:EQUATIONwhere || • || 𝐹 is the Frobenius norm of a matrix, and 𝑤 𝑠𝑝𝑎 is the sparsity hyperparameter.Finally, we sum up the three loss functions to obtain the ultimate loss function:EQUATION) 
experiments: 
 We conducted experiments to address the following research questions:• RQ1: Is our proposed MVCD framework more effective than traditional and state-of-the-art baselines? • RQ2: Does our proposed model effectively utilize multimodal information, and do the individual modules within the model provide substantial contributions? • RQ3: Does the model demonstrate effectiveness in scenarios with limited availability of comments?We establish a comprehensive benchmark for controversy detection by conducting experiments using multiple representative methods as baselines, including uni-modal and multi-modal models.Uni-modal. Due to the predominant focus of previous research on textual modalities, our experiments include several uni-modal baselines, particularly those based on text modality. We utilize BERT [15] and RoBERTa [33] as representative baselines for pretrained language models. For the pretrained image model, we employ ViT [19] . In addition, we incorporate representative models such as TPC-GCN [54] and DTPC-GCN [54] as baselines in the controversy detection task. Furthermore, we use foundation models known for their state-of-the-art capabilities across various downstream tasks, including ChatGLM3-6B [20] and GPT3.5 [6] .Multi-modal. Since multimodal approaches are not yet widely used in controversy detection, we select related multimodal tasks such as fake detection and social bot detection. For the fake detection task, we consider two representative models: MCAN [50] and SVFEND [42] . For the social bot detection task, we include Bot-MoE [34] as a baseline model. Furthermore, we employ state-of-theart foundation models including VideoChat [29] , ChatGLM4 [20] and GPT4 [6] for comparison.Data Preprocessing. To ensure sample balance, we select an equal number of controversial and non-controversial videos from the annotated dataset, resulting in 5,632 videos each. The dataset is divided into training, validation, and test sets using an 8:1:1 ratio. Data preprocessing includes removing hashtags from video descriptions and eliminating stopwords such as mentions (@person) from comments. As for audio, we utilize the Baidu API 7 to perform Automatic Speech Recognition to obtain text.Implementation Details. In our experiments, we employ CN-CLIP [52] to generate image and text embeddings with a fixed vector size of 1024. The Adam [24] optimizer is used to optimize parameters with a learning rate set to 1e-4. The classification dimension is set to 128, and the batch size is set to 128. Training extends for 100 epochs, incorporating early stopping if the validation score does not improve for 10 consecutive epochs. For VideoChat, we employ the gpt-3.5-turbo-16k model for language generation, setting the frame sampling frequency parameter to 4. In the case of GPT3.5 and GPT4V, the temperature is set to 0.7. More details regarding the prompts used in the foundation models are presented in the supplementary materials.Table 3 shows the quantitative results of the evaluation. The experiment results indicate the following observations: MVCD demonstrates superior performance compared to other approaches, validating its effectiveness in capturing important multimodal clues for detecting controversial videos.Surprisingly, the performance of foundation models (GPT3.5, VideoChat, ChatGLM4, and GPT4V) is not ideal, both in uni-modal and multimodal scenarios. This could be attributed to the lack of appropriate training datasets for multimodal controversy detection, which may prevent large models from fully learning diverse controversial scenarios.Regarding SVFEND, which incorporates audio data during the training process, a notable performance disparity is observed when compared to all baselines, except for the foundation models. Remarkably, we observed that the training accuracy of SVFEND achieves an exceptional value of 0.99, without a corresponding proportional increase in testing accuracy. Once we eliminated the audio features from the model, we observed a more reasonable trend in training accuracy, accompanied by an improvement in testing. These findings lead us to speculate that including the audio modality during training may result in overfitting, leading to suboptimal performance in the outcomes.We conduct a series of ablation experiments to evaluate the importance of each modality and module in detecting controversial videos. The results, as shown in Table 4 , indicate that all modules perform well in the multimodal controversy detection task. Notably, when combined as a full model (MVCD), it achieves the highest accuracy of 72.62%. The CGL module exhibits slightly superior performance compared to the others, highlighting the importance of modeling the relationships between videos and comments. Furthermore, considering the involvement of multiple modalities in multimodal controversy detection, we performed ablation experiments on various data features. The experimental results indicate that each data feature plays a role, with notable contributions from profile and video features. During the initial stages of video publication, despite limited interaction and comments, detecting controversies is crucial for enhancing content quality and fostering audience engagement. To assess the model's performance under such circumstances, we evaluate its effectiveness using comments posted within 5 hours after the video is released. We compare two models: the joint model (CGL & IEL) that incorporates Contextual Graph Learning (CGL) and Inconsistency Enhanced Learning (IEL) with comments playing a pivotal role, and the full model MVCD. The experimental results are presented in Table 5 . The joint model (CGL & IEL) achieves relatively low accuracy rates of 50.62% and 50.88% within 0 and 1 hour, respectively. However, the full model MVCD, aided by the VGI module that effectively utilizes video content for prediction, yields more effective results and mitigates the significant drop in performance. This suggests that multimodal content plays a significant role in situations where comments are limited. 
conclusion and outlooks: 
 In this work, we released a comprehensive Multimodal Controversy Dataset (MMCD) in Chinese. To gain a thorough understanding of the characteristics exhibited by controversial videos, we have conducted an exploratory analysis of MMCD. Additionally, to facilitate further research in this field, we have proposed a Multi-view Controversy Detection framework, which effectively captures controversies presented within the video content itself, as well as those arising from the interaction between the video and its associated comments, or among the comments themselves. Extensive experiments showed the effectiveness of the proposed framework. Future research should explore integrating summarized viewpoints in videos or comments for valuable controversy detection. Detecting support and opposition in viewpoints is crucial for controversy detection. Emphasizing ethical implications and explainability of model predictions is essential, especially in sensitive contexts.  
 
 Appendix: Upon acceptance of this paper, the codebase will be made publicly available at https: //anonymous.4open.science/r/MM_Controversy_Detection_Released-DE6A.https://www.theguardian.comhttps://www.emol.com/https://www.toutiao.comhttps://www.douyin.comhttps://vop.baidu.com/server_api† Data formats are abbreviated for simplicity (D: Description, C: Comment, P: Profile, R: ASR Text, K: Keyword). 


Decision: The reviewers are unanimous in their opinion that the paper should be accepted. Most of them are particularly positive about the introduced MMCD dataset, as well as the proposed MVCD approach. In addition, the reviewers agree that the paper is definitely relevant to the scope of ACM MM, clear and well structured. On the other hand, the reviewers have also expressed concerns about e.g. definition of controversy, code availability and the potential ethical issues. 

Taking into account the initial reviews and the rebuttal, I believe that the paper should be accepted.

Reviews: review: paper_summary: paper_summary: The manuscript introduces one database of images and text for controversy detection using the Chinese language. In addition, it presents one architecture for this objective.
strength_weakness: strength_weakness: The paper is very well written and structured leading to a manuscript of high quality. The context is in general concrete and the authors managed to comprehensively describe their technical contributions. Even if there are issues concerning the results, the latter can be considered sufficient in terms of adequacy. Reference list is more than complete and to be precise, it should be shortened as it contains some obsolete references which has no use in the manuscript.
limitations: limitations: Despite its high quality as it concerns the manuscript itself, there are some major issues that should be considered in general: 
-	Whether something is controversy or not, from my perspective it is a subjective matter. Despite the correct approach of the authors to involve in the annotation process various and diverse individuals, parameter of culture plays a pivotal role in how each individual considers a thread as a controversy post. Hence, if the individuals are of the same nationality, most likely they will produce a biased dataset. Unless if we consider that the “way of thinking” and the culture of a nationality cannot affect the categorization of a post as a controversy. 
-	As the proposed database and model, focuses on the Chinse language, its application is extremely limited. MMCD includes textual embeddings for the Chinese language. MVCD can detect a controversy only for the Chinese language. This limits extremely the application.
-	In line 222, the authors claimed that their approach is unique. At first, it would be expected that there will be no comparison with similar multi-modal controversy detection models which is not the case. In addition, after a short research, such an approach would be reference Mariconti, E., Suarez-Tangil, G., Blackburn, J., De Cristofaro, E., Kourtellis, N., Leontiadis, I., ... & Stringhini, G. (2019). " You Know What to Do" Proactive Detection of YouTube Videos Targeted by Coordinated Hate Attacks. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-21. I suppose others could also be found. 
-	There is no details provided concerning the creation of the datasets and crawling these data and more specifically for any ethics related concerns, keywords used etc.
-	No information provided how these “sentiment analysis” and its results were incorporated (Lines 442-452).
-	Many parameters were provided in Section which also refer to Figure 5. The latter should be updated with the parameters (inputs/outputs per component).
-	Based on the descriptions of each model (section 4), it is assumed that eventually the proposed framework is just a pipeline. Though, the authors did not provide any justification for selecting each approach for each pipeline component. 
-	It is mentioned that the used models for text embeddings are pre-trained. Though it is not clarified on which language assuming Chinese. Correct? 
-	From my perspective, experimental results and more specific their commentary should be expanded. Apart from that, the comparison is uneven as all multi-modal approaches are not actually “multi-modal” as claimed by the authors (check my previous comment). In addition, fake detection (see line 743) is not so relevant with the controversy detection so how relevant is the comparison? 
-	There is confusion of the main objective of the paper. In the title, the authors focusing on the dataset while “Experimental results” focus on the proposed framework undermining the dataset and the title per se.
paper_summary: paper_summary: This work constructs a large-scale Multimodal Controversial Dataset (MMCD) in Chinese and proposes a novel framework named Multi-view Controversy Detection (MVCD) to effectively model controversies. The collected dataset from Douyin, a Chinese video platform, has been manually annotated, with annotators assessing the controversy within the video itself, the controversy between the video and its comments, and the controversy among the comments. The videos are diverse, covering 14 different domains. Additionally, the authors provided a statistical analysis of the videos and their popularity in the dataset, alongside sentiment analysis. This analysis provides a good overview of the MMCD dataset.

Regarding the detection approach, the authors propose a complex architecture consisting of five components. Three of the components capture the different aspects of controversy (within the video, between the video and comments, and among comments), while the final component integrates the features of the previous components and predicts whether the video is controversial. The choices for each component are quite reasonable and state-of-the-art, ranging from Mixture of Experts (MoE) to Graph Convolutional Networks. The baselines for comparison are both unimodal and multimodal, and the proposed architecture outperforms all of them. The ablation study is thorough and provides useful insights, as the complexity of the proposed architecture makes it necessary to understand and justify each part. Finally, the third research question for early controversy prediction is quite interesting, proving the potential for real-world applications.
strength_weakness: strength_weakness: 1) The dataset from Douyin is diverse, covering 14 domains, and has been manually annotated for various aspects of controversy.

2) The proposed architecture outperforms both unimodal and multimodal baselines.

3) The ablation study is thorough and provides useful insights into the architecture's effectiveness
limitations: limitations: The complexity of the architecture makes it difficult to understand the overall architecture, and despite the ablation study, it remains challenging to assess the sensitivity of the prediction.
paper_summary: paper_summary: This paper introduces the MMCD, a Multimodal Controversy Dataset in Chinese, and provides a thorough analysis of this newly constructed dataset. The authors also propose a Multi-View Controversy Detection(MVCD) framework that captures controversies presented within the video content itself, as well as those arising from the interaction between the video and its associated comments or among the comments themselves. Extensive experiments demonstrate the effectiveness of the proposed framework.
strength_weakness: strength_weakness: 1. **New Dataset:** The authors constructed a large-scale Chinese multimodal controversy dataset, MMCD. If made publicly available, this dataset would significantly contribute to the academic community's research on controversy detection in short videos, particularly within the Chinese context.
2. **Well-Written Manuscript:** The paper is well-organized and clearly written, making it easy for readers to follow.
3. **Rich Dataset Analysis:** The authors provide a comprehensive analysis of their dataset, offering valuable insights and findings that can inform and inspire further research in controversy detection.
4. **Meaningful Experiments:** The paper includes extensive experiments that effectively demonstrate the proposed framework's performance. Each experiment contributes to a deeper understanding of the model's capabilities and the importance of its components.
limitations: limitations: 1. **Data Analysis Enhancement:** The data analysis could be further improved. For instance, providing a quantitative breakdown of the number of videos across different domains before analyzing the distribution of controversy data would give readers a clearer context.
2. **Ethical Concerns:** The model uses nicknames directly, which raises potential ethical issues. The authors should consider addressing these concerns and discuss how they ensure the privacy and ethical handling of data.
3. **Code Availability:** Although the authors have provided an anonymous GitHub link, they did not include their code upon submission. Making the code available would greatly benefit the research community by allowing others to replicate and build upon their work.
4. **Case Studies:** Including case studies would strengthen the paper. Specific examples showing how different components like MAL, CGL, and IEL, contribute to the final controversy detection would provide practical insights into the model's application and effectiveness.
paper_summary: paper_summary: This paper introduces a novel multimodal social video dataset MMCD for controversy detection, and a multimodal framework, named MVCD, for detecting controversaries.
strength_weakness: strength_weakness: 1.	The introduced MMCD dataset is novel, and is expected to advance this research field.
2.	The proposed MVCD outperforms the baselines by a large margin.
limitations: limitations: 1.	In the Introduction, I would recommend to brief the components of MVCD.
2.	In Section 3.2, it seems that all the annotators are students from the same university, this may potentially introduce biases in the annotated label.
3.	In the baselines, are MCAN and SVFEND designed to process Chinese? If they are not designed to process Chinese, do you modify their components to better process Chinese. If not, the experiments may not be fair, because the proposed method leverages a pretrained Chinese model CN-CLIP to process Chinese text.