Generalized News Event Discovery via Dynamic Augmentation and Entropy Optimization

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 03:46:04

Abstract: News event discovery refers to the identification and detection of news events using multimodal data on social media. Currently, most works assume that the test set consists of known events. However, in real life, the emergence of new events is more frequent, which invalidates this assumption. In this paper, we propose a Dynamic Augmentation and Entropy Optimization (DAEO) model to address the scenario of generalized news event discovery, which requires the model to not only identify known events but also distinguish various new events. Specifically, we first introduce a multimodal augmentation module, which utilizes adversarial learning to enhance the multimodal representation capability. Secondly, we design an adaptive entropy optimization strategy combined with a self-distillation method, which uses multi-view pseudo-label consistency to improve the model's performance on both known and new events. In addition, we collect a multimodal news event discovery (MNED) dataset of 161,350 samples annotated with 66 real-world events. Extensive experimental results on the MNED dataset demonstrate the effectiveness of our proposed method. Our dataset is available on https://anonymous.4open.science/r/2FF5.

Main paper: 
 related work 2.1 news event discovery: 
 The exploration of news event discovery initially stemmed from the domain of topic detection and tracking [2] . With the rise of the internet, an increasing number of researchers have shifted their focus towards utilizing single-modal information present on social media, such as images or text, to facilitate the discovery of news events [12, 21, 22, 26, 28, 38] . For instance, Zaharieva et al. [38] utilized the image information for the detection of specific social events. Lee et al. [12] employed a naive Bayes multinomial classifier for identifying distinct trending topics. Despite their contributions, these single-modal approaches face inherent limitations when compared to analyses that incorporate multimodal data, as accurately detecting news events often requires a synthesis of various eventrelated elements. Therefore, researchers began to delve into news event discovery based on multimodal data [13, 14, 19, [33] [34] [35] 37] . For example, Li et al. [13] proposed a transformer-based conditioned variational autoencoder to jointly model the textual information, visual information and label information for incomplete social event classification. Lin et al. [14] designed a multi-modal fusion with external knowledge to address the out-of-distribution issue in news event detection. These approaches, however, assume that the training and test events remain consistent (i.e., a closed set), which diminishes their efficacy in the face of new, unseen events. Recently, Qian et al. [24] introduced an open-world social event classifer model towards an open setting of news event discovery, which can not only distinguish already occurred events but also identify whether an event is unknown when new events emerge. Nonetheless, the practicality of this approach faces challenges as the number of new events continuously increases over time, making the simple binary distinction between whether an event is new or not insufficient.The field of generalized category discovery has recently emerged, focusing on classifying known categories while also identifying unseen, new categories. The pioneering work in this domain was conducted by Vaze et al. [30] , who introduced the idea of leveraging a universal feature representation to discover new categories. Specifically, they proposed fine-tuning a pre-trained DINO ViT [6] using a combination of one supervised and one self-supervised contrastive method. This approach is further complemented by a semi-supervised clustering for label assignment. In addition, the authors extended UNO [7] and RankStats [8] for this task, which were originally designed for novel class discovery [7, 39] . However, these methods employ a two-step training process, involving feature learning and clustering, which could potentially be sub-optimal.To address this, Wen et al. [32] suggested parametric approaches that construct a trainable classifier, enabling the joint optimization of the entire network. Similar to the idea behind DINO ViT [6] , their method used the generation of pseudo cluster labels to guide the learning of new categories. This work sparked a series of follow-up studies [20, 31] . For example, Wang et al. [31] proposed the use of CLIP-generated text to guide image learning for category discovery. Nevertheless, it is challenging to apply these methods directly to generalized news event discovery, which involves multimodal data and higher-level event labels. This is because these methods rely on image data and pre-trained models developed primarily for similar tasks, which emphasizes the need for specialized adaptation in generalized news event discovery. 
preliminaries and problem statement: 
 Problem 1 (Generalized News Event Discovery). Given a dataset 𝐷 contains two parts: 𝐷 𝐿 containing known events and 𝐷 𝑈 including both known and new events, organized chronologically. A model is expected to be developed that can accurately categorize both known and new events in 𝐷 𝑈 . More specifically, 𝐷 𝐿 = {(𝑥 𝑖 , 𝑦 𝑖 )} 𝑁 𝑖=1 constitutes a labeled dataset containing multimodal instances 𝑥 𝑖 , each labeled with 𝑦 𝑖 from the set 𝑌 𝐿 of known event categories. 𝐷 𝑈 = {(𝑥 𝑗 )} 𝑀 𝑗=1 represents an unlabeled dataset with multimodal instances 𝑥 𝑗 , which are to be associated with labels from an expanded set 𝑌 𝑈 . The set 𝑌 𝑈 includes new, unseen event categories denoted by 𝑌 𝑛𝑒𝑤 , and a subset of 𝑌 𝐿 , designated as 𝑌 𝑓 𝑢𝑡𝑢𝑟𝑒 , which represents a subset of 𝑌 𝐿 that will continue to happen in the future. Hence, the relationship 𝑌 𝑈 = 𝑌 𝑛𝑒𝑤 ∪ 𝑌 𝑓 𝑢𝑡𝑢𝑟𝑒 , with 𝑌 𝑓 𝑢𝑡𝑢𝑟𝑒 ⊆ 𝑌 𝐿 as not all events from 𝑌 𝐿 are expected to reoccur. During training, the model is concurrently trained on both 𝐷 𝐿 , to learn from the historical occurrence of events, and 𝐷 𝑈 , to anticipate and categorize future, unseen events.In addition, in order to ensure that there are enough event types and relationships that can be used for generalized news event discovery, we define three types of news events based on their temporal attributes: short-term, cyclical, and long-term events. The following are the formal definitions:Definition 1: (Short-term Event). A short-term event is characterized by its ephemeral nature, typically unfolding and concluding within a brief time span. Examples of such events include natural disasters, sudden political upheavals, or unexpected public incidents. These events are transient and unpredictable, hence they have a high probability of falling into both 𝑌 𝐿 \ 𝑌 𝑓 𝑢𝑡𝑢𝑟𝑒 (elements present in 𝑌 𝐿 but absent in 𝑌 𝑓 𝑢𝑡𝑢𝑟𝑒 ) and 𝑌 𝑛𝑒𝑤 since they may not have occurred in the past or might represent entirely new scenarios.Definition 2: (Cyclical Event). Cyclical events are those that occur at regular intervals, marked by their predictability and periodicity. An example of a cyclical event is the Olympic Games, which recur on a four-year cycle. These events are anticipated and are typically encompassed within 𝑌 𝑓 𝑢𝑡𝑢𝑟𝑒 due to their recurrent nature.Definition 3: (Long-term Event). Long-term events span extended periods, often unfolding over months, years, or even decades. Wars, economic recessions, or major policy reforms are examples of long-term events. These events persist over such durations that they may be present in both 𝑌 𝐿 and 𝑌 𝑈 . 
methodology 4.1 overview of the framework: 
 As illustrated in Figure 2 , our DAEO model begins by leveraging a pretrained CLIP model [25] to extract features from both images and texts, which are then concatenated to form multimodal event features 𝐸. Specifically, to enable self-learning from unlabeled data, we apply random data augmentation to the images of the input posts to obtain an augmented post for distillation learning. The multimodal augmentation module then employs adversarial learning to generate robust multimodal augmented features 𝐸 𝐴𝑢𝑔 , which enhances the classifier's ability to distinguish between similar events. Then, we adopt a multilayer perceptron (MLP) as classifier 𝑓 to obtain the output. For labeled data, we employ standard supervised learning techniques using the labels; for unlabeled data, we utilize distillation learning for training. Additionally, the adaptive entropy optimization module uses the generated multi-view pseudo-labels for consistency checking to selectively optimize entropy. This approach not only encourages the discovery of new events but also improves the accuracy of known events. According to [30] , it is crucial to adopt a robust pretrained model to discover new category, like DINO ViT [6] . However, most pretrained models are primarily focused on image data. Thanks to the cross-modal alignment training on very large-scale image-text pairs, CLIP [25] demonstrates strong zero-shot performance, evidencing its powerful generalization capability for multi-modal joint embedding. Therefore, given an input sample 𝑥 𝑖 , we utilize the pretrained CLIP ViT-B/16 model to generate features for both the images and texts. These features are then concatenated to form our multimodal event feature 𝐸 𝑖 , which can be represented as:EQUATIONIn generalized news event discovery, it is important for a model to distinguish similar news events finely, such as different earthquakes in disaster events. Previous methods [30] utilize the supervised contrastive learning and self-contrastive learning method to widen the decision margins between different categories. However, applying random data augmentation for contrastive learning on single modalities, such as text or images, does not seem to enhance model performance for multimodal data (see Sec. 5.6). A possible reason is that random augmentation, especially for text, might lead to the loss of event-related clues, causing negative optimization. For news event discovery tasks, the relationship between images and text can be complementary, related, or unrelated.In our model, we adopt a different approach to learn robust features, i.e., the multimodal augmentation module, by generating multimodal augmented features through the adversarial method [15] at the feature level. On one hand, we aim for the generated multimodal augmented features to closely approach the decision boundary, which improves the classifier's ability to distinguish between similar events. On the other hand, we strive to ensure that the generated multimodal features retain the original event semantics, which prevents negative optimization.As shown in Figure 3 , we employ a Variational Autoencoder (VAE) model [10] as the generative model, denoted as 𝐺, which includes an encoder and a decoder. The VAE model has been proven effective in generating features. We utilize the KL divergence [9] to make the encoder's output as close to a standard Gaussian distribution as possible, which can be formulated as:EQUATION)where 𝜇 and 𝜎 are the mean and standard deviation parameters output by the encoder, respectively. Furthermore, we use a residual module to retain more of the original multimodal feature semantics.The augmented features 𝐸 𝐴𝑢𝑔 𝑖 can be formulated as:EQUATIONFor learning to generate robust multimodal augmented features, we perform the adversarial training consisting of two parts. In the first part, as shown in Figure 2 , we fix the parameters of the multimodal augmentation module 𝐺 and train the CLIP and classifier model 𝑓 to minimize the cross-entropy loss between the output and the true event labels, which ensures that augmented features retain their original semantics. It can be formulated as:EQUATIONwhere ℓ 𝑐𝑒 (•) represents the cross-entropy loss function.In the second part, as shown in Figure 3 , we fix the parameters of the CLIP and classifier models and train 𝐺, on one hand, maximize the cross-entropy loss between the output and the true event labels to generate more discriminative features closed to the decision boundary, and on the other hand, minimize the consistency loss to align the semantics of the augmented and original multimodal feature outputs. The consistency loss can be formulated as:EQUATIONTo achieve the adversarial goal, we want the optimal parameters θ𝐶𝐿𝐼𝑃 , θ𝐺 and θ 𝑓 to jointly satisfyEQUATION( θ𝐺 ) = arg maxEQUATIONEQUATIONIn this way, the generated features 𝐸 𝐴𝑢𝑔 𝑖 will be close to the decision boundary, which further helps the classifier 𝑓 to distinguish the class with some ambiguous decision boundaries.To identify new categories, we train a unified prototypical classification head for all new and known classes using a self-distillation framework. For self-distillation, we perform simple augmentations on images to obtain augmented images. Considering the potential for existing random text augmentation methods to change semantics and cause negative optimization, we compose augmented multimodal data directly from augmented images and original texts. Through the CLIP model, we obtain two different views of multimodal features, 𝐸 𝑖 and 𝐸 ′ 𝑖 . Then, we map these multimodal features to 𝐾-dimensional vectors as outputs using a function 𝑓 , where 𝐾 = |𝑌 𝐿 ∪ 𝑌 𝑈 | is the total number of event categories. For labeled data, we optimize using a cross-entropy function in Eq. 8. For unlabeled data, we employ self-distillation learning. Specifically, we first randomly initialize a set of prototypes 𝐶 = {𝑐 1 , ..., 𝑐 𝐾 }, each During training, we compute the cosine similarity between the output features and prototypes to obtain soft labels 𝑝 𝑖 /𝑞 𝑖 for each view, which can be formulated as:EQUATION)where 𝜏 is a temperature parameter for 𝑝 𝑖 and a sharper version for another view 𝑞 𝑖 . The distillation loss can be formulated as:EQUATIONWe also adopt a entropy maximization regularizer [4] for the unsupervised objective, which can be formulated as:EQUATIONHowever, we found that maximizing entropy, while encouraging the exploration of new categories, also decreases the model's confidence in known categories, ultimately sacrificing accuracy on known categories (see Sec. 5.6).To address this issue, we propose an adaptive entropy optimization strategy, aiming for the model to actively explore new categories while maintaining accuracy on known categories. Specifically, we use pseudo-label consistency across four views to decide on entropy optimization. For a sample, on one hand, we generate two pseudo-labels using 𝑝 𝑖 and its augmented view 𝑞 𝑖 ; on the other hand, we generate 𝑝 𝐴𝑢𝑔 𝑖 and 𝑞 𝐴𝑢𝑔 𝑖 as two additional views to obtain two more pseudo-labels using the multimodal augmentation module mentioned earlier, which provides a more challenging perspective as the generated feature, encouraged by adversarial learning, are more diverse and hence more discriminative. We then use a consistency checker to perform consistency checks on the pseudo-labels from these four different views for entropy optimization, which can be formulated as:EQUATION)where 𝑛 represents the number of consistency for the pseudo-labels, 𝛼 and 𝛽 are hyperparameters. Through this strategy, when the model's predictions are completely consistent across different views, we increase the model's confidence in its judgment by minimizing entropy; when there is a discrepancy in the model's judgments across views, we encourage further exploration by maximizing entropy as we want the model to explore new events as much as possible when there is uncertainty, rather than blindly gravitating towards known events.In this study, we optimize a minimax problem via a straightforward back-propagation way. To summarize the previous discussions, the overall objective function of DAEO can be formulated as follows: 3: Compute the pseudo-labels from four different views. 4: Perform a consistency check for these pseudo-labels and compute the adaptive entropy loss 𝐿 𝐴𝑑𝑎𝑝𝑡 according to Eq. 12.( θ𝐶𝐿𝐼𝑃 , θ 𝑓 ) = arg min5: Compute the cross-entropy loss 𝐿 𝐶𝐸 and 𝐿 𝐴𝑢𝑔 𝐶𝐸 and distill loss 𝐿 𝐷𝑖𝑠𝑡𝑖𝑙𝑙 according to Eq. 8, Eq. 4 and Eq. 10, respectively. 6: Optimize the objective in Eq. 13. 7: Recompute the multi-modal features 𝐸 𝑖 and the augmented multi-modal features 𝐸 𝐴𝑢𝑔 𝑖 according to Eq. 1 and Eq. 3, respectively.8: Recompute the cross-entropy loss 𝐿 𝐴𝑢𝑔 𝐶𝐸 , KL divergence loss 𝐿 𝐾𝐿 and consistency loss 𝐿 𝐶𝑜𝑛𝑠𝑖𝑠 according to Eq. 4, Eq. 2 and Eq. 5, respectively. 9: Optimize the objective in Eq. 14. end whileEQUATIONThe detailed algorithm for the DAEO method is presented in Algorithm 1. 
collection of news: 
 Events. The collection of news events plays a crucial role in generalized news event discovery, demanding a diverse array of relationships among different news events to encompass various possibilities. For instance, when using time as a divider to separate the training and test sets, the relationship between events can be identical, subset, intersecting, or entirely distinct, depending on the type and time of the event. In our paper, we define short-term events, cyclical events, and long-term events, which are designed to cover various event relationships and align with real-world scenarios. Specifically, we collect a variety of events ranging from 2010 to 2023 for each type of event from a crowdsourced platform Wikipedia, e.g., short-term events include natural, human-made disasters, etc.; cyclical events encompass sports competitions, political elections, etc.; long-term events involve political conflicts, social movements, etc. Ultimately, our collection comprises 66 news events, including 42 short-term events, 13 cyclical events and 11 long-term events.For data collection and statistics, we select Twitter as our primary source due to its extensive user base. We employ event-related hashtags and temporal searches to avoid oversimplification of the task. For longterm events, we sample important sub-events based on Wikipedia, e.g., "Syrian Civil War" containing "Ghouta Chemical Attack", "US Troops Withdrawing from Northern Syria", and so on. Subsequently, we filter out single-modality data samples and manually verified the semantic relevance of the samples for the corresponding event, resulting in a multimodal news event discovery (MNED) dataset of 161,350 samples. Data statistics and sample distribution are shown in Table 1 and Figure 4 . Further details are available in Appendix.Partitioning. Different from other classification tasks, the generalized news event discovery task inherently involves a temporal dimension. Therefore, we organize all posts chronologically and then split the dataset into training and test sets based on sequential proportions. For this purpose, we divide the dataset into training and test sets by selecting three different time points-corresponding to 25%, 50%, and 75% of the timeline of the collected data-to determine the chronological length of the training set relative to the entire dataset. As for the validation set, we allocate 20% of the training set, chosen randomly across categories. The specifics of this division are summarized in Table 2 .To evaluate our model's performance, we employ a clustering accuracy (ACC) followed by [30] . This metric is calculated as follows:EQUATIONwhere 𝑃 represents the set of all possible permutations that align the model's predicted labels ŷ𝑖 with the actual ground truth labels 𝑦 𝑖 , utilizing the Hungarian method [11] for optimal matching. We apply this metric across three sets: the complete unlabeled set denoted as "All", a subset called "Known" which contains samples from classes already known to the model, and "New", comprising samples from classes not previously seen by the model.We utilize the CLIP ViT-B/16 backbone to train all methods, with fine-tuning the final block and linear projection layer of the text and visual encoders. The SGD optimizer [3] is employed with an initial learning rate of 0.001 and then decayed following a cosine schedule. The models are trained over 100 epochs with a batch size of 128. In alignment with [32] , the temperature value for distillation learning is set to 0.1 and the sharper version starts at 0.07, then is gradually warmed up to 0.04 using a cosine schedule in the starting 10 epochs. The hyperparameters 𝛼 and 𝛽 are set to 0.03 and 2.3, respectively. For non-English text, we utilize the Google Translation API1 to translate the content into English. For posts containing multiple images, we only use the first image. The process of tuning and testing is carried out on a separate validation set, facilitating the selection of the best hyperparameters for optimal performance. Additionally, more extended experiments are provided in Appendix.To investigate the effectiveness of our proposed method, we compare it with three different baseline approaches to provide a comprehensive evaluation. 1) K-means method [16] . This baseline extracts the image and text features from the pretrained CLIP model and concatenates them to form the multimodal features, followed by the K-means clustering algorithm. 2) novel category discovery baselines (UNO [7] and RankStats [8] ). These are strong baselines from the field of novel category discovery. Following the setup in [30] , we configure one classification head to the total number of classes in order to adapt these models to fit the task. 3) the state-ofthe-art methods in generalized category discovery (GCD [30] and SimGCD [32] ). GCD utilizes semi-supervised K-means clustering based on learned features, and SimGCD employs a parametric classifier for distillation learning, which has demonstrated impressive results across various image recognition tasks. The proposed DAEO model contains different modules. To validate their effectiveness, we conduct ablation studies on these components. We denote Multimodal Augmentation as "MA", the entropy minimization and maximization terms in 𝐿 𝐴𝑑𝑎𝑝𝑡 as "Entmin" and "Entmax", respectively, and "Adapt" to represent 𝐿 𝐴𝑑𝑎𝑝𝑡 , with "Ctr" indicating self-contrastive learning and supervised contrastive learning. From the results in Table 5 , we have the following observations: 1) The absence of multimodal augmentation (#1 and #7) leads to a decrease in accuracy, especially for new events, which underscores the contribution of the multimodal augmentation module in learning more robust features. 2) By comparing #4 and #5, we note that the entropy maximization term boosts the model's performance on recognizing new events but adversely affects its ability to identify known events. The inclusion of 𝐿 𝐴𝑑𝑎𝑝𝑡 and Entmin (#4 and #7) not only retains the model's capacity to recognize new events but also improves its accuracy on known events. This demonstrates 3) The addition of self-contrastive learning and supervised contrastive learning (#6 and #7) does not enhance our model's performance, which could be attributed to negative optimization introduced by random augmentations.In addition, we also investigate the effect of the model's backbone, the handling of multilingual text in the dataset, the conditions for 𝐿 𝐴𝑑𝑎𝑝𝑡 and the performance of our model under different event types. 1) As shown in Table 4a , employing larger pretrained models as the backbone improves performance, underscoring the importance of a robust pretrained model. 2) Regarding multilingual text processing, we experiment with using the M-CLIP model [5] , which is pretrained on multiple languages. According to the results in Table 4b , utilizing such model does not outperform a straightforward approach of employing the Google Translate API for language translation, thus we select the translation method. 3) For the condition of 𝐿 𝐴𝑑𝑎𝑝𝑡 , loosening the criteria (i.e., using a consistency threshold across different views to determine entropy minimization/maximization) leads to reduced recognition rates for new events, as shown in Table 4c . Therefore, we select the condition of consistency across all views. 4) As shown in Table 6 , the model has a high recognition rate for cyclical events, due to their high degree of similarity and predictable recurrence. However, for long-term events, the ongoing evolution of the events makes the recognition of even known events as challenging as that of short-term events. To delve into the impact of parameters 𝛼 and 𝛽 on the model's performance, we conducted experiments varying 𝛼 from 2.0 to 2.4 and 𝛽 from 0 to 0.04. As depicted in Figure 5 , we observe that an increase in 𝛼 tends to enhance the accuracy for new events at the expense of slightly reducing accuracy for known events, while 𝛽 exhibits an inverse relationship. Overall, the model demonstrates moderate sensitivity to these parameters, leading to the selection of 𝛼 = 2.3 and 𝛽 = 0.01 as the optimal settings. Regarding the parameter 𝐾, which denotes the number of prototypes, we assume that the number of events is known following [32] in our model. We investigate its effect on our model using different values. As shown in Figure 5 , although the performance on new events slightly decreases with increasing 𝐾 values, the fluctuation remains minimal, which shows our model's robustness to variations in the number of prototypes.To further investigate the effectiveness of our proposed method, we employ t-SNE [29] visualization to illustrate the multimodal event features learned by the model. We select eight similar events, including both known and new events, for visualization. As shown in Figure 6 , we have the following observations: 1) Compared to SimGCD, the features from our proposed method have clearer boundaries between different events, which proves the effectiveness of our approach. 2) For similar events, such as attack events, our model demonstrates a strong capability to differentiate between them, which is attributed to our multimodal augmentation module that utilizes adversarial learning to generate discriminative features. Despite the excellent performance of DAEO, Figure 7 shows three failure cases from different event types. We observe that their misclassification mainly stems from the lack of distinctive elements in the provided images and texts. Specifically, the first case comes from the "2020 Beirut Explosion". Due to the absence of key information about the Beirut explosion, the event is mistakenly classified as a general explosion event. The second case, "FIFA World Cup", included a team photo, which is common in other sports events, such as the "Olympic Games". The third case, "Hong Kong Protests", featured many protesting people, leading the model to mistakenly categorize it as an attack event. These failure cases illustrate the complexity and challenges of the generalized news event discovery. 
conclusion: 
 In this paper, we introduce a Dynamic Augmentation and Entropy Optimization (DAEO) model designed specifically to tackle the challenges of generalized news event discovery. A multimodal augmentation module is designed to employ adversarial learning to generate distinctive multimodal features, which improves the model's ability to discern between similar event categories. An adaptive entropy optimization strategy with a self-distillation method leverages pseudo-labels from different views to adaptively optimize entropy, thereby enhancing the model's ability in recognizing both new and known events. Additionally, we contribute to the field by introducing the Multimodal News Event Discovery (MNED) dataset, which contains various event types and serves as a valuable resource for researchers. Extensive experiments conducted on the MNED dataset validate the effectiveness of our proposed model.  
: 
 The current research [1, 13, 14, 36, 40] in news event discovery primarily focuses on utilizing multimodal approaches due to the richer information provided by the multimodal data. However, a significant limitation of these studies is their reliance on a closed-set assumption, which greatly diminishes their applicability in practical scenarios. As illustrated in Figure 1a , under a closed setting, the focus is mainly on identifying news events that are already known, like cyclical or long-term events, which have happened before. This kind of approach falls short when it comes to detecting novel events that emerge over time. In response to this limitation, some researchers [24] have proposed shifting towards an open setting, aiming to identify novel events as they occur as shown in Figure 1b . Yet, as the volume of new events grows in real life, merely distinguishing whether an event is unknown or not is often insufficient. This has led to the exploration of the generalized news event discovery problem, which seeks to extend beyond the binary classification of new events as either known or unknown. This task aims at not only recognizing previously occurred news events but also differentiating among events that have not yet occurred, referring to this capability as general category discovery [30] . As shown in Figure 1c , it requires the model to both identify known events and categorize new events.However, the task of generalized news event discovery presents several challenges. The first challenge lies in dealing with multimodal features. When identifying news events, it's possible that only one modality, either text or image, provides useful information, or both modalities offer complementary insights. This variability requires the effective integration and utilization of each modality, especially for events that are closely related or similar in nature. The second challenge involves utilizing knowledge of previously occurred events to identify new events. This necessitates a model capable of distinguishing subtle differences between known and new events. Lastly, the challenge of dataset scarcity compounds the difficulty of this task. A comprehensive dataset, rich in both volume and variety of events, including temporal information, is crucial for this task. As illustrated in Figure 1 , temporal information plays a crucial role in the division of datasets. Unfortunately, existing datasets [14, 24] lack this temporal metadata, leading to the use of random splits for training and test sets, which can not reflect the real-world scenario where events unfold over time.To address the aforementioned challenges, we introduce a Dynamic Augmentation and Entropy Optimization (DAEO) model. For the first challenge, we design a multimodal augmentation module to learn more robust multimodal event features, which implicitly leverages the label information of events to learn the relationship between different modalities. It utilizes adversarial learning to not only encourage the generation of multimodal features that can distinguish between different similar events but also ensure the generated features are as diverse as possible. For the second challenge, we learn a unified prototypical classification head for all new and known classes with self-distillation learning. Unlike previous methods [32] that used entropy maximization for all samples, we introduce an adaptive entropy optimization technique. Specifically, we generate various pseudo labels using a multi-view approach, including single-modal random augmentations (e.g., image augmentations) and outputs from the multimodal augmentation module. Then, when there is consistency across multiple views, the model is optimized to minimize entropy, thereby enhancing confidence in identified known events. Conversely, when views differ, entropy maximization is employed to encourage further exploration of the new events. Furthermore, we collect a multimodal news event discovery (MNED) dataset for generalized news event discovery from Twitter, comprising 161,350 multimodal samples annotated with 66 real-world events. Reflecting the temporal characteristics of news events, we define and collect three types of news events: short-term, cyclical, and long-term events. To ensure diversity, each event type encompasses a broad range of sub-events, including short-term events like natural and man-made disasters, terrorist attacks; cyclical events such as sporting events, political elections, international summits; and long-term events covering political conflicts, economic/social crises, and environmental/health issues.The contributions of this paper can be summarized as follows:• We formulate the task of generalized news event discovery and introduce a Dynamic Augmentation and Entropy Optimization (DAEO) model designed to tackle this task. • We propose a multimodal augmentation module and an adaptive entropy optimization strategy aimed at improving the representation of multimodal features and enhancing the ability to uncover new events, respectively. • We collect a comprehensive multimodal news event discovery (MNED) dataset deigned for news event discovery, which encompasses a wide array of events categorized into longterm, cyclical, and short-term events, providing a rich resource for the research community. • Extensive experimental results on the MNED dataset demonstrate the effectiveness of our proposed model. 
 
 Appendix: https://cloud.google.com/translate 


Decision: News event Discovery is a highly subjective area of research.  The area chair is not surprised that the paper got such complementary feedback from the reviewers. Overall, the area chair recommends the paper as a poster.

Reviews: review: paper_summary: paper_summary: This paper introduces the Dynamic Augmentation and Entropy Optimization (DAEO) model for generalized news event discovery, which aims to identify both known and new events from multimodal social media data. The key contributions include:

1. Formulation of the generalized news event discovery problem and the collection of a comprehensive Multimodal News Event Discovery (MNED) dataset.

2. A multimodal augmentation module that generates robust augmented features using adversarial learning to distinguish between similar events.

3. An adaptive entropy optimization strategy that selectively optimizes entropy based on multi-view pseudo-label consistency to balance known and new event recognition.

4. A self-distillation learning framework that trains a unified prototypical classification head for all classes using pseudo-labels from different views.
strength_weakness: strength_weakness: This paper introduces a novel framework for essentially topic recognition and discovery, which is a fairly important task in multimedia discovery. The authors integrated two important modules, namely the multimodal augmentation and the adaptive entropy optimization module. Their introduction in fairly well justified, especially for the AEO module, which seems to be the highlight of this work. Although EO is conventionally, the authors have introduced an additional adaptivity which balances between known and unknowed features. All modules are also well integrated via the adversarial learning paradigm. Finally, the collection and hopefully eventual publication of the collected dataset should also be useful for future researchers.
limitations: limitations: This is a relatively solid work and I don't have much complaints about it in general; the only lacking aspect which can be mentioned is the lack of comparison with an alternative dataset, which makes the presented results slightly thin; hopefully the authors can address this shortcoming in the rebuttal?
paper_summary: paper_summary: The paper studies a challenging issue of unknown events to news discovery and proposes a Dynamic Augmentation and Entropy Optimization (DAEO) model for the task. The proposed DAEO introduces a multimodal augmentation module and an adaptive entropy optimization strategy to improve multimodal representation and uncover new events.
strength_weakness: strength_weakness: 1. The paper studies a significant and challenging issue in news event discovery and proposes a self-motivated DAEO model with dynamic augmentation and adaptive entropy optimization.
2. The paper is well-organized and easy to follow.
3. The paper delivers a new dataset MNED for news event discovery, contributing to the event discovery community.
limitations: limitations: 1. The notation delta in Table 3 is explained and confusing.
2. The paper fails to detail the previous work on discovering new events and analyses the correlations between the paper and the previous studies.
3. The paper lacks experiments to interpret or visualize the capability of the proposed model in handling new news events. Such experiments are beneficial for understanding the benefits and contributions of the proposed model.
paper_summary: paper_summary: This paper introduces a groundbreaking approach to News Event Discovery (NED) that addresses the challenge of detecting both familiar and novel events in a multimodal context.  Departing from conventional assumptions, the authors propose a Dynamic Augmentation and Entropy Optimization (DAEO) framework, designed to cope with the dynamic nature of real-life news occurrences.
strength_weakness: strength_weakness: The idea of the division of different events makes sense for generalized news event discovery. And the design of the MNED dataset can better enrich the research in this field.
limitations: limitations: 1.	Descriptive Contradiction: The details in figure 2 and figure 3 are contradictory, and the description in METHODOLOGY is a little vague for figures, so it is difficult for readers to understand all the detail of the method proposed by author according to the description and figures.
2.	Ablation Study: In Table 4a, the author may consider utilizing larger pretrained models as the backbone, such as Vit-H, due to its broader application and superior performance.
3.	Reproduction: Will the authors open-source the code they implement? It is important for the reproduction of this paper and further research.
paper_summary: paper_summary: The paper introduces the Dynamic Augmentation and Entropy Optimization (DAEO) model for news event discovery on social media, capable of identifying both known and new events using multimodal data. It features an adversarial learning-based augmentation module and an adaptive entropy optimization strategy with self-distillation. The researchers also compiled the Multimodal News Event Discovery (MNED) dataset, containing 161,350 samples of 66 real-world events. Experiments on this dataset demonstrate the model's effectiveness, offering a significant contribution to multimedia applications and data integration innovations.
strength_weakness: strength_weakness: 1.	Comprehensive Work: The paper provides a thorough and detailed study. The model design incorporates multiple modules to effectively process news event data. Additionally, the authors have developed a dedicated dataset and performed relevant partitioning (varying proportions and different types), demonstrating substantial engineering effort.
2.	Effective Model Results: The paper mentions that the experimental results on the MNED dataset demonstrate the effectiveness of the proposed model. This indicates that the model performs well on the dataset provided.
3.	Strong Logical Structure: The paper is logically structured with a clear problem background, module introduction, and experimental analysis. Overall, the writing is coherent, and the content is sufficiently detailed.
limitations: limitations: 1.	Baseline Selection Needs Improvement: From the results presented in Table 3 (particularly for 25% and 50%), it is evident that some baselines perform particularly poorly. The authors' analysis of the experimental results is also insufficient. For instance, the authors did not analyze the results for RankStats and UNO, despite describing them as strong baselines in the field of novel category discovery. However, their performance in the "New" category is significantly worse than others.
2.	Motivation for Dataset Creation Needs Clarification: The paper mentions that temporal information is important in the dataset but does not elaborate on the motivation behind this. The authors divided the dataset into Short-term, Cyclic, and Long-term events and split it based on time. However, the paper merely presents the corresponding results without explaining the rationale behind this approach. The motivation and benefits of this division should be clearly articulated.
3.	Design of Augmentation Modules: In section 4.3, the authors mention that applying random data augmentation on single modalities does not enhance model performance and choose to use feature-level augmentation. However, in section 4.4, they augment directly from augmented images and original texts, which differs from the method in the previous section. The choice of these two different augmentation methods seems inconsistent even though they are in different parts of the model.