OpenRaiser/Intern-Atlas huggingface dataset:
Reinforcement learning-based counter-misinformation response generation: a case study of COVID-19 vaccine misinformation
Authors: Bing He, Mustaque Ahamad, Srijan Kumar
Venue: Conference WWW 2023
Year: 2023

The spread of online misinformation threatens public health, democracy, and the broader society. While professional fact-checkers form the first line of defense by fact-checking popular false claims, they do not engage directly in conversations with misinformation spreaders. On the other hand, non-expert ordinary users act as eyes-on-the-ground who proactively counter misinformation -- recent research has shown that 96% counter-misinformation responses are made by ordinary users. However, research also found that 2/3 times, these responses are rude and lack evidence. This work seeks to create a counter-misinformation response generation model to empower users to effectively correct misinformation. This objective is challenging due to the absence of datasets containing ground-truth of ideal counter-misinformation responses, and the lack of models that can generate responses backed by communication theories. In this work, we create two novel datasets of misinformation and counter-misinformation response pairs from in-the-wild social media and crowdsourcing from college-educated students. We annotate the collected data to distinguish poor from ideal responses that are factual, polite, and refute misinformation. We propose MisinfoCorrect, a reinforcement learning-based framework that learns to generate counter-misinformation responses for an input misinformation post. The model rewards the generator to increase the politeness, factuality, and refutation attitude while retaining text fluency and relevancy. Quantitative and qualitative evaluation shows that our model outperforms several baselines by generating high-quality counter-responses. This work illustrates the promise of generative text models for social good -- here, to help create a safe and reliable information ecosystem. The code and data is accessible on https://github.com/claws-lab/MisinfoCorrect.
Music retrieval and recommendation: A tutorial overview
Authors: Peter Knees, Markus Schedl
Venue: Conference SIGIR 2015
Year: 2015

In this tutorial, we give an introduction to the field of and state of the art in music information retrieval (MIR). The tutorial particularly spotlights the question of music similarity, which is an essential aspect in music retrieval and recommendation. Three factors play a central role in MIR research: (1) the music content, i.e., the audio signal itself, (2) the music context, i.e., metadata in the widest sense, and (3) the listeners and their contexts, manifested in user-music interaction traces. We review approaches that extract features from all three data sources and combinations thereof and show how these features can be used for (large-scale) music indexing, music description, music similarity measurement, and recommendation. These methods are further showcased in a number of popular music applications, such as automatic playlist generation and personalized radio stationing, location-aware music recommendation, music search engines, and intelligent browsing interfaces. Additionally, related topics such as music identification, automatic music accompaniment and score following, and search and retrieval in the music production domain are discussed.
Beyond the first law of geography: Learning representations of satellite imagery by leveraging point-of-interests
Authors: Yanxin Xi, Tong Li, Huandong Wang, Yong Li, Sasu Tarkoma, Pan Hui
Venue: Conference WWW 2022
Year: 2022

Satellite imagery depicts the earth’s surface remotely and provides comprehensive information for many applications, such as land use monitoring and urban planning. Existing studies on unsupervised representation learning for satellite images only take into account the images’ geographic information, ignoring human activity factors. To bridge this gap, we propose using Point-of-Interest (POI) data to capture human factors and design a contrastive learning-based framework to consolidate the representation of satellite imagery with POI information. Also, we design an attention model that merges the representations from the geographic and POI perspectives adaptively. On the basis of real-world datasets collected from Beijing, we evaluate our method for predicting socioeconomic indicators. The results show that the representation containing POI information outperforms the geographic representation in estimating commercial activity-related indicators. Our proposed framework can estimate the socioeconomic indicators with an R2 of 0.874 and outperforms the baseline methods.
Unsupervised Extractive Text Summarization with Distance-Augmented Sentence Graphs
Authors: Jingzhou Liu, Dominic J. D. Hughes, Yiming Yang 0002
Venue: Conference SIGIR 2021
Year: 2021

Supervised summarization has made significant improvements in recent years by leveraging cutting-edge deep learning technologies. However, the true success of supervised methods relies on the availability of large quantity of human-generated summaries of documents, which is highly costly and difficult to obtain in general. This paper proposes an unsupervised approach to extractive text summarization, which uses an automatically constructed sentence graph from each document to select salient sentences for summarization based on both the similarities and relative distances in the neighborhood of each sentences. We further generalize our approach from single-document summarization to a multi-document setting, by aggregating document-level graphs via proximity-based cross-document edges. In our experiments on benchmark datasets, the proposed approach achieved competitive or better results than previous state-of-the-art unsupervised extractive summarization methods in both single-document and multi-document settings, and the performance is competitive to strong supervised baselines.
A mrf-based approach for real-time subway monitoring
Authors: Nikos Paragios, Visvanathan Ramesh
Venue: Conference CVPR 2001
Year: 2001

There has been an increase in the use of video surveillance and monitoring in public areas to improve safety and security. Change detection and crowding/congestion density estimation are two sub-tasks in a subway monitoring system. We propose a method that decomposes this problem into two steps. The first step consists of a change detection algorithm that distinguishes the background from the foreground. This is done using a discontinuity preserving MRF-based approach where the information from different sources (background subtraction, intensity modeling) is combined with spatial constraints to provide a smooth motion detection map. Then, the obtained change detection map is combined with a geometry module that performs a soft auto-calibration to estimate a measure of congestion of the observed area (platform). Extensive experimental results in a metro station of a metropolitan city demonstrates the performance and the potential of our method.
Image and video understanding for recommendation and spam detection systems
Authors: Aman Gupta, Sirjan Kafle, Di Wen 0003, Dylan Wang, Sumit Srivastava, Suhit Sinha, Nikita Gupta, Bharat Jain, Ananth Sankar, Liang Zhang 0021
Venue: Conference KDD 2020
Year: 2020

Image and video-based content has become ever present in a variety of domains like news, entertainment and education. Users typically discover and engage with content via search and recommendation systems. It is also important to serve high quality data to users by filtering out irrelevant or harmful content. Thus, there is an increasing need to leverage the rich information in image and video content in order to power systems for search and recommendation. At the same time, the effectiveness and efficiency of these systems has been accelerated by the availability of large-scale labeled datasets and sophisticated deep learning-based models.
Face relighting with radiance environment maps
Authors: Zhen Wen, Zicheng Liu 0001, Thomas S. Huang
Venue: Conference CVPR 2003
Year: 2003

A radiance environment map pre-integrates a constant surface reflectance with the lighting environment. It has been used to generate photo-realistic rendering at interactive speed. However, one of its limitations is that each radiance environment map can only render the object, which has the same surface reflectance as what it integrates. We present a ratio-image based technique to use a radiance environment map to render diffuse objects with different surface reflectance properties. This method has the advantage that it does not require the separation of illumination from reflectance, and it is simple to implement and runs at interactive speed. In order to use this technique for human face relighting, we have developed a technique that uses spherical harmonics to approximate the radiance environment map for any given image of a face. Thus we are able to relight face images when the lighting environment rotates. Another benefit of the radiance environment map is that we can interactively modify lighting by changing the coefficients of the spherical harmonics basis. Finally we can modify the lighting condition of one person's face so that it matches the new lighting condition of a different person's face image assuming the two faces have similar skin albedos.
MobileSteward: Integrating Multiple App-Oriented Agents with Self-Evolution to Automate Cross-App Instructions
Authors: Yuxuan Liu, Hongda Sun, Wei Liu, Jian Luan, Bo Du, Rui Yan
Venue: Conference KDD 2025
Year: 2025

Mobile phone agents can assist people in automating daily tasks on their phones, which have emerged as a pivotal research spotlight. However, existing procedure-oriented agents struggle with cross-app instructions, due to the following challenges: (1) complex task relationships, (2) diverse app environment, and (3) error propagation and information loss in multi-step execution. Drawing inspiration from object-oriented programming principles, we recognize that object-oriented solutions is more suitable for cross-app instruction. To address these challenges, we propose a self-evolving multi-agent framework named MobileSteward, which integrates multiple app-oriented StaffAgents coordinated by a centralized StewardAgent. We design three specialized modules in MobileSteward: (1) Dynamic Recruitment generates a scheduling graph guided by information flow to explicitly associate tasks among apps. (2) Assigned Execution assigns the task to app-oriented StaffAgents, each equipped with app-specialized expertise to address the diversity between apps. (3) Adjusted Evaluation conducts evaluation to provide reflection tips or deliver key information, which alleviates error propagation and information loss during multi-step execution. To continuously improve the performance of MobileSteward, we develop a Memory-based Self-evolution mechanism, which summarizes the experience from successful execution, to improve the performance of MobileSteward. We establish the first English Cross-APP Benchmark (CAPBench) in the real-world environment to evaluate the agents' capabilities of solving complex cross-app instructions. Experimental results demonstrate that MobileSteward achieves the best performance compared to both single-agent and multi-agent frameworks, highlighting the superiority of MobileSteward in better handling user instructions with diverse complexity.
Pst: measuring skill proficiency in programming exercise process via programming skill tracing
Authors: Ruixin Li, Yu Yin, Le Dai, Shuanghong Shen, Xin Lin, Yu Su, Enhong Chen
Venue: Conference SIGIR 2022
Year: 2022

Programming has become an important skill for individuals nowadays. For the demand to improve personal programming skill, tracking programming skill proficiency is getting more and more important. However, few researchers pay attention to measuring the programming skill of learners. Most of existing studies on learner capability portrait only made use of the exercise results, while the rich behavioral information contained in programming exercise process remains unused. Therefore, we propose a model that measures skill proficiency in programming exercise process named Programming Skill Tracing (PST). We designed Code Information Graph (CIG) to represent the feature of learners' solution code, and Code Tracing Graph (CTG) to measure the changes between the adjacent submissions. Furthermore, we divided programming skill into programming knowledge and coding ability to get more fine-grained assessment. Finally, we conducted various experiments to verify the effectiveness and interpretability of our PST model.
Representative Negative Instance Generation for Online Ad Targeting
Authors: Yuhan Quan, Jingtao Ding, Depeng Jin, Jianbo Yang, Xing Zhou, Yong Li
Venue: Conference CIKM 2020
Year: 2020

Online ad targeting can be formulated as a problem of learning the relevance ranking among possible audiences for a given ad. It has to deal with the massive number of negative,i.e., non-interacted, instances in impression data due to the nature of this service, and thus suffers from data imbalance problem. In this work, we tackle this problem by improving the quality of negative instances used in training the targeting model. We propose to enhance the generalization capability by introducing unobserved data as possible negative instances, and extract more reliable negative instances from the observed negatives in impression data. However, this idea is non-trivial to implement because of the limited learning signal and existing noise signal. To this end, we design a novel RNIG method (short for Representative Negative Instance Generator) to leverage feature matching technique. It aims to generate reliable negative instances that are similar to the observed negatives and further improves the representativeness of generated negatives by matching the most important feature. Extensive experiments on the real-world ad targeting dataset show that our RNIG model has achieved a relative improvement of more than 5%.
Urban traffic accident risk prediction revisited: Regionality, proximity, similarity and sparsity
Authors: Minxiao Chen, Haitao Yuan, Nan Jiang, Zhifeng Bao, Shangguang Wang
Venue: Conference CIKM 2024
Year: 2024

Traffic accidents pose a significant risk to human health and property safety. Therefore, to prevent traffic accidents, predicting their risks has garnered growing interest. We argue that a desired prediction solution should demonstrate resilience to the complexity of traffic accidents. In particular, it should adequately consider the regional background, accurately capture both spatial proximity and semantic similarity, and effectively address the sparsity of traffic accidents. However, these factors are often overlooked or difficult to incorporate. In this paper, we propose a novel multi-granularity hierarchical spatio-temporal network. Initially, we innovate by incorporating remote sensing data, facilitating the creation of hierarchical multi-granularity structure and the comprehension of regional background. We construct multiple high-level risk prediction tasks to enhance model's ability to cope with sparsity. Subsequently, to capture both spatial proximity and semantic similarity, region feature and multi-view graph undergo encoding processes to distill effective representations. Additionally, we propose message passing and adaptive temporal attention module that bridges different granularities and dynamically captures time correlations inherent in traffic accident patterns. At last, a multivariate hierarchical loss function is devised considering the complexity of the prediction purpose. Extensive experiments on two real datasets verify the superiority of our model against the state-of-the-art methods.
Enhancing the Robustness via Adversarial Learning and Joint Spatial-Temporal Embeddings in Traffic Forecasting
Authors: Juyong Jiang, Binqing Wu, Ling Chen, Kai Zhang, Sunghun Kim
Venue: Conference CIKM 2023
Year: 2023

Traffic forecasting is an essential problem in urban planning and computing. The complex dynamic spatial-temporal dependencies among traffic objects (e.g., sensors and road segments) have been calling for highly flexible models; unfortunately, sophisticated models may suffer from poor robustness especially in capturing the trend of the time series (1st-order derivatives with time), leading to unrealistic forecasts. To address the challenge of balancing dynamics and robustness, we propose TrendGCN, a new scheme that extends the flexibility of GCNs and the distribution-preserving capacity of generative and adversarial loss for handling sequential data with inherent statistical correlations. On the one hand, our model simultaneously incorporates spatial (node-wise) embeddings and temporal (time-wise) embeddings to account for heterogeneous space-and-time convolutions; on the other hand, it uses GAN structure to systematically evaluate statistical consistencies between the real and the predicted time series in terms of both the temporal trending and the complex spatial-temporal dependencies. Compared with traditional approaches that handle step-wise predictive errors independently, our approach can produce more realistic and robust forecasts. Experiments on six benchmark traffic forecasting datasets and theoretical analysis both demonstrate the superiority and the state-of-the-art performance of TrendGCN. Source code is available at https://github.com/juyongjiang/TrendGCN.
Gradient-Directed Composition of Multi-Exposure Images
Authors: Wei Zhang 0021, Wai-kuen Cham
Venue: Conference CVPR 2010
Year: 2010

In this paper, we present a simple yet effective method that takes advantage of the gradient information to accomplish the multi-exposure image composition in both static and dynamic scenes. Given multiple images with different exposures, the proposed approach is capable of producing a pleasant tonemapped-like high dynamic range (HDR) image by compositing them seamlessly with the guidance of gradient-based quality assessment. Especially, two novel quality measures: visibility and consistency, are developed based on the observations of gradient changes among different exposures. Experiments in various static and dynamic scenes are conducted to demonstrate the effectiveness of the proposed method.
KAE-Informer: A Knowledge Auto-Embedding Informer for Forecasting Long-Term Workloads of Microservices
Authors: Qin Hua, Dingyu Yang, Shiyou Qian, Hanwen Hu, Jian Cao, Guangtao Xue
Venue: Conference WWW 2023
Year: 2023

Accurately forecasting workloads in terms of throughput that is quantified as queries per second (QPS) is essential for microservices to elastically adjust their resource allocations. However, long-term QPS prediction is challenging in two aspects: 1) generality across various services with different temporal patterns, 2) characterization of intricate QPS sequences which are entangled by multiple components. In this paper, we propose a knowledge auto-embedding Informer network (KAE-Informer) for forecasting the long-term QPS sequences of microservices. By analyzing a large number of microservice traces, we discover that there are two main decomposable and predictable components in QPS sequences, namely global trend & dominant periodicity (TP) and low-frequency residual patterns with long-range dependencies. These two components are important for accurately forecasting long-term QPS. First, KAE-Informer embeds the knowledge of TP components through mathematical modeling. Second, KAE-Informer designs a convolution ProbSparse self-attention mechanism and a multi-layer event discrimination scheme to extract and embed the knowledge of local context awareness and event regression effect implied in residual components, respectively. We conduct experiments based on three real datasets including a QPS dataset collected from 40 microservices. The experiment results show that KAE-Informer achieves a reduction of MAPE, MAE and RMSE by about 16.6%, 17.6% and 23.1% respectively, compared to the state-of-the-art models.
Automated spatio-temporal synchronous modeling with multiple graphs for traffic prediction
Authors: Fuxian Li, Huan Yan, Guangyin Jin, Yue Liu, Yong Li, Depeng Jin
Venue: Conference CIKM 2022
Year: 2022

Traffic prediction plays an important role in many intelligent transportation systems. Many existing works design static neural network architecture to capture complex spatio-temporal correlations, which is hard to adapt to different datasets. Although recent neural architecture search approaches have addressed this problem, it still adopts a coarse-grained search with pre-defined and fixed components in the search space for spatio-temporal modeling. In this paper, we propose a novel neural architecture search framework, entitled AutoSTS, for automated spatio-temporal synchronous modeling in traffic prediction. To be specific, we design a graph neural network (GNN) based architecture search module to capture localized spatio-temporal correlations, where multiple graphs built from different perspectives are jointly utilized to find a better message passing way for mining such correlations. Further, we propose a convolutional neural network (CNN) based architecture search module to capture temporal dependencies with various ranges, where gated temporal convolutions with different kernel sizes and convolution types are designed in search space. Extensive experiments on six public datasets demonstrate that our model can achieve 4%-10% improvements compared with other methods.
Reinforcement Feature Transformation for Polymer Property Performance Prediction
Authors: Xuanming Hu, Dongjie Wang, Wangyang Ying, Yanjie Fu
Venue: Conference CIKM 2024
Year: 2024

Polymer property performance prediction aims to forecast specific features or attributes of polymers, which has become an efficient ap- proach to measuring their performance. However, existing machine learning models face challenges in effectively learning polymer representations due to low-quality polymer datasets, which conse- quently impact their overall performance. This study focuses on improving polymer property performance prediction tasks by re- constructing an optimal and explainable descriptor representation space. Nevertheless, prior research such as feature engineering and representation learning can only partially solve this task since they are either labor-incentive or unexplainable. This raises two issues: 1) automatic transformation and 2) explainable enhancement. To tackle these issues, we propose our unique Traceable Group-wise Reinforcement Generation Perspective. Specifically, we redefine the reconstruction of the representation space into an interactive pro- cess, combining nested generation and selection. Generation creates meaningful descriptors, and selection eliminates redundancies to control descriptor sizes. Our approach employs cascading reinforce- ment learning with three Markov Decision Processes, automating descriptor and operation selection, and descriptor crossing. We utilize a group-wise generation strategy to explore and enhance reward signals for cascading agents. Ultimately, we conduct experi- ments to indicate the effectiveness of our proposed framework.