OpenRaiser/Intern-Atlas huggingface dataset:
Citytraffic: Modeling citywide traffic via neural memorization and generalization approach
Authors: Xiuwen Yi, Zhewen Duan, Ting Li, Tianrui Li, Junbo Zhang, Yu Zheng
Venue: Conference CIKM 2019
Year: 2019

With the increasing vehicles on the road, it is becoming more and more important to sense citywide traffic, which is of great benefit to the government's policy-making and people's decision making. Currently, traffic speed and volume information are mostly derived from GPS trajectories data and volume sensor records respectively. Unfortunately, speed and volume information suffer from serious data missing problem. Speed can be absent at arbitrary road segment and time slot, while volume is only recorded by limited volume sensors. For modeling citywide traffic, inspired by the observations of missing patterns and prior knowledge about traffic, we propose a neural memorization and generalization approach to infer the missing speed and volume, which mainly consists of a memorization module for speed inference and a generalization module for volume inference. Considering the temporal closeness and period properties, memorization module takes advantage of neural multi-head self-attention architecture to memorize the intrinsic correlations from historical traffic information. Generalization module adopts neural key-value attention architecture to generalize the extrinsic dependencies among volume sensors by exploiting road contexts. We conduct extensive experiments on two real-world datasets in two cities, Guiyang and Jinan, and the experimental results consistently demonstrate the advantages of our approach. We have developed a real-time system on the cloud, entitled CityTraffic, providing citywide traffic speed and volume information and fine-grained pollutant emission of vehicles in Guiyang city.
Harnessing empathy and ethics for relevance detection and information categorization in climate and covid-19 tweets
Authors: Apoorva Upadhyaya, Wolfgang Nejdl, Marco Fisichella
Venue: Conference CIKM 2024
Year: 2024

In this work, we aim to understand the general public perception of societal issues related to the current climate crisis and the COVID-19 pandemic on Twitter (X). Social media discussions on such matters often lead to misleading information, resulting in delays in initiatives proposed by governments or policymakers. Hence, we focus on extracting relevant information from the conversations on climate change and COVID that could be useful for authorities to curb the spread of potentially biased information by proposing the classification tasks of relevance detection (RD) and information categorization (IC). We first curate the datasets for the RD and IC tasks for the climate domain and extend the COVID-19 benchmark attention-worthy Twitter dataset for the IC task through manual annotation. We initially conduct experiments with LLMs and observe that LLMs can extract the relevant information in zero and few-shot settings based on multi-perspective reasoning in the form of cognitive empathy and ethical standards, but still perform worse than fine-tuned small language models. Based on the initial findings, we conclude that LLMs may not be the best extractor of relevant information, but induce cognitive empathy and ethical reasonings that can intuitively guide supervised models. To achieve this idea, we develop a cognitive empathy and ethical reasoning-based multi-tasking pipelined network for RD and IC tasks. Our proposed approach provides valuable insights that could be useful in real-world scenarios for governments, policymakers, and other researchers to decode the overall public outlook on societal issues.
Collabequality: A crowd-ai collaborative learning framework to address class-wise inequality in web-based disaster response
Authors: Yang Zhang, Lanyu Shang, Ruohan Zong, Huimin Zeng, Zhenrui Yue, Dong Wang
Venue: Conference WWW 2023
Year: 2023

Web-based disaster response (WebDR) is emerging as a pervasive approach to acquire real-time situation awareness of disaster events by collecting timely observations from the Web (e.g., social media). This paper studies a class-wise inequality problem in WebDR applications where the objective is to address the limitation of current WebDR solutions that often have imbalanced classification performance across different classes. To address such a limitation, this paper explores the collaborative strengths of the diversified yet complementary biases of AI and crowdsourced human intelligence to ensure a more balanced and accurate performance for WebDR applications. However, two critical challenges exist: 1) it is difficult to identify the imbalanced AI results without knowing the ground-truth WebDR labels a priori; ii) it is non-trivial to address the class-wise inequality problem using potentially imperfect crowd labels. To address the above challenges, we develop CollabEquality, an inequality-aware crowd-AI collaborative learning framework that carefully models the inequality bias of both AI and human intelligence from crowdsourcing systems into a principled learning framework. Extensive experiments on two real-world WebDR applications demonstrate that CollabEquality consistently outperforms the state-of-the-art baselines by significantly reducing class-wise inequality while improving the WebDR classification accuracy.
Covariance discriminative learning: A natural and efficient approach to image set classification
Authors: Ruiping Wang 0001, Huimin Guo, Larry S. Davis, Qionghai Dai
Venue: Conference CVPR 2012
Year: 2012

We propose a novel discriminative learning approach to image set classification by modeling the image set with its natural second-order statistic, i.e. covariance matrix. Since nonsingular covariance matrices, a.k.a. symmetric positive definite (SPD) matrices, lie on a Riemannian manifold, classical learning algorithms cannot be directly utilized to classify points on the manifold. By exploring an efficient metric for the SPD matrices, i.e., Log-Euclidean Distance (LED), we derive a kernel function that explicitly maps the covariance matrix from the Riemannian manifold to a Euclidean space. With this explicit mapping, any learning method devoted to vector space can be exploited in either its linear or kernel formulation. Linear Discriminant Analysis (LDA) and Partial Least Squares (PLS) are considered in this paper for their feasibility for our specific problem. We further investigate the conventional linear subspace based set modeling technique and cast it in a unified framework with our covariance matrix based modeling. The proposed method is evaluated on two tasks: face recognition and object categorization. Extensive experimental results show not only the superiority of our method over state-of-the-art ones in both accuracy and efficiency, but also its stability to two real challenges: noisy set data and varying set size.
New localization frameworks: User-centric approaches to source localization in real-world propagation scenarios
Authors: Dongpeng Hou, Yuchen Wang, Chao Gao, Xianghua Li, Zhen Wang
Venue: Conference CIKM 2024
Year: 2024

Source localization in social platforms is critical for managing and controlling the misinformation spreading. Despite all the recent advancements, existing methods do not consider the dynamic and heterogeneous propagation behaviors of users and are developed based on simulated data with strong model assumptions, limiting the application in real-world scenarios. This research addresses this limitation by presenting a novel framework for source localization, grounded in real-world propagation cascades from platforms like Weibo and Twitter. What's more, recognizing the user-driven nature of users in information spread, we systematically crawl and integrate user-specific profiles, offering a realistic understanding of user-driven propagation dynamics. In summary, by developing datasets derived from real-world propagation cascades, we set a precedent in enhancing the authenticity and practice of source identification for social media. Our comprehensive experiments not only validate the feasibility and rationale of our novel user-centric localization approaches but also emphasize the significance of considering user profiles in real-world propagation scenarios. The code is available at https://github.com/cgao-comp/NFSL.
Sketch-based anomaly detection in streaming graphs
Authors: Siddharth Bhatia 0001, Mohit Wadhwa, Kenji Kawaguchi, Neil Shah, Philip S. Yu, Bryan Hooi
Venue: Conference KDD 2023
Year: 2023

Given a stream of graph edges from a dynamic graph, how can we assign anomaly scores to edges and subgraphs in an online manner, for the purpose of detecting unusual behavior, using constant time and memory? For example, in intrusion detection, existing work seeks to detect either anomalous edges or anomalous subgraphs, but not both. In this paper, we first extend the count-min sketch data structure to a higher-order sketch. This higher-order sketch has the useful property of preserving the dense subgraph structure (dense subgraphs in the input turn into dense submatrices in the data structure). We then propose 4 online algorithms that utilize this enhanced data structure, which (a) detect both edge and graph anomalies; (b) process each edge and graph in constant memory and constant update time per newly arriving edge, and; (c) outperform state-of-the-art baselines on 4 real-world datasets. Our method is the first streaming approach that incorporates dense subgraph search to detect graph anomalies in constant memory and time.
Feedback-guided anomaly discovery via online optimization
Authors: Md Amran Siddiqui, Alan Fern, Thomas G. Dietterich, Ryan Wright, Alec Theriault, David W. Archer
Venue: Conference KDD 2018
Year: 2018

Anomaly detectors are often used to produce a ranked list of statistical anomalies, which are examined by human analysts in order to extract the actual anomalies of interest. This can be exceedingly difficult and time consuming when most high-ranking anomalies are false positives and not interesting from an application perspective. In this paper, we study how to reduce the analyst's effort by incorporating their feedback about whether the anomalies they investigate are of interest or not. In particular, the feedback will be used to adjust the anomaly ranking after every analyst interaction, ideally moving anomalies of interest closer to the top. Our main contribution is to formulate this problem within the framework of online convex optimization, which yields an efficient and extremely simple approach to incorporating feedback compared to the prior state-of-the-art. We instantiate this approach for the powerful class of tree-based anomaly detectors and conduct experiments on a range of benchmark datasets. The results demonstrate the utility of incorporating feedback and advantages of our approach over the state-of-the-art. In addition, we present results on a significant cybersecurity application where the goal is to detect red-team attacks in real system audit data. We show that our approach for incorporating feedback is able to significantly reduce the time required to identify malicious system entities across multiple attacks on multiple operating systems.
Unsupervised heterogeneous graph rewriting attack via node clustering
Authors: Haosen Wang, Can Xu, Chenglong Shi, Pengfei Zheng, Shiming Zhang, Minhao Cheng, Hongyang Chen 0001
Venue: Conference KDD 2024
Year: 2024

Self-supervised learning (SSL) has become one of the most popular learning paradigms and has achieved remarkable success in the graph field. Recently, a series of pre-training studies on heterogeneous graphs (HGs) using SSL have been proposed considering the heterogeneity of real-world graph data. However, verification of the robustness of heterogeneous graph pre-training is still a research gap. Most existing researches focus on supervised attacks on graphs, which are limited to a specific scenario and will not work when labels are not available. In this paper, we propose a novel unsupervised heterogeneous graph rewriting attack via node clustering (HGAC) that can effectively attack HG pre-training models without using labels. Specifically, a heterogeneous edge rewriting strategy is designed to ensure the rationality and concealment of the attacks. Then, a tailored heterogeneous graph contrastive learning (HGCL) is used as a surrogate model. Moreover, we leverage node clustering results of the clean HGs as the pseudo-labels to guide the optimization of structural attacks. Extensive experiments exhibit powerful attack performances of our HGAC on various downstream tasks (i.e., node classification, node clustering, metapath prediction, and visualization) under poisoning attack and evasion attack.
Maximizing malicious influence in node injection attack
Authors: Xiao Zhang, Peng Bao, Shirui Pan
Venue: Conference WSDM 2024
Year: 2024

Graph neural networks (GNNs) have achieved impressive performance in various graph-related tasks. However, recent studies have found that GNNs are vulnerable to adversarial attacks. Node injection attacks (NIA) become an emerging scenario of graph adversarial attacks, where the attacks are performed by injecting malicious nodes into the original graph instead of directly modifying it. In this paper, we focus on a more realistic scenario of NIA, where the attacker is only allowed to inject a small number of nodes to degrade the performance of GNNs with very limited information. We analyze the susceptibility of nodes, and based on this we propose a global node injection attack framework, MaxiMal, to maximize malicious information under a strict black-box setting. MaxiMal first introduces a susceptible-reverse influence sampling strategy to select neighbor nodes that are able to spread malicious information widely. Then contrastive loss is introduced to optimize the objective by updating the edges and features of the injected nodes. Extensive experiments on three benchmark datasets demonstrate the superiority of our proposed MaxiMal over the state-of-the-art approaches.
Posted price mechanisms for online allocation with diseconomies of scale
Authors: Hossein Nekouyan Jazi, Bo Sun, Raouf Boutaba, Xiaoqi Tan
Venue: Conference WWW 2025
Year: 2025

This paper addresses the online $k$-selection problem with diseconomies of scale (OSDoS), where a seller seeks to maximize social welfare by optimally pricing items for sequentially arriving buyers, accounting for increasing marginal production costs. Previous studies have investigated deterministic dynamic pricing mechanisms for such settings. However, significant challenges remain, particularly in achieving optimality with small or finite inventories and developing effective randomized posted price mechanisms. To bridge this gap, we propose a novel randomized dynamic pricing mechanism for OSDoS, providing a tighter lower bound on the competitive ratio compared to prior work. Our approach ensures optimal performance in small inventory settings (i.e., when $k$ is small) and surpasses existing online mechanisms in large inventory settings (i.e., when $k$ is large), leading to the best-known posted price mechanism for optimizing online selection and allocation with diseconomies of scale across varying inventory sizes.
Forward once for all: Structural parameterized adaptation for efficient cloud-coordinated on-device recommendation
Authors: Kairui Fu, Zheqi Lv, Shengyu Zhang, Fan Wu, Kun Kuang
Venue: Conference KDD 2025
Year: 2025

In cloud-centric recommender system, regular data exchanges between user devices and cloud could potentially elevate bandwidth demands and privacy risks. On-device recommendation emerges as a viable solution by performing reranking locally to alleviate these concerns. Existing methods primarily focus on developing local adaptive parameters, while potentially neglecting the critical role of tailor-made model architecture. Insights from broader research domains suggest that varying data distributions might favor distinct architectures for better fitting. In addition, imposing a uniform model structure across heterogeneous devices may result in risking inefficacy on less capable devices or sub-optimal performance on those with sufficient capabilities. In response to these gaps, our paper introduces Forward-OFA, a novel approach for the dynamic construction of device-specific networks (both structure and parameters). Forward-OFA employs a structure controller to selectively determine whether each block needs to be assembled for each device. However, during the training of the structure controller, these assembled heterogeneous structures are jointly optimized, where the co-adaption among blocks might encounter gradient conflicts. To mitigate this, Forward-OFA is designed to establish a structure-guided mapping of real-time behaviors to individual parameters of assembled networks. Structure-related parameters and parallel components within the mapper prevent each part from receiving heterogeneous gradients from others, thus bypassing the gradient conflicts for coupled optimization. Besides, direct mapping enables Forward-OFA to achieve adaptation through only one forward pass, allowing for swift adaptation to changing interests and eliminating the requirement for on-device backpropagation. Further sophisticated design protects user privacy and makes the consumption of additional modules on device negligible. Experiments on real-world datasets demonstrate the effectiveness and efficiency of Forward-OFA.
Collaborative sequence prediction for sequential recommender
Authors: Shuzi Niu, Rongzhi Zhang
Venue: Conference CIKM 2017
Year: 2017

With the surge of deep learning, more and more attention has been put on the sequential recommender. It can be casted as sequence prediction problem, where we will predict the next item given the previous items. RNN approaches are able to capture the global sequential features from the data compared with the local features derived in Markov Chain methods. However, both approaches rely on the independence of users' sequences, which are not true in practice. We propose to formulate the sequential recommendation problem as collaborative sequence prediction problem to take the dependency of users' sequences into account. In order to solve the collaborative sequence prediction problem, we define the dynamic neighborhood relationship between users and introduce manifold regularization to RNN on the basis of the multi-facets of collaborative filtering, referred to as MrRNN. Experimental results on benchmark datasets show that our approach outperforms the state-of-the-art baselines.
On-device learning for model personalization with large-scale cloud-coordinated domain adaption
Authors: Yikai Yan, Chaoyue Niu, Renjie Gu, Fan Wu 0006, Shaojie Tang 0001, Lifeng Hua, Chengfei Lyu, Guihai Chen
Venue: Conference KDD 2022
Year: 2022

Cloud-based learning is currently the mainstream in both academia and industry. However, the global data distribution, as a mixture of all the users' data distributions, for training a global model may deviate from each user's local distribution for inference, making the global model non-optimal for each individual user. To mitigate distribution discrepancy, on-device training over local data for model personalization is a potential solution, but suffers from serious overfitting. In this work, we propose a new device-cloud collaborative learning framework under the paradigm of domain adaption, called MPDA, to break the dilemmas of purely cloud-based learning and on-device training. From the perspective of a certain user, the general idea of MPDA is to retrieve some similar data from the cloud's global pool, which functions as large-scale source domains, to augment the user's local data as the target domain. The key principle of choosing which outside data depends on whether the model trained over these data can generalize well over the local data. We theoretically analyze that MPDA can reduce distribution discrepancy and overfitting risk. We also extensively evaluate over the public MovieLens 20M and Amazon Electronics datasets, as well as an industrial dataset collected from Mobile Taobao over a period of 30 days. We finally build a device-tunnel-cloud system pipeline, deploy MPDA in the icon area of Mobile Taobao for click-through rate prediction, and conduct online A/B testing. Both offline and online results demonstrate that MPDA outperforms the baselines of cloud-based learning and on-device training only over local data, from multiple offline and online metrics.
Scaling sequential recommendation models with transformers
Authors: Pablo Zivic, Hernán Ceferino Vázquez, Jorge Sánchez
Venue: Conference SIGIR 2024
Year: 2024

Modeling user preferences has been mainly addressed by looking at users' interaction history with the different elements available in the system. Tailoring content to individual preferences based on historical data is the main goal of sequential recommendation. The nature of the problem, as well as the good performance observed across various domains, has motivated the use of the transformer architecture, which has proven effective in leveraging increasingly larger amounts of training data when accompanied by an increase in the number of model parameters. This scaling behavior has brought a great deal of attention, as it provides valuable guidance in the design and training of even larger models. Taking inspiration from the scaling laws observed in training large language models, we explore similar principles for sequential recommendation. Addressing scalability in this context requires special considerations as some particularities of the problem depart from the language modeling case. These particularities originate in the nature of the content catalogs, which are significantly larger than the vocabularies used for language and might change over time. In our case, we start from a well-known transformer-based model from the literature and make two crucial modifications. First, we pivot from the traditional representation of catalog items as trainable embeddings to representations computed with a trainable feature extractor, making the parameter count independent of the number of items in the catalog. Second, we propose a contrastive learning formulation that provides us with a better representation of the catalog diversity. We demonstrate that, under this setting, we can train our models effectively on increasingly larger datasets under a common experimental setup. We use the full Amazon Product Data dataset, which has only been partially explored in other studies, and reveal scaling behaviors similar to those found in language models. Compute-optimal training is possible but requires a careful analysis of the compute-performance trade-offs specific to the application. We also show that performance scaling translates to downstream tasks by fine-tuning larger pre-trained models on smaller task-specific domains. Our approach and findings provide a strategic roadmap for model training and deployment in real high-dimensional preference spaces, facilitating better training and inference efficiency. We hope this paper bridges the gap between the potential of transformers and the intrinsic complexities of high-dimensional sequential recommendation in real-world recommender systems. Code and models can be found at https://github.com/mercadolibre/srt.