OpenRaiser/Intern-Atlas huggingface dataset:
Unsupervised Anomaly Detection & Diagnosis: A Stein Variational Gradient Descent Approach
Authors: Zhichao Chen, Leilei Ding, Jianmin Huang, Zhixuan Chu, Qingyang Dai, Hao Wang
Venue: CIKM
Year: 2023

Detecting and diagnosing anomalies in observational data plays a crucial role in various real-world applications, such as e-commerce applet maintenance. Unsupervised machine learning techniques are typically employed for anomaly detection and diagnosis due to their convenience and independence from labeled data. Density estimation (DE), as one of the most widely used unsupervised machine learning techniques for anomaly detection, can be categorized into kernel density estimation (KDE)-based methods and normalizing flow (NF)-based methods. While KDE-based methods offer fast computation speed, they often ignore the complex manifold structure present in observational data. On the other hand, NF-based methods address the manifold issue but suffer from longer computation times. In this study, we propose a novel DE-based anomaly detection & diagnosis method using Stein Variational Gradient Descent (SVGD), aiming to leverage the strengths of KDE and NF approaches. Firstly, we rigorously derive the DE capability of SVGD through mathematical analysis. Subsequently, we demonstrate the ability of the SVGD method to perform anomaly diagnosis based on input feature attribution. Finally, to validate the effectiveness of our approach, we conduct experiments using synthetic, benchmark, and industrial datasets. The results demonstrate the superior performance and practical applicability of our proposed method.
Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
Authors: Yuyan Chen, Qiang Fu, Ge Fan, Lun Du, Jian-Guang Lou, Shi Han, Dongmei Zhang, Zhixu Li, Yanghua Xiao
Venue: CIKM
Year: 2023

Recent years, Pre-trained Language models (PLMs) have swept into various fields of artificial intelligence and achieved great success. However, most PLMs, such as T5 and GPT3, have a huge amount of parameters, fine-tuning them is often expensive and time consuming, and storing them takes up a lot of space. Therefore, it is necessary to adopt a parameter-efficient approach to reduce parameters of PLMs in fine-tuning without compromising their performance in downstream tasks. In this paper, we design a novel adapter which only acts on self-attention outputs in PLMs. This adapter adopts element-wise linear transformation using Hadamard product, hence named as Hadamard adapter, requires the fewest parameters compared to previous parameter-efficient adapters. In addition, we also summarize some tuning patterns for Hadamard adapter shared by various downstream tasks, expecting to provide some guidance for further parameter reduction with shared adapters in future studies. The experiments conducted on the widely-used GLUE benchmark with several SOTA PLMs prove that the Hadamard adapter achieves competitive performance with only 0.033% parameters compared with full fine-tuning, and it has the fewest parameters compared with other adapters. Moreover, we further find that there is also some redundant layers in the Hadamard adapter which can be removed to achieve more parameter efficiency with only 0.022% parameters.
Meta-Transfer-Learning for Time Series Data with Extreme Events: An Application to Water Temperature Prediction
Authors: Shengyu Chen, Nasrin Kalanat, Simon N. Topp, Jeffrey M. Sadler, Yiqun Xie, Zhe Jiang, Xiaowei Jia
Venue: CIKM
Year: 2023

This paper proposes a meta-transfer-learning method for predicting daily maximum water temperature in stream networks with explicit modeling of extreme events. Accurate prediction of these extreme events is challenging because of their sparsity in the training data and their distinct responses to external drivers when compared to non-extreme observations. To overcome these challenges, we propose a sample reweighting strategy to escalate the importance of extreme events in the training process while preserving the predictive performance in normal time periods. The sample weight for each training data point is estimated as the similarity with the target test data point using contextual information and physical simulation. The obtained sample weight values are then used to fine-tune the initial model to transfer it to the test data. This method is further enhanced by an extreme value theory-based loss function to enforce the distribution of extreme data points and accelerated by a clustering algorithm based on the estimated similarities. Additionally, we introduce an online learning strategy to further refine the predictive model using newly collected observed data. The experimental results using real stream data from the Delaware River Basin over the past 36 years demonstrate that our meta-transfer-learning method produces more accurate predictions in both normal and extreme time periods when compared to baselines without the sample re-weighting scheme. The similarity learning method can reveal meaningful relationships amongst data points. We also show that the clustering algorithm can be used to accelerate the prediction while not compromising the predictive performance. The online learning strategy is shown to further improve predictive performance using recently observed data.
Segment Augmentation and Prediction Consistency Neural Network for Multi-label Unknown Intent Detection
Authors: Miaoxin Chen, Cao Liu, Boqi Dai, Hai-Tao Zheng, Ting Song, Jiansong Chen, Guanglu Wan, Rui Xie
Venue: CIKM
Year: 2023

Multi-label unknown intent detection is a challenging task where each utterance may contain not only multiple known but also unknown intents. To tackle this challenge, pioneers proposed to predict the intent number of the utterance first, then compare it with the results of known intent matching to decide whether the utterance contains unknown intent(s). Though they have made remarkable progress on this task, their method still suffers from two important issues: 1) It is inadequate to extract multiple intents using only utterance encoding; 2) Optimizing two sub-tasks (intent number prediction and known intent matching) independently leads to inconsistent predictions. In this paper, we propose to incorporate segment augmentation rather than only use utterance encoding to better detect multiple intents. We also design a prediction consistency module to bridge the gap between the two sub-tasks. Empirical results on MultiWOZ2.3 show that our method achieves state-of-the-art performance and improves the best baseline significantly.
Region Profile Enhanced Urban Spatio-Temporal Prediction via Adaptive Meta-Learning
Authors: Jie Chen, Tong Liu, Ruiyuan Li
Venue: CIKM
Year: 2023

Urban spatio-temporal (ST) prediction plays a crucial role in smart city construction. Due to the high cost of ST data collection, improving ST prediction in a lack of data is significant. For this purpose, existing meta-learning methods have been demonstrated powerful by learning an initial network from training tasks and adjusting to target tasks with limited data. However, such shared knowledge from a set of tasks may contain irrelevant noise due to the gap of region-varying ST dynamics, resulting in the negative transfer issue. As a revelation of regional functional patterns, region profiles give rise to the diversity of ST dynamics. Thus, we design a novel adaptive meta-optimized model MetaRSTP, which conducts the initial prediction model in a finer-granularity of region level with region profiles as semantic evidence. To enhance the expressiveness of profiles, we firstly build a semantic alignment space to explore the inter-view co-semantics. Fusing it with view-specific uniqueness, the multi-view region profiles can be better applied in urban tasks. Then, a regional bias generator derives non-shared parameters in terms of profiles, which alleviates the divergence among regions. We set a new meta-learning strategy as initialize the network with fixed generalizable parameters and region-adaptive bias, thus enhancing the personalized prediction performance even in few-shot scenarios. Extensive experiments on real-world datasets illustrate the effectiveness of our MetaRSTP and our learned region profiles.
TCCM: Time and Content-Aware Causal Model for Unbiased News Recommendation
Authors: Yewang Chen, Weiyao Ye, Guipeng Xv, Chen Lin, Xiaomin Zhu
Venue: CIKM
Year: 2023

Popularity bias significantly impacts news recommendation systems, as popular news articles receive more exposure and are often delivered to irrelevant users, resulting in unsatisfactory performance. Existing methods have not adequately addressed the issue of popularity bias in news recommendations, largely due to the neglect of the time factor and the impact of news content on popularity. In this paper, we propose a novel approach called Time and Content-aware Causal Model, namely TCCM. It models the effects of three factors on user interaction behavior, i.e., the time factor, the news popularity, and the matching between news content and user interest. TCCM also estimates news popularity more accurately by incorporating the news content, i.e., the popularity of entity and words. Causal intervention techniques are applied to obtain debiased recommendations. Extensive experiments on well-known benchmark datasets demonstrate that the proposed approach outperforms a range of state-of-the-art techniques.
Identify Risky Rules to Reduce Side Effects in Association Rule Hiding
Authors: Peng Cheng
Venue: CIKM
Year: 2023

Data sharing is necessary for many practical applications. People do, however, frequently worry about the problem of privacy leaking.This study focuses on preventing the disclosure of sensitive information using association rules and frequent itemsets, which are frequently utilized in numerous applications. How to minimize side effects while hiding, particularly side effects on non-sensitive knowledge, is the difficult part of the problem. The majority of association rule hiding techniques currently in use solely consider reducing side effects on frequent itemsets (patterns), rather than rules, in order to conceal sensitive rules by reducing the statistical disclosure of the itemsets that generate such rules. In this study, we provide a concealment technique utilizing potentially risky rules to lessen adverse impacts on non-sensitive rules, not only itemsets. In addition, this method can be tailored to conceal sensitive itemsets, instead of rules. Extensive experiments show that in most cases the proposed solution can bring fewer side effects on rules, frequent patterns or data quality than existing methods.
MI-DPG: Decomposable Parameter Generation Network Based on Mutual Information for Multi-Scenario Recommendation
Authors: Wenzhuo Cheng, Ke Ding, Xin Dong, Yong He, Liang Zhang, Linjian Mo
Venue: CIKM
Year: 2023

Conversion rate (CVR) prediction models play a vital role in recommendation systems. Recent research shows that learning a unified model to serve multiple scenarios is effective for improving overall performance. However, it remains challenging to improve model prediction performance across scenarios at low model parameter cost, and current solutions are hard to robustly model multi-scenario diversity. In this paper, we propose MI-DPG for the multi-scenario CVR prediction, which learns scenario-conditioned dynamic model parameters for each scenario in a more efficient and effective manner. Specifically, we introduce an auxiliary network to generate scenario-conditioned dynamic weighting matrices, which are obtained by combining decomposed scenario-specific and scenario-shared low-rank matrices with parameter efficiency. For each scenario, weighting the backbone model parameters by the weighting matrix helps to specialize the model parameters for different scenarios. It can not only modulate the complete parameter space of the backbone model but also improve the model effectiveness. Furthermore, we design a mutual information regularization to enhance the diversity of model parameters across scenarios by maximizing the mutual information between the scenario-aware input and the scenario-conditioned dynamic weighting matrix. Experiments from three real-world datasets show that MI-DPG outperforms previous multi-scenario recommendation models.
DAS-CL: Towards Multimodal Machine Translation via Dual-Level Asymmetric Contrastive Learning
Authors: Xuxin Cheng, Zhihong Zhu, Yaowei Li, Hongxiang Li, Yuexian Zou
Venue: CIKM
Year: 2023

Multimodal machine translation (MMT) aims to exploit visual information to improve neural machine translation (NMT). It has been demonstrated that image captioning and object detection can further improve MMT. In this paper, to leverage image captioning and object detection more effectively, we propose a Dual-level ASymmetric Contrastive Learning (DAS-CL) framework. Specifically, we leverage image captioning and object detection to generate more pairs of visual inputs and textual inputs. At the utterance level, we introduce an image captioning model to generate more coarse-grained pairs. At the word level, we introduce an object detection model to generate more fine-grained pairs. To mitigate the negative impact of noise in generated pairs, we apply asymmetric contrastive learning at these two levels. Experiments on the Multi30K dataset of three translation directions demonstrate that DAS-CL significantly outperforms existing MMT frameworks and achieves new state-of-the-art performance. More encouragingly, further analysis displays that DAS-CL is more robust to irrelevant visual information.
ParkFlow: Intelligent Dispersal for Mitigating Parking Shortages Using Multi-Granular Spatial-Temporal Analysis
Authors: Yang Fan Chiang, Chun-Wei Shen, Jhe-Wei Tsai, Pei-Xuan Li, Tzu-Chang Lee, Hsun-Ping Hsieh
Venue: CIKM
Year: 2023

Parking behaviors near popular destinations often exhibit a preference for proximity, resulting in poor habits, limited parking availability, and a range of consequential issues such as traffic chaos, economic challenges due to congestion, and imbalanced parking utilization. Taiwan has also faced escalating challenges in this regard. To effectively address these issues, the Government of Taiwan has initiated the Smart City program, encompassing various initiatives to enhance urban functionality. One notable solution implemented under this program is the Smart Parking Meter System (SPMS), designed to enhance the overall parking experience. The SPMS incorporates intelligent billing and secure parking data transmission, ensuring a safer and improved parking environment. In this paper, we propose ParkFlow, a comprehensive software-based solution that seamlessly integrates with smart parking hardware, presenting a holistic approach to tackling these challenges. ParkFlow intelligently disperses parking shortages in highly frequented areas and addresses the problem from multiple perspectives, including user, engineering, and government scenarios. By exploring and addressing these scenarios, we aim to provide valuable insights and inspiration to regions worldwide grappling with similar parking-related difficulties. Based on historical data analysis, the implementation of ParkFlow in resolving the parking imbalance problem is anticipated to lead to a significant increase of up to 10% to 20% in available parking hours in popular areas of Tainan, Taiwan. ParkFlow is in the process of being integrated into the Tainan City Government's Parking application, indicating its potential to address real-world parking challenges.
PCT-CycleGAN: Paired Complementary Temporal Cycle-Consistent Adversarial Networks for Radar-Based Precipitation Nowcasting
Authors: Jaeho Choi, Yura Kim, Kwang-Ho Kim, Sung-Hwa Jung, Ikhyun Cho
Venue: CIKM
Year: 2023

The precipitation nowcasting methods have been elaborated over the centuries because rain has a crucial impact on human life. Not only quantitative precipitation forecast (QPF) models and convolutional long short-term memory (ConvLSTM), but also various sophisticated methods such as the latest MetNet-2 are emerging. In this paper, we propose a paired complementary temporal cycle-consistent adversarial networks (PCT-CycleGAN) for radar-based precipitation nowcasting, inspired by cycle-consistent adversarial networks (CycleGAN), which shows strong performance in image-to-image translation. PCT-CycleGAN generates temporal causality using two generator networks with forward and backward temporal dynamics in paired complementary cycles. Each generator network learns a huge number of one-to-one mappings about time-dependent radar-based precipitation data to approximate a mapping function representing the temporal dynamics in each direction. To create robust temporal causality between paired complementary cycles, novel connection loss is proposed. And torrential loss to cover exceptional heavy rain events is also proposed. The generator network learning forward temporal dynamics in PCT-CycleGAN generates radar-based precipitation data 10 minutes from the current time. Also, it provides a reliable prediction of up to 2 hours with iterative forecasting. The superiority of PCT-CycleGAN is demonstrated through qualitative and quantitative comparisons with several previous methods.
Patient Clustering via Integrated Profiling of Clinical and Digital Data
Authors: Dongjin Choi, Andy Xiang, Ozgur Ozturk, Deep Shrestha, Barry L. Drake, Hamid Haidarian, Faizan Javed, Haesun Park
Venue: CIKM
Year: 2023

We introduce a novel profile-based patient clustering model designed for healthcare clinical data. By utilizing a method grounded on constrained low-rank approximation, our model takes advantage of patients' clinical data and digital interaction data, including browsing and search, to construct patient profiles. As a result of the method, nonnegative embedding vectors are generated, serving as a low-dimensional representation of the patients. Our model was assessed using real-world patient data from a healthcare web portal, with a comprehensive evaluation approach which considered clustering and recommendation capabilities. In comparison to other baselines, our approach demonstrated superior performance in terms of clustering coherence and recommendation accuracy.
Delivery Optimized Discovery in Behavioral User Segmentation under Budget Constraint
Authors: Harshita Chopra, Atanu R. Sinha, Sunav Choudhary, Ryan A. Rossi, Paavan Kumar Indela, Veda Pranav Parwatala, Srinjayee Paul, Aurghya Maiti
Venue: CIKM
Year: 2023

Users' behavioral footprints online enable firms to discover behavior-based user segments (or, segments) and deliver segment specific messages to users. Following the discovery of segments, delivery of messages to users through preferred media channels like Facebook and Google can be challenging, as only a portion of users in a behavior segment find match in a medium, and only a fraction of those matched actually see the message (exposure). Even high quality discovery becomes futile when delivery fails. Many sophisticated algorithms exist for discovering behavioral segments; however, these ignore the delivery component. The problem is compounded because (i) the discovery is performed on the behavior data space in firms' data (e.g., user clicks), while the delivery is predicated on the static data space (e.g., geo, age) as defined by media; and (ii) firms work under budget constraint. We introduce a stochastic optimization based algorithm for delivery optimized discovery of behavioral user segments and offer new metrics to address the joint optimization. We leverage optimization under a budget constraint for delivery combined with a learning-based component for discovery. Extensive experiments on a public dataset from Google and a proprietary dataset show the effectiveness of our approach by simultaneously improving delivery metrics, reducing budget spend and achieving strong predictive performance in discovery.
Incorporating Co-purchase Correlation for Next-basket Recommendation
Authors: Yu Hao Chou, Pu-Jen Cheng
Venue: CIKM
Year: 2023

Next-basket recommendation (NBR) aims to recommend a set of items that users would most likely purchase together. Existing approaches use deep learning to capture basket-level preference and traditional statistical methods to model user behavior sequences. However, these methods neglect the correlation of co-purchase items among users. We, therefore, propose a novel model that incorporates Co-purchase Correlation with Bidirectional Transformer (CCBT) to enhance item representation by exploiting the correlation among users' baskets. The results of experiments conducted on four real-world datasets demonstrate the proposed model outperforms state-of-the-art NBR methods. The relative improvement for Recall@20 ranges from 11% to 27%.
Continually-Adaptive Representation Learning Framework for Time-Sensitive Healthcare Applications
Authors: Akash Choudhuri, Hankyu Jang, Alberto M. Segre, Philip M. Polgreen, Kishlay Jha, Bijaya Adhikari
Venue: CIKM
Year: 2023

Continual learning has emerged as a powerful approach to address the challenges of non-stationary environments, allowing machine learning models to adapt to new data while retaining the previously acquired knowledge. In time-sensitive healthcare applications, where entities such as physicians, hospital rooms, and medications exhibit continuous changes over time, continual learning holds great promise, yet its application remains relatively unexplored. This paper aims to bridge this gap by proposing a novel framework, i.e., Continually-Adaptive Representation Learning, designed to adapt representations in response to changing data distributions in evolving healthcare applications. Specifically, the proposed approach develops a continual learning strategy wherein the context information (e.g., interactions) of healthcare entities is exploited to continually identify and retrain the representations of those entities whose context evolved over time. Moreover, different from existing approaches, the proposed approach leverages the valuable patient information present in clinical notes to generate accurate and robust healthcare embeddings. Notably, the proposed continually-adaptive representations have practical benefits in low-resource clinical settings where it is difficult to training machine learning models from scratch to accommodate the newly available data streams. Experimental evaluations on real-world healthcare datasets demonstrate the effectiveness of our approach in time-sensitive healthcare applications such as Clostridioides difficile (C.diff) Infection (CDI) incidence prediction task and medical intensive care unit transfer prediction task.
Learning Invariant Representations for New Product Sales Forecasting via Multi-Granularity Adversarial Learning
Authors: Zhenzhen Chu, Chengyu Wang, Cen Chen, Dawei Cheng, Yuqi Liang, Weining Qian
Venue: CIKM
Year: 2023

Sales forecasting during the launch of new products has always been a challenging task, due to the lack of historical sales data. The dynamic market environment and consumer preferences also increase the uncertainty of predictions. Large chains face even greater difficulties due to their extensive presence across various regions. Traditional time-series forecasting methods usually rely on statistical models and empirical judgments, which are difficult to handle large, variable data and often fail to achieve satisfactory performance for new products. In this paper, we propose a Multi-granularity AdversaRial Learning framework (MARL) to leverage knowledge from old products and improve the quality of invariant representations for more accurate sales predictions. To evaluate our proposed method, we conducted extensive experiments on both a real-world dataset from a prominent international Café chain and a public dataset. The results demonstrated that our method is more effective than the existing state-of-the-art baselines for new product sales forecasting.