OpenRaiser/Intern-Atlas huggingface dataset:
Spatio-Temporal Transformer Network with Physical Knowledge Distillation for Weather Forecasting
Authors: Jing He, Junzhong Ji, Minglong Lei
Venue: CIKM
Year: 2024

Weather forecasting has become a popular research topic recently, which mainly benefits from the development of spatio-temporal neural networks to effectively extract useful patterns from weather data. Generally, the weather changes in the meteorological system are governed by physical principles. However, it is challenging for spatio-temporal methods to capture the physical knowledge of meteorological dynamics. To address this problem, we propose in this paper a spatio-temporal Transformer network with physical knowledge distillation (PKD-STTN) for weather forecasting. First, the teacher network is implemented by a differential equation network that models weather changes by the potential energy in the atmosphere to reveal the physical mechanism of atmospheric movements. Second, the student network uses a spatio-temporal Transformer that concurrently utilizes three attention modules to comprehensively capture the semantic spatial correlation, geographical spatial correlation, and temporal correlation from weather data. Finally, the physical knowledge of the teacher network is transferred to the student network by inserting a distillation position encoding into the Transformer. Notice that the output of the teacher network is distilled to the position encoding rather than the output of the student network, which can largely utilize physical knowledge without influencing the feature extraction process of Transformers. Experiments on benchmark datasets show that the proposed method can effectively utilize physical principles of weather changes and has obvious performance advantages compared with several strong baselines.
Cryptocurrency Price Forecasting using Variational Autoencoder with Versatile Quantile Modeling
Authors: Sungchul Hong, Seunghwan An, Jong-June Jeon
Venue: CIKM
Year: 2024

In recent years, there has been a growing interest in probabilistic forecasting methods that offer more comprehensive insights by considering prediction uncertainties rather than point estimates. This paper introduces a novel variational autoencoder learning framework for multivariate distributional forecasting. Our approach employs distributional learning to directly estimate the cumulative distribution function of future time series conditional distributions using the continuous ranked probability score. By incorporating a temporal structure within the latent space and utilizing versatile quantile models, such as the generalized lambda distribution, we enable distributional forecasting by generating synthetic time series data for future time points. To assess the effectiveness of our method, we conduct experiments using a multivariate dataset of real cryptocurrency prices, demonstrating its superiority in forecasting high-volatility scenarios.
A Contextual Combinatorial Semi-Bandit Approach to Network Bottleneck Identification
Authors: Fazeleh Sadat Hoseini, Niklas Åkerblom, Morteza Haghir Chehreghani
Venue: CIKM
Year: 2024

Bottleneck identification is a challenging task in network analysis, especially when the network is not fully specified. To address this task, we develop a unified online learning framework based on combinatorial semi-bandits that performs bottleneck identification alongside learning the specifications of the underlying network. Within this framework, we adapt and investigate several combinatorial semi-bandit methods such as epsilon-greedy, LinUCB, BayesUCB, and Thompson Sampling. Our framework is able to employ contextual information in the form of contextual bandits. We evaluate our framework on the real-world application of road networks and demonstrate its effectiveness in different settings.
Nonparametric Estimation of Non-Smooth Divergences
Authors: M. Mahbub Hossain, Alan Wisler, Kevin R. Moon
Venue: CIKM
Year: 2024

Nonparametric estimation of information divergence functionals between two probability densities is an important problem in machine learning. Several estimators exist that guarantee the parametric rate of mean squared error (MSE) of O(1/N) under various assumptions on the smoothness and boundary of the underlying densities, with N being the number of samples. In particular, previous work on ensemble estimation theory derived ensemble estimators of divergence functionals that achieve the parametric rate without requiring knowledge of the densities' support set and are simple to implement. However, these and most other methods all assume some level of differentiability of the divergence functional. This excludes important divergence functionals such as the total variation distance and the Bayes error rate. Here, we show empirically that the ensemble estimation approach for smooth functionals can be applied to less smooth functionals and obtain good convergence rates, suggesting a gap in current theory.
Enhanced Retrieval Effectiveness through Selective Query Generation
Authors: Seyed Mohammad Hosseini, Negar Arabzadeh, Morteza Zihayat, Ebrahim Bagheri
Venue: CIKM
Year: 2024

Prior research has demonstrated that reformulation of queries can significantly enhance retrieval effectiveness. Despite notable successes in neural-based query reformulation methods, identifying optimal reformulations that cover the same information need while enhancing retrieval effectiveness is still challenging. This paper introduces a two-step query reformulation framework for generating and selecting optimal target query variants which not only achieve higher retrieval performance but also preserve the original query's information need. Our comprehensive evaluations on the MS MARCO dataset and TREC Deep Learning tracks demonstrate substantial improvements over original query's performance.
PIECE: Protagonist Identification and Event Chronology Extraction for Enhanced Timeline Summarization
Authors: Tz-Huan Hsu, Li-Hsuan Chin, Yen-Hao Huang, Yi-Shin Chen
Venue: CIKM
Year: 2024

Timeline summarization involves condensing events from news articles to illustrate the temporal development of a specific topic. Traditional methods often extract events based on the number of related reports but tend to overlook the movement of protagonists, the leading actors participating in events that shape the progression of the topic. This oversight can result in the extraction of sensationalized events unrelated to the topic's progression, distracting readers from tracking the topic's development. To address this limitation, we propose a novel strategy that identifies protagonists through dependency relations and tracks changes in the context surrounding them over time using a multi-faceted temporal graph. This temporal graph is a sequence of graphs that effectively captures information progression and shifts over time. Our approach aims to build a biographical timeline with accurate chronology by identifying and following the movement of protagonists. Our experiments demonstrate that our method, PIECE, outperforms previous approaches in date assignment for timeline summarization across different language datasets.
Accurate Neural Network Option Pricing Methods with Control Variate Techniques and Data Synthesis/Cleaning with Financial Rationality
Authors: Chia-Wei Hsu, Tian-Shyr Dai, Chuan-Ju Wang, Ying-Ping Chen
Venue: CIKM
Year: 2024

This paper enhances option pricing accuracy by incorporating financial expertise into a neural network (NN) design and optimizing data sample quality through cleaning and synthesis. Instead of directly estimating option values (OVs) with NNs, we leverage the concept of control variate by decomposing OVs as time values (TVs) estimated by NNs, plus the analytically solvable intrinsic values (IVs). TV surface can be decomposed into two scenarios with very different properties, and we design two NNs according to our derived no-arbitrage constraints for these two scenarios. To alleviate learning inaccuracy due to the kink of the TV surface along the scenario boundary, we synthesize training samples based on our derived constraints to smoothly extend the surface for each scenario. On the other hand, irrational option quotes commonly found in illiquid markets incur uneven surfaces, significantly deteriorating NN predictability. We develop a learnable data-cleaning method to remove potentially irrational quotes spotted by no-arbitrage constraints properly. Besides, unnecessary data syntheses proposed in previous literature can also be removed by incorporating corresponding constraints into our NN to enhance training efficiency. Comprehensive experiments on liquid S&P 500 and illiquid TAIEX option markets examine the superiority of our approach.
The Devil is in the Sources! Knowledge Enhanced Cross-Domain Recommendation in an Information Bottleneck Perspective
Authors: Binbin Hu, Weifan Wang, Shuhan Wang, Ziqi Liu, Bin Shen, Yong He, Jiawei Chen
Venue: CIKM
Year: 2024

Cross-domain Recommendation (CDR) aims to alleviate the data sparsity and the cold-start problems in traditional recommender systems by leveraging knowledge from an informative source domain. However, previously proposed CDR models pursue an imprudent assumption that the entire information from the source domain is equally contributed to the target domain, neglecting the evil part that is completely irrelevant to users' intrinsic interest. To address this concern, in this paper, we propose a novel knowledge enhanced cross-domain recommendation framework named CoTrans, which remolds the core procedures of CDR models with: Compression on the knowledge from the source domain and Transfer of the purity to the target domain. Specifically, following the theory of Graph Information Bottleneck, CoTrans first compresses the source behaviors with the perception of information from the target domain. Then to preserve all the important information for the CDR task, the feedback signals from both domains are utilized to promote the effectiveness of the transfer procedure. Additionally, a knowledge-enhanced encoder is employed to narrow gaps caused by the non-overlapped items across separate domains. Comprehensive experiments on three widely used cross-domain datasets demonstrate that CoTrans significantly outperforms both single-domain and state-of-the-art cross-domain recommendation approaches.
DAMOCRO: A Data Migration Framework Using Online Classification and Reordering
Authors: Zhongxin Hu, Kaiyu Li, Xingjian Mao, Jingfeng Pan, Yunfei Peng, Aijun An, Xiaohui Yu, Dariusz Jania
Venue: CIKM
Year: 2024

This paper introduces DAMOCRO, a data migration framework using online classification and tuple reordering to improve throughput and decrease the costs of data migration. The DAMOCRO workflow consists of four main steps. First, it classifies records into subgroups to maximize the similarity within each group. Next, it reorders tuples within these groups, ensuring that similar tuples are adjacent. Subsequently, column-wise compression is applied to each group. Finally, the compressed data is transferred from the source to the target machine. The initial two steps enhance the compression ratio, thereby boosting throughput and reducing costs. Our evaluations on five real-world datasets and two benchmark datasets, show that the online classification process in DAMOCRO improves throughput by more than 24% and reduces costs by over 19% compared to baselines. Besides, implementing reordering based on functional dependencies brings an additional cost reduction ranging from 10% to 60%, while also enhancing throughput.
DIIT: A Domain-Invariant Information Transfer Method for Industrial Cross-Domain Recommendation
Authors: Heyuan Huang, Xingyu Lou, Chaochao Chen, Pengxiang Cheng, Yue Xin, Chengwei He, Xiang Liu, Jun Wang
Venue: CIKM
Year: 2024

Cross-Domain Recommendation (CDR) have received widespread attention due to their ability to utilize rich information across domains. However, most existing CDR methods assume an ideal static condition that is not practical in industrial recommendation systems (RS). Therefore, simply applying existing CDR methods in the industrial RS environment may lead to low effectiveness and efficiency. To fill this gap, we propose DIIT, an end-to-end Domain-Invariant Information Transfer method for industrial cross-domain recommendation. Specifically, We first simulate the industrial RS environment that maintains respective models in multiple domains, each of them is trained in the incremental mode. Then, for improving the effectiveness, we design two extractors to fully extract domain-invariant information from the latest source domain models at the domain level and the representation level respectively. Finally, for improving the efficiency, we design a migrator to transfer the extracted information to the latest target domain model, which only need the target domain model for inference. Experiments conducted on one production dataset and two public datasets verify the effectiveness and efficiency of DIIT.
Application of Large Language Models in Chemistry Reaction Data Extraction and Cleaning
Authors: Xiaobao Huang, Mihir Surve, Yuhan Liu, Tengfei Luo, Olaf Wiest, Xiangliang Zhang, Nitesh V. Chawla
Venue: CIKM
Year: 2024

Chemical reaction data has existed and still largely exists in unstructured forms. But curating such information into datasets suitable for tasks such as yield and reaction outcome prediction is impractical via manual curation and not possible to automate through programmatic means alone. Large language models (LLMs) have emerged as potent tools, showcasing remarkable capabilities in processing textual information and therefore could be extremely useful in automating this process. To address the challenge of unstructured data, we manually curated a dataset of structured chemical reaction data to fine-tune and evaluate LLMs. We propose a paradigm that leverages prompt-tuning, fine-tuning techniques, and a verifier to check the extracted information. We evaluate the capabilities of various LLMs, including LLAMA-2 and GPT models with different parameter counts, on the data extraction task. Our results show that prompt tuning of GPT-4 yields the best accuracy and evaluation results. Fine-tuning LLAMA-2 models with hundreds of samples does enable them and organize scientific material according to user-defined schemas better though. This workflow shows an adaptable approach for chemical reaction data extraction but also highlights the challenges associated with nuance in chemical information. We open-sourced our code at https://github.com/joker-bruce/LLM_Extraction_Chem.
Robust Sequence-Based Self-Supervised Representation Learning for Anti-Money Laundering
Authors: Shuaibin Huang, Yun Xiong, Yi Xie, Tianyu Qiu, Guangzhong Wang
Venue: CIKM
Year: 2024

As online transactions rapidly increase, money laundering has become more difficult to detect, rendering traditional rule-based algorithms inadequate for the current severe laundering landscape. Although efforts have been made to model user behavior sequences for detecting money laundering, these approaches still fall short in scenarios with extremely low anomaly rates. In our anti-money laundering practices, we have identified the following three challenges: weak perception of intensity, scarce labels, poor representation robustness. In this paper, we present CLeAR, a novel robust sequence-based self-supervised Representation Learning framework for Anti-Money Laundering. To address the weak perception of intensity, we devise an Intensity-Aware Transformer to better capture the nuances of user behavior sequences. By introducing sequence-based Contrastive Learning into this task, we effectively tackle the issue of scarce labels and enhance sequence modeling. Additionally, we developed two self-supervised learning tasks-next behavior matching and sub-sequence matching-that significantly enhance the overall robustness of representation. After rigorous experiments across datasets of various scales, CLeAR consistently delivers exceptional performance, even under the extremely low anomaly rates that closely mimic real-world conditions.
A Payment Transaction Pre-training Model for Fraud Transaction Detection
Authors: Wenxi Huang, Zhangyi Zhao, Xiaojun Chen, Qin Zhang, Mark Junjie Li, Hanjing Su, Qingyao Wu
Venue: CIKM
Year: 2024

The surge in merchant fraud poses a significant threat to market order and consumer security. Effective security monitoring for merchants is crucial in safeguarding the digital life ecosystem and users' financial well-being. Detecting daily fraudulent payment transactions, a challenging task for current methods, requires efficient transformation of transactions into embeddings, especially in representing merchants based on their behavioral transactions. To address this, we propose the Grouping Sampling-based Sequence Generation (GSSG) method to generate meaningful sequences, enabling interactions among correlated transactions. We introduce Hierarchical Embedding Learning (HEL) and Hierarchical Masking pre-training (HMP) for the effective representation of hierarchical structures within flat transaction sequences. Pretrained on WeChat Pay data, our model, PTP, demonstrates superior performance in downstream fraud transaction detection, especially in few-shot learning scenarios, showcasing great potential in payment transaction scenarios.
To Explore or Exploit? A Gradient-informed Framework to Address the Feedback Loop for Graph based Recommendation
Authors: Zhigang Huangfu, Binbin Hu, Zhengwei Wu, Fengyu Han, Gong-Duo Zhang, Lihong Gu, Zhiqiang Zhang
Venue: CIKM
Year: 2024

Graph-based Recommendation Systems (GRSs) have gained prominence for their ability to enhance the accuracy and effectiveness of recommender systems by exploiting structural relationships in user-item interaction data. Despite their advanced capabilities, we find GRSs are susceptible to feedback-loop phenomena that disproportionately diminish the visibility of new and long-tail items, leading to a homogenization of recommendations and the potential emergence of echo chambers. To mitigate this feedback-loop issue, exploration and exploitation (E&E) strategies have been extensively researched. However, conventional E&E methods rest on the assumption that recommendations are independent and identically distributed-an assumption that is not valid for GRSs. To forge an effective E&E approach tailored to GRSs, we introduce a novel framework, the GRADient-informed Exploration and Exploitation (GRADE), designed to adaptively seek out underrepresented or new items with promising rewards. Our method evaluates the potential benefit of exploring an item by assessing the change in the system's empirical risk error pre- and post-exposure. For practical implementation, we approximate this measure using the gradients of potential edges and model parameters, alongside their associated uncertainties. We then orchestrate the balance between exploration and exploitation utilizing Thompson sampling and the Upper Confidence Bound (UCB) strategy. Empirical tests on datasets from two industrial environments demonstrate that GRADE consistently outperforms existing state-of-the-art methods. Additionally, our approach has been successfully integrated into actual industrial systems.
LEX-GNN: Label-Exploring Graph Neural Network for Accurate Fraud Detection
Authors: Woochang Hyun, Insoo Lee, Bongwon Suh
Venue: CIKM
Year: 2024

Graph-based fraud detection faces significant challenges, such as severe class imbalance, inconsistent connections due to the scarcity of fraudulent nodes, and the camouflage of these nodes appearing like benign nodes. Existing studies often adopt the approach of filtering similar nodes to strengthen the homophily assumption of graph neural networks. However, to effectively address these issues, it is important to distinguish and adaptively utilize the labels of neighboring nodes. In this study, we propose the Label-Exploring Graph Neural Network (LEX-GNN), designed to enhance fraud detection by actively leveraging labeled node information. The core idea is that the manner of message passing and reception should vary depending on the node types. Specifically, we first predict the labels of nodes based on their original or previous representations. Subsequently, each node transmits differently processed messages according to its probability of being fraudulent. Finally, target nodes also receive the messages differently depending on their pre-predicted probability. Extensive experimental results on real-world benchmarks demonstrate that LEX-GNN outperforms existing state-of-the-art baselines. Our code is available at https://github.com/wdhyun/LEX-GNN.
Human-in-the-Loop Feature Discovery for Tabular Data
Authors: Andra Ionescu, Zeger Mouw, Efthimia Aivaloglou, Rihan Hai, Asterios Katsifodimos
Venue: CIKM
Year: 2024

In recent years, researchers have developed several methods to automate discovering datasets and augmenting features for training Machine Learning (ML) models. Together with feature selection, these efforts have paved the way towards what is termed the feature discovery process. Data scientists and engineers use automated feature discovery over tabular datasets to add new features from different sources and enrich training data. By surveying data practitioners, we have observed that automated feature discovery approaches do not allow data scientists to use their domain knowledge during the feature discovery process. In addition, automated feature discovery methods can leak private features or introduce biased ones. In this paper, we introduce the first user-driven human-in-the-loop feature discovery method called HILAutoFeat. We demonstrate the capabilities of HILAutoFeat, which effectively combines automated feature discovery with user-driven insights. Our demonstration is centred around two scenarios: (i) an automated feature discovery scenario -- HILAutoFeat acts as a steward in a large data lake where the user is unaware of the quality and relevance of the data, and (ii) a scenario where HILAutoFeat and the user work together -- the user drives the feature discovery process by adding his domain and business knowledge, while HILAutoFeat performs the intensive computations.
PISeL: Pipelining DNN Inference for Serverless Computing
Authors: Masoud Rahimi Jafari, Jianchang Su, Yifan Zhang, Oliver Wang, Wei Zhang
Venue: CIKM
Year: 2024

Serverless computing offers resource efficiency, cost efficiency, and a "pay-as-you-go" pricing model, which makes it highly attractive to both users and cloud providers. However, serverless computing faces serious cold start problem, especially for deep neural network (DNN) inference, which requires low latency. Existing cold start optimization focuses only on quick container start and fast runtime and library loading. However, DNN application bootstrap (DNN framework load and start, model initialization, model download, deserialization and copy) is the leading factor during the overall cold start time. As the model size grows, the application-level bootstrap becomes more severe. We present PISeL, a generic and fast application-level cold-start optimization mechanism for DNN inference. We propose a layer-grouping mechanism and policy to pipeline model download, model deserialization and copy and request execution. The grouping policy strikes a balance that minimizes both pipeline bubble risk and synchronization overhead. The pipelining process is transparent to a variety of DNN jobs and is implemented with the hook point in a lightweight manner. PISeL not only greatly reduces the cold start time, but also the peek memory usage which can easily incur OOM (out of memory) problem. Our experiments show that PISeL accelerates cold start time with all experimented system configurations and DNN models. PISeL can speed up cold start times by 37% and 63% using PyTorch framework executed on CPU and GPU and also 29% and 33% using TensorFlow framework executed on CPU and GPU. Furthermore, PISeL reduces maximum memory usage by up to 59% and 30% using PyTorch and TensorFlow frameworks.