OpenRaiser/Intern-Atlas huggingface dataset:
ALDF: An Adaptive Logical Decision Framework for Multimodal Named Entity Recognition
Authors: Guohui Ding, Tianhao Jiang, Rui Zhou, Qian Gao
Venue: CIKM
Year: 2024

Multimodal Named Entity Recognition (MNER) aims to achieve more accurate entity recognition by incorporating image information to assist text, which is particularly significant on social media platforms. Current research disproportionately emphasizes enhancing text with images, overlooking that the core of the NER task remains textual. The modal differences between images and text inevitably introduces noise when incorporating image information. Therefore, when textual information is sufficient to independently complete the NER task, the introduction of image information is unnecessary. This paper proposes an Adaptive Logical Decision Framework (ALDF) capable of determining the sufficiency of textual information in NER tasks, deciding whether to introduce image information, avoiding unnecessary noise, and focusing more on information-scarce entities when introducing image information. Specifically, we designed a Logic Reasoning Neural Network (LRNN) that uses an evidence-theory-based method to simulate human decision-making logic and generate decision support degrees for deciding whether image information should participate in the recognition task. When incorporating image information, we utilize the generated decision support degrees to guide the multi-head self-attention mechanism, enhancing the model's focus on information-scarce entities. Additionally, we employ a modality-aware progressive training method that can use decision information in real-time during multimodal training and reduce information redundancy between modalities. Extensive experiments demonstrate that our model achieves state-of-the-art performance on popular public datasets.
Fairness in Large Language Models in Three Hours
Authors: Thang Viet Doan, Zichong Wang, Nhat Nguyen Minh Hoang, Wenbin Zhang
Venue: CIKM
Year: 2024

Large Language Models (LLMs) have demonstrated remarkable success across various domains but often lack fairness considerations, potentially leading to discriminatory outcomes against marginalized populations. Unlike fairness in traditional machine learning, fairness in LLMs involves unique backgrounds, taxonomies, and fulfillment techniques. This tutorial provides a systematic overview of recent advances in the literature concerning fair LLMs, beginning with real-world case studies to introduce LLMs, followed by an analysis of bias causes therein. The concept of fairness in LLMs is then explored, summarizing the strategies for evaluating bias and the algorithms designed to promote fairness. Additionally, resources for assessing bias in LLMs, including toolkits and datasets, are compiled, and current research challenges and open questions in the field are discussed. The repository is available at https://github.com/LavinWong/Fairness-in-Large-Language-Models.
Efficient Global Message Passing for Heterophilous Graphs
Authors: Yanfei Dong, Mohammed Haroon Dupty, Lambert Deng, Yong Liang Goh, Wee Sun Lee
Venue: CIKM
Year: 2024

We investigate Graph Neural Networks (GNNs) on heterophilous graphs for node classification. To address the scarcity of useful local information in heterophilous neighborhood, it is often essential to explore global interactions. However, many existing methods in this endeavor are computationally expensive and may suffer from issues like oversquashing. In addition, earlier studies show that GNNs can be outperformed by Multi-Layer Perceptrons on heterophilous graphs, indicating insufficient exploitation of node feature information. To address these limitations, we propose Prototype Mediated GNN (PM-GNN), a novel framework which efficiently captures global feature information using class prototypes. PM-GNN learns multiple class prototypes for each class from raw node features with a soft k-means clustering mechanism. These prototypes are then transferred onto node embeddings via explicit message passing, bypassing local neighborhoods and mitigating oversquashing. PM-GNN can scale to large graphs, outperforming strong baselines on multiple heterophilous datasets.
PIXEL: Prompt-based Zero-shot Hashing via Visual and Textual Semantic Alignment
Authors: Zeyu Dong, Qingqing Long, Yihang Zhou, Pengfei Wang, Zhihong Zhu, Xiao Luo, Yidong Wang, Pengyang Wang, Yuanchun Zhou
Venue: CIKM
Year: 2024

Zero-Shot Hashing (ZSH) has aroused significant attention due to its efficiency and generalizability in multi-modal retrieval scenarios, which aims to encode semantic information into hash codes without needing unseen labeled training samples. In addition to commonly used visual images as visual semantics and class labels as global semantics, the corresponding attribute descriptions contain critical local semantics with detailed information. However, most existing methods focus on leveraging the extracted attribute numerical values, without exploring the textual semantics in attribute descriptions. To bridge this gap, in this paper, we propose Prompt-based zero-shot hashing via vIsual and teXtual sEmantic aLignment, namely PIXEL. Concretely, we design the attribute prompt template depending on attribute descriptions to make the model capture the corresponding local semantics. Then, achieving the textual embedding and visual embedding, we proposed an alignment module to model the intra- and inter-class contrastive distances. In addition, the attribute-wise constraint and class-wise constraint are utilized to collaboratively learn the hash code, image representation, and visual attributes more effectively. Finally, extensive experimental results demonstrate the superiority of PIXEL.
Boosting Certificate Robustness for Time Series Classification with Efficient Self-Ensemble
Authors: Chang George Dong, Zhengyang David Li, Liangwei Nathan Zheng, Weitong Chen, Wei Emma Zhang
Venue: CIKM
Year: 2024

Recently, the issue of adversarial robustness in the time series domain has garnered significant attention. However, the available defense mechanisms remain limited, with adversarial training being the predominant approach, though it does not provide theoretical guarantees. Randomized Smoothing has emerged as a standout method due to its ability to certify a provable lower bound on robustness radius under lp-ball attacks. Recognizing its success, research in the time series domain has started focusing on these aspects. However, existing research predominantly focuses on time series forecasting, or under the non-lp robustness in statistic feature augmentation for time series classification (TSC). Our review found that Randomized Smoothing performs modestly in TSC, struggling to provide effective assurances on datasets with poor robustness. Therefore, we propose a self-ensemble method to enhance the lower bound of the probability confidence of predicted labels by reducing the variance of classification margins, thereby certifying a larger radius. This approach also addresses the computational overhead issue of Deep Ensemble (DE) while remaining competitive and, in some cases, outperforming it in terms of robustness. Both theoretical analysis and experimental results validate the effectiveness of our method, demonstrating superior performance in robustness testing compared to baseline approaches.
On the Use of Large Language Models for Table Tasks
Authors: Yuyang Dong, Masafumi Oyamada, Chuan Xiao, Haochen Zhang
Venue: CIKM
Year: 2024

The proliferation of large language models (LLMs) has catalyzed a diverse array of applications. This tutorial delves into the application of LLMs for tabular data and targets a variety of table-related tasks, such as table understanding, text-to-SQL conversion, and tabular data preprocessing. It surveys LLM solutions to these tasks in five classes, categorized by their underpinning techniques: prompting, fine-tuning, RAG, agents, and multimodal methods. It discusses how LLMs offer innovative ways to interpret, augment, query, and cleanse tabular data, featuring academic contributions and their practical use in the industrial sector. It emphasizes the versatility and effectiveness of LLMs in handling complex table tasks, showcasing their ability to improve data quality, enhance analytical capabilities, and facilitate more intuitive data interactions. By surveying different approaches, this tutorial highlights the strengths of LLMs in enriching table tasks with more accuracy and usability, setting a foundation for future research and application in data science and AI-driven analytics. Presentation slides for this tutorial will be available at: https://dongyuyang.github.io/tableLLM-tutorial/ .
FZR: Enhancing Knowledge Transfer via Shared Factors Composition in Zero-Shot Relational Learning
Authors: Zhijun Dong, Likang Wu, Kai Zhang, Ye Liu, Yanghai Zhang, Zhi Li, Hongke Zhao, Enhong Chen
Venue: CIKM
Year: 2024

Zero-Shot Relational Learning (ZSRL), strives to predict relations that have not been observed during training, presenting a considerable challenge in terms of model generalization. Existing ZSRL methods usually utilize the prior knowledge of labels (e.g., text description, ontological schema) to enable knowledge transfer by learned features. Nonetheless, these methods remain limited to calculating the surface features exhibited by relations, failing to fully explore their underlying driving factors. This leads to insufficient discrimination between the shared and distinctive inherent components among relations, which consequently impedes the cognitive understanding required for advanced reasoning. In our study, we aim to identify and utilize shared factors that widely exist in the prior knowledge of classes to learn enhanced semantic representations via shared factors composition, and develop our Factor-based ZSRL framework (FZR) with Generative Adversarial Networks (GANs) to bridge inequality between seen and unseen classes. FZR is designed to restructure the semantic space in such a way that it captures the essence of relation formation, thereby facilitating superior knowledge transfer in zero-shot scenarios. We conduct extensive experiments and evaluate our model on real-world datasets, and the results clearly demonstrate the effectiveness of the proposed model in zero-shot relational learning tasks.
Enhancing Deep Entity Resolution with Integrated Blocker-Matcher Training: Balancing Consensus and Discrepancy
Authors: Wenzhou Dou, Derong Shen, Xiangmin Zhou, Hui Bai, Yue Kou, Tiezheng Nie, Hang Cui, Ge Yu
Venue: CIKM
Year: 2024

Deep entity resolution (ER) identifies matching entities across data sources using techniques based on deep learning. It involves two steps: a blocker for identifying the potential matches to generate the candidate pairs, and a matcher for accurately distinguishing the matches and non-matches among these candidate pairs. Recent deep ER approaches utilize pretrained language models (PLMs) to extract similarity features for blocking and matching, achieving state-of-the-art performance. However, they often fail to balance the consensus and discrepancy between the blocker and matcher, emphasizing the consensus while neglecting the discrepancy. This paper proposes MutualER, a deep entity resolution framework that integrates and jointly trains the blocker and matcher, balancing both the consensus and discrepancy between them. Specifically, we firstly introduce a lightweight PLM in siamese structure for the blocker and a heavier PLM in cross structure or an autoregressive large language model (LLM) for the matcher. Two optimization techniques named Mutual Sample Selection (MSS) and Similarity Knowledge Transferring (SKT) are designed to jointly train the blocker and matcher. MSS enables the blocker and matcher to mutually select the customized training samples for each other to maintain the discrepancy, while SKT allows them to share the similarity knowledge for improving their blocking and matching capabilities respectively to maintain the consensus. Extensive experiments on five datasets demonstrate that MutualER significantly outperforms existing PLM-based and LLM-based approaches, achieving leading performance in both effectiveness and efficiency.
Towards Uncertainty Quantification for Time Series Segmentation
Authors: Erick Draayer, Huiping Cao
Venue: CIKM
Year: 2024

Time Series Segmentation (TSS) is a data mining task widely used in many applications to generate a set of change points for a time series. Current TSS performance analyses focus on accuracy and, therefore, fail to fully evaluate the reliability and originality of a segmentation. We investigate using uncertainty quantification (UQ) to fully evaluate TSS performance. We propose UQ-TSS, a framework to quantify uncertainties surrounding TSS. UQ-TSS captures uncertainties from different sources in an integrative manner. It incorporates a novel TS augmentation algorithm to address inherent uncertainty in the data. It uses ensemble learning in a novel way to create samples and estimate the probability distributions of changepoint presence and locations. We demonstrate the ability of UQ-TSS to guide hyperparameter selection, refine segmentations, and determine an algorithm's suitability for segmenting without the need for ground truth. We validate these claims through extensive experimentation using several well-established TSS algorithms and datasets.
Explainable Stock Price Movement Prediction using Contrastive Learning
Authors: Kelvin Du, Rui Mao, Frank Xing, Erik Cambria
Venue: CIKM
Year: 2024

Predicting stock price movements is a high-stakes task that demands explainability for human decision-makers. A key shortcoming in current methods is treating sub-predictions independently, without learning from accumulated experiences. We propose a novel triplet network for contrastive learning to enhance the explainability of stock movement prediction by considering instances of "integrated textual information and quantitative indicators". We refer to the target past-l-day tweet-price time series as the "anchor instance". Each anchor instance is paired with a "positive instance" characterized by highly correlated return trends yet significant differences across the entire feature space, and a "negative instance" that exhibits similar return trends along with high proximity in the feature space. The model is designed with the objective of (1) minimizing the cross entropy loss between input logits and target, (2) minimizing the distance between the anchor instances and positive instances, and (3) maximizing the distance between the anchor instances and negative instances. Our framework's effectiveness is demonstrated through extensive testing, showing superior performance on stock prediction benchmarks.
iMIRACLE: An Iterative Multi-View Graph Neural Network to Model Intercellular Gene Regulation From Spatial Transcriptomic Data
Authors: Ziheng Duan, Siwei Xu, Cheyu Lee, Dylan Riffle, Jing Zhang
Venue: CIKM
Year: 2024

Spatial transcriptomics has transformed genomic research by measuring spatially resolved gene expressions, allowing us to investigate how cells adapt to their microenvironment via modulating their expressed genes. This essential process usually starts from cell-cell communication (CCC) via ligand-receptor (LR) interaction, leading to regulatory changes within the receiver cell. However, few methods were developed to connect them to provide biological insights into intercellular regulation. To fill this gap, we propose iMiracle, an iterative multi-view graph neural network that models each cell's intercellular regulation with three key features. Firstly, iMiracle integrates inter- and intra-cellular networks to jointly estimate cell-type- and micro-environment-driven gene expressions. Optionally, it allows prior knowledge of intra-cellular networks as pre-structured masks to maintain biological relevance. Secondly, iMiracle employs iterative learning to overcome the sparsity of spatial transcriptomic data and gradually fill in the missing edges in the CCC network. Thirdly, iMiracle infers a cell-specific ligand-gene regulatory score based on the contributions of different LR pairs to interpret inter-cellular regulation. We applied iMiracle to nine simulated and eight real datasets from three sequencing platforms and demonstrated that iMiracle consistently outperformed ten methods in gene expression imputation and four methods in regulatory score inference. Lastly, we developed iMiracle as an open-source software and anticipate that it can be a powerful tool in decoding the complexities of inter-cellular transcriptional regulation.
Integrating Fair Representation Learning with Fairness Regularization for Intersectional Group Fairness
Authors: David Quashigah Dzakpasu, Jixue Liu, Jiuyong Li, Lin Liu
Venue: CIKM
Year: 2024

In the pursuit of intersectional group fairness in machine learning models, significant attention has been directed towards fair representation learning methods. These methods aim to mitigate bias in training data by encoding data effectively while removing sensitive attribute information. However, existing fair representation learning methods often assume that decoupling sensitive attribute information from the latent representation will automatically lead to fairness on any downstream tasks learnt on the non-sensitive subspace of the latent representation. Nonetheless, biases can persist even when using representations devoid of sensitive attribute information. This is due to the learning algorithm's influence during downstream task training. In this paper, we propose a method dubbed FairReg which integrates fairness regularization with fair representation learning. This unified approach creates a more comprehensive and robust framework for ensuring intersectional group fairness in machine learning models. Empirical evaluations conducted on two real-world depression prediction datasets demonstrate the effectiveness of our method in improving intersectional group fairness compared to existing approaches.
Accurate Embedding-based Log Determinant Optimization
Authors: Daye Eun, Byungkon Kang
Venue: CIKM
Year: 2024

Many tangible and intangible objects are represented as itemsets; i.e., composition of individual items. In this paper, we address the problem of finding the embedding of such items so as to use those embeddings in tasks like missing item prediction. We approach this problem by means of determinantal point process (DPP) in order to reflect the diversity within each set. Doing so requires an optimization of a log determinant of a symmetric positive definite (SPD) matrix. The standard practice to achieve this is to perform a low-rank decomposition of the matrix and derive update rules for the low rank matrix. In this work, we propose to approach this problem by means of item embedding. That is, we will learn the SPD matrix by trying to find the right vector representations for the given data for a fixed kernel function. To this end, we propose a novel algorithm to accurately compute the gradients of the log determinant with respect to the embedding vectors. We also show that our approach outperforms Autodiff-based learning in terms of gradient direction and running time, and that other general log determinant optimization problems can be addressed.
Effective Clean-Label Backdoor Attacks on Graph Neural Networks
Authors: Xuanhao Fan, Enyan Dai
Venue: CIKM
Year: 2024

Graph Neural Networks (GNNs) have achieved remarkable success across various domains, yet recent studies have exposed their vulnerability to backdoor attacks. Backdoor attacks inject triggers into the training set to poison the model, with adversaries typically relabeling training samples with backdoor triggers to a target label. This leads a GNN trained on the poisoned dataset to misclassify any test sample containing the backdoor trigger as the target label. However, relabeling not only increases the cost of the attack but also raises the risk of detection. Therefore, our study focuses on clean-label backdoor attacks, which do not require modify the labels of trigger-attached samples in the training phase. Specifically, we employ a novel method to select effective poisoned samples belonging to the target class. An adaptive trigger generator is furthest deployed to high attack success rates under a small backdoor budget. Our experiments on four public datasets validate the effectiveness of our proposed attack.
A Spatio-Temporal Diffusion Model for Missing and Real-Time Financial Data Inference
Authors: Yupeng Fang, Ruirui Liu, Huichou Huang, Peilin Zhao, Qingyao Wu
Venue: CIKM
Year: 2024

Missing values and unreleased figures are common but highly important for backtesting and real-time analysis in the financial industry, yet underexploited in the existing literature. In this paper, we focus on the issue of empirical asset pricing, where the cross-section of future asset returns is a function of lagged firm characteristics that vary in time frequencies and missing ratios. Most of the existing imputation methods cannot fully capture the complex and evolving spatio-temporal relations among firm-level characteristics. In particular, these methods fail to explicitly consider the spatial relations and feature structure in the stock network where we have to process granular data of thousands of stocks and hundreds of characteristics for each stock. To address these challenges, we propose a spatio-temporal diffusion model (STDM) that gradually recovers the masked financial data conditioning on high-dimensional stock-and-characteristics historical data. We propose characteristic-specific projection to construct characteristic-level features at both ends of the STDM, meanwhile maintaining firm-level features in the middle of the STDM to largely reduce the computational memory. Moreover, along with the temporal attention, we design a spatial graph convolutional network, making it computationally efficient and effective to learn time-varying spatio-temporal interdependence across firms. We further employ an implicit sampler that greatly accelerates the inference procedure so that the STDM is able to produce high-quality point and density estimates of missing and real-time firm characteristics within a few steps. We evaluate our model on the most comprehensive open-source dataset 'OSAP' and generate state-of-the-art performance in extensive experiments.
Progressive Multimodal Pivot Learning: Towards Semantic Discordance Understanding as Humans
Authors: Junlin Fang, Wenya Wang, Tianze Luo, Yanyong Huang, Fengmao Lv
Venue: CIKM
Year: 2024

Multimodal recognition can achieve enhanced performance by leveraging the complementary information from different modalities. However, in real-world scenarios, multimodal samples often express discordant semantic meanings across modalities, lacking evident complementary information. Unlike humans who can easily understand the intrinsic semantic information of these semantically discordant samples, existing multimodal recognition models show poor performance on them. With the motivation of improving the robustness of multimodal recognition models in practical scenarios, this work poses a new challenge in multimodal recognition, which is coined as Semantic Discordance Understanding. Unlike existing works only focusing on detecting semantically discordant samples as noisy data, this new challenge requires deep models to follow humans' ability in understanding the inherent semantic meanings of semantically discordant samples. To address this challenge, we further propose the Progressive Multimodal Pivot Learning (PMPL) approach by introducing a learnable pivot memory to explore the inherent semantics meaning hidden under discordant modalities. To this end, our approach inserts Pivot Memory Learning (PML) modules into multiple layers of unimodal foundation models to progressively trade-off the conflict information across modalities. By introducing the multimodal pivot learning paradigm for multimodal recognition, the proposed PMPL approach can alleviate the negative effect of semantic discordance caused by the cross-modal information exchange mechanism of existing multimodal recognition models. Experiments on different benchmarks validate the superiority of our approach. Code is available at https://github.com/tiggers23/PMPL.