OpenRaiser/Intern-Atlas huggingface dataset:
GLFNet: Global and Local Frequency-domain Network for Long-term Time Series Forecasting
Authors: Xucheng Zhou, Yuwen Liu, Lianyong Qi, Xiaolong Xu, Wanchun Dou, Xuyun Zhang, Yang Zhang, Xiaokang Zhou
Venue: CIKM
Year: 2024

Recently, patch-based transformer methods have demonstrated strong effectiveness in time series forecasting. However, the complexity of self-attention imposes demands on memory and compute resources. In addition, though patches can capture comprehensive temporal information while preserving locality, temporal information within patches remains important for time series prediction. The existing methods mainly focus on modeling long-term dependencies across patches, while paying little attention to the short-term dependencies within patches. In this paper, we propose the Global and Local Frequency-domain Network (GLFNet), a novel architecture that efficiently learns global time dependencies and local time relationships in the frequency domain. Specifically, we design a frequency filtering layer to learn the temporal interactions instead of self-attention. Then we devise a dual filtering block consisting of global filter block and local filter block which learns the global dependencies across patches and local dependencies within patches. Experiments on seven benchmark datasets demonstrate that our approach achieve superior performance with improved efficiency.
Learning Cross-modal Knowledge Reasoning and Heuristic-prompt for Visual-language Navigation
Authors: Dongming Zhou, Zhengbin Pang, Wei Li
Venue: CIKM
Year: 2024

Visual language navigation is an exciting and challenging multi-modal task. Most existing research focuses on the fusion of visual features and semantic space, which ignoring the importance of local highlight features and semantic knowledge alignment in images for agent navigation. Therefore, this paper proposes a novel visual language model combining Knowledge-augmented Reasoning and Soft-Prompt (KRSP) learning. First, we perform fine-grained processing of local regions in the image and to map context image features and text knowledge to the same common sub-space. We focus on regional knowledge to increase the model reasoning ability. Next, soft-prompt learning aligns keywords and sub-visual information in instruction features to solve the path mismatch problem in coarse-grained instructions. We use a large-scale pre-training model CoCoOp to collect highly matched soft action prompts into a unified instruction set. Finally, we propose a general cross-modal feature alignment loss function. The potential semantic correlation between sub-visual information and instruction space is closer through the penalty mechanism of the alignment function. This paper verifies the method effectiveness on the R2R and REVERIE datasets, and the experimental results show that KRSP achieves state-of-the-art performance. Among them, the KRSP of SPL evaluation metric increased by 4.5% in unseen scenarios.
LST2A: Lexical-Syntactic Targeted Adversarial Attack for Texts
Authors: Guanghao Zhou, Panjia Qiu, Mingyuan Fan, Cen Chen, Yaliang Li, Wenmeng Zhou
Venue: CIKM
Year: 2024

Textual adversarial attack in black-box scenarios is a challenging task, as only the predicted label is available, and the text space is discrete and non-differentiable. Current research in this area is still in its infancy and mostly focuses on untargeted attack, lacking the capability to control the labels of the generated adversarial examples. Meanwhile, existing textual adversarial attack methods primarily rely on word substitution operations to maintain semantic similarity between the adversarial and original examples, which greatly limits the search space for adversarial examples. To address these issues, we propose a novel Lexical-Syntactic Targeted Adversarial Attack method tailored for the black-box settings, referred to as LST2A. Our approach involves adversarial perturbations at different levels of granularities, i.e., word-level with word substitution operations and syntactic-level through rewriting the syntax of the examples. Specifically, we first embed the entire text into the embedding layer of a masked language model, and then optimize perturbations at the word level within the hidden state to generate adversarial examples with the target label. For examples that are difficult to attack successfully with only word-level perturbations at higher semantic similarity thresholds, we leverage Large Language Model (LLM) to introduce syntactic-level perturbations to these examples, making them more vulnerable to the decision boundary of the victim model. Subsequently, we re-optimize the word-level perturbations for these vulnerable examples. Extensive experiments and human evaluations demonstrate that our proposed method consistently outperforms the state-of-the-art baselines, crafting smoother, more grammatically correct adversarial examples.
Understanding and Modeling Job Marketplace with Pretrained Language Models
Authors: Yaochen Zhu, Liang Wu, Binchi Zhang, Song Wang, Qi Guo, Liangjie Hong, Luke Simon, Jundong Li
Venue: CIKM
Year: 2024

Job marketplace is a heterogeneous graph composed of interactions among members (job-seekers), companies, and jobs. Understanding and modeling job marketplace can benefit both job seekers and employers, ultimately contributing to the greater good of the society. However, existing graph neural network (GNN)-based methods have shallow understandings of the associated textual features and heterogeneous relations. To address the above challenges, we propose PLM4Job, a job marketplace foundation model that tightly couples pretrained language models (PLM) with job market graph, aiming to fully utilize the pretrained knowledge and reasoning ability to model member/job textual features as well as various member-job relations simultaneously. In the pretraining phase, we propose a heterogeneous ego-graph-based prompting strategy to model and aggregate member/job textual features based on the topological structure around the target member/job node, where entity type embeddings and graph positional embeddings are introduced accordingly to model different entities and their heterogeneous relations. Meanwhile, a proximity-aware attention alignment strategy is designed to dynamically adjust the attention of the PLM on ego-graph node tokens in the prompt, such that the attention can be better aligned with job marketplace semantics. Extensive experiments at LinkedIn demonstrate the effectiveness of PLM4Job.
Submodular Optimization: Variants, Theory and Applications
Authors: Yanhui Zhu
Venue: CIKM
Year: 2024

Submodular function optimization is a fundamental tool in modeling complex interactions in machine learning and graph mining problems. We propose to study constrained submodular optimization to improve the current state of the art. Our goals are to design evolutionary algorithms with stronger approximation guarantees, the study of submodular maximization under submodular constraints, fairness in submodular optimization, and k-submodular optimization. We begin by exploring a basic submodular maximization problem in the context of viral marketing/information diffusion in social networks. From there, we broaden our scope to include a broader range of scenarios and formulate them into more optimization problems. Looking ahead, we plan to tackle scalable submodular optimization problems in fairness, dynamic constraints, dynamic streams, and distributed fashion. Lastly, we discuss a series of real-world applications that can be formulated as submodular optimization problems. In the future, we aim to apply algorithmic ideas to solve more real-world problems.
SaLa: Scenario-aware Label Graph Interaction for Multi-intent Spoken Language Understanding
Authors: Zhihong Zhu, Xuxin Cheng, Zhanpeng Chen, Zhichang Wang, Zhiqi Huang, Yuexian Zou
Venue: CIKM
Year: 2024

Recent joint models for multi-intent detection and slot filling (a.k.a multi-intent SLU) have obtained promising results by leveraging the semantic similarities or co-occurrence relationships between intent and slot labels. However, a critical aspect frequently neglected by current models is the significant correlations between label co-occurrences and specific scenarios, such as watching a movie or booking a ticket, which is essential for understanding user utterances in multi-intent SLU. In this paper, we propose a new framework dubbed SALA (short for Scenario- a ware Label gr a ph interaction), which effectively captures the dynamic co-occurrence relationships among labels across various scenarios, employing a strategy akin to a divide-and-conquer approach. Concretely, SALA first autonomously classifies the scenario of utterances, and tracks the co-occurring labels by maintaining a unique co-occurrence matrix for each scenario during the training phase. These scenario-independent co-occurrence matrices are further employed to guide the interactions among label representations through graph propagation to conduct accurate prediction. Extensive experiments on two multi-intent SLU benchmark datasets demonstrate the superiority of our SALA. More strikingly, SALA also attains competitive results on four extra single-intent and multi-domain SLU benchmark datasets, demonstrating its strong generalizability.
Collaborative Scope: Encountering the Substitution Effect within the Delivery Scope in Online Food Delivery Platform
Authors: Yida Zhu, Liying Chen, Chen Zheng, Jia Shi, Daping Xiong, Zewen Huang, Shihao Ren, Shuiping Chen, Jinghua Hao, Renqing He
Venue: CIKM
Year: 2024

Online food delivery (OFD) services, known for offering varied meals at home, have gained global popularity. Meituan has recently ventured into the affordable market segment with its "Pinhaofan'' service, highlighting the imperative to delivery efficiency. To achieve this, delivery scope is regarded as one of the most effective operational tools. The delivery scope of a merchant refers to the geo-graphical area where they can serve customers. Current methods for generating delivery scopes primarily focus on optimizing a single merchant's efficiency or rely on manual delineated from the merchant's perspective, neglecting the merchant substitution effect and potentially resulting in order loss. In this paper, we propose a novel method, named Collaborative Scope, which views the delivery scope as an assortment optimization problem, considering the substitution effect between merchants from the user's perspective. We introduce the discrete choice model of econometrics and use the Enhanced Multinomial Logit Model to predict user conversion rates in the merchant list. Next, we formulate the delivery scope optimization problem of multiple merchants as a mixed integer programming problem. The city-wide solution of this problem, owing to the large-scale combinatorial optimization triggered by high-dimensional decision variables, incurs high computational complexity. To address this, we propose an approximate solution to the original problem through a first-order Taylor series approximation, which significantly reduces the computation complexity at the expense of a slight decrease in solution accuracy. Offline and online A/B test results indicate that, compared to existing methods, Collaborative Scope significantly improves delivery efficiency by reducing delivery difficulty without hurt of order volume. Notably, Collaborative Scope is currently deployed on "Pinhaofan'', serving tens of millions of online users.
L-APPLE: Language-agnostic Prototype Prefix Learning for Cross-lingual Event Detection
Authors: Ziqin Zhu, Xutan Peng, Qian Li, Cheng Ji, Qingyun Sun, Jianxin Li
Venue: CIKM
Year: 2024

Cross-lingual event detection (CLED) is a challenging information extraction task in which a model is trained in one language and evaluated in another. Most recent methods attack CLED by aligning source and target language representations based on fine-tuning multilingual pre-trained language models. However, they need to modify all the model parameters and store a complete copy for each source-target language pair, which is resource-intensive and requires significant memory. In contrast, prefix-tuning is a more lightweight alternative, but it relies solely on the labeled source language data during training, limiting its performance. To address the above problems, we propose a novel framework for CLED with Language-agnostic Prototypical Prefix-Learning (L-APPLE), which can integrate language-agnostic event information with prefix-tuning. In detail, inspired by vanilla prompt methods, L-APPLE divides the prefix into two parts: one optimized as continuous word embeddings while the other generated with cross-lingual aligned event prototypes. Meanwhile, we employ language alignment with contrastive learning to acquire cross-lingual aligned event prototypes, and finally, parameters are optimized using both task and alignment loss. The evaluation of public CLED benchmarks demonstrates that L-APPLE achieves significant improvements in CLED with only less than 0.1% of the parameters optimized compared to previous fine-tuning methods.
EMERGE: Enhancing Multimodal Electronic Health Records Predictive Modeling with Retrieval-Augmented Generation
Authors: Yinghao Zhu, Changyu Ren, Zixiang Wang, Xiaochen Zheng, Shiyun Xie, Junlan Feng, Xi Zhu, Zhoujun Li, Liantao Ma, Chengwei Pan
Venue: CIKM
Year: 2024

The integration of multimodal Electronic Health Records (EHR) data has significantly advanced clinical predictive capabilities. Existing models, which utilize clinical notes and multivariate time-series EHR data, often fall short of incorporating the necessary medical context for accurate clinical tasks, while previous approaches with knowledge graphs (KGs) primarily focus on structured knowledge extraction. In response, we propose EMERGE, a Retrieval-Augmented Generation (RAG) driven framework to enhance multimodal EHR predictive modeling. We extract entities from both time-series data and clinical notes by prompting Large Language Models (LLMs) and align them with professional PrimeKG, ensuring consistency. In addition to triplet relationships, we incorporate entities' definitions and descriptions for richer semantics. The extracted knowledge is then used to generate task-relevant summaries of patients' health statuses. Finally, we fuse the summary with other modalities using an adaptive multimodal fusion network with cross-attention. Extensive experiments on the MIMIC-III and MIMIC-IV datasets' in-hospital mortality and 30-day readmission tasks demonstrate the superior performance of the EMERGE framework over baseline models. Comprehensive ablation studies and analysis highlight the efficacy of each designed module and robustness to data sparsity. EMERGE contributes to refining the utilization of multimodal EHR data in healthcare, bridging the gap with nuanced medical contexts essential for informed clinical predictions. We have publicly released the code at https://github.com/yhzhu99/EMERGE.
PRISM: Mitigating EHR Data Sparsity via Learning from Missing Feature Calibrated Prototype Patient Representations
Authors: Yinghao Zhu, Zixiang Wang, Long He, Shiyun Xie, Xiaochen Zheng, Liantao Ma, Chengwei Pan
Venue: CIKM
Year: 2024

Electronic Health Records (EHRs) provide valuable patient data but often suffer from sparsity issue, posing significant challenges in predictive modeling. Conventional imputation methods inadequately distinguish between real and imputed data, leading to potential inaccuracies of patient representations. To address these issues, we introduce PRISM, a framework that indirectly imputes data through prototype representations of similar patients, thus ensuring denser and more accurate embeddings. PRISM also includes a feature confidence learner module, which evaluates the reliability of each feature considering missing statuses. Additionally, it incorporates a new patient similarity metric that accounts for feature confidence, avoiding overreliance on imprecise imputed values. Our extensive experiments on the MIMIC-III, MIMIC-IV, PhysioNet Challenge 2012, eICU datasets demonstrate PRISM's superior performance in predicting in-hospital mortality and 30-day readmission tasks, showcasing its effectiveness in handling EHR data sparsity. For the sake of reproducibility and further research, we have publicly released the code at https://github.com/yhzhu99/PRISM.
Not All Negatives are Equally Negative: Soft Contrastive Learning for Unsupervised Sentence Representations
Authors: Haojie Zhuang, Wei Emma Zhang, Jian Yang, Weitong Chen, Quan Z. Sheng
Venue: CIKM
Year: 2024

Contrastive learning has been extensively studied in sentence representation learning as it demonstrates effectiveness in various downstream applications, where the same sentence with different dropout masks (or other augmentation methods) is considered as positive pair while taking other sentences in the same mini-batch as negative pairs. However, these methods mostly treat all negative examples equally and overlook the different similarities between the negative examples and the anchors, which thus fail to capture the fine-grained semantic information of the sentences. To address this issue, we explicitly differentiate the negative examples by their similarities with the anchor, and thus propose a simple yet effective method SoftCSE that individualizes either the weight or temperature of each negative pair in the standard InfoNCE loss according to the similarities of the negative examples and the anchors. We further provide the theoretical analysis of our methods to show why and how SoftCSE works, including the optimal solution, gradient analysis and the connection with other loss. Empirically, we conduct extensive experiments on semantic textual similarity (STS) and transfer (TR) tasks, as well as text retrieval and reranking, where we observe significant performance improvements compared to strong baseline models.
MV-BART: Multi-view BART for Multi-modal Sarcasm Detection
Authors: Xingjie Zhuang, Fengling Zhou, Zhixin Li
Venue: CIKM
Year: 2024

Understanding emotions in dialogue is an essential part of human communication, its an extremely complex cognitive process involving cross-modal interactions and cross-emotional associations, and multimodal sarcasm detection is an emerging but challenging research task in this process aiming at video discourse incorporating appropriate contextual information and external knowledge and identifying sarcasm by understanding both verbal and non-verbal components. However, existing research primarily focuses on constructing multimodal fusion representations and capturing incongruity between modalities as indicative cues for recognizing sarcasm, which relies on a fixed network design architecture that is difficult to cope with complex and diverse satirical scenarios in real life. As humans, we rely on the combination of visual and auditory cues, such as facial expressions and intonations, to understand information. Our brains are implicitly trained to integrate information from multiple senses to form a comprehensive understanding of conveyed messages, a process known as multi-sensory integration. The combination of different modalities not only provides additional information but also amplifies the information conveyed by each modality relative to others. Therefore, dynamic variations in the weights of different modalities play a crucial role in multi-modal understanding. From this perspective, we propose a new framework called Multi-view BART(MV-BART), which is capable of exploiting multi-granularity cues from multiple viewpoints and dynamically adjusting the view weights, applied to different sarcastic scenarios. It is worth mentioning that we analyze the proposed framework by testing it on several benchmark datasets, and the results outperform the existing state-of-the-art.
Mastodoner: A Command-line Tool and Python Library for Public Data Collection from Mastodon
Authors: Haris Bin Zia, Ignacio Castro, Gareth Tyson
Venue: CIKM
Year: 2024

This paper introduces Mastodoner, a command-line tool and Python library aimed at simplifying access to public data on Mastodon, a prominent player in the Fediverse --- a decentralized network of interconnected social media platforms. Mastodoner addresses the challenges posed by Mastodon's decentralized nature by providing a unified interface for data collection, instance discovery, and secure data sharing. Through examples and demonstrations, this paper illustrates Mastodoner's capabilities in facilitating researchers' access to and analysis of public Mastodon data, thus advancing research in decentralized social media analytics. The tool and documentation are available at: https://github.com/harisbinzia/mastodoner.
Collecting and Analyzing Public Data from Mastodon
Authors: Haris Bin Zia, Ignacio Castro, Gareth Tyson
Venue: CIKM
Year: 2024

Understanding online behaviors, communities, and trends through social media analytics is becoming increasingly important. Recent changes in the accessibility of platforms like Twitter have made Mastodon a valuable alternative for researchers. In this tutorial, we will explore methods for collecting and analyzing public data from Mastodon, a decentralized micro-blogging social network. Participants will learn about the architecture of Mastodon, techniques and best practices for data collection, and various analytical methods to derive insights from the collected data. This session aims to equip researchers with the skills necessary to harness the potential of Mastodon data in computational social science and social data science research.
DetCat: Detecting Categorical Outliers in Relational Datasets
Authors: Arthur Zylinski, Abdulhakim Ali Qahtan
Venue: CIKM
Year: 2024

Poor data quality significantly affects different data analytics tasks, leading to inaccurate decisions and poor predictions of the machine learning models. Outliers represent one of the most common data glitches that impact data quality. While detecting outliers in numerical data has been extensively studied, few attempts were made to solve the problem of detecting categorical outliers. In this paper, we introduce DetCat for detecting categorical outliers in relational datasets, by utilizing the syntactic structure of the values. For a given attribute, DetCat identifies a set of patterns that represents the majority of the values as dominating patterns. Data values that cannot be generated by the dominating patterns are declared as outliers. The demo will show the effectiveness of our tool in detecting categorical outliers and discovering the syntactical data patterns.