OpenRaiser/Intern-Atlas huggingface dataset:
Personalized Interest Sustainability Modeling for Sequential POI Recommendation
Authors: Zewen Long, Liang Wang, Qiang Liu, Shu Wu
Venue: CIKM
Year: 2023

Sequential point-of-interest (POI) recommendation endeavors to capture users' dynamic interests based on their historical check-ins, subsequently predicting the next POIs that they are most likely to visit.Existing methods conventionally capture users' personalized dynamic interests from their chronological sequences of visited POIs. However, these methods fail to explicitly consider personalized interest sustainability, which means whether each user's interest in specific POIs will sustain beyond the training time. In this work, we propose a personalized INterest Sustainability modeling framework for sequential POI REcommendation, INSPIRE for brevity. Different from existing methods that directly recommend next POIs through users' historical trajectories, our proposed INSPIRE focuses on users' personalized interest sustainability. Specifically, we first develop a new task to predict whether each user will visit the POIs in the recent period of the training time. Afterwards, to remedy the sparsity issue of users' check-in history, we propose to augment users' check-in history in three ways: geographical, intrinsic, and extrinsic schemes. Extensive experiments are conducted on two real-world datasets and results show that INSPIRE outperforms existing next POI solutions.
BRep-BERT: Pre-training Boundary Representation BERT with Sub-graph Node Contrastive Learning
Authors: Yunzhong Lou, Xueyang Li, Haotian Chen, Xiangdong Zhou
Venue: CIKM
Year: 2023

Obtaining effective entity feature representations is crucial in the field of Boundary Representation (B-Rep), a key parametric representation method in Computer-Aided Design (CAD). However, the lack of labeled large-scale database and the scarcity of task-specific label sets pose significant challenges. To address these problems, we propose an innovative unsupervised neural network approach called BRep-BERT, which extends the concept of BERT to the B-Rep domain. Specifically, we utilize Graph Neural Network (GNN) Tokenizer to generate discrete entity labels with geometric and structural semantic information. We construct new entity representation sequences based on the structural relationships and pre-train the model through the Masked Entity Modeling (MEM) task. To address the attention sparsity issue in large-scale geometric models, we incorporate graph structure information and learnable relative position encoding into the attention module to optimize feature updates. Additionally, we employ geometric sub-graphs and multi-level contrastive learning techniques to enhance the model's ability to learn regional features. Comparisons with previous methods demonstrate that BRep-BERT achieves the state-of-the-art performance on both full-data training and few-shot learning tasks across multiple B-Rep datasets. Particularly, BRep-BERT outperforms previous methods significantly in the few-shot learning scenarios. Comprehensive experiments demonstrate the substantial advantages and potential of BRep-BERT in handling B-Rep data representation. Code will be released at https://github.com/louyz1026/Brep_Bert.
Selecting Walk Schemes for Database Embedding
Authors: Yuval Lev Lubarsky, Jan Tönshoff, Martin Grohe, Benny Kimelfeld
Venue: CIKM
Year: 2023

Machinery for data analysis often requires a numeric representation of the input. Towards that, a common practice is to embed components of structured data into a high-dimensional vector space. We study the embedding of the tuples of a relational database, where existing techniques are often based on optimization tasks over a collection of random walks from the database. The focus of this paper is on the recent FoRWaRD algorithm that is designed for dynamic databases, where walks are sampled by following foreign keys between tuples. Importantly, different walks have different schemas, or ?walk schemes," that are derived by listing the relations and attributes along the walk. Also importantly, different walk schemes describe relationships of different natures in the database. We show that by focusing on a few informative walk schemes, we can obtain tuple embedding significantly faster, while retaining the quality. We define the problem of scheme selection for tuple embedding, devise several approaches and strategies for scheme selection, and conduct a thorough empirical study of the performance over a collection of downstream tasks. Our results confirm that with effective strategies for scheme selection, we can obtain high-quality embeddings considerably (e.g., three times) faster, preserve the extensibility to newly inserted tuples, and even achieve an increase in the precision of some tasks.
Can Embeddings Analysis Explain Large Language Model Ranking?
Authors: Claudio Lucchese, Giorgia Minello, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, Alberto Veneri
Venue: CIKM
Year: 2023

Understanding the behavior of deep neural networks for Information Retrieval (IR) is crucial to improve trust in these effective models. Current popular approaches to diagnose the predictions made by deep neural networks are mainly based on: i) the adherence of the retrieval model to some axiomatic property of the IR system, ii) the generation of free-text explanations, or iii) feature importance attributions. In this work, we propose a novel approach that analyzes the changes of document and query embeddings in the latent space and that might explain the inner workings of IR large pre-trained language models. In particular, we focus on predicting query/document relevance, and we characterize the predictions by analyzing the topological arrangement of the embeddings in their latent space and their evolution while passing through the layers of the network. We show that there exists a link between the embedding adjustment and the predicted score, based on how tokens cluster in the embedding space. This novel approach, grounded in the query and document tokens interplay over the latent space, provides a new perspective on neural ranker explanation and a promising strategy for improving the efficiency of the models and Query Performance Prediction (QPP).
ClinicalRisk: A New Therapy-related Clinical Trial Dataset for Predicting Trial Status and Failure Reasons
Authors: Junyu Luo, Zhi Qiao, Lucas Glass, Cao Xiao, Fenglong Ma
Venue: CIKM
Year: 2023

Clinical trials aim to study new tests and evaluate their effects on human health outcomes, which has a huge market size. However, carrying out clinical trials is expensive and time-consuming and often ends in no results. It will revolutionize clinical practice if we can develop an effective model to automatically estimate the status of a clinical trial and find out possible failure reasons. However, it is challenging to develop such a model because of the lack of a benchmark dataset. To address these challenges, in this paper, we first build a new dataset by extracting the publicly available clinical trial reports from ClinicalTrials.gov. The associated status of each report is treated as the status label. To analyze the failure reasons, domain experts help us manually annotate each failed report based on the description associated with it. More importantly, we examine several state-of-the-art text classification baselines on this task and find out that the unique format of the clinical trial protocols plays an essential role in affecting prediction accuracy, demonstrating the need for specially designed clinical trial classification models.
pADR: Towards Personalized Adverse Drug Reaction Prediction by Modeling Multi-sourced Data
Authors: Junyu Luo, Cheng Qian, Xiaochen Wang, Lucas Glass, Fenglong Ma
Venue: CIKM
Year: 2023

Predicting adverse drug reactions (ADRs) of drugs is one of the most critical steps in drug development. By pre-estimating the adverse reactions, researchers and drug development companies can greatly prevent the potential ADR risks and tragedies. However, the current ADR prediction methods suffer from several limitations. First, the prediction results are based on pure drug-related information, which makes them impossible to be directly applied for the personalized ADR prediction task. The lack of personalization of models also makes rare adverse events hard to be predicted. Therefore, it is of great interest to develop a new personalized ADR prediction method by introducing additional sources, e.g., patient health records. However, few methods have tried to use additional sources. In the meantime, the variety of different source formats and structures makes this task more challenging. To address the above challenges, we propose a novel personalized multi-sourced-based drug adverse reaction prediction model named pADR. pADR first works on every single source to transform them into proper representations. Next, a hierarchical multi-sourced Transformer is designed to automatically model the interactions between different sources and fuse them together for the final adverse event prediction. Experimental results on a new multi-sourced ADR prediction dataset show that pADR1 outperforms state-of-the-art drug-based baselines. Moreover, the case and ablation studies also illustrate the effectiveness of our proposed fusion strategies and the reasonableness of each module design.
FAF: A Risk Detection Framework on Industry-Scale Graphs
Authors: Yice Luo, Guannan Wang, Yongchao Liu, Jiaxin Yue, Weihong Cheng, Binjie Fei
Venue: CIKM
Year: 2023

It is neither effective nor profitable for individuals to attempt to bypass Ant Group's comprehensive risk control system. However, there are still criminals (such as underground loan sharks and professional hackers,), who may teach borrowers how to use hacking techniques to circumvent Ant Group and its partners' risk control system. Despite the fact that our risk control system has greatly reduced fraud losses for merchants in practice, a significant number of intermediary-related frauds still occur. During our investigation into fraud events at Zhima Credit Renting (ZCR), we discovered that more than 30 percent of fraud cases were directly linked to malicious intermediaries. To address this issue, we propose an anti-abettor fraud detection framework, called FAF (Fraud-Abettor-Fraud), specifically designed to combat intermediary-related frauds on an industry-wide scale. We have developed a series of algorithms under the FAF framework, which outperform commonly used risk detection methods and meet real-world business requirements. In this paper, we use ZCR's risk management application as a real-world example-which has deployed FAF for over 1 year-to demonstrate the superiority of the FAF framework compared to existing methods.
Context-Aware Prompt for Generation-based Event Argument Extraction with Diffusion Models
Authors: Lei Luo, Yajing Xu
Venue: CIKM
Year: 2023

Event argument extraction (EAE) has attracted increasing attention via generation-based methods. However, most existing works tend to independently extract arguments for each role, ignoring the correlation between different arguments, especially in long contexts. Motivated by these observations and the high-quality generation results of recent diffusion models, we propose an effective model called PGAD (Context-Aware Prompt for Generation-based EAE with Diffusion models) for both sentence-level and document-level EAE. In PGAD, a text diffusion model is designed to generate diverse context-aware prompt representations in conjunction with a series of random Gaussian noise. Firstly, cross-attention is employed between the designed prompt and input context within the text diffusion model in order to generate the context-aware prompt. Through this interaction, the context-aware prompt is able to capture multiple role-specific argument span queriers. Secondly, the context-aware prompt is aligned with the context to generate event arguments by joint optimization. Extensive experiments on three publicly available EAE datasets demonstrate the superiority of our proposed PGAD model over existing approaches.
Integrating Priors into Domain Adaptation Based on Evidence Theory
Authors: Ying Lv, Jianpeng Ma, Yiqiu Zhang, Gang Xu
Venue: CIKM
Year: 2023

Domain adaptation aims to build up a learning model for target domain by leveraging transferable knowledge from different but related source domains. Existing domain adaptation methods generally transfer the knowledge from source domain to target domain through measuring the consistency between the different domains. Under this strategy, if the data of source domain is not sufficient to guarantee the consistency, the transferable knowledge will be very limited. On the other hand, we often have priors about target domain which facilitate knowledge transfer but are neglected in the extant domain adaptation methods. To tackle the problems, we integrate the priors of target domain into transfer process and propose a domain adaptation method based on evidence evidence theory. We represent the priors with evidential belief function and reformulate the domain adaptation objective based on likelihood principle, in which the priors are used to adjust transferred knowledge to suit for target domain. Based on this, we propose an improved coordinate ascent algorithm to optimize likelihood objective of domain adaption. Experimental results on both text and image datasets validate that the proposed method is effective to improve the knowledge transferability in domain adaptation, especially when the source domain is limited.
DCGNN: Dual-Channel Graph Neural Network for Social Bot Detection
Authors: Nuoyan Lyu, Bingbing Xu, Fangda Guo, Huawei Shen
Venue: CIKM
Year: 2023

The importance of social bot detection has been increasingly recognized due to its profound impact on information dissemination. Existing methodologies can be categorized into feature engineering and deep learning-based methods, which mainly focus on static features, e.g., post characteristics and user profiles.However, existing methods often overlook the burst phenomena when distinguishing social bots and genuine users, i.e, the sudden and intense activity or behavior of bots after prolonged inter. Through comprehensive analysis, we find that both burst behavior and static features play pivotal roles in social bot detection. To capture such properties, the dual-channel GNN (DCGNN) is proposed which consists of a burst-aware channel with an adaptive-pass filter and a static-aware channel with a low-pass filter to model user characteristics effectively. Experimental results demonstrate the superiority of this method over competitive baselines.
A Principled Decomposition of Pointwise Mutual Information for Intention Template Discovery
Authors: Denghao Ma, Kevin Chen-Chuan Chang, Yueguo Chen, Xueqiang Lv, Liang Shen
Venue: CIKM
Year: 2023

With the rise of Artificial Intelligence (AI), question answering systems have become common for users to interact with computers, e.g., ChatGPT and Siri. These systems require a substantial amount of labeled data to train their models. However, the labeled data is scarce and challenging to be constructed. The construction process typically involves two stages: discovering potential sample candidates and manually labeling these candidates. To discover high-quality candidate samples, we study the intention paraphrase template discovery task: Given some seed questions or templates of an intention, discover new paraphrase templates that describe the intention and are diverse to the seeds enough in text. As the first exploration of the task, we identify the new quality requirements, i.e., relevance, divergence and popularity, and identify the new challenges, i.e., the paradox of divergent yet relevant paraphrases, and the conflict of popular yet relevant paraphrases. To untangle the paradox of divergent yet relevant paraphrases, in which the traditional bag of words falls short, we develop usage-centric modeling, which represents a question/template/answer as a bag of usages that users engaged (e.g., up-votes), and uses a usage-flow graph to interrelate templates, questions and answers. To balance the conflict of popular yet relevant paraphrases, we propose a new and principled decomposition for the well-known Pointwise Mutual Information from the usage perspective (usage-PMI), and then develop a Bayesian inference framework over the usage-flow graph to estimate the usage-PMI. Extensive experiments over three large CQA corpora show strong performance advantage over the baselines adopted from paraphrase identification task. We release 885,000 paraphrase templates of high quality discovered by our proposed PMI decomposition model, and the data is available in site https://github.com/Para-Questions/Intention\_template\_discovery.
Dynamic Group Parameter Modeling for Click-Through-Rate Prediction
Authors: Xuan Ma, Jian Wang, Zhiyuan Chen, Zehua Zhang, Jie He, Changping Peng, Zhangang Lin, Jingping Shao
Venue: CIKM
Year: 2023

It is noted that Click-Through-Rate(CTR) prediction plays an important part in recommendation systems and online advertising. Over the past few years, numerous studies have been conducted to improve the accuracy of CTR prediction by exploring data inherent patterns. These studies indicate that training CTR models with group-specific parameters on divided data groups can lead to significant improvements. However, most works generally divide groups manually with some prior knowledge, and such a fixed group division method may hinder the expression of user common interests. To address this limitation, we propose a novel group parameter modeling method, where the user group division and group parameter learning processes are completed in an automatic and dynamic way. Our method employs a three-stage approach, consisting of group information selection, group representation learning, and group parameter generation, which allows the efficient expression of user common interests. We conduct experiments on both public datasets and industrial datasets, and the experimental results demonstrate the effectiveness of our method. We have also deployed the model in an online advertising system and observed significant improvements in both CTR and Revenue Per Mille (RPM).
MultiCAD: Contrastive Representation Learning for Multi-modal 3D Computer-Aided Design Models
Authors: Weijian Ma, Minyang Xu, Xueyang Li, Xiangdong Zhou
Venue: CIKM
Year: 2023

CAD models are multimodal data where information and knowledge contained in construction sequences and shapes are complementary to each other and representation learning methods should consider both of them. Such traits have been neglected in previous methods learning unimodal representations. To leverage the information from both modalities, we develop a multimodal contrastive learning strategy where features from different modalities interact via contrastive learning paradigm, driven by a novel multimodal contrastive loss. Two pretext tasks on both geometry and sequence domains are designed along with a two-stage training strategy to make the representation focus on encoding geometric details and decoding representations into construction sequences, thus being more applicable to downstream tasks such as multimodal retrieval and CAD sequence reconstruction. Experimental results show that the performance of our multimodal representation learning scheme has surpassed the baselines and unimodal methods significantly.
Age-Aware Guidance via Masking-Based Attention in Face Aging
Authors: Junyeong Maeng, Kwanseok Oh, Heung-Il Suk
Venue: CIKM
Year: 2023

Face age transformation aims to convert reference images into synthesized images so that they portray the specified target ages. The crux of this task is to change only age-related areas of the given image while maintaining the age-irrelevant areas unchanged. Nevertheless, a common limitation among most existing models is the struggle to generate high-quality aging images that effectively consider both crucial properties. To address this problem, we propose a novel GAN-based face-aging framework that utilizes age-aware Guidance via Masking-Based Attention (GMBA). Specifically, we devise an age-aware guidance module to adjust age-relevant and age-irrelevant attributes within the image seamlessly. By virtue of its capability, it enables the model to produce realistic age-transformed images that certainly preserve the input's identities while delicately imposing age-related properties. Experimental results show that our proposed GMBA outperformed other state-of-the-art methods in terms of identity preservation and accurate age conversion, as well as providing superior visual quality for age-transformed images.
Application and Evaluation of Large Language Models for the Generation of Survey Questions
Authors: Antonio Maiorino, Zoe Padgett, Chun Wang, Misha Yakubovskiy, Peng Jiang
Venue: CIKM
Year: 2023

Generative Language Models have shown promising results in various domains, and some of the most successful applications are related to "concept expansion", which is the task of generating extensive text based on concise instructions provided through a "seed" prompt. In this presentation we will discuss the recent work conducted by the Data Science team at SurveyMonkey, where we have recently introduced a new feature that harnesses Generative AI models to streamline the survey design process. With this feature users can effortlessly initiate this process by specifying their desired objectives through a prompt, allowing them to automate the creation of surveys that include the critical aspects they wish to investigate. We will share our findings regarding some of the challenges encountered during the development of this feature. These include techniques for conditioning the model outputs, integrating generated text with industry-standard questions, fine-tuning Language Models using semi-synthetic Data Generation techniques, and more. Moreover, we will showcase the Evaluation Methodology that we have developed to measure the quality of the generated surveys across several dimensions. This evaluation process is crucial in ensuring that the generated surveys align well with user expectations and serve their intended purpose effectively. Our goal is to demonstrate the promising potential of Generative Language Models in the context of Survey Research, and we believe that sharing our learnings on these challenges and how we addressed them will be useful for practitioners working with Language Models on similar problems.
The 3rd International Workshop on Mining and Learning in the Legal Domain
Authors: Masoud Makrehchi, Dell Zhang, Alina Petrova, John Armour
Venue: CIKM
Year: 2023

The increasing accessibility of legal corpora and databases create opportunities to develop data-driven techniques and advanced tools that can facilitate a variety of tasks in the legal domain, such as legal search and research, legal document review and summary, legal contract drafting, and legal outcome prediction. Compared with other application domains, the legal domain is characterized by the huge scale of natural language text data, the high complexity of specialist knowledge, and the critical importance of ethical considerations. The MLLD workshop aims to bring together researchers and practitioners to share the latest research findings and innovative approaches in employing data mining, machine learning, information retrieval, and knowledge management techniques to transform the legal sector. Building upon the previous successes, the third edition of the MLLD workshop will emphasize the exploration of new research opportunities brought about by recent rapid advances in Large Language Models and Generative AI. We encourage submissions that intersect computer science and law, from both academia and industry, embodying the interdisciplinary spirit of CIKM.