OpenRaiser/Intern-Atlas huggingface dataset:
Geometry Interaction Augmented Graph Collaborative Filtering
Authors: Jie Xu, Chaozhuo Li
Venue: CIKM
Year: 2023

Graph collaborative filtering, which could capture the abundant collaborative signal from the high-order connectivity of the tree-likeness user-item interaction graph, has received considerable research attention recently. Most graph collaborative filtering methods embed graphs in the Euclidean spaces, but that could have high distortion when embedding graphs with tree-likeness structure. Recently, some researchers address this problem by learning the feature representations in the hyperbolic spaces. However, because the user-item interaction graphs also have cyclic structure, the high-order collaborative signal cannot be well captured by hyperbolic spaces. From this point of view, neither Euclidean spaces nor hyperbolic spaces can capture the full information from the complexity of user-item interactions. Therefore, how to construct a suitable embedding space for graph collaboration filtering is an important problem. In this paper, we analyze the properties of hyperbolic geometry in graph collaborative filtering tasks and proposed a novel geometry interaction augmented graph collaborative filtering (GeoGCF) method, which leverages both Euclidean and hyperbolic geometry to model the user-item interactions. Experimental results show the effectiveness of the proposed method.
DynED: Dynamic Ensemble Diversification in Data Stream Classification
Authors: Soheil Abadifard, Sepehr Bakhshi, Sanaz Gheibuni, Fazli Can
Venue: CIKM
Year: 2023

Ensemble methods are commonly used in classification due to their remarkable performance. Achieving high accuracy in a data stream environment is a challenging task considering disruptive changes in the data distribution, also known as concept drift. A greater diversity of ensemble components is known to enhance prediction accuracy in such settings. Despite the diversity of components within an ensemble, not all contribute as expected to its overall performance. This necessitates a method for selecting components that exhibit high performance and diversity. We present a novel ensemble construction and maintenance approach based on MMR (Maximal Marginal Relevance) that dynamically combines the diversity and prediction accuracy of components during the process of structuring an ensemble. The experimental results on both four real and 11 synthetic datasets demonstrate that the proposed approach (DynED) provides a higher average mean accuracy compared to the five state-of-the-art baselines.
Towards Effective Modeling and Exploitation of Search and User Context in Conversational Information Retrieval
Authors: Praveen Acharya
Venue: CIKM
Year: 2023

Conversational information retrieval has garnered considerable attention in recent years. A major challenge in conversational search is formulating the most effective query during the dialogue between the searcher and the conversational agent. Unlike traditional information retrieval systems that assume users can independently create queries, conversational settings allow agents to assist users in query formulation. This alleviates the burden on users by leveraging the multi-turn nature of the conversation to aid them in reaching their information goals. Conversational context plays a vital role in the query process. In this work, we focus on understanding and leveraging conversational context from two dimensions: conversational history and knowledge history. The goal is to identify and model the relevant portions from the search dialogue and the knowledge history and to use this within the search process to improve the overall performance of conversational information retrieval systems.
Optimizing for Member Value in an Edge Building Marketplace
Authors: Ayan Acharya, Siyuan Gao, Ankan Saha, Borja Ocejo, Kinjal Basu, Sathiya Keerthi Selvaraj, Rahul Mazumder, Aman Gupta, Parag Agrawal
Venue: CIKM
Year: 2023

Social networks are prosperous marketplaces where creators and consumers congregate to share and consume various content. In general, products that rank content for distribution (such as newsfeeds, stories, and notifications) and are related to edge recommendations (such as connect to members, follow celebrities or groups or hashtags) optimize the experience of active users. Typically, such users generate ample interaction data amenable to accurate model training and prediction. In contrast, we prioritize enhancing the experience of inactive members (IMs) who do not have a rich connection network. We formulate strategies for recommending superior edges to help members grow their connection network. Adapting the recommendations provides enormous value to the IMs and can significantly influence their future behaviour and engagement with the ecosystem. To that end, we propose a general and scalable multi-objective optimization (MOO) framework to provide more value to IMs as invitation recipients on LinkedIn, a professional network with over 900M members. To deal with the enormous scale, we formulate the problem as a massive constrained linear optimization involving billions of variables and millions of constraints and efficiently solve it using accelerated gradient descent,making this the largest deployment of LP-based recommender systems worldwide. Furthermore, the proposed MOO paradigm can solve the general problem of matching different types of entities in an m-sided marketplace. Finally, we discuss the challenges and benefits of implementing and ramping our method in production at scale at LinkedIn and report our findings about the core business metrics related to users' engagement and network health.
Enhancing E-commerce Product Search through Reinforcement Learning-Powered Query Reformulation
Authors: Sanjay Agrawal, Srujana Merugu, Vivek Sembium
Venue: CIKM
Year: 2023

Query reformulation (QR) is a widely used technique in web and product search. In QR, we map a poorly formed or low coverage user query to a few semantically similar queries that are rich in product coverage, thereby enabling effective targeted searches with less cognitive load on the user. Recent QR approaches based on generative language models are superior to informational retrieval-based methods but exhibit key limitations: (i) generated reformulations often have low lexical diversity and fail to retrieve a large set of relevant products of a wider variety, (ii) the training objective of generative models does not incorporate a our goal of improving product coverage. In this paper, we propose RLQR (Reinforcement Learning for Query Reformulations), for generating high quality diverse reformulations which aim to maximize the product coverage (number of distinct relevant products returned). We evaluate our approach against supervised generative models and strong RL-based methods. Our experiments demonstrate a 28.6% increase in product coverage compared to a standard generative model, outperforming SOTA benchmarks by a significant margin. We also conduct our experiments on an external Amazon shopping dataset and demonstrate increased product coverage over SOTA algorithms.
SNAKE Challenge: Sanitization Algorithms under Attack
Authors: Tristan Allard, Louis Béziaud, Sébastien Gambs
Venue: CIKM
Year: 2023

While there were already some privacy challenges organized in the domain of data sanitization, they have mainly focused on the defense side of the problem. To favor the organization of successful challenges focusing on attacks, we introduce the SNAKE framework that is designed to facilitate the organization of challenges dedicated to attacking existing data sanitization mechanisms. In particular, it enables to easily automate the redundant tasks that are inherent to any such challenge and exhibits the following salient features: genericity with respect to attacks, ease of use and extensibility. We propose to demonstrate the main features of the SNAKE framework through a specific instantiation focusing on membership inference attacks over differentially-private synthetic data generation schemes. This instance of the SNAKE framework is currently being used for supporting a challenge co-located with APVP 2023 (the French workshop on the protection of privacy).
Some Useful Things to Know When Combining IR and NLP: the Easy, the Hard and the Ugly
Authors: Omar Alonso, Kenneth Church
Venue: CIKM
Year: 2023

Deep nets such as GPT are at the core of the current advances in many systems and applications. Things are moving very fast, and it appears that techniques are out of date within weeks. How can we take advantage of new discoveries and incorporate them into our existing work? Are these radical new developments, repetitions of older concepts, or both? In this tutorial, we aim to bring interested researchers and practitioners up to speed on the recent and ongoing techniques around ML and Deep learning in the context of IR and NLP. Additionally, our goal is to clarify terminology, emphasize fundamentals, and outline new research opportunities.
Nowcast-to-Forecast: Token-Based Multiple Remote Sensing Data Fusion for Precipitation Forecast
Authors: Sojung An
Venue: CIKM
Year: 2023

Accurate short-term precipitation forecast is of social and economic significance for preventing severe weather damage. Deep learning has been rapidly adopted in nowcasting based on weather radar, which plays a key role in preventing dangerous weather conditions such as torrential rainfall. However, the limited observation range of the radar imposes constraints on shorter forecast lead times. Securing a sufficient lead time for timely flood warnings and emergency responses is crucial. Here, we propose a novel GAN-based framework that combines radar and satellite data to extend forecast lead time. First, we tokenize the satellite image to align with radar dimensions and combine the satellite and radar data. We then apply positional encoding to add positional information. Second, we design the self-conditioned generator to estimate distributions of various rainfall intensities. Finally, we employ Gaussian Fourier features to map the input noise into a continuous representation. The proposed framework realistically and accurately produces time series images of various precipitation types. Furthermore, our multisource data-driven system outperforms numerical weather prediction at forecasts of up to 6 hours in South Korea.
Combining Inductive and Deductive Reasoning for Query Answering over Incomplete Knowledge Graphs
Authors: Medina Andresel, Trung-Kien Tran, Csaba Domokos, Pasquale Minervini, Daria Stepanova
Venue: CIKM
Year: 2023

Current methods for embedding-based query answering over incomplete Knowledge Graphs (KGs) only focus on inductive reasoning, i.e., predicting answers by learning patterns from the data, and lack the complementary ability to do deductive reasoning, which requires the application of domain knowledge to infer further information. To address this shortcoming, we investigate the problem of incorporating ontologies into embedding-based query answering models by defining the task of embedding-based ontology-mediated query answering. We propose various integration strategies into prominent representatives of embedding models that involve (1) different ontology-driven data augmentation techniques and (2) adaptation of the loss function to enforce the ontology axioms. We design novel benchmarks for the considered task based on the LUBM and the NELL KGs and evaluate our methods on them. The achieved improvements in the setting that requires both inductive and deductive reasoning are from 20% to 55% in HITS@3.
Noisy Perturbations for Estimating Query Difficulty in Dense Retrievers
Authors: Negar Arabzadeh, Radin Hamidi Rad, Maryam Khodabakhsh, Ebrahim Bagheri
Venue: CIKM
Year: 2023

Estimating query difficulty, also known as Query Performance Prediction (QPP), is concerned with assessing the retrieval quality of a ranking method for an input query. Most traditional unsupervised frequency-based models and many recent supervised neural methods have been designed specifically for predicting the performance of sparse retrievers such as BM25. In this paper we propose an unsupervised QPP method for dense neural retrievers which operates by redefining the well-known concept of query robustness i.e., a more robust query to perturbations is an easier query to handle. We propose to generate query perturbations for measuring query robustness by systematically injecting noise into the contextualized neural representation of each query. We then compare the retrieved list for the original query with that of the perturbed query as a way to measure query robustness. Our experiments on four different query sets including MS MARCO, TREC Deep Learning track 2019 and 2020 and TREC DL-Hard show consistently improved performance on linear and ranking correlation metrics over the state of the art.
HOVER: Homophilic Oversampling via Edge Removal for Class-Imbalanced Bot Detection on Graphs
Authors: Bradley Ashmore, Lingwei Chen
Venue: CIKM
Year: 2023

As malicious bots reside in a network to disrupt network stability, graph neural networks (GNNs) have emerged as one of the most popular bot detection methods. However, in most cases these graphs are significantly class-imbalanced. To address this issue, graph oversampling has recently been proposed to synthesize nodes and edges, which still suffers from graph heterophily, leading to suboptimal performance. In this paper, we propose HOVER, which implements Homophilic Oversampling Via Edge Removal for bot detection on graphs. Instead of oversampling nodes and edges within initial graph structure, HOVER designs a simple edge removal method with heuristic criteria to mitigate heterophily and learn distinguishable node embeddings, which are then used to oversample minority bots to generate a balanced class distribution without edge synthesis. Experiments on TON IoT networks demonstrate the state-of-the-art performance of HOVER on bot detection with high graph heterophily and extreme class imbalance.
CLosER: Conversational Legal Longformer with Expertise-Aware Passage Response Ranker for Long Contexts
Authors: Arian Askari, Mohammad Aliannejadi, Amin Abolghasemi, Evangelos Kanoulas, Suzan Verberne
Venue: CIKM
Year: 2023

In this paper, we investigate the task of response ranking in conversational legal search. We propose a novel method for conversational passage response retrieval (ConvPR) for long conversations in domains with mixed levels of expertise. Conversational legal search is challenging because the domain includes long, multi-participant dialogues with domain-specific language. Furthermore, as opposed to other domains, there typically is a large knowledge gap between the questioner (a layperson) and the responders (lawyers), participating in the same conversation. We collect and release a large-scale real-world dataset called LegalConv with nearly one million legal conversations from a legal community question answering (CQA) platform. We address the particular challenges of processing legal conversations, with our novel Conversational Legal Longformer with Expertise-Aware Response Ranker, called CLosER. The proposed method has two main innovations compared to state-of-the-art methods for ConvPR: (i) Expertise-Aware Post-Training; a learning objective that takes into account the knowledge gap difference between participants to the conversation; and (ii) a simple but effective strategy for re-ordering the context utterances in long conversations to overcome the limitations of the sparse attention mechanism of the Longformer architecture. Evaluation on LegalConv shows that our proposed method substantially and significantly outperforms existing state-of-the-art models on the response selection task. Our analysis indicates that our Expertise-Aware PostTraining, i.e., continued pre-training or domain/task adaptation, plays an important role in the achieved effectiveness. Our proposed method is generalizable to other tasks with domain-specific challenges and can facilitate future research on conversational search in other domains.
A Test Collection of Synthetic Documents for Training Rankers: ChatGPT vs. Human Experts
Authors: Arian Askari, Mohammad Aliannejadi, Evangelos Kanoulas, Suzan Verberne
Venue: CIKM
Year: 2023

In this resource paper, we investigate the usefulness of generative Large Language Models (LLMs) in generating training data for cross-encoder re-rankers in a novel direction: generating synthetic documents instead of synthetic queries. We introduce a new dataset, ChatGPT-RetrievalQA, and compare the effectiveness of strong models fine-tuned on both LLM-generated and human-generated data. We build ChatGPT-RetrievalQA based on an existing dataset, human ChatGPT Comparison Corpus (HC3), consisting of public question collections with human responses and answers from ChatGPT. We fine-tune a range of cross-encoder re-rankers on either human-generated or ChatGPT-generated data. Our evaluation on MS MARCO DEV, TREC DL'19, and TREC DL'20 demonstrates that cross-encoder re-ranking models trained on LLM-generated responses are significantly more effective for out-of-domain re-ranking than those trained on human responses. For in-domain re-ranking, the human-trained re-rankers outperform the LLM-trained re-rankers. Our novel findings suggest that generative LLMs have high potential in generating training data for neural retrieval models and can be used to augment training data, especially in domains with smaller amounts of labeled data. We believe that our dataset, ChatGPT-RetrievalQA, presents various opportunities for analyzing and improving rankers with human and synthetic data. We release our data, code, and model checkpoints for future work.
MODIMO: Workshop on Multi-Omics Data Integration for Modelling Biological Systems
Authors: Simone Avesani, Vincenzo Bonnici, Simone Pernice, Marco Beccuti, Rosalba Giugno
Venue: CIKM
Year: 2023

Multi-omics analysis aims at extracting previously uncovered biological knowledge by integrating information across multiple single-omic sources. Past approaches have focused on the simultaneous analysis of a small number of omic data sets. Current challenges face the problem of integrating multiple omic sources into a unified complex model, or of combining already available tools for two-by-two omics analyses and merging their outcomes. By doing so and leveraging integrated system-level knowledge, multi-omic approaches ought to enable the development of better qualitative and quantitative models for descriptive and predictive analyses. To move this area forward, new statistical and algorithmic frameworks are needed, for example for generalizing classical graph theory results to heterogeneous networks, and applying them to diverse problems such as drug repurposing or understanding the immune response to infections. Thus, in short, this workshop aims at investigating novel methodologies for providing crucial insights into multi-omics data management, integration, and analysis to enable biological discoveries. The workshop will be sponsored by the InfoLife CINI National Laboratory (https://www.consorzio-cini.it/index.php/en/ ).
Towards Improving Accuracy and Computation Cost Optimization of Recommendation Systems
Authors: Abdelghani Azri
Venue: CIKM
Year: 2023

It is hard to avoid recommender systems (RS) these days which play a vital role in various domains, such as e-commerce, online streaming platforms, and personalized content delivery. These systems assist users in discovering relevant items based on their preferences and past interactions. A variety of methods are developed by RS communities, to address the accuracy issue. But, most of these methods are sequential and omit the item features and user side information which contains relevant and rich information that can increase the accuracy of these systems. However, enhancing the accuracy of recommendations often comes at the expense of increased computational costs. This PhD thesis aims to address the challenge of improving the accuracy of RS following a hybrid approach that allows leveraging user-item features based on some relevant state-of-the-art models and using deep learning techniques such as the contractive autoencoder (CAE) while optimizing the cost of computation using parallel/distributed paradigms.
Accelerating Concept Learning via Sampling
Authors: Alkid Baci, Stefan Heindorf
Venue: CIKM
Year: 2023

Node classification is an important task in many fields, e.g., predicting entity types in knowledge graphs, classifying papers in citation graphs, or classifying nodes in social networks. In many cases, it is crucial to explain why certain predictions are made. Towards this end, concept learning has been proposed as a means of interpretable node classification: given positive and negative examples in a knowledge base, concepts in description logics are learned that serve as classification models. However, state-of-the-art concept learners, including EvoLearner and CELOE exhibit long runtimes. In this paper, we propose to accelerate concept learning with graph sampling techniques. We experiment with seven techniques and tailor them to the setting of concept learning. In our experiments, we achieve a reduction in training size by over 90% while maintaining a high predictive performance.
Comparative Analysis of Open Source and Commercial Embedding Models for Question Answering
Authors: Georgios Balikas
Venue: CIKM
Year: 2023

In this industry track presentation, we will provide a comprehensive tour of the best performing embedding models for question answering, as determined by the Massive Text Embedding Benchmark1. We showcase these models while also considering solutions offered by OpenAI and Cohere, renowned for their state-of-the-art performance. Through rigorous evaluations on internal Salesforce datasets tailored for Question Answering on Knowledge articles, we compare the performance of these models using standardized metrics. Our analysis sheds light on the current state-of-the-art in question answering using embedding models across three diverse domains. We hope that this talk's outcomes will empower practitioners and researchers to make informed decisions when selecting the most suitable solution for their specific requirements.
Tutorial on User Simulation for Evaluating Information Access Systems
Authors: Krisztian Balog, ChengXiang Zhai
Venue: CIKM
Year: 2023

With the emergence of various information access systems exhibiting increasing complexity, there is a critical need for sound and scalable means of automatic evaluation. To address this challenge, user simulation emerges as a promising solution. This half-day tutorial focuses on providing a thorough understanding of user simulation techniques designed specifically for evaluation purposes. We systematically review major research progress, covering both general frameworks for designing user simulators, and specific models and algorithms for simulating user interactions with search engines, recommender systems, and conversational assistants. We also highlight some important future research directions.