Reviewerly/exHarmony huggingface dataset:
Gen-IR@SIGIR 2023: The First Workshop on Generative Information Retrieval

Generative information retrieval (IR) has experienced substantial growth across multiple research communities (e.g., information retrieval, computer vision, natural language processing, and machine learning), and has been highly visible in the popular press. Theoretical, empirical, and actual user-facing products have been released that retrieve documents (via generation) or directly generate answers given an input request. We would like to investigate whether end-to-end generative models are just another trend or, as some claim, a paradigm change for IR. This necessitates new metrics, theoretical grounding, evaluation methods, task definitions, models, user interfaces, etc. The goal of this workshop1 is to focus on previously explored Generative IR techniques like document retrieval and direct Grounded Answer Generation, while also offering a venue for the discussion and exploration of how Generative IR can be applied to new domains like recommendation systems, summarization, etc. The format of the workshop is interactive, including roundtable and keynote sessions and tends to avoid the one-sided dialogue of a mini-conference.
Reproducibility, Replicability, and Insights into Dense Multi-Representation Retrieval Models: from ColBERT to Col*

Dense multi-representation retrieval models, exemplified as ColBERT, estimate the relevance between a query and a document based on the similarity of their contextualised token-level embeddings. Indeed, by using contextualised token embeddings, dense retrieval, conducted as either exact or semantic matches, can result in increased effectiveness for both in-domain and out-of-domain retrieval tasks, indicating that it is an important model to study. However, the exact role that these semantic matches play is not yet well investigated. For instance, although tokenisation is one of the crucial design choices for various pretrained language models, its impact on the matching behaviour has not been examined in detail. In this work, we inspect the reproducibility and replicability of the contextualised late interaction mechanism by extending ColBERT to Col⋆ which implements the late interaction mechanism across various pretrained models and different types of tokenisers. As different tokenisation methods can directly impact the matching behaviour within the late interaction mechanism, we study the nature of matches occurring in different Col⋆ models, and further quantify the contribution of lexical and semantic matching on retrieval effectiveness. Overall, our experiments successfully reproduce the performance of ColBERT on various query sets, and replicate the late interaction mechanism upon different pretrained models with different tokenisers. Moreover, our experimental results yield new insights, such as: (i) semantic matching behaviour varies across different tokenisers; (ii) more specifically, high-frequency tokens tend to perform semantic matching than other token families; (iii) late interaction mechanism benefits more from lexical matching than semantic matching; (iv) special tokens, such as [CLS], play a very important role in late interaction.
On Stance Detection in Image Retrieval for Argumentation

Given a text query on a controversial topic, the task of Image Retrieval for Argumentation is to rank images according to how well they can be used to support a discussion on the topic. An important subtask therein is to determine the stance of the retrieved images, i.e., whether an image supports the pro or con side of the topic. In this paper, we conduct a comprehensive reproducibility study of the state of the art as represented by the CLEF'22 Touché lab and an in-house extension of it. Based on the submitted approaches, we developed a unified and modular retrieval process and reimplemented the submitted approaches according to this process. Through this unified reproduction (which also includes models not previously considered), we achieve an effectiveness improvement in argumentative image detection of up to 0.832 [email protected] However, despite this reproduction success, our study also revealed a previously unknown negative result: for stance detection, none of the reproduced or new approaches can convincingly beat a random baseline. To understand the apparent challenges inherent to image stance detection, we conduct a thorough error analysis and provide insight into potential new ways to approach this task.
Knowledge-enhanced Multi-View Graph Neural Networks for Session-based Recommendation

Session-based recommendation (SBR) has received increasing attention to predict the next item via extracting and integrating both global and local item-item relationships. However, there still exist some deficiencies in current works when capturing these two kinds of relationships. For global item-item relationships, the global graph constructed by most SBR is a pseudo-global graph, which may cause redundant mining of sequence relationships. For local item-item relationships, conventional SBR only mines the sequence patterns while ignoring the feature patterns, which may introduce noise when learning users' interests. To address these problems, we propose a novel Knowledge-enhanced Multi-View Graph Neural Network (KMVG) by constructing three views, namely knowledge view, session view, and pairwise view. Specifically, benefiting from the rich semantic information in the knowledge graph (KG), we build a genuine global graph that is sequence-independent based on KG to mine the global item-item relationships in the knowledge view. Then, a session view is utilized to capture the contextual transitions among items as the sequence patterns of local item-item relationships, and a pairwise view is used to explore the feature commonality within a session as the feature patterns of the local item-item relationships. Extensive experiments on three real-world public datasets demonstrate the superiority of KMVG, showing that it outperforms the state-of-the-art baselines. Further analysis also reveals the effectiveness of KMVG in exploiting the item-item relationships under multiple views.
Adaptive Popularity Debiasing Aggregator for Graph Collaborative Filtering

The graph neural network-based collaborative filtering (CF) models user-item interactions as a bipartite graph and performs iterative aggregation to enhance performance. Unfortunately, the aggregation process may amplify the popularity bias, which impedes user engagement with niche (unpopular) items. While some efforts have studied the popularity bias in CF, they often focus on modifying loss functions, which can not fully address the popularity bias in GNN-based CF models. This is because the debiasing loss can be falsely backpropagated to non-target nodes during the backward pass of the aggregation.
Large Language Models are Versatile Decomposers: Decomposing Evidence and Questions for Table-based Reasoning

Table-based reasoning has shown remarkable progress in a wide range of table-based tasks. It is a challenging task, which requires reasoning over both free-form natural language (NL) questions and (semi-)structured tabular data. However, previous table-based reasoning solutions usually suffer from significant performance degradation on ''huge'' evidence (tables). In addition, most existing methods struggle to reason over complex questions since the essential information is scattered in different places. To alleviate the above challenges, we exploit large language models (LLMs) as decomposers for effective table-based reasoning, which (i) decompose huge evidence (a huge table) into sub-evidence (a small table) to mitigate the interference of useless information for table reasoning, and (ii) decompose a complex question into simpler sub-questions for text reasoning. First, we use a powerful LLM to decompose the evidence involved in the current question into the sub-evidence that retains the relevant information and excludes the remaining irrelevant information from the ''huge'' evidence. Second, we propose a novel ''parsing-execution-filling'' strategy to decompose a complex question into simper step-by-step sub-questions by generating intermediate SQL queries as a bridge to produce numerical and logical sub-questions with a powerful LLM. Finally, we leverage the decomposed sub-evidence and sub-questions to get the final answer with a few in-context prompting examples. Extensive experiments on three benchmark datasets (TabFact, WikiTableQuestion, and FetaQA) demonstrate that our method achieves significantly better results than competitive baselines for table-based reasoning. Notably, our method outperforms human performance for the first time on the TabFact dataset. In addition to impressive overall performance, our method also has the advantage of interpretability, where the returned results are to some extent tractable with the generated sub-evidence and sub-questions. For reproducibility, we release our source code and data at: https://github.com/AlibabaResearch/DAMO-ConvAI.
A Static Pruning Study on Sparse Neural Retrievers

Sparse neural retrievers, such as DeepImpact, uniCOIL and SPLADE, have been introduced recently as an efficient and effective way to perform retrieval with inverted indexes. They aim to learn term importance and, in some cases, document expansions, to provide a more effective document ranking compared to traditional bag-of-words retrieval models such as BM25. However, these sparse neural retrievers have been shown to increase the computational costs and latency of query processing compared to their classical counterparts. To mitigate this, we apply a well-known family of techniques for boosting the efficiency of query processing over inverted indexes: static pruning. We experiment with three static pruning strategies, namely document-centric, term-centric and agnostic pruning, and we assess, over diverse datasets, that these techniques still work with sparse neural retrievers. In particular, static pruning achieves 2x speedup with negligible effectiveness loss (≤ 2% drop) and, depending on the use case, even 4x speedup with minimal impact on the effectiveness (≤ 8% drop). Moreover, we show that neural rerankers are robust to candidates from statically pruned indexes.
When Newer is Not Better: Does Deep Learning Really Benefit Recommendation From Implicit Feedback?

In recent years, neural models have been repeatedly touted to exhibit state-of-the-art performance in recommendation. Nevertheless, multiple recent studies have revealed that the reported state-of-the-art results of many neural recommendation models cannot be reliably replicated. A primary reason is that existing evaluations are performed under various inconsistent protocols. Correspondingly, these replicability issues make it difficult to understand how much benefit we can actually gain from these neural models. It then becomes clear that a fair and comprehensive performance comparison between traditional and neural models is needed.
Wisdom of Crowds and Fine-Grained Learning for Serendipity Recommendations

Serendipity is a notion that means an unexpected but valuable discovery. Due to its elusive and subjective nature, serendipity is difficult to study even with today's advances in machine learning and deep learning techniques. Both ground truth data collecting and model developing are the open research questions. This paper addresses both the data and the model challenges for identifying serendipity in recommender systems. For the ground truth data collecting, it proposes a new and scalable approach by using both user generated reviews and a crowd sourcing method. The result is a large-scale ground truth data on serendipity. For model developing, it designed a self-enhanced module to learn the fine-grained facets of serendipity in order to mitigate the inherent data sparsity problem in any serendipity ground truth dataset. The self-enhanced module is general enough to be applied with many base deep learning models for serendipity. A series of experiments have been conducted. As the result, a base deep learning model trained on our collected ground truth data, as well as with the help of the self-enhanced module, outperforms the state-of-the-art baseline models in predicting serendipity.
RL4RS: A Real-World Dataset for Reinforcement Learning based Recommender System

Reinforcement learning based recommender systems (RL-based RS) aim at learning a good policy from a batch of collected data, by casting recommendations to multi-step decision-making tasks. However, current RL-based RS research commonly has a large reality gap. In this paper, we introduce the first open-source real-world dataset, RL4RS, hoping to replace the artificial datasets and semi-simulated RS datasets previous studies used due to the resource limitation of the RL-based RS domain. Unlike academic RL research, RL-based RS suffers from the difficulties of being well-validated before deployment. We attempt to propose a new systematic evaluation framework, including evaluation of environment simulation, evaluation on environments, and counterfactual policy evaluation. In summary, the RL4RS (Reinforcement Learning for Recommender Systems), a new resource with special concerns on the reality gaps, contains two real-world datasets, data understanding tools, tuned simulation environments, related advanced RL baselines, batch RL baselines, and counterfactual policy evaluation algorithms. The RL4RS suite can be found at https://github.com/fuxiAIlab/RL4RS.
Review-based Multi-intention Contrastive Learning for Recommendation

Real recommendation systems contain various features, which are often high-dimensional, sparse, and difficult to learn effectively. In addition to numerical features, user reviews contain rich semantic information including user preferences, which are used as auxiliary features by researchers. The methods of supplementing data features based on reviews have certain effects. However, most of them simply concatenate review representations and other features together, without considering that the text representation contains a lot of noise information. In addition, the important intentions contained in user reviews are not modeled effectively. In order to solve the above problems, we propose a novel Review-based Multi-intention Contrastive Learning (RMCL) method. In detail, RMCL proposes an intention representation method based on mixed Gaussian distribution hypothesis. Further, RMCL adopts a multi-intention contrastive strategy, which establishes a fine-grained connection between user reviews and item reviews. Extensive experiments on five real-world datasets demonstrate significant improvements of our proposed RMCL model over the state-of-the-art methods.
BiTimeBERT: Extending Pre-Trained Language Representations with Bi-Temporal Information

Time is an important aspect of documents and is used in a range of NLP and IR tasks. In this work, we investigate methods for incorporating temporal information during pre-training to further improve the performance on time-related tasks. Compared with common pre-trained language models like BERT which utilize synchronic document collections (e.g., BookCorpus and Wikipedia) as the training corpora, we use long-span temporal news article collection for building word representations. We introduce BiTimeBERT, a novel language representation model trained on a temporal collection of news articles via two new pre-training tasks, which harnesses two distinct temporal signals to construct time-aware language representations. The experimental results show that BiTimeBERT consistently outperforms BERT and other existing pre-trained models with substantial gains on different downstream NLP tasks and applications for which time is of importance (e.g., the accuracy improvement over BERT is 155% on the event time estimation task).
Multivariate Representation Learning for Information Retrieval

Dense retrieval models use bi-encoder network architectures for learning query and document representations. These representations are often in the form of a vector representation and their similarities are often computed using the dot product function. In this paper, we propose a new representation learning framework for dense retrieval. Instead of learning a vector for each query and document, our framework learns a multivariate distribution and uses negative multivariate KL divergence to compute the similarity between distributions. For simplicity and efficiency reasons, we assume that the distributions are multivariate normals and then train large language models to produce mean and variance vectors for these distributions. We provide a theoretical foundation for the proposed framework and show that it can be seamlessly integrated into the existing approximate nearest neighbor algorithms to perform retrieval efficiently. We conduct an extensive suite of experiments on a wide range of datasets, and demonstrate significant improvements compared to competitive dense retrieval models.
Quantifying and Leveraging User Fatigue for Interventions in Recommender Systems

Predicting churn and designing intervention strategies are crucial for online platforms to maintain user engagement. We hypothesize that predicting churn, i.e. users leaving from the system without further return, is often a delayed act, and it might get too late for the system to intervene. We propose detecting early signs of users losing interest, allowing time for intervention, and introduce a new formulation ofuser fatigue as short-term dissatisfaction, providing early signals to predict long-term churn. We identify behavioral signals predicting fatigue and develop models for fatigue prediction. Furthermore, we leverage the predicted fatigue estimates to develop fatigue-aware ad-load balancing intervention strategy that reduces churn, improving short- and long-term user retention. Results from deployed recommendation system and multiple live A/B tests across over 80 million users generating over 200 million sessions highlight gains for user engagement and platform strategic metrics.
Exploring the Potential of Augmented Reality (AR) and Virtual Reality (VR) in Education

The integration of technology into education has revolutionized traditional learning methods and opened new avenues for immersive and interactive experiences. Augmented reality (AR) and virtual reality (VR) are two emerging technologies that have gained significant attention in education. This paper will explore the potential of AR and VR in education by examining their applications, benefits, and challenges. Augmented reality (AR) and virtual reality (VR) have the potential to revolutionize education. AR augments the real world with digital content, while VR creates immersive virtual environments. These technologies promote engagement, enable experiential learning, and personalize education. AR and VR enable collaboration, accessibility, and creativity. As hardware evolves and integration improves, the impact of AR and VR in education will continue to grow and change the way students learn and educators teach. This summary explores the potential of augmented reality (AR) and virtual reality (VR) in education. It highlights the transformative impact of these technologies on traditional teaching methods. AR and VR provide immersive and engaging experiences that enhance students' understanding, retention, and application of knowledge. They enable experiential learning, personalized and adaptive approaches, collaboration, and integration. In addition, AR and VR foster creativity and problem-solving skills and bridge the gap between theory and practice. As technology advances, the integration of AR and VR into education offers promising prospects for revolutionizing learning experiences
Short-Term Photovoltaic Power Plant Output Forecasting Using Sky Images and Deep Learning

With the steady increase in the use of renewable energy sources in the energy sector, new challenges arise, especially the unpredictability of these energy sources. This uncertainty complicates the management, planning, and development of energy systems. An effective solution to these challenges is short-term forecasting of the output of photovoltaic power plants. In this paper, a novel method for short-term production prediction was explored which involves continuous photography of the sky above the photovoltaic power plant. By analyzing a series of sky images, patterns can be identified to help predict future photovoltaic power generation. A hybrid model that integrates both a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) for short-term production forecasting was developed and tested. This model effectively detects spatial and temporal patterns from images and power output data, displaying considerable prediction accuracy. In particular, a 74% correlation was found between the model’s predictions and actual future production values, demonstrating the model’s efficiency. The results of this paper suggest that the hybrid CNN-LSTM model offers an improvement in prediction accuracy and practicality compared to traditional forecasting methods. This paper highlights the potential of Deep Learning in improving renewable energy practices, particularly in power prediction, contributing to the overall sustainability of power systems.
TDAN: Transferable Domain Adversarial Network for Link Prediction in Heterogeneous Social Networks

Link prediction has received increased attention in social network analysis. One of the unique challenges in heterogeneous social networks is link prediction in new link types without verified link information, such as recommending products to new overseas groups. Existing link prediction models tend to learn type-specific knowledge on specific link types and predict missing or future links on the same link types. However, because of the uncertainty of new link types in the evolving process of social networks, it is difficult to collect sufficient verified link information in new link types. Therefore, we propose the Transferable Domain Adversarial Network ( TDAN ) based on transfer learning to handle the challenge. TDAN exploits transferable type-shared knowledge in historical link types to help predict the unobserved links in new link types. TDAN mainly comprises a structural encoder, a domain discriminator, and an optimization decoder. The structural encoder learns the link representations in a heterogeneous social network. Subsequently, to learn transferable type-shared knowledge, the domain discriminator distinguishes link representations into different link types while minimizing the differences between type-specific knowledge in adversarial training. Inspired by the denoising auto-encoder, the optimization decoder reconstructs the learned type-shared knowledge to eliminate the noise generated during the adversarial training. Extensive experiments on Facebook and YouTube show that TDAN can outperform the state-of-the-art models.
Robust Engineering of Maximally Entangled States by Identical Particle Interferometry

We propose a procedure for the robust preparation of maximally entangled states of identical fermionic qubits, studying the role played by particle statistics in the process. The protocol exploits externally activated noisy channels to reset the system to a known state. The subsequent interference effects generated at a beam splitter result in a mixture of maximally entangled Bell states and NOON states. We also discuss how every maximally entangled state of two fermionic qubits distributed over two spatial modes can be obtained from one another by fermionic passive optical transformations. Using a pseudospin-insensitive, non-absorbing, parity check detector, the proposed technique is thus shown to deterministically prepare any arbitrary maximally entangled state of two identical fermions. These results extend recent findings related to bosonic qubits. Finally, we analyze the performance of the protocol for both bosons and fermions when the externally activated noisy channels are not used and the two qubits undergo standard types of noise. The results supply further insights towards viable strategies for noise-protected entanglement exploitable in quantum-enhanced technologies.