pwc-archive/papers-with-abstracts huggingface dataset:
Statistical ranking with dynamic covariates
Authors: Pinjun Dong, Ruijian Han, Binyan Jiang, Yiming Xu
Date: 2024-06-24 00:00:00

We introduce a general covariate-assisted statistical ranking model within the Plackett--Luce framework. Unlike previous studies focusing on individual effects with fixed covariates, our model allows covariates to vary across comparisons. This added flexibility enhances model fitting yet brings significant challenges in analysis. This paper addresses these challenges in the context of maximum likelihood estimation (MLE). We first provide sufficient and necessary conditions for both model identifiability and the unique existence of the MLE. Then, we develop an efficient alternating maximization algorithm to compute the MLE. Under suitable assumptions on the design of comparison graphs and covariates, we establish a uniform consistency result for the MLE, with convergence rates determined by the asymptotic graph connectivity. We also construct random designs where the proposed assumptions hold almost surely. Numerical studies are conducted to support our findings and demonstrate the model's application to real-world datasets, including horse racing and tennis competitions.
Reinterpreting Economic Complexity: A co-clustering approach
Authors: Carlo Bottai, Jacopo Di Iorio, Martina Iori
Date: 2024-06-23 00:00:00
Tasks: Clustering

Economic growth results from countries' accumulation of organizational and technological capabilities. The Economic and Product Complexity Indices, introduced as an attempt to measure these capabilities from a country's basket of exported products, have become popular to study economic development, the geography of innovation, and industrial policies. Despite this reception, the interpretation of these indicators proved difficult. Although the original Method of Reflections suggested a direct interconnection between country and product metrics, it has been proved that the Economic and Product Complexity Indices result from a spectral clustering algorithm that separately groups similar countries or similar products, respectively. This recent approach to economic and product complexity conflicts with the original one and treats separately countries and products. However, building on previous interpretations of the indices and the recent evolution in spectral clustering, we show that these indices simultaneously identify two co-clusters of similar countries and products. This viewpoint reconciles the spectral clustering interpretation of the indices with the original Method of Reflections interpretation. By proving the often neglected intimate relationship between country and product complexity, this approach emphasizes the role of a selected set of products in determining economic development while extending the range of applications of these indicators in economics.
Hidden Variables unseen by Random Forests
Authors: Ricardo Blum, Munir Hiabu, Enno Mammen, Joseph Theo Meyer
Date: 2024-06-19 00:00:00

Random Forests are widely claimed to capture interactions well. However, some simple examples suggest that they perform poorly in the presence of certain pure interactions that the conventional CART criterion struggles to capture during tree construction. We argue that simple alternative partitioning schemes used in the tree growing procedure can enhance identification of these interactions. In a simulation study we compare these variants to conventional Random Forests and Extremely Randomized trees. Our results validate that the modifications considered enhance the model's fitting ability in scenarios where pure interactions play a crucial role.
Meta-experiments: Improving experimentation through experimentation
Authors: Melanie J. I. Müller
Date: 2024-06-24 00:00:00

A/B testing is widexly used in the industry to optimize customer facing websites. Many companies employ experimentation specialists to facilitate and improve the process of A/B testing. Here, we present the application of A/B testing to this improvement effort itself, by running experiments on the experimentation process, which we call 'meta-experiments'. We discuss the challenges of this approach using the example of one of our meta-experiments, which helped experimenters to run more sufficiently powered A/B tests. We also point out the benefits of 'dog fooding' for the experimentation specialists when running their own experiments.
Star+: A New Multi-Domain Model for CTR Prediction
Authors: Çağrı Yeşil, Kaya Turgut
Date: 2024-06-24 00:00:00
Tasks: Click-Through Rate Prediction, Multi-Task Learning, Prediction, Recommendation Systems

In this paper, we introduce Star+, a novel multi-domain model for click-through rate (CTR) prediction inspired by the Star model. Traditional single-domain approaches and existing multi-task learning techniques face challenges in multi-domain environments due to their inability to capture domain-specific data distributions and complex inter-domain relationships. Star+ addresses these limitations by enhancing the interaction between shared and domain-specific information through various fusion strategies, such as add, adaptive add, concatenation, and gating fusions, to find the optimal balance between domain-specific and shared information. We also investigate the impact of different normalization techniques, including layer normalization, batch normalization, and partition normalization, on the performance of our model. Our extensive experiments on both industrial and public datasets demonstrate that Star+ significantly improves prediction accuracy and efficiency. This work contributes to the advancement of recommendation systems by providing a robust, scalable, and adaptive solution for multi-domain environments.
Context-augmented Retrieval: A Novel Framework for Fast Information Retrieval based Response Generation using Large Language Model
Authors: Sai Ganesh, Anupam Purwar, Gautam B
Date: 2024-06-24 00:00:00
Tasks: Answer Generation, Information Retrieval, Language Modeling, Language Modelling, Large Language Model, Question Answering, RAG, Response Generation, Retrieval, Retrieval-augmented Generation, text-classification, Text Classification

Generating high-quality answers consistently by providing contextual information embedded in the prompt passed to the Large Language Model (LLM) is dependent on the quality of information retrieval. As the corpus of contextual information grows, the answer/inference quality of Retrieval Augmented Generation (RAG) based Question Answering (QA) systems declines. This work solves this problem by combining classical text classification with the Large Language Model (LLM) to enable quick information retrieval from the vector store and ensure the relevancy of retrieved information. For the same, this work proposes a new approach Context Augmented retrieval (CAR), where partitioning of vector database by real-time classification of information flowing into the corpus is done. CAR demonstrates good quality answer generation along with significant reduction in information retrieval and answer generation time.
On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Models
Authors: Dongyang Li, Junbing Yan, Taolin Zhang, Chengyu Wang, Xiaofeng He, Longtao Huang, Hui Xue, Jun Huang
Date: 2024-06-24 00:00:00
Tasks: RAG, Retrieval, Retrieval-augmented Generation, World Knowledge

Retrieval augmented generation (RAG) exhibits outstanding performance in promoting the knowledge capabilities of large language models (LLMs) with retrieved documents related to user queries. However, RAG only focuses on improving the response quality of LLMs via enhancing queries indiscriminately with retrieved information, paying little attention to what type of knowledge LLMs really need to answer original queries more accurately. In this paper, we suggest that long-tail knowledge is crucial for RAG as LLMs have already remembered common world knowledge during large-scale pre-training. Based on our observation, we propose a simple but effective long-tail knowledge detection method for LLMs. Specifically, the novel Generative Expected Calibration Error (GECE) metric is derived to measure the ``long-tailness'' of knowledge based on both statistics and semantics. Hence, we retrieve relevant documents and infuse them into the model for patching knowledge loopholes only when the input query relates to long-tail knowledge. Experiments show that, compared to existing RAG pipelines, our method achieves over 4x speedup in average inference time and consistent performance improvement in downstream tasks.
A Survey on Intent-aware Recommender Systems
Authors: Dietmar Jannach, Markus Zanker
Date: 2024-06-24 00:00:00
Tasks: Recommendation Systems, Survey

Many modern online services feature personalized recommendations. A central challenge when providing such recommendations is that the reason why an individual user accesses the service may change from visit to visit or even during an ongoing usage session. To be effective, a recommender system should therefore aim to take the users' probable intent of using the service at a certain point in time into account. In recent years, researchers have thus started to address this challenge by incorporating intent-awareness into recommender systems. Correspondingly, a number of technical approaches were put forward, including diversification techniques, intent prediction models or latent intent modeling approaches. In this paper, we survey and categorize existing approaches to building the next generation of Intent-Aware Recommender Systems (IARS). Based on an analysis of current evaluation practices, we outline open gaps and possible future directions in this area, which in particular include the consideration of additional interaction signals and contextual information to further improve the effectiveness of such systems.
A Mechanism for Optimizing Media Recommender Systems
Authors: Brian McFadden
Date: 2024-06-23 00:00:00
Tasks: Recommendation Systems

A mechanism is described that addresses the fundamental trade off between media producers who want to increase reach and consumers who provide attention based on the rate of utility received, and where overreach negatively impacts that rate. An optimal solution can be achieved when the media source considers the impact of overreach in a cost function used in determining the optimal distribution of content to maximize individual consumer utility and participation. The result is a Nash equilibrium between producer and consumer that is also Pareto efficient. Comparison with the literature on Recommender systems highlights the advantages of the mechanism, including identifying an optimal content volume for the consumer and improvements for optimizing with multiple objectives. A practical algorithm for generating the optimal distribution for each consumer is provided.
Evaluating D-MERIT of Partial-annotation on Information Retrieval
Authors: Royi Rassin, Yaron Fairstein, Oren Kalinsky, Guy Kushilevitz, Nachshon Cohen, Alexander Libov, Yoav Goldberg
Date: 2024-06-23 00:00:00
Tasks: Information Retrieval, Passage Retrieval, Retrieval, Text Retrieval

Retrieval models are often evaluated on partially-annotated datasets. Each query is mapped to a few relevant texts and the remaining corpus is assumed to be irrelevant. As a result, models that successfully retrieve false negatives are punished in evaluation. Unfortunately, completely annotating all texts for every query is not resource efficient. In this work, we show that using partially-annotated datasets in evaluation can paint a distorted picture. We curate D-MERIT, a passage retrieval evaluation set from Wikipedia, aspiring to contain all relevant passages for each query. Queries describe a group (e.g., "journals about linguistics") and relevant passages are evidence that entities belong to the group (e.g., a passage indicating that "Language" is a journal about linguistics). We show that evaluating on a dataset containing annotations for only a subset of the relevant passages might result in misleading ranking of the retrieval systems and that as more relevant texts are included in the evaluation set, the rankings converge. We propose our dataset as a resource for evaluation and our study as a recommendation for balance between resource-efficiency and reliable evaluation when annotating evaluation sets for text retrieval.
Learning k-Determinantal Point Processes for Personalized Ranking
Authors: Yuli Liu, Christian Walder, Lexing Xie
Date: 2024-06-23 00:00:00
Tasks: Diversity, Point Processes, Recommendation Systems

The key to personalized recommendation is to predict a personalized ranking on a catalog of items by modeling the user's preferences. There are many personalized ranking approaches for item recommendation from implicit feedback like Bayesian Personalized Ranking (BPR) and listwise ranking. Despite these methods have shown performance benefits, there are still limitations affecting recommendation performance. First, none of them directly optimize ranking of sets, causing inadequate exploitation of correlations among multiple items. Second, the diversity aspect of recommendations is insufficiently addressed compared to relevance. In this work, we present a new optimization criterion LkP based on set probability comparison for personalized ranking that moves beyond traditional ranking-based methods. It formalizes set-level relevance and diversity ranking comparisons through a Determinantal Point Process (DPP) kernel decomposition. To confer ranking interpretability to the DPP set probabilities and prioritize the practicality of LkP, we condition the standard DPP on the cardinality k of the DPP-distributed set, known as k-DPP, a less-explored extension of DPP. The generic stochastic gradient descent based technique can be directly applied to optimizing models that employ LkP. We implement LkP in the context of both Matrix Factorization (MF) and neural networks approaches, on three real-world datasets, obtaining improved relevance and diversity performances. LkP is broadly applicable, and when applied to existing recommendation models it also yields strong performance improvements, suggesting that LkP holds significant value to the field of recommender systems.
FIRST: Faster Improved Listwise Reranking with Single Token Decoding
Authors: Revanth Gangi Reddy, JaeHyeok Doo, Yifei Xu, Md Arafat Sultan, Deevya Swain, Avirup Sil, Heng Ji
Date: 2024-06-21 00:00:00
Tasks: Information Retrieval, Language Modeling, Language Modelling, Learning-To-Rank, Reranking

Large Language Models (LLMs) have significantly advanced the field of information retrieval, particularly for reranking. Listwise LLM rerankers have showcased superior performance and generalizability compared to existing supervised approaches. However, conventional listwise LLM reranking methods lack efficiency as they provide ranking output in the form of a generated ordered sequence of candidate passage identifiers. Further, they are trained with the typical language modeling objective, which treats all ranking errors uniformly--potentially at the cost of misranking highly relevant passages. Addressing these limitations, we introduce FIRST, a novel listwise LLM reranking approach leveraging the output logits of the first generated identifier to directly obtain a ranked ordering of the candidates. Further, we incorporate a learning-to-rank loss during training, prioritizing ranking accuracy for the more relevant passages. Empirical results demonstrate that FIRST accelerates inference by 50% while maintaining a robust ranking performance with gains across the BEIR benchmark. Finally, to illustrate the practical effectiveness of listwise LLM rerankers, we investigate their application in providing relevance feedback for retrievers during inference. Our results show that LLM rerankers can provide a stronger distillation signal compared to cross-encoders, yielding substantial improvements in retriever recall after relevance feedback.
Food Pairing Unveiled: Exploring Recipe Creation Dynamics through Recommender Systems
Authors: Giovanni Palermo, Claudio Caprioli, Giambattista Albora
Date: 2024-06-21 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

In the early 2000s, renowned chef Heston Blumenthal formulated his "food pairing" hypothesis, positing that if foods share many flavor compounds, then they tend to taste good when eaten together. In 2011, Ahn et al. conducted a study using a dataset of recipes, ingredients, and flavor compounds, finding that, in Western cuisine, ingredients in recipes often share more flavor compounds than expected by chance, indicating a natural tendency towards food pairing. Building upon Ahn's research, our work applies state-of-the-art collaborative filtering techniques to the dataset, providing a tool that can recommend new foods to add in recipes, retrieve missing ingredients and advise against certain combinations. We create our recommender in two ways, by taking into account ingredients appearances in recipes or shared flavor compounds between foods. While our analysis confirms the existence of food pairing, the recipe-based recommender performs significantly better than the flavor-based one, leading to the conclusion that food pairing is just one of the principles to take into account when creating recipes. Furthermore, and more interestingly, we find that food pairing in data is mostly due to trivial couplings of very similar ingredients, leading to a reconsideration of its current role in recipes, from being an already existing feature to a key to open up new scenarios in gastronomy. Our flavor-based recommender can thus leverage this novel concept and provide a new tool to lead culinary innovation.
AdFlush: A Real-World Deployable Machine Learning Solution for Effective Advertisement and Web Tracker Prevention
Authors: Kiho Lee, Chaejin Lim, Beomjin Jin, TaeYoung Kim, Hyoungshick Kim
Date: 2024-05-13 00:00:00
Proceeding: WWW: Proceedings of the ACM on Web Conference 2024 5
Tasks: Blocking, CPU

Conventional ad blocking and tracking prevention tools often fall short in addressing web content manipulation. Machine learning approaches have been proposed to enhance detection accuracy, yet aspects of practical deployment have frequently been overlooked. This paper introduces AdFlush, a novel machine learning model for real-world browsers. To develop AdFlush, we evaluated the effectiveness of 883 features, ultimately selecting 27 key features for optimal performance. We tested AdFlush on a dataset of 10,000 real-world websites, achieving an F1 score of 0.98, thereby outperforming AdGraph (F1 score: 0.93), WebGraph (F1 score: 0.90), and WTAgraph (F1 score: 0.84). Additionally, AdFlush significantly reduces computational overhead, requiring 56% less CPU and 80% less memory than AdGraph. We also assessed AdFlush's robustness against adversarial manipulations, demonstrating superior resilience with F1 scores ranging from 0.89 to 0.98, surpassing the performance of AdGraph and WebGraph, which recorded F1 scores between 0.81 and 0.87. A six-month longitudinal study confirmed that AdFlush maintains a high F1 score above 0.97 without the need for retraining, underscoring its effectiveness.
Mental Disorder Classification via Temporal Representation of Text
Authors: Raja Kumar, Kishan Maharaj, Ashita Saxena, Pushpak Bhattacharyya
Date: 2024-06-15 00:00:00
Tasks: Classification, Language Modelling

Mental disorders pose a global challenge, aggravated by the shortage of qualified mental health professionals. Mental disorder prediction from social media posts by current LLMs is challenging due to the complexities of sequential text data and the limited context length of language models. Current language model-based approaches split a single data instance into multiple chunks to compensate for limited context size. The predictive model is then applied to each chunk individually, and the most voted output is selected as the final prediction. This results in the loss of inter-post dependencies and important time variant information, leading to poor performance. We propose a novel framework which first compresses the large sequence of chronologically ordered social media posts into a series of numbers. We then use this time variant representation for mental disorder classification. We demonstrate the generalization capabilities of our framework by outperforming the current SOTA in three different mental conditions: depression, self-harm, and anorexia, with an absolute improvement of 5% in the F1 score. We investigate the situation where current data instances fall within the context length of language models and present empirical results highlighting the importance of temporal properties of textual data. Furthermore, we utilize the proposed framework for a cross-domain study, exploring commonalities across disorders and the possibility of inter-domain data usage.
Accumulating Word Representations in Multi-level Context Integration for ERC Task
Authors: Jieying Xue, Phuong Minh Nguyen, Matheny Blake, Nguyen Minh Le
Date: 2023-11-06 00:00:00
Proceeding: International Conference on Knowledge and Systems Engineering (KSE) 2023 11
Tasks: Emotion Recognition, Emotion Recognition in Conversation, Sentence

Emotion Recognition in Conversations (ERC) has attracted augmented interest recently because of its pronounced adaptability, which is to forecast the sentiment label for each utterance given a conversation as context. In order to identify the emotion of a focal sentence, it is crucial to model its meaning fused with contextual information. Many recent studies have focused on capturing different types of context as supporting information and integrated it in various ways: local and global contexts or at the speaker level through intra-speaker and inter-speaker integration. However, the importance of word representations after context integration has not been investigated completely, while word information is also essential to reflect the speaker's emotions in the conversation. Therefore, in this work, we endeavor to investigate the impact of accumulating word vector representations on sentence modeling fused with multi-level contextual integration. To this end, we propose an effective method for sentence modeling in ERC tasks and achieve competitive state-of-the-art results across four widely recognized bench-mark datasets: Iemocap, MELD, EmoryNLP, and DailyDialog. Our source code can be accessed via the following link: github.com/yingjie7/per_erc/tree/AccumWR.
SegHist: A General Segmentation-based Framework for Chinese Historical Document Text Line Detection
Authors: Xingjian Hu, Baole Wei, Liangcai Gao, Jun Wang
Date: 2024-06-17 00:00:00
Tasks: Line Detection, Text Detection

Text line detection is a key task in historical document analysis facing many challenges of arbitrary-shaped text lines, dense texts, and text lines with high aspect ratios, etc. In this paper, we propose a general framework for historical document text detection (SegHist), enabling existing segmentation-based text detection methods to effectively address the challenges, especially text lines with high aspect ratios. Integrating the SegHist framework with the commonly used method DB++, we develop DB-SegHist. This approach achieves SOTA on the CHDAC, MTHv2, and competitive results on HDRC datasets, with a significant improvement of 1.19% on the most challenging CHDAC dataset which features more text lines with high aspect ratios. Moreover, our method attains SOTA on rotated MTHv2 and rotated HDRC, demonstrating its rotational robustness. The code is available at https://github.com/LumionHXJ/SegHist.