pwc-archive/papers-with-abstracts huggingface dataset:
QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errors
Authors: Kepu Zhang, Zhongxiang Sun, Weijie Yu, Xiaoxue Zang, Kai Zheng, Yang song, Han Li, Jun Xu
Date: 2025-04-05 00:00:00
Tasks: Contrastive Learning, RAG, Retrieval, Retrieval-augmented Generation

Retriever-augmented generation (RAG) has become a widely adopted approach for enhancing the factual accuracy of large language models (LLMs). While current benchmarks evaluate the performance of RAG methods from various perspectives, they share a common assumption that user queries used for retrieval are error-free. However, in real-world interactions between users and LLMs, query entry errors such as keyboard proximity errors, visual similarity errors, and spelling errors are frequent. The impact of these errors on current RAG methods against such errors remains largely unexplored. To bridge this gap, we propose QE-RAG, the first robust RAG benchmark designed specifically to evaluate performance against query entry errors. We augment six widely used datasets by injecting three common types of query entry errors into randomly selected user queries at rates of 20\% and 40\%, simulating typical user behavior in real-world scenarios. We analyze the impact of these errors on LLM outputs and find that corrupted queries degrade model performance, which can be mitigated through query correction and training a robust retriever for retrieving relevant documents. Based on these insights, we propose a contrastive learning-based robust retriever training method and a retrieval-augmented query correction method. Extensive in-domain and cross-domain experiments reveal that: (1) state-of-the-art RAG methods including sequential, branching, and iterative methods, exhibit poor robustness to query entry errors; (2) our method significantly enhances the robustness of RAG when handling query entry errors and it's compatible with existing RAG methods, further improving their robustness.
Towards Robust Offline Evaluation: A Causal and Information Theoretic Framework for Debiasing Ranking Systems
Authors: Seyedeh Baharan Khatami, Sayan Chakraborty, Ruomeng Xu, Babak Salimi
Date: 2025-04-04 00:00:00
Tasks: Retrieval

Evaluating retrieval-ranking systems is crucial for developing high-performing models. While online A/B testing is the gold standard, its high cost and risks to user experience require effective offline methods. However, relying on historical interaction data introduces biases-such as selection, exposure, conformity, and position biases-that distort evaluation metrics, driven by the Missing-Not-At-Random (MNAR) nature of user interactions and favoring popular or frequently exposed items over true user preferences. We propose a novel framework for robust offline evaluation of retrieval-ranking systems, transforming MNAR data into Missing-At-Random (MAR) through reweighting combined with black-box optimization, guided by neural estimation of information-theoretic metrics. Our contributions include (1) a causal formulation for addressing offline evaluation biases, (2) a system-agnostic debiasing framework, and (3) empirical validation of its effectiveness. This framework enables more accurate, fair, and generalizable evaluations, enhancing model assessment before deployment.
Counterfactual Inference under Thompson Sampling
Authors: Olivier Jeunen
Date: 2025-04-03 00:00:00
Tasks: Causal Inference, counterfactual, Counterfactual Inference, Decision Making, Decision Making Under Uncertainty, Recommendation Systems, Sequential Decision Making, Thompson Sampling

Recommender systems exemplify sequential decision-making under uncertainty, strategically deciding what content to serve to users, to optimise a range of potential objectives. To balance the explore-exploit trade-off successfully, Thompson sampling provides a natural and widespread paradigm to probabilistically select which action to take. Questions of causal and counterfactual inference, which underpin use-cases like offline evaluation, are not straightforward to answer in these contexts. Specifically, whilst most existing estimators rely on action propensities, these are not readily available under Thompson sampling procedures. We derive exact and efficiently computable expressions for action propensities under a variety of parameter and outcome distributions, enabling the use of off-policy estimators in Thompson sampling scenarios. This opens up a range of practical use-cases where counterfactual inference is crucial, including unbiased offline evaluation of recommender systems, as well as general applications of causal inference in online advertising, personalisation, and beyond.
Generate the browsing process for short-video recommendation
Authors: Chao Feng, Yanze Zhang, Chenghao Zhang
Date: 2025-04-02 00:00:00

This paper introduces a new model to generate the browsing process for short-video recommendation and proposes a novel Segment Content Aware Model via User Engagement Feedback (SCAM) for watch time prediction in video recommendation. Unlike existing methods that rely on multimodal features for video content understanding, SCAM implicitly models video content through users' historical watching behavior, enabling segment-level understanding without complex multimodal data. By dividing videos into segments based on duration and employing a Transformer-like architecture, SCAM captures the sequential dependence between segments while mitigating duration bias. Extensive experiments on industrial-scale and public datasets demonstrate SCAM's state-of-the-art performance in watch time prediction. The proposed approach offers a scalable and effective solution for video recommendation by leveraging segment-level modeling and users' engagement feedback.
Using Multiple Outcomes to Adjust Standard Errors for Spatial Correlation
Authors: Stefano DellaVigna, Guido Imbens, Woojin Kim, David M. Ritzwoller
Date: 2025-04-17 00:00:00

Empirical research in economics often examines the behavior of agents located in a geographic space. In such cases, statistical inference is complicated by the interdependence of economic outcomes across locations. A common approach to account for this dependence is to cluster standard errors based on a predefined geographic partition. A second strategy is to model dependence in terms of the distance between units. Dependence, however, does not necessarily stop at borders and is typically not determined by distance alone. This paper introduces a method that leverages observations of multiple outcomes to adjust standard errors for cross-sectional dependence. Specifically, a researcher, while interested in a particular outcome variable, often observes dozens of other variables for the same units. We show that these outcomes can be used to estimate dependence under the assumption that the cross-sectional correlation structure is shared across outcomes. We develop a procedure, which we call Thresholding Multiple Outcomes (TMO), that uses this estimate to adjust standard errors in a given regression setting. We show that adjustments of this form can lead to sizable reductions in the bias of standard errors in calibrated U.S. county-level regressions. Re-analyzing nine recent papers, we find that the proposed correction can make a substantial difference in practice.
How Much Weak Overlap Can Doubly Robust T-Statistics Handle?
Authors: Jacob Dorn
Date: 2025-04-17 00:00:00
Tasks: regression

In the presence of sufficiently weak overlap, it is known that no regular root-n-consistent estimators exist and standard estimators may fail to be asymptotically normal. This paper shows that a thresholded version of the standard doubly robust estimator is asymptotically normal with well-calibrated Wald confidence intervals even when constructed using nonparametric estimates of the propensity score and conditional mean outcome. The analysis implies a cost of weak overlap in terms of black-box nuisance rates, borne when the semiparametric bound is infinite, and the contribution of outcome smoothness to the outcome regression rate, which is incurred even when the semiparametric bound is finite. As a byproduct of this analysis, I show that under weak overlap, the optimal global regression rate is the same as the optimal pointwise regression rate, without the usual polylogarithmic penalty. The high-level conditions yield new rules of thumb for thresholding in practice. In simulations, thresholded AIPW can exhibit moderate overrejection in small samples, but I am unable to reject a null hypothesis of exact coverage in large samples. In an empirical application, the clipped AIPW estimator that targets the standard average treatment effect yields similar precision to a heuristic 10% fixed-trimming approach that changes the target sample.
The heterogeneous causal effects of the EU's Cohesion Fund
Authors: Angelos Alexopoulos, Ilias Kostarakos, Christos Mylonakis, Petros Varthalitis
Date: 2025-04-17 00:00:00
Tasks: Causal Inference

This paper quantifies the causal effect of cohesion policy on EU regional output and investment focusing on one of its least studied instruments, i.e., the Cohesion Fund (CF). We employ modern causal inference methods to estimate not only the local average treatment effect but also its time-varying and heterogeneous effects across regions. Utilizing this method, we propose a novel framework for evaluating the effectiveness of CF as an EU cohesion policy tool. Specifically, we estimate the time varying distribution of the CF's causal effects across EU regions and derive key distribution metrics useful for policy evaluation. Our analysis shows that relying solely on average treatment effects masks significant heterogeneity and can lead to misleading conclusions about the effectiveness of the EU's cohesion policy. We find that the impact of the CF is frontloaded, peaking within the first seven years after a region's initial inclusion in the program. The distribution of the effects during this first seven-year cycle of funding is right skewed with relatively thick tails. This indicates positive effects but unevenly distributed across regions. Moreover, the magnitude of the CF effect is inversely related to a region's relative position in the initial distribution of output, i.e., relatively poorer recipient regions experience higher effects compared to relatively richer regions. Finally, we find a non-linear relationship with diminishing returns, whereby the impact of CF declines as the ratio of CF funds received to a region's gross value added (GVA) increases.
A Proposed Hybrid Recommender System for Tourism Industry in Iraq Using Evolutionary Apriori and K-means Algorithms
Authors: Bryar A. Hassan, Alla A. Hassan, Joan Lu, Aram M. Ahmed, Tarik A. Rashid
Date: 2025-04-01 00:00:00
Tasks: Recommendation Systems

The rapid proliferation of tourism data across sectors, including accommodations, cultural sites, and events, has made it increasingly challenging for travelers to identify relevant and personalized recommendations. While traditional recommender systems such as collaborative, content-based, and context-aware systems offer partial solutions, they often struggle with issues like data sparsity and overspecialization. This study proposes a novel hybrid recommender system that combines evolutionary Apriori and K-means clustering algorithms to improve recommendation accuracy and efficiency in the tourism domain. Designed specifically to address the diverse and dynamic tourism landscape in Iraq, the system provides personalized recommendations and clusters of tourist destinations tailored to user preferences and contextual information. To evaluate the systems performance, experiments were conducted on an augmented dataset representative of Iraqs tourism activity, comparing the proposed system with existing methods. Results indicate that the proposed hybrid system significantly reduces execution time by 27-56% and space consumption by 24-31%, while achieving consistently lower Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) values, thereby enhancing prediction accuracy. This approach offers a scalable, context-aware framework that is well-suited for application in regions where tourism data is limited, such as Iraq, ultimately advancing tourism recommender systems by addressing their limitations in complex and data-scarce environments.
Evaluation of the phi-3-mini SLM for identification of texts related to medicine, health, and sports injuries
Authors: Chris Brogly, Saif Rjaibi, Charlotte Liang, Erica Lam, Edward Wang, Adam Levitan, Sarah Paleczny, Michael Cusimano
Date: 2025-03-31 00:00:00
Tasks: 4k, MedQA

Small Language Models (SLMs) have potential to be used for automatically labelling and identifying aspects of text data for medicine/health-related purposes from documents and the web. As their resource requirements are significantly lower than Large Language Models (LLMs), these can be deployed potentially on more types of devices. SLMs often are benchmarked on health/medicine-related tasks, such as MedQA, although performance on these can vary especially depending on the size of the model in terms of number of parameters. Furthermore, these test results may not necessarily reflect real-world performance regarding the automatic labelling or identification of texts in documents and the web. As a result, we compared topic-relatedness scores from Microsofts phi-3-mini-4k-instruct SLM to the topic-relatedness scores from 7 human evaluators on 1144 samples of medical/health-related texts and 1117 samples of sports injury-related texts. These texts were from a larger dataset of about 9 million news headlines, each of which were processed and assigned scores by phi-3-mini-4k-instruct. Our sample was selected (filtered) based on 1 (low filtering) or more (high filtering) Boolean conditions on the phi-3 SLM scores. We found low-moderate significant correlations between the scores from the SLM and human evaluators for sports injury texts with low filtering (\r{ho} = 0.3413, p < 0.001) and medicine/health texts with high filtering (\r{ho} = 0.3854, p < 0.001), and low significant correlation for medicine/health texts with low filtering (\r{ho} = 0.2255, p < 0.001). There was negligible, insignificant correlation for sports injury-related texts with high filtering (\r{ho} = 0.0318, p = 0.4466).
WebMap -- Large Language Model-assisted Semantic Link Induction in the Web
Authors: Shiraj Pokharel, Georg P. Roßrucker, Mario M. Kubek
Date: 2025-03-31 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model

Carrying out research tasks is only inadequately supported, if not hindered, by current web search engines. This paper therefore proposes functional extensions of WebMap, a semantically induced overlay linking structure on the web to inherently facilitate research activities. These add-ons support the dynamic determination and regrouping of document clusters, the creation of a semantic signpost in the web, and the interactive tracing of topics back to their origins.
InteractiveSurvey: An LLM-based Personalized and Interactive Survey Paper Generation System
Authors: Zhiyuan Wen, Jiannong Cao, Zian Wang, Beichen Guo, Ruosong Yang, Shuaiqi Liu
Date: 2025-03-31 00:00:00
Tasks: Paper generation, RAG, Retrieval, Retrieval-augmented Generation, Survey

The exponential growth of academic literature creates urgent demands for comprehensive survey papers, yet manual writing remains time-consuming and labor-intensive. Recent advances in large language models (LLMs) and retrieval-augmented generation (RAG) facilitate studies in synthesizing survey papers from multiple references, but most existing works restrict users to title-only inputs and fixed outputs, neglecting the personalized process of survey paper writing. In this paper, we introduce InteractiveSurvey - an LLM-based personalized and interactive survey paper generation system. InteractiveSurvey can generate structured, multi-modal survey papers with reference categorizations from multiple reference papers through both online retrieval and user uploads. More importantly, users can customize and refine intermediate components continuously during generation, including reference categorization, outline, and survey content through an intuitive interface. Evaluations of content quality, time efficiency, and user studies show that InteractiveSurvey is an easy-to-use survey generation system that outperforms most LLMs and existing methods in output content quality while remaining highly time-efficient.
UltraRAG: A Modular and Automated Toolkit for Adaptive Retrieval-Augmented Generation
Authors: Yuxuan Chen, Dewen Guo, Sen Mei, Xinze Li, Hao Chen, Yishan Li, YiXuan Wang, Chaoyue Tang, Ruobing Wang, Dingjun Wu, Yukun Yan, Zhenghao Liu, Shi Yu, Zhiyuan Liu, Maosong Sun
Date: 2025-03-31 00:00:00
Tasks: RAG, Retrieval, Retrieval-augmented Generation

Retrieval-Augmented Generation (RAG) significantly enhances the performance of large language models (LLMs) in downstream tasks by integrating external knowledge. To facilitate researchers in deploying RAG systems, various RAG toolkits have been introduced. However, many existing RAG toolkits lack support for knowledge adaptation tailored to specific application scenarios. To address this limitation, we propose UltraRAG, a RAG toolkit that automates knowledge adaptation throughout the entire workflow, from data construction and training to evaluation, while ensuring ease of use. UltraRAG features a user-friendly WebUI that streamlines the RAG process, allowing users to build and optimize systems without coding expertise. It supports multimodal input and provides comprehensive tools for managing the knowledge base. With its highly modular architecture, UltraRAG delivers an end-to-end development solution, enabling seamless knowledge adaptation across diverse user scenarios. The code, demonstration videos, and installable package for UltraRAG are publicly available at https://github.com/OpenBMB/UltraRAG.
Hyper-RAG: Combating LLM Hallucinations using Hypergraph-Driven Retrieval-Augmented Generation
Authors: Yifan Feng, Hao Hu, Xingliang Hou, Shiquan Liu, Shihui Ying, Shaoyi Du, Han Hu, Yue Gao
Date: 2025-03-30 00:00:00
Tasks: RAG, Retrieval, Retrieval-augmented Generation

Large language models (LLMs) have transformed various sectors, including education, finance, and medicine, by enhancing content generation and decision-making processes. However, their integration into the medical field is cautious due to hallucinations, instances where generated content deviates from factual accuracy, potentially leading to adverse outcomes. To address this, we introduce Hyper-RAG, a hypergraph-driven Retrieval-Augmented Generation method that comprehensively captures both pairwise and beyond-pairwise correlations in domain-specific knowledge, thereby mitigating hallucinations. Experiments on the NeurologyCrop dataset with six prominent LLMs demonstrated that Hyper-RAG improves accuracy by an average of 12.3% over direct LLM use and outperforms Graph RAG and Light RAG by 6.3% and 6.0%, respectively. Additionally, Hyper-RAG maintained stable performance with increasing query complexity, unlike existing methods which declined. Further validation across nine diverse datasets showed a 35.5% performance improvement over Light RAG using a selection-based assessment. The lightweight variant, Hyper-RAG-Lite, achieved twice the retrieval speed and a 3.3% performance boost compared with Light RAG. These results confirm Hyper-RAG's effectiveness in enhancing LLM reliability and reducing hallucinations, making it a robust solution for high-stakes applications like medical diagnostics.
The impact of institutional quality on the relation between FDI and house prices in ASEAN emerging countries
Authors: Hoang-Anh Le
Date: 2025-04-18 00:00:00

This study investigates the relationship between house prices and capital inflows in ASEAN emerging economies, and the impact of institutional quality on that relation. Using a unique balanced panel data set of six emerging countries in ASEAN from 2009 to 2019, we employ various econometric techniques to examine the impact of foreign direct investment (FDI) on the house price index. Our findings indicate a long-run relationship and Granger causality from FDI to the house price index in these markets, and we also find evidence of co-movement between the stock price index and the house price index. Additionally, our results suggest that better institutions reduce the impact of FDI on host country housing markets in the context of ASEAN emerging economies. This is one of the first studies to shed light on the role of institutional quality in the effect of FDIs on housing prices in this region.
Balancing Engagement and Polarization: Multi-Objective Alignment of News Content Using LLMs
Authors: Mengjie, Cheng, Elie Ofek, Hema Yoganarasimhan
Date: 2025-04-18 00:00:00
Tasks: Articles

We study how media firms can use LLMs to generate news content that aligns with multiple objectives -- making content more engaging while maintaining a preferred level of polarization/slant consistent with the firm's editorial policy. Using news articles from The New York Times, we first show that more engaging human-written content tends to be more polarizing. Further, naively employing LLMs (with prompts or standard Direct Preference Optimization approaches) to generate more engaging content can also increase polarization. This has an important managerial and policy implication: using LLMs without building in controls for limiting slant can exacerbate news media polarization. We present a constructive solution to this problem based on the Multi-Objective Direct Preference Optimization (MODPO) algorithm, a novel approach that integrates Direct Preference Optimization with multi-objective optimization techniques. We build on open-source LLMs and develop a new language model that simultaneously makes content more engaging while maintaining a preferred editorial stance. Our model achieves this by modifying content characteristics strongly associated with polarization but that have a relatively smaller impact on engagement. Our approach and findings apply to other settings where firms seek to use LLMs for content creation to achieve multiple objectives, e.g., advertising and social media.
A Framework for Lightweight Responsible Prompting Recommendation
Authors: Tiago Machado, Sara E. Berger, Cassia Sanctos, Vagner Figueiredo de Santana, Lemara Williams, Zhaoqing Wu
Date: 2025-03-29 00:00:00
Tasks: Sentence, Sentence Embeddings

Computer Science and Design practitioners have been researching and proposing alternatives for a dearth of recommendations, standards, or best practices in user interfaces for decades. Now, with the advent of generative Artificial Intelligence (GenAI), we have yet again an emerging, powerful technology that lacks sufficient guidance in terms of possible interactions, inputs, and outcomes. In this context, this work proposes a lightweight framework for responsible prompting recommendation to be added before the prompt is sent to GenAI. The framework is comprised of (1) a human-curated dataset for recommendations, (2) a red team dataset for assessing recommendations, (3) a sentence transformer for semantics mapping, (4) a similarity metric to map input prompt to recommendations, (5) a set of similarity thresholds, (6) quantized sentence embeddings, (7) a recommendation engine, and (8) an evaluation step to use the red team dataset. With the proposed framework and open-source system, the contributions presented can be applied in multiple contexts where end-users can benefit from guidance for interacting with GenAI in a more responsible way, recommending positive values to be added and harmful sentences to be removed.