pwc-archive/papers-with-abstracts huggingface dataset:
Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection
Authors: Yachao Zhao, Bo wang, Yan Wang
Date: 2025-01-04 00:00:00

Large Language Models (LLMs) have been shown to exhibit various biases and stereotypes in their generated content. While extensive research has investigated bias in LLMs, prior work has predominantly focused on explicit bias, leaving the more nuanced implicit biases largely unexplored. This paper presents a systematic framework grounded in social psychology theories to investigate and compare explicit and implicit biases in LLMs. We propose a novel "self-reflection" based evaluation framework that operates in two phases: first measuring implicit bias through simulated psychological assessment methods, then evaluating explicit bias by prompting LLMs to analyze their own generated content. Through extensive experiments on state-of-the-art LLMs across multiple social dimensions, we demonstrate that LLMs exhibit a substantial inconsistency between explicit and implicit biases, where explicit biases manifest as mild stereotypes while implicit biases show strong stereotypes. Furthermore, we investigate the underlying factors contributing to this explicit-implicit bias inconsistency. Our experiments examine the effects of training data scale, model parameters, and alignment techniques. Results indicate that while explicit bias diminishes with increased training data and model size, implicit bias exhibits a contrasting upward trend. Notably, contemporary alignment methods (e.g., RLHF, DPO) effectively suppress explicit bias but show limited efficacy in mitigating implicit bias. These findings suggest that while scaling up models and alignment training can address explicit bias, the challenge of implicit bias requires novel approaches beyond current methodologies.
Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and Trends
Authors: Camille Barboule, Benjamin Piwowarski, Yoan Chabot
Date: 2025-01-04 00:00:00
Tasks: document understanding, Question Answering, Survey

Using Large Language Models (LLMs) for Visually-rich Document Understanding (VrDU) has significantly improved performance on tasks requiring both comprehension and generation, such as question answering, albeit introducing new challenges. This survey explains how VrDU models enhanced by LLMs function, covering methods for integrating VrD features into LLMs and highlighting key challenges.
Personalized Graph-Based Retrieval for Large Language Models
Authors: Steven Au, Cameron J. Dimacali, Ojasmitha Pedirappagari, Namyong Park, Franck Dernoncourt, Yu Wang, Nikos Kanakaris, Hanieh Deilamsalehy, Ryan A. Rossi, Nesreen K. Ahmed
Date: 2025-01-04 00:00:00
Tasks: Knowledge Graphs, Retrieval, Retrieval-augmented Generation, Text Generation

As large language models (LLMs) evolve, their ability to deliver personalized and context-aware responses offers transformative potential for improving user experiences. Existing personalization approaches, however, often rely solely on user history to augment the prompt, limiting their effectiveness in generating tailored outputs, especially in cold-start scenarios with sparse data. To address these limitations, we propose Personalized Graph-based Retrieval-Augmented Generation (PGraphRAG), a framework that leverages user-centric knowledge graphs to enrich personalization. By directly integrating structured user knowledge into the retrieval process and augmenting prompts with user-relevant context, PGraphRAG enhances contextual understanding and output quality. We also introduce the Personalized Graph-based Benchmark for Text Generation, designed to evaluate personalized text generation tasks in real-world settings where user history is sparse or unavailable. Experimental results show that PGraphRAG significantly outperforms state-of-the-art personalization methods across diverse tasks, demonstrating the unique advantages of graph-based retrieval for personalization.
Applying Text Mining to Analyze Human Question Asking in Creativity Research
Authors: Anna Wróblewska, Marceli Korbin, Yoed N. Kenett, Daniel Dan, Maria Ganzha, Marcin Paprzycki
Date: 2025-01-03 00:00:00

Creativity relates to the ability to generate novel and effective ideas in the areas of interest. How are such creative ideas generated? One possible mechanism that supports creative ideation and is gaining increased empirical attention is by asking questions. Question asking is a likely cognitive mechanism that allows defining problems, facilitating creative problem solving. However, much is unknown about the exact role of questions in creativity. This work presents an attempt to apply text mining methods to measure the cognitive potential of questions, taking into account, among others, (a) question type, (b) question complexity, and (c) the content of the answer. This contribution summarizes the history of question mining as a part of creativity research, along with the natural language processing methods deemed useful or helpful in the study. In addition, a novel approach is proposed, implemented, and applied to five datasets. The experimental results obtained are comprehensively analyzed, suggesting that natural language processing has a role to play in creative research.
Instruction-Following Pruning for Large Language Models
Authors: Bairu Hou, Qibin Chen, Jianyu Wang, Guoli Yin, Chong Wang, Nan Du, Ruoming Pang, Shiyu Chang, Tao Lei
Date: 2025-01-03 00:00:00
Tasks: Instruction Following, Math

With the rapid scaling of large language models (LLMs), structured pruning has become a widely used technique to learn efficient, smaller models from larger ones, delivering superior performance compared to training similarly sized models from scratch. In this paper, we move beyond the traditional static pruning approach of determining a fixed pruning mask for a model, and propose a dynamic approach to structured pruning. In our method, the pruning mask is input-dependent and adapts dynamically based on the information described in a user instruction. Our approach, termed "instruction-following pruning", introduces a sparse mask predictor that takes the user instruction as input and dynamically selects the most relevant model parameters for the given task. To identify and activate effective parameters, we jointly optimize the sparse mask predictor and the LLM, leveraging both instruction-following data and the pre-training corpus. Experimental results demonstrate the effectiveness of our approach on a wide range of evaluation benchmarks. For example, our 3B activated model improves over the 3B dense model by 5-8 points of absolute margin on domains such as math and coding, and rivals the performance of a 9B model.
AGGA: A Dataset of Academic Guidelines for Generative AI and Large Language Models
Authors: Junfeng Jiao, Saleh Afroogh, Kevin Chen, David Atkinson, Amit Dhurandhar
Date: 2025-01-03 00:00:00

This study introduces AGGA, a dataset comprising 80 academic guidelines for the use of Generative AIs (GAIs) and Large Language Models (LLMs) in academic settings, meticulously collected from official university websites. The dataset contains 188,674 words and serves as a valuable resource for natural language processing tasks commonly applied in requirements engineering, such as model synthesis, abstraction identification, and document structure assessment. Additionally, AGGA can be further annotated to function as a benchmark for various tasks, including ambiguity detection, requirements categorization, and the identification of equivalent requirements. Our methodologically rigorous approach ensured a thorough examination, with a selection of universities that represent a diverse range of global institutions, including top-ranked universities across six continents. The dataset captures perspectives from a variety of academic fields, including humanities, technology, and both public and private institutions, offering a broad spectrum of insights into the integration of GAIs and LLMs in academia.
CarbonChat: Large Language Model-Based Corporate Carbon Emission Analysis and Climate Knowledge Q&A System
Authors: Zhixuan Cao, Ming Han, Jingtao Wang, Meng Jia
Date: 2025-01-03 00:00:00
Tasks: Chunking, Hallucination, Intent Recognition, Language Modeling, Language Modelling, Large Language Model, Retrieval, Retrieval-augmented Generation

As the impact of global climate change intensifies, corporate carbon emissions have become a focal point of global attention. In response to issues such as the lag in climate change knowledge updates within large language models, the lack of specialization and accuracy in traditional augmented generation architectures for complex problems, and the high cost and time consumption of sustainability report analysis, this paper proposes CarbonChat: Large Language Model-based corporate carbon emission analysis and climate knowledge Q&A system, aimed at achieving precise carbon emission analysis and policy understanding.First, a diversified index module construction method is proposed to handle the segmentation of rule-based and long-text documents, as well as the extraction of structured data, thereby optimizing the parsing of key information.Second, an enhanced self-prompt retrieval-augmented generation architecture is designed, integrating intent recognition, structured reasoning chains, hybrid retrieval, and Text2SQL, improving the efficiency of semantic understanding and query conversion.Next, based on the greenhouse gas accounting framework, 14 dimensions are established for carbon emission analysis, enabling report summarization, relevance evaluation, and customized responses.Finally, through a multi-layer chunking mechanism, timestamps, and hallucination detection features, the accuracy and verifiability of the analysis results are ensured, reducing hallucination rates and enhancing the precision of the responses.
Boosting Explainability through Selective Rationalization in Pre-trained Language Models
Authors: Libing Yuan, Shuaibo Hu, Kui Yu, Le Wu
Date: 2025-01-03 00:00:00

The widespread application of pre-trained language models (PLMs) in natural language processing (NLP) has led to increasing concerns about their explainability. Selective rationalization is a self-explanatory framework that selects human-intelligible input subsets as rationales for predictions. Recent studies have shown that applying existing rationalization frameworks to PLMs will result in severe degeneration and failure problems, producing sub-optimal or meaningless rationales. Such failures severely damage trust in rationalization methods and constrain the application of rationalization techniques on PLMs. In this paper, we find that the homogeneity of tokens in the sentences produced by PLMs is the primary contributor to these problems. To address these challenges, we propose a method named Pre-trained Language Model's Rationalization (PLMR), which splits PLMs into a generator and a predictor to deal with NLP tasks while providing interpretable rationales. The generator in PLMR also alleviates homogeneity by pruning irrelevant tokens, while the predictor uses full-text information to standardize predictions. Experiments conducted on two widely used datasets across multiple PLMs demonstrate the effectiveness of the proposed method PLMR in addressing the challenge of applying selective rationalization to PLMs. Codes: https://github.com/ylb777/PLMR.
Enhancing Uncertainty Modeling with Semantic Graph for Hallucination Detection
Authors: Kedi Chen, Qin Chen, Jie zhou, Xinqi Tao, Bowen Ding, Jingwen Xie, Mingchen Xie, Peilong Li, Feng Zheng, Liang He
Date: 2025-01-02 00:00:00
Tasks: Hallucination, Sentence

Large Language Models (LLMs) are prone to hallucination with non-factual or unfaithful statements, which undermines the applications in real-world scenarios. Recent researches focus on uncertainty-based hallucination detection, which utilizes the output probability of LLMs for uncertainty calculation and does not rely on external knowledge or frequent sampling from LLMs. Whereas, most approaches merely consider the uncertainty of each independent token, while the intricate semantic relations among tokens and sentences are not well studied, which limits the detection of hallucination that spans over multiple tokens and sentences in the passage. In this paper, we propose a method to enhance uncertainty modeling with semantic graph for hallucination detection. Specifically, we first construct a semantic graph that well captures the relations among entity tokens and sentences. Then, we incorporate the relations between two entities for uncertainty propagation to enhance sentence-level hallucination detection. Given that hallucination occurs due to the conflict between sentences, we further present a graph-based uncertainty calibration method that integrates the contradiction probability of the sentence with its neighbors in the semantic graph for uncertainty calculation. Extensive experiments on two datasets show the great advantages of our proposed approach. In particular, we obtain substantial improvements with 19.78% in passage-level hallucination detection.
Evolving Generalizable Parallel Algorithm Portfolios via Domain-Agnostic Instance Generation
Authors: Zhiyuan Wang, Shengcai Liu, Peng Yang, Ke Tang
Date: 2025-01-06 00:00:00

Generalization is the core objective when training optimizers from data. However, limited training instances often constrain the generalization capability of the trained optimizers. Co-evolutionary approaches address this challenge by simultaneously evolving a parallel algorithm portfolio (PAP) and an instance population to eventually obtain PAPs with good generalization. Yet, when applied to a specific problem class, these approaches have a major limitation. They require practitioners to provide instance generators specially tailored to the problem class, which is often non-trivial to design. This work proposes a general-purpose, off-the-shelf PAP construction approach, named domain-agnostic co-evolution of parameterized search (DACE), for binary optimization problems where decision variables take values of 0 or 1. The key novelty of DACE lies in its neural network-based domain-agnostic instance representation and generation mechanism that eliminates the need for domain-specific instance generators. The strong generality of DACE is validated across three real-world binary optimization problems: the complementary influence maximization problem (CIMP), the compiler arguments optimization problem (CAOP), and the contamination control problem (CCP). Given only a small set of training instances from these problem classes, DACE, without requiring domain knowledge, constructs PAPs with even better generalization performance than existing approaches on all three classes, despite their use of domain-specific instance generators.
Resolving the Exploration-Exploitation Dilemma in Evolutionary Algorithms: A Novel Human-Centered Framework
Authors: Ehsan Shams
Date: 2025-01-04 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary Algorithms (EAs) are widely employed tools for complex search and optimization tasks; however, the absence of an overarching operational framework that permits a systematic regulation of the exploration-exploitation tradeoff--critical for efficient convergence--restricts the full actualization of their potential, leading to the so-called exploration-exploitation dilemma in algorithm design. A systematic resolution to this dilemma requires: (1) an independent yet coordinated control over exploration and exploitation, and (2) an explicit, computationally feasible, adaptive regulation mechanism. The current, almost decentralized, traditional parameter tuning-centeric approach--lacks the foundation to satisfy these requirements under encoding-imposed structural constraints. We propose a Human-Centered Two-Phase Search (HCTPS) framework, in which the actualization of (1) and (2) is enabled through an external configuration variable--the Search Space Control Parameter (SSCP). As the sole control knob of HCTPS, the SSCP centralizes exploration adjustments, sparing users from micromanaging traditional parameters with unintelligible interdependencies. To this construct, the human user serves as a meta-parameter, adaptively steering the regulatory process via SSCP adjustments. We prove that the HCTPS strictly surpasses the current approach in terms of search space coverage without disrupting the EAs' inherent convergence mechanisms, demonstrate a concrete instantiation of it--using the Genetic Algorithm as the underlying heuristic on a suite of global benchmark unconstrained optimization problems, provide a through assessment of the proposed framework, and envision future research directions. Any search algorithm prone to this dilemma can be applied in light of the proposed framework, being algorithm-agnostic by design.
Coarsened confounding for causal effects: a large-sample framework
Authors: Debashis Ghosh, Lei Wang
Date: 2025-01-06 00:00:00
Tasks: Causal Inference

There has been widespread use of causal inference methods for the rigorous analysis of observational studies and to identify policy evaluations. In this article, we consider coarsened exact matching, developed in Iacus et al. (2011). While they developed some statistical properties, in this article, we study the approach using asymptotics based on a superpopulation inferential framework. This methodology is generalized to what we termed as coarsened confounding, for which we propose two new algorithms. We develop asymptotic results for the average causal effect estimator as well as providing conditions for consistency. In addition, we provide an asymptotic justification for the variance formulae in Iacus et al. (2011). A bias correction technique is proposed, and we apply the proposed methodology to data from two well-known observational studi
Exact Matching in Correlated Networks with Node Attributes for Improved Community Recovery
Authors: Joonhyuk Yang, Hye Won Chung
Date: 2025-01-06 00:00:00
Tasks: Attribute, Community Detection, Graph Matching, Stochastic Block Model

We study community detection in multiple networks whose nodes and edges are jointly correlated. This setting arises naturally in applications such as social platforms, where a shared set of users may exhibit both correlated friendship patterns and correlated attributes across different platforms. Extending the classical Stochastic Block Model (SBM) and its contextual counterpart (CSBM), we introduce the correlated CSBM, which incorporates structural and attribute correlations across graphs. To build intuition, we first analyze correlated Gaussian Mixture Models, wherein only correlated node attributes are available without edges, and identify the conditions under which an estimator minimizing the distance between attributes achieves exact matching of nodes across the two databases. For correlated CSBMs, we develop a two-step procedure that first applies $k$-core matching to most nodes using edge information, then refines the matching for the remaining unmatched nodes by leveraging their attributes with a distance-based estimator. We identify the conditions under which the algorithm recovers the exact node correspondence, enabling us to merge the correlated edges and average the correlated attributes for enhanced community detection. Crucially, by aligning and combining graphs, we identify regimes in which community detection is impossible in a single graph but becomes feasible when side information from correlated graphs is incorporated. Our results illustrate how the interplay between graph matching and community recovery can boost performance, broadening the scope of multi-graph, attribute-based community detection.
A Stable Measure for Conditional Periodicity of Time Series using Persistent Homology
Authors: Bala Krishnamoorthy, Elizabeth P. Thompson
Date: 2025-01-06 00:00:00
Tasks: Dimensionality Reduction, Dynamic Time Warping, Time Series

Given a pair of time series, we study how the periodicity of one influences the periodicity of the other. There are several known methods to measure the similarity between a pair of time series, such as cross-correlation, coherence, cross-recurrence, and dynamic time warping. But we have yet to find any measures with theoretical stability results. Persistence homology has been utilized to construct a scoring function with theoretical guarantees of stability that quantifies the periodicity of a single univariate time series f1, denoted score(f1). Building on this concept, we propose a conditional periodicity score that quantifies the periodicity of one univariate time series f1 given another f2, denoted score(f1|f2), and derive theoretical stability results for the same. With the use of dimension reduction in mind, we prove a new stability result for score(f1|f2) under principal component analysis (PCA) when we use the projections of the time series embeddings onto their respective first K principal components. We show that the change in our score is bounded by a function of the eigenvalues corresponding to the remaining (unused) N-K principal components and hence is small when the first K principal components capture most of the variation in the time series embeddings. Finally we derive a lower bound on the minimum embedding dimension to use in our pipeline which guarantees that any two such embeddings give scores that are within a given epsilon of each other. We present a procedure for computing conditional periodicity scores and implement it on several pairs of synthetic signals. We experimentally compare our similarity measure to the most-similar statistical measure of cross-recurrence, and show the increased accuracy and stability of our score when predicting and measuring whether or not the periodicities of two time series are similar.
Simultaneous analysis of approximate leave-one-out cross-validation and mean-field inference
Authors: Pierre C Bellec
Date: 2025-01-05 00:00:00

Approximate Leave-One-Out Cross-Validation (ALO-CV) is a method that has been proposed to estimate the generalization error of a regularized estimator in the high-dimensional regime where dimension and sample size are of the same order, the so called ``proportional regime''. A new analysis is developed to derive the consistency of ALO-CV for non-differentiable regularizer under Gaussian covariates and strong-convexity of the regularizer. Using a conditioning argument, the difference between the ALO-CV weights and their counterparts in mean-field inference is shown to be small. Combined with upper bounds between the mean-field inference estimate and the leave-one-out quantity, this provides a proof that ALO-CV approximates the leave-one-out quantity as well up to negligible error terms. Linear models with square loss, robust linear regression and single-index models are explicitly treated.