pwc-archive/papers-with-abstracts huggingface dataset:
Distributionally Robust Active Learning for Gaussian Process Regression
Authors: Shion Takeno, Yoshito Okura, Yu Inatsu, Aoyama Tatsuya, Tomonari Tanaka, Akahane Satoshi, Hiroyuki Hanada, Noriaki Hashimoto, Taro Murayama, Hanju Lee, Shinya Kojima, Ichiro Takeuchi
Date: 2025-02-24 00:00:00
Tasks: Active Learning, GPR, Prediction, regression

Gaussian process regression (GPR) or kernel ridge regression is a widely used and powerful tool for nonlinear prediction. Therefore, active learning (AL) for GPR, which actively collects data labels to achieve an accurate prediction with fewer data labels, is an important problem. However, existing AL methods do not theoretically guarantee prediction accuracy for target distribution. Furthermore, as discussed in the distributionally robust learning literature, specifying the target distribution is often difficult. Thus, this paper proposes two AL methods that effectively reduce the worst-case expected error for GPR, which is the worst-case expectation in target distribution candidates. We show an upper bound of the worst-case expected squared error, which suggests that the error will be arbitrarily small by a finite number of data labels under mild conditions. Finally, we demonstrate the effectiveness of the proposed methods through synthetic and real-world datasets.
Functional BART with Shape Priors: A Bayesian Tree Approach to Constrained Functional Regression
Authors: Jiahao Cao, Shiyuan He, Bohai Zhang
Date: 2025-02-24 00:00:00
Tasks: regression

Motivated by the remarkable success of Bayesian additive regression trees (BART) in regression modelling, we propose a novel nonparametric Bayesian method, termed Functional BART (FBART), tailored specifically for function-on-scalar regression. FBART leverages spline-based representations for functional responses coupled with a flexible tree-based partitioning structure, effectively capturing complex and heterogeneous relationships between response curves and scalar predictors. To facilitate efficient posterior inference, we develop a customized Bayesian backfitting algorithm. Additionally, we extend FBART by introducing shape constraints (e.g., monotonicity or convexity) on the response curves, enabling enhanced estimation and prediction when prior shape information is available. The use of shape priors ensures that posterior samples respect the specified functional constraints. Under mild regularity conditions, we establish posterior convergence rates for both FBART and its shape-constrained variant, demonstrating rate adaptivity to unknown smoothness. Extensive simulation studies and analyses of two real datasets illustrate the superior estimation accuracy and predictive performance of our proposed methods compared to existing state-of-the-art alternatives.
Achieving Fair PCA Using Joint Eigenvalue Decomposition
Authors: Vidhi Rathore, Naresh Manwani
Date: 2025-02-24 00:00:00
Tasks: Dimensionality Reduction, Fairness

Principal Component Analysis (PCA) is a widely used method for dimensionality reduction, but it often overlooks fairness, especially when working with data that includes demographic characteristics. This can lead to biased representations that disproportionately affect certain groups. To address this issue, our approach incorporates Joint Eigenvalue Decomposition (JEVD), a technique that enables the simultaneous diagonalization of multiple matrices, ensuring fair and efficient representations. We formally show that the optimal solution of JEVD leads to a fair PCA solution. By integrating JEVD with PCA, we strike an optimal balance between preserving data structure and promoting fairness across diverse groups. We demonstrate that our method outperforms existing baseline approaches in fairness and representational quality on various datasets. It retains the core advantages of PCA while ensuring that sensitive demographic attributes do not create disparities in the reduced representation.
Deep Minimax Classifiers for Imbalanced Datasets with a Small Number of Minority Samples
Authors: Hansung Choi, Daewon Seo
Date: 2025-02-24 00:00:00

The concept of a minimax classifier is well-established in statistical decision theory, but its implementation via neural networks remains challenging, particularly in scenarios with imbalanced training data having a limited number of samples for minority classes. To address this issue, we propose a novel minimax learning algorithm designed to minimize the risk of worst-performing classes. Our algorithm iterates through two steps: a minimization step that trains the model based on a selected target prior, and a maximization step that updates the target prior towards the adversarial prior for the trained model. In the minimization, we introduce a targeted logit-adjustment loss function that efficiently identifies optimal decision boundaries under the target prior. Moreover, based on a new prior-dependent generalization bound that we obtained, we theoretically prove that our loss function has a better generalization capability than existing loss functions. During the maximization, we refine the target prior by shifting it towards the adversarial prior, depending on the worst-performing classes rather than on per-class risk estimates. Our maximization method is particularly robust in the regime of a small number of samples. Additionally, to adapt to overparameterized neural networks, we partition the entire training dataset into two subsets: one for model training during the minimization step and the other for updating the target prior during the maximization step. Our proposed algorithm has a provable convergence property, and empirical results indicate that our algorithm performs better than or is comparable to existing methods. All codes are publicly available at https://github.com/hansung-choi/TLA-linear-ascent.
Advancing Eurasia Fire Understanding Through Machine Learning Techniques
Authors: Boris Kriuk
Date: 2025-02-24 00:00:00
Tasks: Management, Weather Forecasting

Modern fire management systems increasingly rely on satellite data and weather forecasting; however, access to comprehensive datasets remains limited due to proprietary restrictions. Despite the ecological significance of wildfires, large-scale, multi-regional research is constrained by data scarcity. Russian diverse ecosystems play a crucial role in shaping Eurasian fire dynamics, yet they remain underexplored. This study addresses existing gaps by introducing an open-access dataset that captures detailed fire incidents alongside corresponding meteorological conditions. We present one of the most extensive datasets available for wildfire analysis in Russia, covering 13 consecutive months of observations. Leveraging machine learning techniques, we conduct exploratory data analysis and develop predictive models to identify key fire behavior patterns across different fire categories and ecosystems. Our results highlight the critical influence of environmental factor patterns on fire occurrence and spread behavior. By improving the understanding of wildfire dynamics in Eurasia, this work contributes to more effective, data-driven approaches for proactive fire management in the face of evolving environmental conditions.
Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
Authors: Zixuan Gong, Xiaolin Hu, Huayi Tang, Yong liu
Date: 2025-02-24 00:00:00
Tasks: Generalization Bounds, In-Context Learning

Large language models (LLMs) have demonstrated remarkable in-context learning (ICL) abilities. However, existing theoretical analysis of ICL primarily exhibits two limitations: (a) Limited i.i.d. Setting. Most studies focus on supervised function learning tasks where prompts are constructed with i.i.d. input-label pairs. This i.i.d. assumption diverges significantly from real language learning scenarios where prompt tokens are interdependent. (b) Lack of Emergence Explanation. Most literature answers what ICL does from an implicit optimization perspective but falls short in elucidating how ICL emerges and the impact of pre-training phase on ICL. In our paper, to extend (a), we adopt a more practical paradigm, auto-regressive next-token prediction (AR-NTP), which closely aligns with the actual training of language models. Specifically, within AR-NTP, we emphasize prompt token-dependency, which involves predicting each subsequent token based on the preceding sequence. To address (b), we formalize a systematic pre-training and ICL framework, highlighting the layer-wise structure of sequences and topics, alongside a two-level expectation. In conclusion, we present data-dependent, topic-dependent and optimization-dependent PAC-Bayesian generalization bounds for pre-trained LLMs, investigating that ICL emerges from the generalization of sequences and topics. Our theory is supported by experiments on numerical linear dynamic systems, synthetic GINC and real-world language datasets.
Data Analysis Prediction over Multiple Unseen Datasets: A Vector Embedding Approach
Authors: Andreas Loizou, Dimitrios Tsoumakos
Date: 2025-02-24 00:00:00

The massive increase in the data volume and dataset availability for analysts compels researchers to focus on data content and select high-quality datasets to enhance the performance of analytics operators. While selecting the highest quality data for analysis highly increases task accuracy and efficiency, it is still a hard task, especially when the number of available inputs is very large. To address this issue, we propose a novel methodology that infers the outcome of analytics operators by creating a model from datasets similar to the queried one. Dataset similarity is performed via projecting each dataset to a vector embedding representation. The vectorization process is performed using our proposed deep learning model NumTabData2Vec, which takes a whole dataset and projects it into a lower vector embedding representation space. Through experimental evaluation, we compare the prediction performance and the execution time of our framework to another state-of-the-art modelling operator framework, illustrating that our approach predicts analytics outcomes accurately. Furthermore, our vectorization model can project different real-world scenarios to a lower vector embedding representation and distinguish between them.
A comparative analysis of rank aggregation methods for the partial label ranking problem
Authors: Jiayi Wang, Juan C. Alfaro, Viktor Bengs
Date: 2025-02-24 00:00:00

The \textit{label ranking} problem is a supervised learning scenario in which the learner predicts a \textit{total order} of the class labels for a given input instance. Recently, research has increasingly focused on the \textit{partial label ranking} problem, a generalization of the label ranking problem that allows \textit{ties} in the predicted orders. So far, most existing learning approaches for the partial label ranking problem rely on approximation algorithms for rank aggregation in the final prediction step. This paper explores several alternative aggregation methods for this critical step, including scoring-based and probabilistic-based rank aggregation approaches. To enhance their suitability for the more general partial label ranking problem, the investigated methods are extended to increase the likelihood of producing ties. Experimental evaluations on standard benchmarks demonstrate that scoring-based variants consistently outperform the current state-of-the-art method in handling incomplete information. In contrast, probabilistic-based variants fail to achieve competitive performance.
The feasibility of multi-graph alignment: a Bayesian approach
Authors: Louis Vassaux, Laurent Massoulié
Date: 2025-02-24 00:00:00

We establish thresholds for the feasibility of random multi-graph alignment in two models. In the Gaussian model, we demonstrate an "all-or-nothing" phenomenon: above a critical threshold, exact alignment is achievable with high probability, while below it, even partial alignment is statistically impossible. In the sparse Erd\H{o}s-R\'enyi model, we rigorously identify a threshold below which no meaningful partial alignment is possible and conjecture that above this threshold, partial alignment can be achieved. To prove these results, we develop a general Bayesian estimation framework over metric spaces, which provides insight into a broader class of high-dimensional statistical problems.
CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thought
Authors: Boxuan Zhang, Ruqi Zhang
Date: 2025-02-24 00:00:00
Tasks: Mathematical Reasoning, Misinformation, Uncertainty Quantification

Large language models (LLMs) excel in many tasks but struggle to accurately quantify uncertainty in their generated responses. This limitation makes it challenging to detect misinformation and ensure reliable decision-making. Existing uncertainty quantification (UQ) methods for LLMs are primarily prompt-wise rather than response-wise, often requiring multiple response samples, which incurs high computational costs. Moreover, LLMs have been shown to be overconfident, particularly when using reasoning steps to derive their answers. In this work, we propose CoT-UQ, a response-wise UQ framework that integrates LLMs' inherent reasoning capabilities through Chain-of-Thought (CoT) into the UQ process. CoT-UQ captures critical information during inference by extracting keywords from each reasoning step and assessing their importance to the final answer. This key reasoning information is then aggregated to produce a final uncertainty estimate. We conduct extensive experiments based on LLaMA Family with model sizes varying from 8B to 13B across logical and mathematical reasoning tasks. Experimental results demonstrate that CoT-UQ significantly outperforms existing UQ methods, achieving an average improvement of 5.9% AUROC compared to current UQ methods. The code is available at: https://github.com/ZBox1005/CoT-UQ.
Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage
Authors: Konstantina Bairaktari, Jiayun Wu, Zhiwei Steven Wu
Date: 2025-02-24 00:00:00
Tasks: Conformal Prediction, Prediction

Conformal prediction is a powerful distribution-free framework for constructing prediction sets with coverage guarantees. Classical methods, such as split conformal prediction, provide marginal coverage, ensuring that the prediction set contains the label of a random test point with a target probability. However, these guarantees may not hold uniformly across different subpopulations, leading to disparities in coverage. Prior work has explored coverage guarantees conditioned on events related to the covariates and label of the test point. We present Kandinsky conformal prediction, a framework that significantly expands the scope of conditional coverage guarantees. In contrast to Mondrian conformal prediction, which restricts its coverage guarantees to disjoint groups -- reminiscent of the rigid, structured grids of Piet Mondrian's art -- our framework flexibly handles overlapping and fractional group memberships defined jointly on covariates and labels, reflecting the layered, intersecting forms in Wassily Kandinsky's compositions. Our algorithm unifies and extends existing methods, encompassing covariate-based group conditional, class conditional, and Mondrian conformal prediction as special cases, while achieving a minimax-optimal high-probability conditional coverage bound. Finally, we demonstrate the practicality of our approach through empirical evaluation on real-world datasets.
Joint Value Estimation and Bidding in Repeated First-Price Auctions
Authors: Yuxiao Wen, Yanjun Han, Zhengyuan Zhou
Date: 2025-02-24 00:00:00
Tasks: Causal Inference

We study regret minimization in repeated first-price auctions (FPAs), where a bidder observes only the realized outcome after each auction -- win or loss. This setup reflects practical scenarios in online display advertising where the actual value of an impression depends on the difference between two potential outcomes, such as clicks or conversion rates, when the auction is won versus lost. We analyze three outcome models: (1) adversarial outcomes without features, (2) linear potential outcomes with features, and (3) linear treatment effects in features. For each setting, we propose algorithms that jointly estimate private values and optimize bidding strategies, achieving near-optimal regret bounds. Notably, our framework enjoys a unique feature that the treatments are also actively chosen, and hence eliminates the need for the overlap condition commonly required in causal inference.
Stronger Neyman Regret Guarantees for Adaptive Experimental Design
Authors: Georgy Noarov, Riccardo Fogliato, Martin Bertran, Aaron Roth
Date: 2025-02-24 00:00:00
Tasks: Experimental Design

We study the design of adaptive, sequential experiments for unbiased average treatment effect (ATE) estimation in the design-based potential outcomes setting. Our goal is to develop adaptive designs offering sublinear Neyman regret, meaning their efficiency must approach that of the hindsight-optimal nonadaptive design. Recent work [Dai et al, 2023] introduced ClipOGD, the first method achieving $\widetilde{O}(\sqrt{T})$ expected Neyman regret under mild conditions. In this work, we propose adaptive designs with substantially stronger Neyman regret guarantees. In particular, we modify ClipOGD to obtain anytime $\widetilde{O}(\log T)$ Neyman regret under natural boundedness assumptions. Further, in the setting where experimental units have pre-treatment covariates, we introduce and study a class of contextual "multigroup" Neyman regret guarantees: Given any set of possibly overlapping groups based on the covariates, the adaptive design outperforms each group's best non-adaptive designs. In particular, we develop a contextual adaptive design with $\widetilde{O}(\sqrt{T})$ anytime multigroup Neyman regret. We empirically validate the proposed designs through an array of experiments.
A Weight Adaptation Trigger Mechanism in Decomposition-based Evolutionary Multi-Objective Optimisation
Authors: Xiaofeng Han, Xiaochen Chu, Tao Chao, Ming Yang, Miqing Li
Date: 2025-02-23 00:00:00
Tasks: Evolutionary Algorithms

Decomposition-based multi-objective evolutionary algorithms (MOEAs) are widely used for solving multi-objective optimisation problems. However, their effectiveness depends on the consistency between the problems Pareto front shape and the weight distribution. Decomposition-based MOEAs, with uniformly distributed weights (in a simplex), perform well on problems with a regular (simplex-like) Pareto front, but not on those with an irregular Pareto front. Previous studies have focused on adapting the weights to approximate the irregular Pareto front during the evolutionary process. However, these adaptations can actually harm the performance on the regular Pareto front via changing the weights during the search process that are eventually the best fit for the Pareto front. In this paper, we propose an algorithm called the weight adaptation trigger mechanism for decomposition-based MOEAs (ATM-MOEA/D) to tackle this issue. ATM-MOEA/D uses an archive to gradually approximate the shape of the Pareto front during the search. When the algorithm detects evolution stagnation (meaning the population no longer improves significantly), it compares the distribution of the population with that of the archive to distinguish between regular and irregular Pareto fronts. Only when an irregular Pareto front is identified, the weights are adapted. Our experimental results show that the proposed algorithm not only performs generally better than seven state-of-the-art weight-adapting methods on irregular Pareto fronts but also is able to achieve the same results as fixed-weight methods like MOEA/D on regular Pareto fronts.
Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Networks
Authors: Amanda Bertschinger, James Bagrow, Joshua Bongard
Date: 2025-02-24 00:00:00
Tasks: Evolutionary Algorithms, Form, regression, Symbolic Regression

Data increasingly abounds, but distilling their underlying relationships down to something interpretable remains challenging. One approach is genetic programming, which `symbolically regresses' a data set down into an equation. However, symbolic regression (SR) faces the issue of requiring training from scratch for each new dataset. To generalize across all datasets, deep learning techniques have been applied to SR. These networks, however, are only able to be trained using a symbolic objective: NN-generated and target equations are symbolically compared. But this does not consider the predictive power of these equations, which could be measured by a behavioral objective that compares the generated equation's predictions to actual data. Here we introduce a method that combines gradient descent and evolutionary computation to yield neural networks that minimize the symbolic and behavioral errors of the equations they generate from data. As a result, these evolved networks are shown to generate more symbolically and behaviorally accurate equations than those generated by networks trained by state-of-the-art gradient based neural symbolic regression methods. We hope this method suggests that evolutionary algorithms, combined with gradient descent, can improve SR results by yielding equations with more accurate form and function.
A Brain-Inspired Perception-Decision Driving Model Based on Neural Pathway Anatomical Alignment
Authors: Haidong Wang, Pengfei Xiao, Ao Liu, Qia Shan, Jianhua Zhang
Date: 2025-02-22 00:00:00
Tasks: Autonomous Driving, Decision Making

In the realm of autonomous driving, conventional approaches for vehicle perception and decision-making primarily rely on sensor input and rule-based algorithms. However, these methodologies often suffer from lack of interpretability and robustness, particularly in intricate traffic scenarios. To tackle this challenge, we propose a novel brain-inspired driving (BID) framework. Diverging from traditional methods, our approach harnesses brain-inspired perception technology to achieve more efficient and robust environmental perception. Additionally, it employs brain-inspired decision-making techniques to facilitate intelligent decision-making. The experimental results show that the performance has been significantly improved across various autonomous driving tasks and achieved the end-to-end autopilot successfully. This contribution not only advances interpretability and robustness but also offers fancy insights and methodologies for further advancing autonomous driving technology.