pwc-archive/papers-with-abstracts huggingface dataset:
StablePCA: Learning Shared Representations across Multiple Sources via Minimax Optimization
Authors: Zhenyu Wang, Molei Liu, Jing Lei, Francis Bach, Zijian Guo
Date: 2025-05-02 00:00:00
Tasks: Fairness

When synthesizing multisource high-dimensional data, a key objective is to extract low-dimensional feature representations that effectively approximate the original features across different sources. Such general feature extraction facilitates the discovery of transferable knowledge, mitigates systematic biases such as batch effects, and promotes fairness. In this paper, we propose Stable Principal Component Analysis (StablePCA), a novel method for group distributionally robust learning of latent representations from high-dimensional multi-source data. A primary challenge in generalizing PCA to the multi-source regime lies in the nonconvexity of the fixed rank constraint, rendering the minimax optimization nonconvex. To address this challenge, we employ the Fantope relaxation, reformulating the problem as a convex minimax optimization, with the objective defined as the maximum loss across sources. To solve the relaxed formulation, we devise an optimistic-gradient Mirror Prox algorithm with explicit closed-form updates. Theoretically, we establish the global convergence of the Mirror Prox algorithm, with the convergence rate provided from the optimization perspective. Furthermore, we offer practical criteria to assess how closely the solution approximates the original nonconvex formulation. Through extensive numerical experiments, we demonstrate StablePCA's high accuracy and efficiency in extracting robust low-dimensional representations across various finite-sample scenarios.
Gaussian Process Policy Iteration with Additive Schwarz Acceleration for Forward and Inverse HJB and Mean Field Game Problems
Authors: Xianjin Yang, Jingguo Zhang
Date: 2025-05-01 00:00:00
Tasks: Computational Efficiency

We propose a Gaussian Process (GP)-based policy iteration framework for addressing both forward and inverse problems in Hamilton--Jacobi--Bellman (HJB) equations and mean field games (MFGs). Policy iteration is formulated as an alternating procedure between solving the value function under a fixed control policy and updating the policy based on the resulting value function. By exploiting the linear structure of GPs for function approximation, each policy evaluation step admits an explicit closed-form solution, eliminating the need for numerical optimization. To improve convergence, we incorporate the additive Schwarz acceleration as a preconditioning step following each policy update. Numerical experiments demonstrate the effectiveness of Schwarz acceleration in improving computational efficiency.
Stochastic Subspace Descent Accelerated via Bi-fidelity Line Search
Authors: Nuojin Cheng, Alireza Doostan, Stephen Becker
Date: 2025-04-30 00:00:00

Efficient optimization remains a fundamental challenge across numerous scientific and engineering domains, especially when objective function and gradient evaluations are computationally expensive. While zeroth-order optimization methods offer effective approaches when gradients are inaccessible, their practical performance can be limited by the high cost associated with function queries. This work introduces the bi-fidelity stochastic subspace descent (BF-SSD) algorithm, a novel zeroth-order optimization method designed to reduce this computational burden. BF-SSD leverages a bi-fidelity framework, constructing a surrogate model from a combination of computationally inexpensive low-fidelity (LF) and accurate high-fidelity (HF) function evaluations. This surrogate model facilitates an efficient backtracking line search for step size selection, for which we provide theoretical convergence guarantees under standard assumptions. We perform a comprehensive empirical evaluation of BF-SSD across four distinct problems: a synthetic optimization benchmark, dual-form kernel ridge regression, black-box adversarial attacks on machine learning models, and transformer-based black-box language model fine-tuning. Numerical results demonstrate that BF-SSD consistently achieves superior optimization performance while requiring significantly fewer HF function evaluations compared to relevant baseline methods. This study highlights the efficacy of integrating bi-fidelity strategies within zeroth-order optimization, positioning BF-SSD as a promising and computationally efficient approach for tackling large-scale, high-dimensional problems encountered in various real-world applications.
Realized Local Volatility Surface
Authors: Yuming Ma, Shintaro Sengoku, Kazuhide Nakata
Date: 2025-04-22 00:00:00
Tasks: counterfactual, Management

For quantitative trading risk management purposes, we present a novel idea: the realized local volatility surface. Concisely, it stands for the conditional expected volatility when sudden market behaviors of the underlying occur. One is able to explore risk management usages by following the orthotical Delta-Gamma dynamic hedging framework. The realized local volatility surface is, mathematically, a generalized Wiener measure from historical prices. It is reconstructed via employing high-frequency trading market data. A Stick-Breaking Gaussian Mixture Model is fitted via Hamiltonian Monte Carlo, producing a local volatility surface with 95% credible intervals. A practically validated Bayesian nonparametric estimation workflow. Empirical results on TSLA high-frequency data illustrate its ability to capture counterfactual volatility. We also discuss its application in improving volatility-based risk management.
Learning to Borrow Features for Improved Detection of Small Objects in Single-Shot Detectors
Authors: Richard Schmit
Date: 2025-04-30 00:00:00
Tasks: Descriptive, Object, object-detection, Object Detection, Robust Object Detection, Small Object Detection

Detecting small objects remains a significant challenge in single-shot object detectors due to the inherent trade-off between spatial resolution and semantic richness in convolutional feature maps. To address this issue, we propose a novel framework that enables small object representations to "borrow" discriminative features from larger, semantically richer instances within the same class. Our architecture introduces three key components: the Feature Matching Block (FMB) to identify semantically similar descriptors across layers, the Feature Representing Block (FRB) to generate enhanced shallow features through weighted aggregation, and the Feature Fusion Block (FFB) to refine feature maps by integrating original, borrowed, and context information. Built upon the SSD framework, our method improves the descriptive capacity of shallow layers while maintaining real-time detection performance. Experimental results demonstrate that our approach significantly boosts small object detection accuracy over baseline methods, offering a promising direction for robust object detection in complex visual environments.
Can Nash inform capital requirements? Allocating systemic risk measures
Authors: Çağın Ararat, Zachary Feinstein
Date: 2025-04-29 00:00:00

Systemic risk measures aggregate the risks from multiple financial institutions to find system-wide capital requirements. Though much attention has been given to assessing the level of systemic risk, less has been given to allocating that risk to the constituent institutions. Within this work, we propose a Nash allocation rule that is inspired by game theory. Intuitively, to construct these capital allocations, the banks compete in a game to reduce their own capital requirements while, simultaneously, maintaining system-level acceptability. We provide sufficient conditions for the existence and uniqueness of Nash allocation rules, and apply our results to the prominent structures used for systemic risk measures in the literature. We demonstrate the efficacy of Nash allocations with numerical case studies using the Eisenberg-Noe aggregation mechanism.
Smooth Integer Encoding via Integral Balance
Authors: Stanislav Semenov
Date: 2025-04-28 00:00:00

We introduce a novel method for encoding integers using smooth real-valued functions whose integral properties implicitly reflect discrete quantities. In contrast to classical representations, where the integer appears as an explicit parameter, our approach encodes the number N in the set of natural numbers through the cumulative balance of a smooth function f_N(t), constructed from localized Gaussian bumps with alternating and decaying coefficients. The total integral I(N) converges to zero as N tends to infinity, and the integer can be recovered as the minimal point of near-cancellation. This method enables continuous and differentiable representations of discrete states, supports recovery through spline-based or analytical inversion, and extends naturally to multidimensional tuples (N1, N2, ...). We analyze the structure and convergence of the encoding series, demonstrate numerical construction of the integral map I(N), and develop procedures for integer recovery via numerical inversion. The resulting framework opens a path toward embedding discrete logic within continuous optimization pipelines, machine learning architectures, and smooth symbolic computation.
Automated decision-making for dynamic task assignment at scale
Authors: Riccardo Lo Bianco, Willem van Jaarsveld, Jeroen Middelhuis, Luca Begnardi, Remco Dijkman
Date: 2025-04-28 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning

The Dynamic Task Assignment Problem (DTAP) concerns matching resources to tasks in real time while minimizing some objectives, like resource costs or task cycle time. In this work, we consider a DTAP variant where every task is a case composed of a stochastic sequence of activities. The DTAP, in this case, involves the decision of which employee to assign to which activity to process requests as quickly as possible. In recent years, Deep Reinforcement Learning (DRL) has emerged as a promising tool for tackling this DTAP variant, but most research is limited to solving small-scale, synthetic problems, neglecting the challenges posed by real-world use cases. To bridge this gap, this work proposes a DRL-based Decision Support System (DSS) for real-world scale DTAPS. To this end, we introduce a DRL agent with two novel elements: a graph structure for observations and actions that can effectively represent any DTAP and a reward function that is provably equivalent to the objective of minimizing the average cycle time of tasks. The combination of these two novelties allows the agent to learn effective and generalizable assignment policies for real-world scale DTAPs. The proposed DSS is evaluated on five DTAP instances whose parameters are extracted from real-world logs through process mining. The experimental evaluation shows how the proposed DRL agent matches or outperforms the best baseline in all DTAP instances and generalizes on different time horizons and across instances.
Sharp higher order convergence rates for the Adam optimizer
Authors: Steffen Dereich, Arnulf Jentzen, Adrian Riekert
Date: 2025-04-28 00:00:00

Gradient descent based optimization methods are the methods of choice to train deep neural networks in machine learning. Beyond the standard gradient descent method, also suitable modified variants of standard gradient descent involving acceleration techniques such as the momentum method and/or adaptivity techniques such as the RMSprop method are frequently considered optimization methods. These days the most popular of such sophisticated optimization schemes is presumably the Adam optimizer that has been proposed in 2014 by Kingma and Ba. A highly relevant topic of research is to investigate the speed of convergence of such optimization methods. In particular, in 1964 Polyak showed that the standard gradient descent method converges in a neighborhood of a strict local minimizer with rate (x - 1)(x + 1)^{-1} while momentum achieves the (optimal) strictly faster convergence rate (\sqrt{x} - 1)(\sqrt{x} + 1)^{-1} where x \in (1,\infty) is the condition number (the ratio of the largest and the smallest eigenvalue) of the Hessian of the objective function at the local minimizer. It is the key contribution of this work to reveal that Adam also converges with the strictly faster convergence rate (\sqrt{x} - 1)(\sqrt{x} + 1)^{-1} while RMSprop only converges with the convergence rate (x - 1)(x + 1)^{-1}.
Convergence Properties of Natural Gradient Descent for Minimizing KL Divergence
Authors: Adwait Datar, Nihat Ay
Date: 2025-04-27 00:00:00

The Kullback-Leibler (KL) divergence plays a central role in probabilistic machine learning, where it commonly serves as the canonical loss function. Optimization in such settings is often performed over the probability simplex, where the choice of parameterization significantly impacts convergence. In this work, we study the problem of minimizing the KL divergence and analyze the behavior of gradient-based optimization algorithms under two dual coordinate systems within the framework of information geometry$-$ the exponential family ($\theta$ coordinates) and the mixture family ($\eta$ coordinates). We compare Euclidean gradient descent (GD) in these coordinates with the coordinate-invariant natural gradient descent (NGD), where the natural gradient is a Riemannian gradient that incorporates the intrinsic geometry of the parameter space. In continuous time, we prove that the convergence rates of GD in the $\theta$ and $\eta$ coordinates provide lower and upper bounds, respectively, on the convergence rate of NGD. Moreover, under affine reparameterizations of the dual coordinates, the convergence rates of GD in $\eta$ and $\theta$ coordinates can be scaled to $2c$ and $\frac{2}{c}$, respectively, for any $c>0$, while NGD maintains a fixed convergence rate of $2$, remaining invariant to such transformations and sandwiched between them. Although this suggests that NGD may not exhibit uniformly superior convergence in continuous time, we demonstrate that its advantages become pronounced in discrete time, where it achieves faster convergence and greater robustness to noise, outperforming GD. Our analysis hinges on bounding the spectrum and condition number of the Hessian of the KL divergence at the optimum, which coincides with the Fisher information matrix.
Optimal dividends for a NatCat insurer in the presence of a climate tipping point
Authors: Hansjoerg Albrecher, Pablo Azcue, Nora Muler
Date: 2025-04-27 00:00:00

We study optimal dividend strategies for an insurance company facing natural catastrophe claims, anticipating the arrival of a climate tipping point after which the claim intensity and/or the claim size distribution of the underlying risks deteriorates irreversibly. Extending earlier literature based on a shot-noise Cox process assumption for claim arrivals, we show that the non-stationary feature of such a tipping point can, in fact, be an advantage for shareholders seeking to maximize expected discounted dividends over the lifetime of the portfolio. Assuming the tipping point arrives according to an Erlang distribution, we demonstrate that the corresponding system of two-dimensional stochastic control problems admits a viscosity solution, which can be numerically approximated using a discretization of the current surplus and the claim intensity level. We also prove uniform convergence of this discrete solution to that of the original continuous problem. The results are illustrated through several numerical examples, and the sensitivity of the optimal dividend strategies to the presence of a climate tipping point is analyzed. In all these examples, it turns out that when the insurance premium is adjusted fairly at the moment of the tipping point, and all quantities are observable, the non-stationarity introduced by the tipping point can, in fact, represent an upward potential for shareholders.
Smooth Approximations of the Rounding Function
Authors: Stanislav Semenov
Date: 2025-04-26 00:00:00

We propose novel smooth approximations to the classical rounding function, suitable for differentiable optimization and machine learning applications. Our constructions are based on two approaches: (1) localized sigmoid window functions centered at each integer, and (2) normalized weighted sums of sigmoid derivatives representing local densities. The first method approximates the step-like behavior of rounding through differences of shifted sigmoids, while the second method achieves smooth interpolation between integers via density-based weighting. Both methods converge pointwise to the classical rounding function as the sharpness parameter k tends to infinity, and allow controlled trade-offs between smoothness and approximation accuracy. We demonstrate that by restricting the summation to a small set of nearest integers, the computational cost remains low without sacrificing precision. These constructions provide fully differentiable alternatives to hard rounding, which are valuable in contexts where gradient-based methods are essential.
Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis
Authors: Damien Martins Gomes
Date: 2025-04-26 00:00:00
Tasks: Computational Efficiency, image-classification, Image Classification, Language Modeling, Language Modelling, Second-order methods

First-order optimization methods remain the standard for training deep neural networks (DNNs). Optimizers like Adam incorporate limited curvature information by preconditioning the stochastic gradient with a diagonal matrix. Despite the widespread adoption of first-order methods, second-order optimization algorithms often exhibit superior convergence compared to methods like Adam and SGD. However, their practicality in training DNNs is still limited by a significantly higher per-iteration computational cost compared to first-order methods. In this thesis, we present AdaFisher, a novel adaptive second-order optimizer that leverages a diagonal block-Kronecker approximation of the Fisher information matrix to adaptively precondition gradients. AdaFisher aims to bridge the gap between the improved convergence and generalization of second-order methods and the computational efficiency needed for training DNNs. Despite the traditionally slower speed of second-order optimizers, AdaFisher is effective for tasks such as image classification and language modeling, exhibiting remarkable stability and robustness during hyperparameter tuning. We demonstrate that AdaFisher outperforms state-of-the-art optimizers in both accuracy and convergence speed. The code is available from https://github.com/AtlasAnalyticsLab/AdaFisher.
Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime
Authors: Raphaël Barboni, Gabriel Peyré, François-Xavier Vialard
Date: 2025-04-25 00:00:00

We study the convergence of gradient methods for the training of mean-field single hidden layer neural networks with square loss. Observing this is a separable non-linear least-square problem which is linear w.r.t. the outer layer's weights, we consider a Variable Projection (VarPro) or two-timescale learning algorithm, thereby eliminating the linear variables and reducing the learning problem to the training of the feature distribution. Whereas most convergence rates or the training of neural networks rely on a neural tangent kernel analysis where features are fixed, we show such a strategy enables provable convergence rates for the sampling of a teacher feature distribution. Precisely, in the limit where the regularization strength vanishes, we show that the dynamic of the feature distribution corresponds to a weighted ultra-fast diffusion equation. Relying on recent results on the asymptotic behavior of such PDEs, we obtain guarantees for the convergence of the trained feature distribution towards the teacher feature distribution in a teacher-student setup.
A multilevel approach to accelerate the training of Transformers
Authors: Guillaume Lauga, Maël Chaumette, Edgar Desainte-Maréville, Étienne Lasalle, Arthur Lebeurrier
Date: 2025-04-24 00:00:00

In this article, we investigate the potential of multilevel approaches to accelerate the training of transformer architectures. Using an ordinary differential equation (ODE) interpretation of these architectures, we propose an appropriate way of varying the discretization of these ODE Transformers in order to accelerate the training. We validate our approach experimentally by a comparison with the standard training procedure.
The Safety-Privacy Tradeoff in Linear Bandits
Authors: Arghavan Zibaie, Spencer Hutchinson, Ramtin Pedarsani, Mahnoosh Alizadeh
Date: 2025-04-23 00:00:00

We consider a collection of linear stochastic bandit problems, each modeling the random response of different agents to proposed interventions, coupled together by a global safety constraint. We assume a central coordinator must choose actions to play on each bandit with the objective of regret minimization, while also ensuring that the expected response of all agents satisfies the global safety constraints at each round, in spite of uncertainty about the bandits' parameters. The agents consider their observed responses to be private and in order to protect their sensitive information, the data sharing with the central coordinator is performed under local differential privacy (LDP). However, providing higher level of privacy to different agents would have consequences in terms of safety and regret. We formalize these tradeoffs by building on the notion of the sharpness of the safety set - a measure of how the geometric properties of the safe set affects the growth of regret - and propose a unilaterally unimprovable vector of privacy levels for different agents given a maximum regret budget.
Seeing Through Risk: A Symbolic Approximation of Prospect Theory
Authors: Ali Arslan Yousaf, Umair Rehman, Muhammad Umair Danish
Date: 2025-04-20 00:00:00
Tasks: Decision Making

We propose a novel symbolic modeling framework for decision-making under risk that merges interpretability with the core insights of Prospect Theory. Our approach replaces opaque utility curves and probability weighting functions with transparent, effect-size-guided features. We mathematically formalize the method, demonstrate its ability to replicate well-known framing and loss-aversion phenomena, and provide an end-to-end empirical validation on synthetic datasets. The resulting model achieves competitive predictive performance while yielding clear coefficients mapped onto psychological constructs, making it suitable for applications ranging from AI safety to economic policy analysis.
Meyer-Zheng topology and multi-asset behavioral portfolio selection under transaction costs
Authors: Artur Sidorenko
Date: 2025-05-03 00:00:00

The paper addresses a singular control problem, where controls are processes of finite variation, with a focus on portfolio investment under transaction costs. We establish the existence of an optimal strategy in the class of randomized strategies for a range of goal functionals, including cumulative prospect theory preferences, subject to the nonnegativity constraint on the portfolio wealth. Unlike the existing approaches, we employ a metrizable Meyer-Zheng topology. Additionally, we investigate the applicability of the Skorokhod convergence for processes, adapted to a specific filtration. The presented framework provides a clear distinction between constraints imposed on the market model and on the goal functional.