pwc-archive/papers-with-abstracts huggingface dataset:
SOFARI-R: High-Dimensional Manifold-Based Inference for Latent Responses
Authors: Zemin Zheng, Xin Zhou, Jinchi Lv
Date: 2025-04-24 00:00:00
Tasks: Multi-Task Learning, Uncertainty Quantification

Data reduction with uncertainty quantification plays a key role in various multi-task learning applications, where large numbers of responses and features are present. To this end, a general framework of high-dimensional manifold-based SOFAR inference (SOFARI) was introduced recently in Zheng, Zhou, Fan and Lv (2024) for interpretable multi-task learning inference focusing on the left factor vectors and singular values exploiting the latent singular value decomposition (SVD) structure. Yet, designing a valid inference procedure on the latent right factor vectors is not straightforward from that of the left ones and can be even more challenging due to asymmetry of left and right singular vectors in the response matrix. To tackle these issues, in this paper we suggest a new method of high-dimensional manifold-based SOFAR inference for latent responses (SOFARI-R), where two variants of SOFARI-R are introduced. The first variant deals with strongly orthogonal factors by coupling left singular vectors with the design matrix and then appropriately rescaling them to generate new Stiefel manifolds. The second variant handles the more general weakly orthogonal factors by employing the hard-thresholded SOFARI estimates and delicately incorporating approximation errors into the distribution. Both variants produce bias-corrected estimators for the latent right factor vectors that enjoy asymptotically normal distributions with justified asymptotic variance estimates. We demonstrate the effectiveness of the newly suggested method using extensive simulation studies and an economic application.
Learning Low-Dimensional Embeddings for Black-Box Optimization
Authors: Riccardo Busetto, Manas Mejari, Marco Forgione, Alberto Bemporad, Dario Piga
Date: 2025-05-02 00:00:00
Tasks: Meta-Learning

When gradient-based methods are impractical, black-box optimization (BBO) provides a valuable alternative. However, BBO often struggles with high-dimensional problems and limited trial budgets. In this work, we propose a novel approach based on meta-learning to pre-compute a reduced-dimensional manifold where optimal points lie for a specific class of optimization problems. When optimizing a new problem instance sampled from the class, black-box optimization is carried out in the reduced-dimensional space, effectively reducing the effort required for finding near-optimal solutions.
Learning Enhanced Ensemble Filters
Authors: Eviatar Bach, Ricardo Baptista, Edoardo Calvello, Bohan Chen, Andrew Stuart
Date: 2025-04-24 00:00:00

The filtering distribution in hidden Markov models evolves according to the law of a mean-field model in state-observation space. The ensemble Kalman filter (EnKF) approximates this mean-field model with an ensemble of interacting particles, employing a Gaussian ansatz for the joint distribution of the state and observation at each observation time. These methods are robust, but the Gaussian ansatz limits accuracy. This shortcoming is addressed by approximating the mean-field evolution using a novel form of neural operator taking probability distributions as input: a measure neural mapping (MNM). A MNM is used to design a novel approach to filtering, the MNM-enhanced ensemble filter (MNMEF), which is defined in both the mean-field limit and for interacting ensemble particle approximations. The ensemble approach uses empirical measures as input to the MNM and is implemented using the set transformer, which is invariant to ensemble permutation and allows for different ensemble sizes. The derivation of methods from a mean-field formulation allows a single parameterization of the algorithm to be deployed at different ensemble sizes. In practice fine-tuning of a small number of parameters, for specific ensemble sizes, further enhances the accuracy of the scheme. The promise of the approach is demonstrated by its superior root mean-square-error performance relative to leading methods in filtering the Lorenz 96 and Kuramoto-Sivashinsky models.
Recursive Algorithms for Sparse Parameter Identification of Multivariate Stochastic Systems with Non-stationary Observations
Authors: Yanxin Fu, Wenxiao Zhao
Date: 2025-05-01 00:00:00

The classical sparse parameter identification methods are usually based on the iterative basis selection such as greedy algorithms, or the numerical optimization of regularized cost functions such as LASSO and Bayesian posterior probability distribution, etc., which, however, are not suitable for online sparsity inference when data arrive sequentially. This paper presents recursive algorithms for sparse parameter identification of multivariate stochastic systems with non-stationary observations. First, a new bivariate criterion function is presented by introducing an auxiliary variable matrix into a weighted $L_1$ regularization criterion. The new criterion function is subsequently decomposed into two solvable subproblems via alternating optimization of the two variable matrices, for which the optimizers can be explicitly formulated into recursive equations. Second, under the non-stationary and non-persistent excitation conditions on the systems, theoretical properties of the recursive algorithms are established. That is, the estimates are proved to be with (i) set convergence, i.e., the accurate estimation of the sparse index set of the unknown parameter matrix, and (ii) parameter convergence, i.e., the consistent estimation for values of the non-zero elements of the unknown parameter matrix. Finally, numerical examples are given to support the theoretical analysis.
Evaluating Uncertainty in Deep Gaussian Processes
Authors: Matthijs van der Lende, Jeremias Lino Ferrao, Niclas Müller-Hof
Date: 2025-04-24 00:00:00
Tasks: Gaussian Processes, Point Processes, Uncertainty Quantification

Reliable uncertainty estimates are crucial in modern machine learning. Deep Gaussian Processes (DGPs) and Deep Sigma Point Processes (DSPPs) extend GPs hierarchically, offering promising methods for uncertainty quantification grounded in Bayesian principles. However, their empirical calibration and robustness under distribution shift relative to baselines like Deep Ensembles remain understudied. This work evaluates these models on regression (CASP dataset) and classification (ESR dataset) tasks, assessing predictive performance (MAE, Accu- racy), calibration using Negative Log-Likelihood (NLL) and Expected Calibration Error (ECE), alongside robustness under various synthetic feature-level distribution shifts. Results indicate DSPPs provide strong in-distribution calibration leveraging their sigma point approximations. However, compared to Deep Ensembles, which demonstrated superior robustness in both per- formance and calibration under the tested shifts, the GP-based methods showed vulnerabilities, exhibiting particular sensitivity in the observed metrics. Our findings underscore ensembles as a robust baseline, suggesting that while deep GP methods offer good in-distribution calibration, their practical robustness under distribution shift requires careful evaluation. To facilitate reproducibility, we make our code available at https://github.com/matthjs/xai-gp.
Neural Networks Enabled Discovery On the Higher-Order Nonlinear Partial Differential Equation of Traffic Dynamics
Authors: Zihang Wei, Yunlong Zhang, Chenxi Liu, Yang Zhou
Date: 2025-05-01 00:00:00

Modeling the traffic dynamics is essential for understanding and predicting the traffic spatiotemporal evolution. However, deriving the partial differential equation (PDE) models that capture these dynamics is challenging due to their potential high order property and nonlinearity. In this paper, we introduce a novel deep learning framework, "TRAFFIC-PDE-LEARN", designed to discover hidden PDE models of traffic network dynamics directly from measurement data. By harnessing the power of the neural network to approximate a spatiotemporal fundamental diagram that facilitates smooth estimation of partial derivatives with low-resolution loop detector data. Furthermore, the use of automatic differentiation enables efficient computation of the necessary partial derivatives through the chain and product rules, while sparse regression techniques facilitate the precise identification of physically interpretable PDE components. Tested on data from a real-world traffic network, our model demonstrates that the underlying PDEs governing traffic dynamics are both high-order and nonlinear. By leveraging the learned dynamics for prediction purposes, the results underscore the effectiveness of our approach and its potential to advance intelligent transportation systems.
Dual Filter: A Mathematical Framework for Inference using Transformer-like Architectures
Authors: Heng-Sheng Chang, Prashant G. Mehta
Date: 2025-05-01 00:00:00
Tasks: Decoder

This paper presents a mathematical framework for causal nonlinear prediction in settings where observations are generated from an underlying hidden Markov model (HMM). Both the problem formulation and the proposed solution are motivated by the decoder-only transformer architecture, in which a finite sequence of observations (tokens) is mapped to the conditional probability of the next token. Our objective is not to construct a mathematical model of a transformer. Rather, our interest lies in deriving, from first principles, transformer-like architectures that solve the prediction problem for which the transformer is designed. The proposed framework is based on an original optimal control approach, where the prediction objective (MMSE) is reformulated as an optimal control problem. An analysis of the optimal control problem is presented leading to a fixed-point equation on the space of probability measures. To solve the fixed-point equation, we introduce the dual filter, an iterative algorithm that closely parallels the architecture of decoder-only transformers. These parallels are discussed in detail along with the relationship to prior work on mathematical modeling of transformers as transport on the space of probability measures. Numerical experiments are provided to illustrate the performance of the algorithm using parameter values used in researchscale transformer models.
Aerial Image Classification in Scarce and Unconstrained Environments via Conformal Prediction
Authors: Farhad Pourkamali-Anaraki
Date: 2025-04-24 00:00:00
Tasks: Conformal Prediction, image-classification, Image Classification, Model Compression, Prediction

This paper presents a comprehensive empirical analysis of conformal prediction methods on a challenging aerial image dataset featuring diverse events in unconstrained environments. Conformal prediction is a powerful post-hoc technique that takes the output of any classifier and transforms it into a set of likely labels, providing a statistical guarantee on the coverage of the true label. Unlike evaluations on standard benchmarks, our study addresses the complexities of data-scarce and highly variable real-world settings. We investigate the effectiveness of leveraging pretrained models (MobileNet, DenseNet, and ResNet), fine-tuned with limited labeled data, to generate informative prediction sets. To further evaluate the impact of calibration, we consider two parallel pipelines (with and without temperature scaling) and assess performance using two key metrics: empirical coverage and average prediction set size. This setup allows us to systematically examine how calibration choices influence the trade-off between reliability and efficiency. Our findings demonstrate that even with relatively small labeled samples and simple nonconformity scores, conformal prediction can yield valuable uncertainty estimates for complex tasks. Moreover, our analysis reveals that while temperature scaling is often employed for calibration, it does not consistently lead to smaller prediction sets, underscoring the importance of careful consideration in its application. Furthermore, our results highlight the significant potential of model compression techniques within the conformal prediction pipeline for deployment in resource-constrained environments. Based on our observations, we advocate for future research to delve into the impact of noisy or ambiguous labels on conformal prediction performance and to explore effective model reduction strategies.
Linear Parameter Varying Attitude Control For CubeSats Using Electrospray Thrusters
Authors: Felix Biertümpfel, Emily Burgin, Hanna-Lee Harjono, Paulo Lozano, Harald Pfifer
Date: 2025-05-01 00:00:00

This paper proposes the design of a single linear parameter-varying (LPV) controller for the attitude control of CubeSats using electro spray thrusters. CubeSat attitude control based on electro spray thrusters faces two main challenges. Firstly, the thruster can only generate a small control torque leading to easily saturating the actuation system. Secondly, CubeSats need to operate multiple different maneuvers from large to small slews to pointing tasks. LPV control is ideally suitable to address these challenges. The proposed design follows a mixed-sensitivity control scheme. The parameter-varying weights depend on the attitude error and are derived from the performance and robustness requirements of individual typical CubeSat maneuvers. The controller is synthesized by minimizing the induced L2-norm of the closed-loop interconnections between the controller and weighted plant. The performance and robustness of the controller is demonstrated on a simulation of the MIT Space Propulsion Lab's Magnetic Levitation CubeSat Testbed.
Dimension-reduced Optimization of Multi-zone Thermostatically Controlled Loads
Authors: Xueyuan Cui, Yi Wang, Bolun Xu
Date: 2025-05-01 00:00:00
Tasks: Computational Efficiency, Dimensionality Reduction, Multi-Task Learning

This study proposes a computationally efficient method for optimizing multi-zone thermostatically controlled loads (TCLs) by leveraging dimensionality reduction through an auto-encoder. We develop a multi-task learning framework to jointly represent latent variables and formulate a state-space model based on observed TCL operation data. This significantly reduces the dimensionality of TCL variables and states while preserving critical nonlinear interdependencies in TCL control. To address various application scenarios, we introduce optimization algorithms based on system identification (OptIden) and system simulation (OptSim) tailored to the latent variable representation. These approaches employ automatic differentiation and zeroth-order techniques, respectively, for efficient implementation. We evaluate the proposed method using a 90-zone apartment prototype, comparing its performance to traditional high-dimensional optimization. Results demonstrate that our approach effectively reduces control costs while achieving significantly higher computational efficiency.
Linear Phase Balancing Scheme using Voltage Unbalance Sensitivities in Multi-phase Power Distribution Grids
Authors: Rahul K. Gupta
Date: 2025-05-01 00:00:00

Power distribution networks, especially in North America, are often unbalanced due to the mix of single-, two- and three-phase networks as well as due to the high penetration of single-phase devices at the distribution level such as electric vehicle (EV) chargers and single-phase solar plants. However, the network operator must adhere to the voltage unbalance levels within the limits specified by IEEE, IEC, and NEMA standards for the safety of the equipment as well as the efficiency of the network operation. Existing works have proposed active and reactive power control in the network to minimize imbalances. However, these optimization problems are highly nonlinear and nonconvex due to the inherent non-linearity of unbalanced metrics and power-flow equations. In this work, we propose a linearization approach of unbalance metrics such as voltage unbalance factors (VUF), phase voltage unbalance rate (PVUR), and line voltage unbalance rate (LVUR) using the first order Taylor's approximation. This linearization is then applied to the phase balancing control scheme; it is formulated as a feedback approach where the linearization is updated successively after the active/reactive control setpoint has been actuated and shows improvement in voltage imbalances. We demonstrate the application of the proposed scheme on a standard IEEE benchmark test case, demonstrating its effectiveness.
Stabilization by Controllers Having Integer Coefficients
Authors: Joowon Lee, Donggil Lee, Junsoo Kim
Date: 2025-05-01 00:00:00

The system property of ``having integer coefficients,'' that is, a transfer function has an integer monic polynomial as its denominator, is significant in the field of encrypted control as it is required for a dynamic controller to be realized over encrypted data. This paper shows that there always exists a controller with integer coefficients stabilizing a given discrete-time linear time-invariant plant. A constructive algorithm to obtain such a controller is provided, along with numerical examples. Furthermore, the proposed method is applied to converting a pre-designed controller to have integer coefficients, while the original performance is preserved in the sense that the transfer function of the closed-loop system remains unchanged.
An introduction to R package `mvs`
Authors: Wouter van Loon
Date: 2025-04-24 00:00:00
Tasks: Ensemble Learning, MULTI-VIEW LEARNING

In biomedical science, a set of objects or persons can often be described by multiple distinct sets of features obtained from different data sources or modalities (called "multi-view data"). Classical machine learning methods ignore the multi-view structure of such data, limiting model interpretability and performance. The R package `mvs` provides methods that were designed specifically for dealing with multi-view data, based on the multi-view stacking (MVS) framework. MVS is a form of supervised (machine) learning used to train multi-view classification or prediction models. MVS works by training a learning algorithm on each view separately, estimating the predictive power of each view-specific model through cross-validation, and then using another learning algorithm to assign weights to the view-specific models based on their estimated predictions. MVS is a form of ensemble learning, dividing the large multi-view learning problem into smaller sub-problems. Most of these sub-problems can be solved in parallel, making it computationally attractive. Additionally, the number of features of the sub-problems is greatly reduced compared with the full multi-view learning problem. This makes MVS especially useful when the total number of features is larger than the number of observations (i.e., high-dimensional data). MVS can still be applied even if the sub-problems are themselves high-dimensional by adding suitable penalty terms to the learning algorithms. Furthermore, MVS can be used to automatically select the views which are most important for prediction. The R package `mvs` makes fitting MVS models, including such penalty terms, easily and openly accessible. `mvs` allows for the fitting of stacked models with any number of levels, with different penalty terms, different outcome distributions, and provides several options for missing data handling.
Hyper-Transforming Latent Diffusion Models
Authors: Ignacio Peis, Batuhan Koyuncu, Isabel Valera, Jes Frellsen
Date: 2025-04-23 00:00:00
Tasks: Computational Efficiency, Decoder

We introduce a novel generative framework for functions by integrating Implicit Neural Representations (INRs) and Transformer-based hypernetworks into latent variable models. Unlike prior approaches that rely on MLP-based hypernetworks with scalability limitations, our method employs a Transformer-based decoder to generate INR parameters from latent variables, addressing both representation capacity and computational efficiency. Our framework extends latent diffusion models (LDMs) to INR generation by replacing standard decoders with a Transformer-based hypernetwork, which can be trained either from scratch or via hyper-transforming-a strategy that fine-tunes only the decoder while freezing the pre-trained latent space. This enables efficient adaptation of existing generative models to INR-based representations without requiring full retraining.
On phase in scaled graphs
Authors: Sebastiaan van den Eijnden, Chao Chen, Koen Scheres, Thomas Chaffey, Alexander Lanzon
Date: 2025-04-30 00:00:00

The scaled graph has been introduced recently as a nonlinear extension of the classical Nyquist plot for linear time-invariant systems. In this paper, we introduce a modified definition for the scaled graph, termed the signed scaled graph (SSG), in which the phase component is characterized by making use of the Hilbert transform. Whereas the original definition of the scaled graph uses unsigned phase angles, the new definition has signed phase angles which ensures the possibility to differentiate between phase-lead and phase-lag properties in a system. Making such distinction is important from both an analysis and a synthesis perspective, and helps in providing tighter stability estimates of feedback interconnections. We show how the proposed SSG leads to intuitive characterizations of positive real and negative imaginary nonlinear systems, and present various interconnection results. We showcase the effectiveness of our results through several motivating examples.
Signal Recovery from Random Dot-Product Graphs Under Local Differential Privacy
Authors: Siddharth Vishwanath, Jonathan Hehir
Date: 2025-04-24 00:00:00
Tasks: Community Detection

We consider the problem of recovering latent information from graphs under $\varepsilon$-edge local differential privacy where the presence of relationships/edges between two users/vertices remains confidential, even from the data curator. For the class of generalized random dot-product graphs, we show that a standard local differential privacy mechanism induces a specific geometric distortion in the latent positions. Leveraging this insight, we show that consistent recovery of the latent positions is achievable by appropriately adjusting the statistical inference procedure for the privatized graph. Furthermore, we prove that our procedure is nearly minimax-optimal under local edge differential privacy constraints. Lastly, we show that this framework allows for consistent recovery of geometric and topological information underlying the latent positions, as encoded in their persistence diagrams. Our results extend previous work from the private community detection literature to a substantially richer class of models and inferential tasks.
Beyond Quadratic Costs: A Bregman Divergence Approach to H$_\infty$ Control
Authors: Joudi Hajar, Reza Ghane, Babak Hassibi
Date: 2025-05-01 00:00:00

This paper presents a novel extension of the H$_\infty$ control framework that generalizes the traditional quadratic cost formulation to accommodate strictly convex, nonquadratic functions for the state, control input, and disturbance. This new formulation not only captures additional noise characteristics but also supports a range of performance objectives-including sparse control, safety constraints, and other tailored behaviors-beyond what is possible with quadratic costs. We derive a closed-form solution of a central controller that minimizes the worst-case performance ratio under the proposed cost structure. Furthermore, we develop Riccati-like equations that impose necessary and sufficient conditions on the nonquadratic cost functions, thereby ensuring the existence of a robust solution. Finally, we rigorously establish Lyapunov stability for the closed-loop system. The proposed framework bridges robust control theory with modern approaches in machine learning and signal processing, offering enhanced flexibility and improved performance in complex control scenarios.