pwc-archive/papers-with-abstracts huggingface dataset:
High-Dynamic Radar Sequence Prediction for Weather Nowcasting Using Spatiotemporal Coherent Gaussian Representation
Authors: Ziye Wang, Yiran Qin, Lin Zeng, Ruimao Zhang
Date: 2025-02-17 00:00:00
Tasks: Mamba

Weather nowcasting is an essential task that involves predicting future radar echo sequences based on current observations, offering significant benefits for disaster management, transportation, and urban planning. Current prediction methods are limited by training and storage efficiency, mainly focusing on 2D spatial predictions at specific altitudes. Meanwhile, 3D volumetric predictions at each timestamp remain largely unexplored. To address such a challenge, we introduce a comprehensive framework for 3D radar sequence prediction in weather nowcasting, using the newly proposed SpatioTemporal Coherent Gaussian Splatting (STC-GS) for dynamic radar representation and GauMamba for efficient and accurate forecasting. Specifically, rather than relying on a 4D Gaussian for dynamic scene reconstruction, STC-GS optimizes 3D scenes at each frame by employing a group of Gaussians while effectively capturing their movements across consecutive frames. It ensures consistent tracking of each Gaussian over time, making it particularly effective for prediction tasks. With the temporally correlated Gaussian groups established, we utilize them to train GauMamba, which integrates a memory mechanism into the Mamba framework. This allows the model to learn the temporal evolution of Gaussian groups while efficiently handling a large volume of Gaussian tokens. As a result, it achieves both efficiency and accuracy in forecasting a wide range of dynamic meteorological radar signals. The experimental results demonstrate that our STC-GS can efficiently represent 3D radar sequences with over $16\times$ higher spatial resolution compared with the existing 3D representation methods, while GauMamba outperforms state-of-the-art methods in forecasting a broad spectrum of high-dynamic weather conditions.
Fractal Generative Models
Authors: Tianhong Li, Qinyi Sun, Lijie Fan, Kaiming He
Date: 2025-02-24 00:00:00
Tasks: Image Generation

Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstracting generative models into atomic generative modules. Analogous to fractals in mathematics, our method constructs a new type of generative model by recursively invoking atomic generative modules, resulting in self-similar fractal architectures that we call fractal generative models. As a running example, we instantiate our fractal framework using autoregressive models as the atomic generative modules and examine it on the challenging task of pixel-by-pixel image generation, demonstrating strong performance in both likelihood estimation and generation quality. We hope this work could open a new paradigm in generative modeling and provide a fertile ground for future research. Code is available at https://github.com/LTH14/fractalgen.
Digital implementations of deep feature extractors are intrinsically informative
Authors: Max Getter
Date: 2025-02-20 00:00:00

Rapid information (energy) propagation in deep feature extractors is crucial to balance computational complexity versus expressiveness as a representation of the input. We prove an upper bound for the speed of energy propagation in a unified framework that covers different neural network models, both over Euclidean and non-Euclidean domains. Additional structural information about the signal domain can be used to explicitly determine or improve the rate of decay. To illustrate this, we show global exponential energy decay for a range of 1) feature extractors with discrete-domain input signals, and 2) convolutional neural networks (CNNs) via scattering over locally compact abelian (LCA) groups.
Is Mathematics Obsolete?
Authors: Jeremy Avigad
Date: 2025-02-03 00:00:00

This is an essay about the value of mathematical and symbolic reasoning in the age of AI.
Training Neural ODEs Using Fully Discretized Simultaneous Optimization
Authors: Mariia Shapovalova, Calvin Tsay
Date: 2025-02-21 00:00:00

Neural Ordinary Differential Equations (Neural ODEs) represent continuous-time dynamics with neural networks, offering advancements for modeling and control tasks. However, training Neural ODEs requires solving differential equations at each epoch, leading to high computational costs. This work investigates simultaneous optimization methods as a faster training alternative. In particular, we employ a collocation-based, fully discretized formulation and use IPOPT--a solver for large-scale nonlinear optimization--to simultaneously optimize collocation coefficients and neural network parameters. Using the Van der Pol Oscillator as a case study, we demonstrate faster convergence compared to traditional training methods. Furthermore, we introduce a decomposition framework utilizing Alternating Direction Method of Multipliers (ADMM) to effectively coordinate sub-models among data batches. Our results show significant potential for (collocation-based) simultaneous Neural ODE training pipelines.
Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
Authors: Hannah Laus, Suzanna Parkinson, Vasileios Charisopoulos, Felix Krahmer, Rebecca Willett
Date: 2025-02-21 00:00:00
Tasks: compressed sensing

Machine learning methods are commonly used to solve inverse problems, wherein an unknown signal must be estimated from few measurements generated via a known acquisition procedure. In particular, neural networks perform well empirically but have limited theoretical guarantees. In this work, we study an underdetermined linear inverse problem that admits several possible solution mappings. A standard remedy (e.g., in compressed sensing) establishing uniqueness of the solution mapping is to assume knowledge of latent low-dimensional structure in the source signal. We ask the following question: do deep neural networks adapt to this low-dimensional structure when trained by gradient descent with weight decay regularization? We prove that mildly overparameterized deep linear networks trained in this manner converge to an approximate solution that accurately solves the inverse problem while implicitly encoding latent subspace structure. To our knowledge, this is the first result to rigorously show that deep linear networks trained with weight decay automatically adapt to latent subspace structure in the data under practical stepsize and weight initialization schemes. Our work highlights that regularization and overparameterization improve generalization, while overparameterization also accelerates convergence during training.
A Data-Driven Real-Time Optimal Power Flow Algorithm Using Local Feedback
Authors: Heng Liang, Yujin Huang, Changhong Zhao
Date: 2025-02-21 00:00:00

The increasing penetration of distributed energy resources (DERs) adds variability as well as fast control capabilities to power networks. Dispatching the DERs based on local information to provide real-time optimal network operation is the desideratum. In this paper, we propose a data-driven real-time algorithm that uses only the local measurements to solve time-varying AC optimal power flow (OPF). Specifically, we design a learnable function that takes the local feedback as input in the algorithm. The learnable function, under certain conditions, will result in a unique stationary point of the algorithm, which in turn transfers the OPF problems to be optimized over the parameters of the function. We then develop a stochastic primal-dual update to solve the variant of the OPF problems based on a deep neural network (DNN) parametrization of the learnable function, which is referred to as the training stage. We also design a gradient-free alternative to bypass the cumbersome gradient calculation of the nonlinear power flow model. The OPF solution-tracking error bound is established in the sense of universal approximation of DNN. Numerical results on the IEEE 37-bus test feeder show that the proposed method can track the time-varying OPF solutions with higher accuracy and faster computation compared to benchmark methods.
Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions
Authors: Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath
Date: 2025-02-21 00:00:00

We study the local geometry of empirical risks in high dimensions via the spectral theory of their Hessian and information matrices. We focus on settings where the data, $(Y_\ell)_{\ell =1}^n\in \mathbb R^d$, are i.i.d. draws of a $k$-component Gaussian mixture model, and the loss depends on the projection of the data into a fixed number of vectors, namely $\mathbf{x}^\top Y$, where $\mathbf{x}\in \mathbb{R}^{d\times C}$ are the parameters, and $C$ need not equal $k$. This setting captures a broad class of problems such as classification by one and two-layer networks and regression on multi-index models. We prove exact formulas for the limits of the empirical spectral distribution and outlier eigenvalues and eigenvectors of such matrices in the proportional asymptotics limit, where the number of samples and dimension $n,d\to\infty$ and $n/d=\phi \in (0,\infty)$. These limits depend on the parameters $\mathbf{x}$ only through the summary statistic of the $(C+k)\times (C+k)$ Gram matrix of the parameters and class means, $\mathbf{G} = (\mathbf{x},\mathbf{\mu})^\top(\mathbf{x},\mathbf{\mu})$. It is known that under general conditions, when $\mathbf{x}$ is trained by stochastic gradient descent, the evolution of these same summary statistics along training converges to the solution of an autonomous system of ODEs, called the effective dynamics. This enables us to connect the spectral theory to the training dynamics. We demonstrate our general results by analyzing the effective spectrum along the effective dynamics in the case of multi-class logistic regression. In this setting, the empirical Hessian and information matrices have substantially different spectra, each with their own static and even dynamical spectral transitions.
Dimension-free bounds in high-dimensional linear regression via error-in-operator approach
Authors: Fedor Noskov, Nikita Puchkin, Vladimir Spokoiny
Date: 2025-02-21 00:00:00

We consider a problem of high-dimensional linear regression with random design. We suggest a novel approach referred to as error-in-operator which does not estimate the design covariance $\Sigma$ directly but incorporates it into empirical risk minimization. We provide an expansion of the excess prediction risk and derive non-asymptotic dimension-free bounds on the leading term and the remainder. This helps us to show that auxiliary variables do not increase the effective dimension of the problem, provided that parameters of the procedure are tuned properly. We also discuss computational aspects of our method and illustrate its performance with numerical experiments.
Tensor Product Neural Networks for Functional ANOVA Model
Authors: Seokhun Park, Insung Kong, Yongchan Choi, Chanmoo Park, Yongdai Kim
Date: 2025-02-21 00:00:00
Tasks: model

Interpretability for machine learning models is becoming more and more important as machine learning models become more complex. The functional ANOVA model, which decomposes a high-dimensional function into a sum of lower dimensional functions (commonly referred to as components), is one of the most popular tools for interpretable AI, and recently, various neural networks have been developed for estimating each component in the functional ANOVA model. However, such neural networks are highly unstable when estimating each component since the components themselves are not uniquely defined. That is, there are multiple functional ANOVA decompositions for a given function. In this paper, we propose a novel neural network which guarantees a unique functional ANOVA decomposition and thus is able to estimate each component stably and accurately. We call our proposed neural network ANOVA Tensor Product Neural Network (ANOVA-TPNN) since it is motivated by the tensor product basis expansion. Theoretically, we prove that ANOVA-TPNN can approximate any smooth function well. Empirically, we show that ANOVA-TPNN provide much more stable estimation of each component and thus much more stable interpretation when training data and initial values of the model parameters vary than existing neural networks do.
Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling
Authors: Yufan Li, Pragya Sur
Date: 2025-02-21 00:00:00
Tasks: Binary Classification

We study the fundamental problem of calibrating a linear binary classifier of the form $\sigma(\hat{w}^\top x)$, where the feature vector $x$ is Gaussian, $\sigma$ is a link function, and $\hat{w}$ is an estimator of the true linear weight $w^\star$. By interpolating with a noninformative $\textit{chance classifier}$, we construct a well-calibrated predictor whose interpolation weight depends on the angle $\angle(\hat{w}, w_\star)$ between the estimator $\hat{w}$ and the true linear weight $w_\star$. We establish that this angular calibration approach is provably well-calibrated in a high-dimensional regime where the number of samples and features both diverge, at a comparable rate. The angle $\angle(\hat{w}, w_\star)$ can be consistently estimated. Furthermore, the resulting predictor is uniquely $\textit{Bregman-optimal}$, minimizing the Bregman divergence to the true label distribution within a suitable class of calibrated predictors. Our work is the first to provide a calibration strategy that satisfies both calibration and optimality properties provably in high dimensions. Additionally, we identify conditions under which a classical Platt-scaling predictor converges to our Bregman-optimal calibrated solution. Thus, Platt-scaling also inherits these desirable properties provably in high dimensions.
Do we really need the Rademacher complexities?
Authors: Daniel Bartl, Shahar Mendelson
Date: 2025-02-21 00:00:00

We study the fundamental problem of learning with respect to the squared loss in a convex class. The state-of-the-art sample complexity estimates in this setting rely on Rademacher complexities, which are generally difficult to control. We prove that, contrary to prevailing belief and under minimal assumptions, the sample complexity is not governed by the Rademacher complexities but rather by the behaviour of the limiting gaussian process. In particular, all such learning problems that have the same $L_2$-structure -- even those with heavy-tailed distributions -- share the same sample complexity. This constitutes the first universality result for general convex learning problems. The proof is based on a novel learning procedure, and its performance is studied by combining optimal mean estimation techniques for real-valued random variables with Talagrand's generic chaining method.
Engineered Zwitterion-Infused Clay Composites with Antibacterial and Antifungal Efficacy
Authors: Suvash Ghimire, Yi Wu, Manjyot Kaur Chug, Elizabeth J. Brisbois, Kyungtae Kim, Kausik Mukhopadhyay
Date: 2025-02-21 00:00:00

Microbes and pathogens play a detrimental role in healing wounds, causing infections like impetigo through bodily fluids and skin and entering the bloodstream through the wounds, thereby hindering the healing process and tissue regeneration. Clay, known for its long history of natural therapeutic use, has emerged as one of the most promising candidates for biomedical applications due to its non-toxic nature, porosity, high surface area, ubiquity, and excellent cation exchange capacity. This study demonstrates an innovative approach to engineering an organo-functionalized, infection-resistant, easy-to-use bandage material from clay, an environmentally benign and sustainable material. The hybrid membranes have been developed using clays, zwitterions, silver ions, and terbinafine hydrochloride (TBH) to impart antibacterial and antifungal efficacy. A critical aspect of this study is embedding organic molecules and metal ions with the clays and releasing them to resist the growth and kill the pathogens. The antimicrobial efficacy of the membranes has been tested using a zone of inhibition study against the most common microbes in skin wounds, viz. S. aureus, E. coli, and C. albicans. Results from our studies not only demonstrate the potential of these hybrid clay membranes as a cost-effective, scalable, and effective solution for treating microbial infections but also instill newer avenues for point-of-care wound-healing treatments, offering hope for improved patient outcomes.
Sparks of cognitive flexibility: self-guided context inference for flexible stimulus-response mapping by attentional routing
Authors: Rowan P. Sommers, Sushrut Thorat, Daniel Anthes, Tim C. Kietzmann
Date: 2025-02-21 00:00:00

Flexible cognition demands discovering hidden rules to quickly adapt stimulus-response mappings. Standard neural networks struggle in such tasks requiring rapid, context-driven remapping. Recently, Hummos (2023) introduced a fast-and-slow learning algorithm to mitigate this shortcoming, but its scalability to complex, image-computable tasks was unclear. Here, we propose the Wisconsin Neural Network (WiNN), which extends Hummos' fast-and-slow learning to image-computable tasks demanding flexible rule-based behavior. WiNN employs a pretrained convolutional neural network for vision, coupled with an adjustable "context state" that guides attention to relevant features. If WiNN produces an incorrect response, it first iteratively updates its context state to refocus attention on task-relevant cues, then performs minimal parameter updates to attention and readout layers. This strategy preserves generalizable representations in the sensory and attention networks, reducing catastrophic forgetting. We evaluate WiNN on an image-based extension of the Wisconsin Card Sorting Task, revealing several markers of cognitive flexibility: (i) WiNN autonomously infers underlying rules, (ii) requires fewer examples to do so than control models reliant on large-scale parameter updates, (iii) can perform context-based rule inference solely via context-state adjustments-further enhanced by slow updates of attention and readout parameters, and (iv) generalizes to unseen compositional rules through context-state updates alone. By blending fast context inference with targeted attentional guidance, WiNN achieves "sparks" of flexibility. This approach offers a path toward context-sensitive models that retain knowledge while rapidly adapting to complex, rule-based tasks.
BAN: Neuroanatomical Aligning in Auditory Recognition between Artificial Neural Network and Human Cortex
Authors: Haidong Wang, Pengfei Xiao, Ao Liu, Jianhua Zhang, Qia Shan
Date: 2025-02-21 00:00:00
Tasks: Genre classification, Music Genre Classification

Drawing inspiration from neurosciences, artificial neural networks (ANNs) have evolved from shallow architectures to highly complex, deep structures, yielding exceptional performance in auditory recognition tasks. However, traditional ANNs often struggle to align with brain regions due to their excessive depth and lack of biologically realistic features, like recurrent connection. To address this, a brain-like auditory network (BAN) is introduced, which incorporates four neuroanatomically mapped areas and recurrent connection, guided by a novel metric called the brain-like auditory score (BAS). BAS serves as a benchmark for evaluating the similarity between BAN and human auditory recognition pathway. We further propose that specific areas in the cerebral cortex, mainly the middle and medial superior temporal (T2/T3) areas, correspond to the designed network structure, drawing parallels with the brain's auditory perception pathway. Our findings suggest that the neuroanatomical similarity in the cortex and auditory classification abilities of the ANN are well-aligned. In addition to delivering excellent performance on a music genre classification task, the BAN demonstrates a high BAS score. In conclusion, this study presents BAN as a recurrent, brain-inspired ANN, representing the first model that mirrors the cortical pathway of auditory recognition.
Applications of Random Matrix Theory in Machine Learning and Brain Mapping
Authors: Katrina Lawrence
Date: 2025-02-05 00:00:00

Brain mapping analyzes the wavelengths of brain signals and outputs them in a map, which is then analyzed by a radiologist. Introducing Machine Learning (ML) into the brain mapping process reduces the variable of human error in reading such maps and increases efficiency. A key area of interest is determining the correlation between the functional areas of the brain on a voxel (3-dimensional pixel) wise basis. This leads to determining how a brain is functioning and can be used to detect diseases, disabilities, and sicknesses. As such, random noise presents a challenge in consistently determining the actual signals from the scan. This paper discusses how an algorithm created by Random Matrix Theory (RMT) can be used as a tool for ML, as it detects the correlation of the functional areas of the brain. Random matrices are simulated to represent the voxel signal intensity strength for each time interval where a stimulus is presented in an fMRI scan. Using the Marchenko-Pastur law for Wishart Matrices, a result of RMT, it was found that no matter what type of noise was added to the random matrices, the observed eigenvalue distribution of the Wishart Matrices would converge to the theoretical distribution. This means that RMT is robust and has a high test-re-test reliability. These results further indicate that a strong correlation exists between the eigenvalues, and hence the functional regions of the brain. Any eigenvalue that differs significantly from those predicted from RMT may indicate the discovery of a new discrete brain network.