pwc-archive/papers-with-abstracts huggingface dataset:
BurTorch: Revisiting Training from First Principles by Coupling Autodiff, Math Optimization, and Systems
Authors: Konstantin Burlachenko, Peter Richtárik
Date: 2025-03-18 00:00:00
Tasks: CPU, Math

In this work, we introduce BurTorch, a compact high-performance framework designed to optimize Deep Learning (DL) training on single-node workstations through an exceptionally efficient CPU-based backpropagation (Rumelhart et al., 1986; Linnainmaa, 1970) implementation. Although modern DL frameworks rely on compilerlike optimizations internally, BurTorch takes a different path. It adopts a minimalist design and demonstrates that, in these circumstances, classical compiled programming languages can play a significant role in DL research. By eliminating the overhead of large frameworks and making efficient implementation choices, BurTorch achieves orders-of-magnitude improvements in performance and memory efficiency when computing $\nabla f(x)$ on a CPU. BurTorch features a compact codebase designed to achieve two key goals simultaneously. First, it provides a user experience similar to script-based programming environments. Second, it dramatically minimizes runtime overheads. In large DL frameworks, the primary source of memory overhead for relatively small computation graphs $f(x)$ is due to feature-heavy implementations. We benchmarked BurTorch against widely used DL frameworks in their execution modes: JAX (Bradbury et al., 2018), PyTorch (Paszke et al., 2019), TensorFlow (Abadi et al., 2016); and several standalone libraries: Autograd (Maclaurin et al., 2015), Micrograd (Karpathy, 2020), Apple MLX (Hannun et al., 2023). For small compute graphs, BurTorch outperforms best-practice solutions by up to $\times 2000$ in runtime and reduces memory consumption by up to $\times 3500$. For a miniaturized GPT-3 model (Brown et al., 2020), BurTorch achieves up to a $\times 20$ speedup and reduces memory up to $\times 80$ compared to PyTorch.
Multitaper-Based Post-Processing of Compact Antenna Responses Obtained in Non-Anechoic Conditions
Authors: Mariusz Dzwonkowski, Adrian Bekasiewicz, Slawomir Koziel
Date: 2025-03-18 00:00:00

The process of developing antenna structures typically involves prototype measurements. While accurate validation of far-field performance can be performed in dedicated facilities like anechoic chambers, high cost of construction and maintenance might not justify their use for teaching, or low-budget research scenarios. Non-anechoic experiments provide a cost-effective alternative, however the performance metrics obtained in such conditions require appropriate correction. In this paper, we consider a multitaper approach for post-processing antenna far-field characteristics measured in challenging, non-anechoic environments. The discussed algorithm enhances one-shot measurements to enable extraction of line-of-sight responses while attenuating interferences from multi-path propagation and the noise from external sources of electromagnetic radiation. The performance of the considered method has been demonstrated in uncontrolled conditions using a compact spline-based monopole. Furthermore, the approach has been favorably validated against the state-of-the-art techniques from the literature.
Fast alignment of heterogeneous images in sliced Wasserstein distance
Authors: Yunpeng Shi, Amit Singer, Eric J. Verbeke
Date: 2025-03-17 00:00:00

Many applications of computer vision rely on the alignment of similar but non-identical images. We present a fast algorithm for aligning heterogeneous images based on optimal transport. Our approach combines the speed of fast Fourier methods with the robustness of sliced probability metrics and allows us to efficiently compute the alignment between two $L \times L$ images using the sliced 2-Wasserstein distance in $O(L^2 \log L)$ operations. We show that our method is robust to translations, rotations and deformations in the images.
Parameter-free structure-texture image decomposition by unrolling
Authors: Laura Girometti, Jean-François Aujol, Antoine Guennec, Yann Traonmilin
Date: 2025-03-17 00:00:00

In this work, we propose a parameter-free and efficient method to tackle the structure-texture image decomposition problem. In particular, we present a neural network LPR-NET based on the unrolling of the Low Patch Rank model. On the one hand, this allows us to automatically learn parameters from data, and on the other hand to be computationally faster while obtaining qualitatively similar results compared to traditional iterative model-based methods. Moreover, despite being trained on synthetic images, numerical experiments show the ability of our network to generalize well when applied to natural images.
Neural network-based Godunov corrections for approximate Riemann solvers using bi-fidelity learning
Authors: Akshay Thakur, Matthew J. Zahr
Date: 2025-03-17 00:00:00

The Riemann problem is fundamental in the computational modeling of hyperbolic partial differential equations, enabling the development of stable and accurate upwind schemes. While exact solvers provide robust upwinding fluxes, their high computational cost necessitates approximate solvers. Although approximate solvers achieve accuracy in many scenarios, they produce inaccurate solutions in certain cases. To overcome this limitation, we propose constructing neural network-based surrogate models, trained using supervised learning, designed to map interior and exterior conservative state variables to the corresponding exact flux. Specifically, we propose two distinct approaches: one utilizing a vanilla neural network and the other employing a bi-fidelity neural network. The performance of the proposed approaches is demonstrated through applications to one-dimensional and two-dimensional partial differential equations, showcasing their robustness and accuracy.
Ensemble Kalman-Bucy filtering for nonlinear model predictive control
Authors: Sebastian Reich
Date: 2025-03-16 00:00:00
Tasks: Model Predictive Control, State Estimation

We consider the problem of optimal control for partially observed dynamical systems. Despite its prevalence in practical applications, there are still very few algorithms available, which take uncertainties in the current state estimates and future observations into account. In other words, most current approaches separate state estimation from the optimal control problem. In this paper, we extend the popular ensemble Kalman filter to receding horizon optimal control problems in the spirit of nonlinear model predictive control. We provide an interacting particle approximation to the forward-backward stochastic differential equations arising from Pontryagin's maximum principle with the forward stochastic differential equation provided by the time-continuous ensemble Kalman-Bucy filter equations. The receding horizon control laws are approximated as linear and are continuously updated as in nonlinear model predictive control. We illustrate the performance of the proposed methodology for an inverted pendulum example.
Towards Learning High-Precision Least Squares Algorithms with Sequence Models
Authors: Jerry Liu, Jessica Grogan, Owen Dugan, Ashish Rao, Simran Arora, Atri Rudra, Christopher Ré
Date: 2025-03-15 00:00:00

This paper investigates whether sequence models can learn to perform numerical algorithms, e.g. gradient descent, on the fundamental problem of least squares. Our goal is to inherit two properties of standard algorithms from numerical analysis: (1) machine precision, i.e. we want to obtain solutions that are accurate to near floating point error, and (2) numerical generality, i.e. we want them to apply broadly across problem instances. We find that prior approaches using Transformers fail to meet these criteria, and identify limitations present in existing architectures and training procedures. First, we show that softmax Transformers struggle to perform high-precision multiplications, which prevents them from precisely learning numerical algorithms. Second, we identify an alternate class of architectures, comprised entirely of polynomials, that can efficiently represent high-precision gradient descent iterates. Finally, we investigate precision bottlenecks during training and address them via a high-precision training recipe that reduces stochastic gradient noise. Our recipe enables us to train two polynomial architectures, gated convolutions and linear attention, to perform gradient descent iterates on least squares problems. For the first time, we demonstrate the ability to train to near machine precision. Applied iteratively, our models obtain 100,000x lower MSE than standard Transformers trained end-to-end and they incur a 10,000x smaller generalization gap on out-of-distribution problems. We make progress towards end-to-end learning of numerical algorithms for least squares.
Improving Wolbachia-Based Control Programs in Urban Settings: Insights from Spatial Modeling
Authors: Daniela Florez, Ricardo Cortez, James M. Hyman, Zhuolin Qu
Date: 2025-03-15 00:00:00

Arboviral diseases remain a major public health concern, particularly in tropical and subtropical regions where mosquito populations thrive. One promising strategy to curb transmission is the release of Aedes aegypti mosquitoes infected with Wolbachia, a bacterium that reduces their ability to spread viruses. However, past large-scale releases have not always been successful, especially in complex urban settings, where restricted access to certain areas often leads to infection establishment failures and wasted resources. To address this, we developed and analyzed a partial differential equation model that simulates how Wolbachia-infected mosquitoes are established in different urban environments. We also explored strategies to improve their success under constraints on release size and the efficacy level of insecticide used for pre-release interventions. Our findings suggest that targeted releases are most effective in areas with limited mosquito movement without additional insecticide use. In higher mosquito dispersal areas, reducing at least 35% of wild mosquitoes before release significantly improves establishment within nine months. Additionally, distributing releases over 2-5 weekly batches enhances success more than a single large release, even without other interventions. These findings offer practical insights for designing cost-effective and efficient Wolbachia-based mosquito control programs, reducing the burden of mosquito-borne diseases on vulnerable communities.
SafeSlice: Enabling SLA-Compliant O-RAN Slicing via Safe Deep Reinforcement Learning
Authors: Ahmad M. Nagib, Hatem Abou-zeid, Hossam S. Hassanein
Date: 2025-03-17 00:00:00
Tasks: Deep Reinforcement Learning

Deep reinforcement learning (DRL)-based slicing policies have shown significant success in simulated environments but face challenges in physical systems such as open radio access networks (O-RANs) due to simulation-to-reality gaps. These policies often lack safety guarantees to ensure compliance with service level agreements (SLAs), such as the strict latency requirements of immersive applications. As a result, a deployed DRL slicing agent may make resource allocation (RA) decisions that degrade system performance, particularly in previously unseen scenarios. Real-world immersive applications require maintaining SLA constraints throughout deployment to prevent risky DRL exploration. In this paper, we propose SafeSlice to address both the cumulative (trajectory-wise) and instantaneous (state-wise) latency constraints of O-RAN slices. We incorporate the cumulative constraints by designing a sigmoid-based risk-sensitive reward function that reflects the slices' latency requirements. Moreover, we build a supervised learning cost model as part of a safety layer that projects the slicing agent's RA actions to the nearest safe actions, fulfilling instantaneous constraints. We conduct an exhaustive experiment that supports multiple services, including real virtual reality (VR) gaming traffic, to investigate the performance of SafeSlice under extreme and changing deployment conditions. SafeSlice achieves reductions of up to 83.23% in average cumulative latency, 93.24% in instantaneous latency violations, and 22.13% in resource consumption compared to the baselines. The results also indicate SafeSlice's robustness to changing the threshold configurations of latency constraints, a vital deployment scenario that will be realized by the O-RAN paradigm to empower mobile network operators (MNOs).
Agentic Search Engine for Real-Time IoT Data
Authors: Abdelrahman Elewah, Khalid Elgazzar
Date: 2025-03-15 00:00:00
Tasks: RAG, Retrieval-augmented Generation

The Internet of Things (IoT) has enabled diverse devices to communicate over the Internet, yet the fragmentation of IoT systems limits seamless data sharing and coordinated management. We have recently introduced SensorsConnect, a unified framework to enable seamless content and sensor data sharing in collaborative IoT systems, inspired by how the World Wide Web (WWW) enabled a shared and accessible space for information among humans. This paper presents the IoT Agentic Search Engine (IoT-ASE), a real-time search engine tailored for IoT environments. IoT-ASE leverages Large Language Models (LLMs) and Retrieval Augmented Generation (RAG) techniques to address the challenge of searching vast, real-time IoT data, enabling it to handle complex queries and deliver accurate, contextually relevant results. We implemented a use-case scenario in Toronto to demonstrate how IoT-ASE can improve service quality recommendations by leveraging real-time IoT data. Our evaluation shows that IoT-ASE achieves a 92\% accuracy in retrieving intent-based services and produces responses that are concise, relevant, and context-aware, outperforming generalized responses from systems like Gemini. These findings highlight the potential IoT-ASE to make real-time IoT data accessible and support effective, real-time decision-making.
Hierarchical Evolutionary Optimization with Predictive Modeling for Stable Delay-Constrained Routing in Vehicular Networks
Authors: Zhang Zhiou, Guo Weian, Zhang Qin, Lin Haibin, Li Dongyang
Date: 2025-03-15 00:00:00

Vehicular Ad Hoc Networks (VANETs) are a cornerstone of intelligent transportation systems, facilitating real-time communication between vehicles and infrastructure. However, the dynamic nature of VANETs introduces significant challenges in routing, especially in minimizing communication delay while ensuring route stability. This paper proposes a hierarchical evolutionary optimization framework for delay-constrained routing in vehicular networks. Leveraging multi-objective optimization, the framework balances delay and stability objectives and incorporates adaptive mechanisms like incremental route adjustments and LSTM-based predictive modeling. Simulation results confirm that the proposed framework maintains low delay and high stability, adapting effectively to frequent topology changes in dynamic vehicular environments.
End-to-End Edge AI Service Provisioning Framework in 6G ORAN
Authors: Yun Tang, Udhaya Chandhar Srinivasan, Benjamin James Scott, Obumneme Umealor, Dennis Kevogo, Weisi Guo
Date: 2025-03-15 00:00:00
Tasks: Large Language Model, Model Selection

With the advent of 6G, Open Radio Access Network (O-RAN) architectures are evolving to support intelligent, adaptive, and automated network orchestration. This paper proposes a novel Edge AI and Network Service Orchestration framework that leverages Large Language Model (LLM) agents deployed as O-RAN rApps. The proposed LLM-agent-powered system enables interactive and intuitive orchestration by translating the user's use case description into deployable AI services and corresponding network configurations. The LLM agent automates multiple tasks, including AI model selection from repositories (e.g., Hugging Face), service deployment, network adaptation, and real-time monitoring via xApps. We implement a prototype using open-source O-RAN projects (OpenAirInterface and FlexRIC) to demonstrate the feasibility and functionality of our framework. Our demonstration showcases the end-to-end flow of AI service orchestration, from user interaction to network adaptation, ensuring Quality of Service (QoS) compliance. This work highlights the potential of integrating LLM-driven automation into 6G O-RAN ecosystems, paving the way for more accessible and efficient edge AI ecosystems.
Mobility-aware Seamless Service Migration and Resource Allocation in Multi-edge IoV Systems
Authors: Zheyi Chen, Sijin Huang, Geyong Min, Zhaolong Ning, Jie Li, Yan Zhang
Date: 2025-03-11 00:00:00
Tasks: Deep Reinforcement Learning, Edge-computing

Mobile Edge Computing (MEC) offers low-latency and high-bandwidth support for Internet-of-Vehicles (IoV) applications. However, due to high vehicle mobility and finite communication coverage of base stations, it is hard to maintain uninterrupted and high-quality services without proper service migration among MEC servers. Existing solutions commonly rely on prior knowledge and rarely consider efficient resource allocation during the service migration process, making it hard to reach optimal performance in dynamic IoV environments. To address these important challenges, we propose SR-CL, a novel mobility-aware seamless Service migration and Resource allocation framework via Convex-optimization-enabled deep reinforcement Learning in multi-edge IoV systems. First, we decouple the Mixed Integer Nonlinear Programming (MINLP) problem of service migration and resource allocation into two sub-problems. Next, we design a new actor-critic-based asynchronous-update deep reinforcement learning method to handle service migration, where the delayed-update actor makes migration decisions and the one-step-update critic evaluates the decisions to guide the policy update. Notably, we theoretically derive the optimal resource allocation with convex optimization for each MEC server, thereby further improving system performance. Using the real-world datasets of vehicle trajectories and testbed, extensive experiments are conducted to verify the effectiveness of the proposed SR-CL. Compared to benchmark methods, the SR-CL achieves superior convergence and delay performance under various scenarios.
PrETi: Predicting Execution Time in Early Stage with LLVM and Machine Learning
Authors: Risheng Xu, Philipp Sieweck, Hermann von Hasseln, Dirk Nowotka
Date: 2025-03-17 00:00:00
Tasks: Prediction

We introduce preti, a novel framework for predicting software execution time during the early stages of development. preti leverages an LLVM-based simulation environment to extract timing-related runtime information, such as the count of executed LLVM IR instructions. This information, combined with historical execution time data, is utilized to train machine learning models for accurate time prediction. To further enhance prediction accuracy, our approach incorporates simulations of cache accesses and branch prediction. The evaluations on public benchmarks demonstrate that preti achieves an average Absolute Percentage Error (APE) of 11.98\%, surpassing state-of-the-art methods. These results underscore the effectiveness and efficiency of preti as a robust solution for early-stage timing analysis.
ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory
Authors: Liangyu Wang, Jie Ren, Hang Xu, Junxiao Wang, Huanyi Xie, David E. Keyes, Di Wang
Date: 2025-03-16 00:00:00
Tasks: CPU, GPU

Fine-tuning large pre-trained LLMs generally demands extensive GPU memory. Traditional first-order optimizers like SGD encounter substantial difficulties due to increased memory requirements from storing activations and gradients during both the forward and backward phases as the model size expands. Alternatively, zeroth-order (ZO) techniques can compute gradients using just forward operations, eliminating the need to store activations. Furthermore, by leveraging CPU capabilities, it's feasible to enhance both the memory and processing power available to a single GPU. We propose a novel framework, ZO2 (Zeroth-Order Offloading), for efficient zeroth-order fine-tuning of LLMs with only limited GPU memory. Our framework dynamically shifts model parameters between the CPU and GPU as required, optimizing computation flow and maximizing GPU usage by minimizing downtime. This integration of parameter adjustments with ZO's double forward operations reduces unnecessary data movement, enhancing the fine-tuning efficacy. Additionally, our framework supports an innovative low-bit precision approach in AMP mode to streamline data exchanges between the CPU and GPU. Employing this approach allows us to fine-tune extraordinarily large models, such as the OPT-175B with more than 175 billion parameters, on a mere 18GB GPU--achievements beyond the reach of traditional methods. Moreover, our framework achieves these results with almost no additional time overhead and absolutely no accuracy loss compared to standard zeroth-order methods. ZO2's code has been open-sourced in https://github.com/liangyuwang/zo2.
Can LLMs Enable Verification in Mainstream Programming?
Authors: Aleksandr Shefer, Igor Engel, Stanislav Alekseev, Daniil Berezun, Ekaterina Verbitskaia, Anton Podkopaev
Date: 2025-03-18 00:00:00
Tasks: Code Generation, HumanEval

Although formal methods are capable of producing reliable software, they have seen minimal adoption in everyday programming. Automatic code generation using large language models is becoming increasingly widespread, but it rarely considers producing strong correctness guarantees. In this study, we explore the ability of LLMs to produce verified code in three verification languages (Dafny, Nagini, and Verus). To do so, we use manually curated datasets derived from the state-ofthe-art Python benchmark, HumanEval. We also assess what types of information are sufficient to achieve good-quality results.
Can LLMs Formally Reason as Abstract Interpreters for Program Analysis?
Authors: Jacqueline L. Mitchell, Brian Hyeongseok Kim, Chenyu Zhou, Chao Wang
Date: 2025-03-16 00:00:00
Tasks: Code Generation

LLMs have demonstrated impressive capabilities in code generation and comprehension, but their potential in being able to perform program analysis in a formal, automatic manner remains under-explored. To that end, we systematically investigate whether LLMs can reason about programs using a program analysis framework called abstract interpretation. We prompt LLMs to follow two different strategies, denoted as Compositional and Fixed Point Equation, to formally reason in the style of abstract interpretation, which has never been done before to the best of our knowledge. We validate our approach using state-of-the-art LLMs on 22 challenging benchmark programs from the Software Verification Competition (SV-COMP) 2019 dataset, widely used in program analysis. Our results show that our strategies are able to elicit abstract interpretation-based reasoning in the tested models, but LLMs are susceptible to logical errors, especially while interpreting complex program structures, as well as general hallucinations. This highlights key areas for improvement in the formal reasoning capabilities of LLMs.