pwc-archive/papers-with-abstracts huggingface dataset:
Adaptive Resource Allocation Optimization Using Large Language Models in Dynamic Wireless Environments
Authors: Hyeonho Noh, Byonghyo Shim, Hyun Jong Yang
Date: 2025-02-04 00:00:00
Tasks: Large Language Model

Deep learning (DL) has made notable progress in addressing complex radio access network control challenges that conventional analytic methods have struggled to solve. However, DL has shown limitations in solving constrained NP-hard problems often encountered in network optimization, such as those involving quality of service (QoS) or discrete variables like user indices. Current solutions rely on domain-specific architectures or heuristic techniques, and a general DL approach for constrained optimization remains undeveloped. Moreover, even minor changes in communication objectives demand time-consuming retraining, limiting their adaptability to dynamic environments where task objectives, constraints, environmental factors, and communication scenarios frequently change. To address these challenges, we propose a large language model for resource allocation optimizer (LLM-RAO), a novel approach that harnesses the capabilities of LLMs to address the complex resource allocation problem while adhering to QoS constraints. By employing a prompt-based tuning strategy to flexibly convey ever-changing task descriptions and requirements to the LLM, LLM-RAO demonstrates robust performance and seamless adaptability in dynamic environments without requiring extensive retraining. Simulation results reveal that LLM-RAO achieves up to a 40% performance enhancement compared to conventional DL methods and up to an $80$\% improvement over analytical approaches. Moreover, in scenarios with fluctuating communication objectives, LLM-RAO attains up to 2.9 times the performance of traditional DL-based networks.
Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification
Authors: Rudolf Reiter, Jasper Hoffmann, Dirk Reinhardt, Florian Messerer, Katrin Baumgärtner, Shamburaj Sawant, Joschka Boedecker, Moritz Diehl, Sebastien Gros
Date: 2025-02-04 00:00:00
Tasks: Autonomous Driving, Model Predictive Control

The fields of MPC and RL consider two successful control techniques for Markov decision processes. Both approaches are derived from similar fundamental principles, and both are widely used in practical applications, including robotics, process control, energy systems, and autonomous driving. Despite their similarities, MPC and RL follow distinct paradigms that emerged from diverse communities and different requirements. Various technical discrepancies, particularly the role of an environment model as part of the algorithm, lead to methodologies with nearly complementary advantages. Due to their orthogonal benefits, research interest in combination methods has recently increased significantly, leading to a large and growing set of complex ideas leveraging MPC and RL. This work illuminates the differences, similarities, and fundamentals that allow for different combination algorithms and categorizes existing work accordingly. Particularly, we focus on the versatile actor-critic RL approach as a basis for our categorization and examine how the online optimization approach of MPC can be used to improve the overall closed-loop performance of a policy.
Sequential Multi-objective Multi-agent Reinforcement Learning Approach for Predictive Maintenance
Authors: Yan Chen, Cheng Liu
Date: 2025-02-04 00:00:00
Tasks: Multi-agent Reinforcement Learning, Multi-Objective Multi-Agent Reinforcement Learning, quantile regression

Existing predictive maintenance (PdM) methods typically focus solely on whether to replace system components without considering the costs incurred by inspection. However, a well-considered approach should be able to minimize Remaining Useful Life (RUL) at engine replacement while maximizing inspection interval. To achieve this, multi-agent reinforcement learning (MARL) can be introduced. However, due to the sequential and mutually constraining nature of these 2 objectives, conventional MARL is not applicable. Therefore, this paper introduces a novel framework and develops a Sequential Multi-objective Multi-agent Proximal Policy Optimization (SMOMA-PPO) algorithm. Furthermore, to provide comprehensive and effective degradation information to RL agents, we also employed Gated Recurrent Unit, quantile regression, and probability distribution fitting to develop a GRU-based RUL Prediction (GRP) model. Experiments demonstrate that the GRP method significantly improves the accuracy of RUL predictions in the later stages of system operation compared to existing methods. When incorporating its output into SMOMA-PPO, we achieve at least a 15% reduction in average RUL without unscheduled replacements (UR), nearly a 10% increase in inspection interval, and an overall decrease in maintenance costs. Importantly, our approach offers a new perspective for addressing multi-objective maintenance planning with sequential constraints, effectively enhancing system reliability and reducing maintenance expenses.
GNN-DT: Graph Neural Network Enhanced Decision Transformer for Efficient Optimization in Dynamic Environments
Authors: Stavros Orfanoudakis, Nanda Kishor Panda, Peter Palensky, Pedro P. Vergara
Date: 2025-02-03 00:00:00
Tasks: Efficient Exploration, Graph Neural Network, Offline RL, Reinforcement Learning (RL)

Reinforcement Learning (RL) methods used for solving real-world optimization problems often involve dynamic state-action spaces, larger scale, and sparse rewards, leading to significant challenges in convergence, scalability, and efficient exploration of the solution space. This study introduces GNN-DT, a novel Decision Transformer (DT) architecture that integrates Graph Neural Network (GNN) embedders with a novel residual connection between input and output tokens crucial for handling dynamic environments. By learning from previously collected trajectories, GNN-DT tackles the sparse rewards limitations of online RL algorithms and delivers high-quality solutions in real-time. We evaluate GNN-DT on the complex electric vehicle (EV) charging optimization problem and prove that its performance is superior and requires significantly fewer training trajectories, thus improving sample efficiency compared to existing DT and offline RL baselines. Furthermore, GNN-DT exhibits robust generalization to unseen environments and larger action spaces, addressing a critical gap in prior offline and online RL approaches.
Towards Data-Driven Multi-Stage OPF
Authors: Oleksii Molodchyk, Philipp Schmitz, Alexander Engelmann, Karl Worthmann, Timm Faulwasser
Date: 2025-02-03 00:00:00

The operation of large-scale power systems is usually scheduled ahead via numerical optimization. However, this requires models of grid topology, line parameters, and bus specifications. Classic approaches first identify the network topology, i.e., the graph of interconnections and the associated impedances. The power generation schedules are then computed by solving a multi-stage optimal power flow (OPF) problem built around the model. In this paper, we explore the prospect of data-driven approaches to multi-stage optimal power flow. Specifically, we leverage recent findings from systems and control to bypass the identification step and to construct the optimization problem directly from data. We illustrate the performance of our method on a 118-bus system and compare it with the classical identification-based approach.
HingePlace: Harnessing the neural thresholding behavior to optimize Transcranial Electrical Stimulation
Authors: Chaitanya Goswami, Pulkit Grover
Date: 2025-02-03 00:00:00

Transcranial Electrical Stimulation (tES) is a neuromodulation technique that utilizes electrodes on the scalp to stimulate target brain regions. tES has shown promise in treating many neurological conditions, such as stroke rehabilitation and chronic pain. Several electrode placement algorithms have been proposed to optimize tES-based therapies by designing multi-electrode montages that create focal neural responses. We first extend a well-known unification result by Fernandez-Corazza et al. to unify all major traditional electrode placement algorithms. We utilize this unification result to identify a common restriction among traditional electrode placement algorithms: they do not harness the thresholding behavior of neural response. Consequently, these algorithms only partially harness the properties of neural response to optimize tES, particularly increasing the focality of neural response. We propose a new electrode placement algorithm, HingePlace, that utilizes a symmetrized hinge loss to harness the thresholding behavior of neural response. We extensively compare the HingePlace algorithm with traditional electrode placement algorithms in two simulation platforms. Across both platforms, we find that HingePlace-designed montages consistently generate more focal neural responses -- by as much as 60% -- than the electrode montages designed by traditional electrode placement algorithms.
Trajectory Map-Matching in Urban Road Networks Based on RSS Measurements
Authors: Zheng Xing, Weibing Zhao
Date: 2025-02-03 00:00:00

This paper proposes an RSS-based approach to reconstruct vehicle trajectories within a road network, enforcing signal propagation rules and vehicle mobility constraints to mitigate the impact of RSS noise and sparsity. The key challenge lies in leveraging latent spatiotemporal correlations within RSS data while navigating complex road networks. To address this, we develop a Hidden Markov Model (HMM)-based RSS embedding (HRE) technique that employs alternating optimization to infer vehicle trajectories from RSS measurements. This model captures spatiotemporal dependencies while a road graph ensures network compliance. Additionally, we introduce a maximum speed-constrained rough trajectory estimation (MSR) method to guide the optimization process, enabling rapid convergence to a favorable local solution.
C codegen considered unnecessary: go directly to binary, do not pass C. Compilation of Julia code for deployment in model-based engineering
Authors: Fredrik Bagge Carlson, Cody Tapscott, Gabriel Baraldi, Chris Rackauckas
Date: 2025-02-03 00:00:00
Tasks: C++ code, Code Generation, State Estimation

Since time immemorial an old adage has always seemed to ring true: you cannot use a high-level productive programming language like Python or R for real-time control and embedded-systems programming, you must rewrite your program in C. We present a counterexample to this mantra by demonstrating how recent compiler developments in the Julia programming language allow users of Julia and the equation-based modeling language ModelingToolkit to compile and deploy binaries for real-time model-based estimation and control. Contrary to the approach taken by a majority of modeling and simulation tools, we do not generate C code, and instead demonstrate how we may use the native Julia code-generation pipeline through LLVM to compile architecture-specific binaries from high-level code. This approach avoids many of the restrictions typically placed on high-level languages to enable C-code generation. As case studies, we include a nonlinear state estimator derived from an equation-based model which is compiled into a program that performs state estimation for deployment onto a Raspberry Pi, as well as a PID controller library implemented in Julia and compiled into a shared library callable from a C program.
Model Order Reduction from Data with Certification
Authors: Behrad Samari, Amy Nejati, Abolfazl Lavaei
Date: 2025-02-03 00:00:00
Tasks: model

Model order reduction (MOR) involves offering low-dimensional models that effectively approximate the behavior of complex high-order systems. Due to potential model complexities and computational costs, designing controllers for high-dimensional systems with complex behaviors can be challenging, rendering MOR a practical alternative to achieve results that closely resemble those of the original complex systems. To construct such effective reduced-order models (ROMs), existing literature generally necessitates precise knowledge of original systems, which is often unavailable in real-world scenarios. This paper introduces a data-driven scheme to construct ROMs of dynamical systems with unknown mathematical models. Our methodology leverages data and establishes similarity relations between output trajectories of unknown systems and their data-driven ROMs via the notion of simulation functions (SFs), capable of formally quantifying their closeness. To achieve this, under a rank condition readily fulfillable using data, we collect only two input-state trajectories from unknown systems to construct both ROMs and SFs, while offering correctness guarantees. We demonstrate that the proposed ROMs derived from data can be leveraged for controller synthesis endeavors while effectively ensuring high-level logic properties over unknown dynamical models. We showcase our data-driven findings across a range of benchmark scenarios involving various unknown physical systems, demonstrating the enforcement of diverse complex properties.
Nonlinear receding-horizon differential game for drone racing along a three-dimensional path
Authors: Kijin Sung, Kenta Hoshino, Akihiko Honda, Takeya Shima, Toshiyuki Ohtsuka
Date: 2025-02-03 00:00:00
Tasks: Model Predictive Control

Drone racing involves high-speed navigation of three-dimensional paths, posing a substantial challenge in control engineering. This study presents a game-theoretic control framework, the nonlinear receding-horizon differential game (NRHDG), designed for competitive drone racing. NRHDG enhances robustness in adversarial settings by predicting and countering an opponent's worst-case behavior in real time. It extends standard nonlinear model predictive control (NMPC), which otherwise assumes a fixed opponent model. First, we develop a novel path-following formulation based on projection point dynamics, eliminating the need for costly distance minimization. Second, we propose a potential function that allows each drone to switch between overtaking and obstructing maneuvers based on real-time race situations. Third, we establish a new performance metric to evaluate NRHDG with NMPC under race scenarios. Simulation results demonstrate that NRHDG outperforms NMPC in terms of both overtaking efficiency and obstructing capabilities.
Learning the Integral Quadratic Constraints on Plant-Model Mismatch
Authors: Wentao Tang
Date: 2025-02-03 00:00:00
Tasks: Learning Theory

While a characterization of plant-model mismatch is necessary for robust control, the mismatch usually can not be described accurately due to the lack of knowledge about the plant model or the complexity of nonlinear plants. Hence, this paper considers this problem in a data-driven way, where the mismatch is captured by parametric forms of integral quadratic constraints (IQCs) and the parameters contained in the IQC equalities are learned from sampled trajectories from the plant. To this end, a one-class support vector machine (OC-SVM) formulation is proposed, and its generalization performance is analyzed based on the statistical learning theory. The proposed approach is demonstrated by a single-input-single-output time delay mismatch and a nonlinear two-phase reactor with a linear nominal model, showing accurate recovery of frequency-domain uncertainties.
An MDP Model for Censoring in Harvesting Sensors: Optimal and Approximated Solutions
Authors: Jesus Fernandez-Bes, Jesus Cid-Sueiro, Antonio G. Marques
Date: 2025-02-02 00:00:00
Tasks: Q-Learning

In this paper, we propose a novel censoring policy for energy-efficient transmissions in energy-harvesting sensors. The problem is formulated as an infinite-horizon Markov Decision Process (MDP). The objective to be optimized is the expected sum of the importance (utility) of all transmitted messages. Assuming that such importance can be evaluated at the transmitting node, we show that, under certain conditions on the battery model, the optimal censoring policy is a threshold function on the importance value. Specifically, messages are transmitted only if their importance is above a threshold whose value depends on the battery level. Exploiting this property, we propose a model-based stochastic scheme that approximates the optimal solution, with less computational complexity and faster convergence speed than a conventional Q-learning algorithm. Numerical experiments in single-hop and multi-hop networks confirm the analytical advantages of the proposed scheme.
IEEEICM25: "Stability of Digital Robust Motion Control Systems with Disturbance Observer"
Authors: Emre Sariyildiz
Date: 2025-02-02 00:00:00

In this paper, new stability analysis methods are proposed for digital robust motion control systems implemented using a disturbance observer.
Optimal Sensor Placement in Power Transformers Using Physics-Informed Neural Networks
Authors: Sirui Li, Federica Bragone, Matthieu Barreau, Tor Laneryd, Kateryna Morozovska
Date: 2025-02-01 00:00:00

Our work aims at simulating and predicting the temperature conditions inside a power transformer using Physics-Informed Neural Networks (PINNs). The predictions obtained are then used to determine the optimal placement for temperature sensors inside the transformer under the constraint of a limited number of sensors, enabling efficient performance monitoring. The method consists of combining PINNs with Mixed Integer Optimization Programming to obtain the optimal temperature reconstruction inside the transformer. First, we extend our PINN model for the thermal modeling of power transformers to solve the heat diffusion equation from 1D to 2D space. Finally, we construct an optimal sensor placement model inside the transformer that can be applied to problems in 1D and 2D.
Enhancing Field-Oriented Control of Electric Drives with Tiny Neural Network Optimized for Micro-controllers
Authors: Martin Joel Mouk Elele, Danilo Pau, Shixin Zhuang, Tullio Facchinetti
Date: 2025-02-01 00:00:00
Tasks: Quantization

The deployment of neural networks on resource-constrained micro-controllers has gained momentum, driving many advancements in Tiny Neural Networks. This paper introduces a tiny feed-forward neural network, TinyFC, integrated into the Field-Oriented Control (FOC) of Permanent Magnet Synchronous Motors (PMSMs). Proportional-Integral (PI) controllers are widely used in FOC for their simplicity, although their limitations in handling nonlinear dynamics hinder precision. To address this issue, a lightweight 1,400 parameters TinyFC was devised to enhance the FOC performance while fitting into the computational and memory constraints of a micro-controller. Advanced optimization techniques, including pruning, hyperparameter tuning, and quantization to 8-bit integers, were applied to reduce the model's footprint while preserving the network effectiveness. Simulation results show the proposed approach significantly reduced overshoot by up to 87.5%, with the pruned model achieving complete overshoot elimination, highlighting the potential of tiny neural networks in real-time motor control applications.
Covariance Analysis of Attitude and Angular Rate Estimation using Accelerometers
Authors: Koya Yamamoto, Patrick Kelly, Manoranjan Majji, Felipe Guzman
Date: 2025-02-01 00:00:00

In this work a method for using accelerometers for the determination of angular velocity and acceleration is presented. Minimum sensor requirements and insights into how an array of accelerometers can be configured to maximize estimator performance are considered. The framework presented utilizes linear least squares to estimate functions that are quadratic in angular velocity. Simple methods for determining the sign of the spin axis and the linearized covariance approximation are presented and found to perform quite effectively when compared to results obtained by Monte Carlo.
Model-Free Predictive Control: Introductory Algebraic Calculations, and a Comparison with HEOL and ANNs
Authors: Cédric Join, Emmanuel Delaleau, Michel Fliess
Date: 2025-02-01 00:00:00
Tasks: Model Predictive Control, Reinforcement Learning (RL)

Model predictive control (MPC) is a popular control engineering practice, but requires a sound knowledge of the model. Model-free predictive control (MFPC), a burning issue today, also related to reinforcement learning (RL) in AI, is reformulated here via a linear differential equation with constant coefficients, thanks to a new perspective on optimal control combined with recent advances in the field of model-free control (MFC). It is replacing Dynamic Programming, the Hamilton-Jacobi-Bellman equation, and Pontryagin's Maximum Principle. The computing burden is low. The implementation is straightforward. Two nonlinear examples, a chemical reactor and a two tank system, are illustrating our approach. A comparison with the HEOL setting, where some expertise of the process model is needed, shows only a slight superiority of the later. A recent identification of the two tank system via a complex ANN architecture might indicate that a full modeling and the corresponding machine learning mechanism are not always necessary neither in control, nor, more generally, in AI.
Provably-Stable Neural Network-Based Control of Nonlinear Systems
Authors: Anran Li, John P. Swensen, Mehdi Hosseinzadeh
Date: 2025-02-01 00:00:00

In recent years, Neural Networks (NNs) have been employed to control nonlinear systems due to their potential capability in dealing with situations that might be difficult for conventional nonlinear control schemes. However, to the best of our knowledge, the current literature on NN-based control lacks theoretical guarantees for stability and tracking performance. This precludes the application of NN-based control schemes to systems where stringent stability and performance guarantees are required. To address this gap, this paper proposes a systematic and comprehensive methodology to design provably-stable NN-based control schemes for affine nonlinear systems. Rigorous analysis is provided to show that the proposed approach guarantees stability of the closed-loop system with the NN in the loop. Also, it is shown that the resulting NN-based control scheme ensures that system states asymptotically converge to a neighborhood around the desired equilibrium point, with a tunable proximity threshold. The proposed methodology is validated and evaluated via simulation studies on an inverted pendulum and experimental studies on a Parrot Bebop 2 drone.
Impulsive Relative Motion Control with Continuous-Time Constraint Satisfaction for Cislunar Space Missions
Authors: Fabio Spada, Purnanand Elango, Behçet Açıkmeşe
Date: 2025-01-31 00:00:00
Tasks: CPU

Recent investments in cislunar applications open new frontiers for space missions within highly nonlinear dynamical regimes. In this paper, we propose a method based on Sequential Convex Programming (SCP) to loiter around a given target with impulsive actuation while satisfying path constraints continuously over the finite time-horizon, i.e., independently of the number of nodes in which domain is discretized. Location, timing, magnitude, and direction of a fixed number of impulses are optimized in a model predictive framework, exploiting the exact nonlinear dynamics of non-stationary orbital regimes. The proposed approach is first validated on a relative orbiting problem with respect to a selenocentric near rectilinear halo orbit. The approach is then compared to a formulation with path constraints imposed only at nodes and with mesh refined to ensure complete satisfaction of path constraints over the continuous-time horizon. CPU time per iteration of 400 ms for the refined-mesh approach reduce to 5.5 ms for the proposed approach.