pwc-archive/papers-with-abstracts huggingface dataset:
Economic Model Predictive Control as a Solution to Markov Decision Processes
Authors: Dirk Reinhardt, Akhil S. Anand, Shambhuraj Sawant, Sebastien Gros
Date: 2024-07-23 00:00:00
Tasks: Model Predictive Control

Markov Decision Processes (MDPs) offer a fairly generic and powerful framework to discuss the notion of optimal policies for dynamic systems, in particular when the dynamics are stochastic. However, computing the optimal policy of an MDP can be very difficult due to the curse of dimensionality present in solving the underlying Bellman equations. Model Predictive Control (MPC) is a very popular technique for building control policies for complex dynamic systems. Historically, MPC has focused on constraint satisfaction and steering dynamic systems towards a user-defined reference. More recently, Economic MPC was proposed as a computationally tractable way of building optimal policies for dynamic systems. When stochsaticity is present, economic MPC is close to the MDP framework. In that context, Economic MPC can be construed as attractable heuristic to provide approximate solutions to MDPs. However, there is arguably a knowledge gap in the literature regarding these approximate solutions and the conditions for an MPC scheme to achieve closed-loop optimality. This chapter aims to clarify this approximation pedagogically, to provide the conditions for MPC to deliver optimal policies, and to explore some of their consequences.
Data-Driven Domestic Flexible Demand: Observations from experiments in cold climate
Authors: Dirk Reinhardt, WenQi Cai, Sebastien Gros
Date: 2024-07-23 00:00:00

In this chapter, we report on our experience with domestic flexible electric energy demand based on a regular commercial (HVAC)-based heating system in a house. Our focus is on investigating the predictability of the energy demand of the heating system and of the thermal response when varying the heating system settings. Being able to form such predictions is crucial for most flexible demand algorithms. We will compare several methods for predicting the thermal and energy response, which either gave good results or which are currently promoted in the literature for controlling buildings. We will report that the stochasticity of a house response is -- in our experience -- the main difficulty in providing domestic flexible demand from heating. The experiments were carried out on a regular house in Norway, equipped with four air-to-air Mitsubishi heat pumps and a high-efficiency balanced ventilation system. The house was equipped with multiple IoT-based climate sensors, real-time power measurement, and the possibility to drive the HVAC system via the IoT. The house is operating on the spot market (Nord Pool NO3) and is exposed to a peak energy demand penalty. Over a period of three years, we have collected data on the house (temperatures, humidity, air quality), real-time power and hourly energy consumption, while applying various flexible demand algorithms responding to the local energy costs. This has produced large variations in the settings of the heating system and energy demand, resulting in rich data for investigating the house response. This chapter aims at providing important insights on providing flexible demand from houses in cold climates.
Evaluating Uncertainties in Electricity Markets via Machine Learning and Quantum Computing
Authors: Shuyang Zhu, Ziqing Zhu, Linghua Zhu, Yujian Ye, Siqi Bu, Sasa Z. Djokic
Date: 2024-07-23 00:00:00
Tasks: Computational Efficiency, Decision Making, Multi-agent Reinforcement Learning

The analysis of decision-making process in electricity markets is crucial for understanding and resolving issues related to market manipulation and reduced social welfare. Traditional Multi-Agent Reinforcement Learning (MARL) method can model decision-making of generation companies (GENCOs), but faces challenges due to uncertainties in policy functions, reward functions, and inter-agent interactions. Quantum computing offers a promising solution to resolve these uncertainties, and this paper introduces the Quantum Multi-Agent Deep Q-Network (Q-MADQN) method, which integrates variational quantum circuits into the traditional MARL framework. The main contributions of the paper are: identifying the correspondence between market uncertainties and quantum properties, proposing the Q-MADQN algorithm for simulating electricity market bidding, and demonstrating that Q-MADQN allows for a more thorough exploration and simulates more potential bidding strategies of profit-oriented GENCOs, compared to conventional methods, without compromising computational efficiency. The proposed method is illustrated on IEEE 30-bus test network, confirming that it offers a more accurate model for simulating complex market dynamics.
Flatness-based motion planning for a non-uniform moving cantilever Euler-Bernoulli beam with a tip-mass
Authors: Soham Chatterjee, Aman Batra, Vivek Natarajan
Date: 2024-07-23 00:00:00
Tasks: Motion Planning

Consider a non-uniform Euler-Bernoulli beam with a tip-mass at one end and a cantilever joint at the other end. The cantilever joint is not fixed and can itself be moved along an axis perpendicular to the beam. The position of the cantilever joint is the control input to the beam. The dynamics of the beam is governed by a coupled PDE-ODE model with boundary input. On a natural state-space, there exists a unique state trajectory for this beam model for every initial state and each smooth control input which is compatible with the initial state. In this paper, we study the motion planning problem of transferring the beam from an initial state to a final state over a prescribed time interval. We address this problem by extending the generating functions approach to flatness-based control, originally proposed in the literature for motion planning of parabolic PDEs, to the beam model. We prove that such a transfer is possible if the initial and final states belong to a certain set, which also contains steady-states of the beam. We illustrate our theoretical results using simulations and experiments.
Addressing Trust Issues for Vehicle to Grid in Distributed Power Grids Using Blockchains
Authors: Yunwang Chen, Xiang Lei, Linni Jian
Date: 2024-07-23 00:00:00

While blockchain offers inherent security, trust issues among stakeholders in vehicle-to-grid (V2G) applications remain unresolved due to a lack of regulatory frameworks and standardization. Additionally, a tailored decentralized privacy-preserved coordination scheme for blockchain in V2G networks is needed to ensure user privacy and efficient energy transactions. This paper proposes a V2G trading and coordination scheme tailored to the decentralized nature of blockchain as well as the interests of stakeholders utilizing smart charging points (SCPs) and Stackelberg game model. Case studies using real-world data from Southern University of Science and Technology demonstrate the efficacy of proposed scheme in reducing EV charging costs and the potential for supporting auxiliary grid services.
Koopman Analytical Modeling of Position and Attitude Dynamics: a Case Study for Quadrotor Control
Authors: Simone Martini, Kimon P. Valavanis, Margareta Stefanovic
Date: 2024-07-23 00:00:00
Tasks: Position

This research presents a novel, analytical, Koopman Operator based formulation for position and attitude dynamics which can be used to derive control strategies for underactuated systems. Compared to data driven Koopman based techniques, the analytical approach presented in this work is model based and allows for an exact linear representation of the original nonlinear position and attitude dynamics. In fact, the resulting infinite dimensional model, defined in the lifted state space, is linear in the autonomous component and state dependent in the control. A boundary study is carried on to define the range of validity of the finite truncation of the Koopman based model followed by a controllability and stabilizability analysis to show the feasibility of employing the derived model for control system design. Compared to existing literature formulation, the presented model results in a better approximation of the original dyanmics using a more compact truncation of the lifted state space. Moreover, the model is derived using the Koopman approach on the entirety of the dynamics and does not require the need of angular velocity dynamic compensation. A case study involving an underactuated quadrotor unmanned aerial vehicle (UAV) is provided to show that, for practical use, a truncated subset of the infinite dimensional model, embeds most of the original nonlinear dynamics and can be used to design linear control strategies in the lifted space which results in nonlinear controllers in the original state space. The main advantages of the presented approach reside in the effective use of linear control strategies for nonlinear plats and the solution of the underactuation problem employing a single control loop.
Online Reduced-Order Data-Enabled Predictive Control
Authors: Amin Vahidi-Moghaddam, Kaixiang Zhang, Xunyuan Yin, Vaibhav Srivastava, Zhaojian Li
Date: 2024-07-22 00:00:00
Tasks: Data Driven Optimal Control

Data-enabled predictive control (DeePC) has garnered significant attention for its ability to achieve safe, data-driven optimal control without relying on explicit system models. Traditional DeePC methods use pre-collected input/output (I/O) data to construct Hankel matrices for online predictive control. However, in systems with evolving dynamics or insufficient pre-collected data, incorporating real-time data into the DeePC framework becomes crucial to enhance control performance. This paper proposes an online DeePC framework for time-varying systems (i.e., systems with evolving dynamics), enabling the algorithm to update the Hankel matrix online by adding real-time informative signals. By exploiting the minimum non-zero singular value of the Hankel matrix, the developed online DeePC selectively integrates informative data and effectively captures evolving system dynamics. Additionally, a numerical singular value decomposition technique is introduced to reduce the computational complexity for updating a reduced-order Hankel matrix. Simulation results on two cases, a linear time-varying system and the vehicle anti-rollover control, demonstrate the effectiveness of the proposed online reduced-order DeePC framework.
Efficient Replay Memory Architectures in Multi-Agent Reinforcement Learning for Traffic Congestion Control
Authors: Mukul Chodhary, Kevin Octavian, SooJean Han
Date: 2024-07-22 00:00:00
Tasks: Decision Making, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Scheduling

Episodic control, inspired by the role of episodic memory in the human brain, has been shown to improve the sample inefficiency of model-free reinforcement learning by reusing high-return past experiences. However, the memory growth of episodic control is undesirable in large-scale multi-agent problems such as vehicle traffic management. This paper proposes a novel replay memory architecture called Dual-Memory Integrated Learning, to augment to multi-agent reinforcement learning methods for congestion control via adaptive light signal scheduling. Our dual-memory architecture mimics two core capabilities of human decision-making. First, it relies on diverse types of memory--semantic and episodic, short-term and long-term--in order to remember high-return states that occur often in the network and filter out states that don't. Second, it employs equivalence classes to group together similar state-action pairs and that can be controlled using the same action (i.e., light signal sequence). Theoretical analyses establish memory growth bounds, and simulation experiments on several intersection networks showcase improved congestion performance (e.g., vehicle throughput) from our method.
An RF-Domain Leakage Cancellation Scheme for FMCW Radars
Authors: Yikuan Chen, Ali Niknejad
Date: 2024-07-22 00:00:00

This paper proposes a novel solution to address the leakage from the transmitter (TX) to the receiver (RX) in frequency-modulated continuous-wave (FMCW) radars. The proposed scheme replicates the leakage using an in-phase and quadrature mixer (IQ-mixer) and performs leakage cancellation in the radio-frequency (RF) domain. This approach utilizes a Wilkinson power combiner after the RX antenna to subtract the replicated leakage signal from the received signal, ensuring that only the true target signal reaches the low-noise amplifier (LNA). This scheme enhances the dynamic range and the receiver's ability to discern proximate targets from previously indistinguishable low beat-frequency clutter. In addition, the proposed technique incorporates a second IQ-mixer based complex modulator in the transmitter to tune the leakage beat frequency. This allows for accurate estimation of the leakage amplitude and phase without additional hardware. Simulation results show more than 20 dB of leakage cancellation.
Cell shape and orientation control galvanotactic accuracy
Authors: Ifunanya Nwogbaga, Brian A. Camley
Date: 2024-07-24 00:00:00

Galvanotaxis is believed to be driven by the redistribution of transmembrane proteins and other molecules, referred to as "sensors", through electrophoresis and electroosmosis. Here, we update our previous model of the limits of galvanotaxis due to stochasticity of sensor movements to account for cell shape and orientation. Computing the Fisher information, we find that cells in principle possess more information about the electric field direction when their long axis is parallel to the field, but that for weak fields maximum-likelihood estimators of the field direction may actually have lower variability when the cell's long axis is perpendicular to the field. In an alternate possibility, we find that if cells instead estimate the field direction by taking the average of all the sensor locations as its directional cue ("vector sum"), this introduces a bias towards the short axis, an effect not present for isotropic cells. We also explore the possibility that cell elongation arises downstream of sensor redistribution. We argue that if sensors migrate to the cell's rear, the cell will expand perpendicular the field - as is more commonly observed - but if sensors migrate to the front, the cell will elongate parallel to the field.
Unraveling Molecular Structure: A Multimodal Spectroscopic Dataset for Chemistry
Authors: Marvin Alberts, Oliver Schilter, Federico Zipoli, Nina Hartrampf, Teodoro Laino
Date: 2024-07-04 00:00:00

Spectroscopic techniques are essential tools for determining the structure of molecules. Different spectroscopic techniques, such as Nuclear magnetic resonance (NMR), Infrared spectroscopy, and Mass Spectrometry, provide insight into the molecular structure, including the presence or absence of functional groups. Chemists leverage the complementary nature of the different methods to their advantage. However, the lack of a comprehensive multimodal dataset, containing spectra from a variety of spectroscopic techniques, has limited machine-learning approaches mostly to single-modality tasks for predicting molecular structures from spectra. Here we introduce a dataset comprising simulated $^1$H-NMR, $^{13}$C-NMR, HSQC-NMR, Infrared, and Mass spectra (positive and negative ion modes) for 790k molecules extracted from chemical reactions in patent data. This dataset enables the development of foundation models for integrating information from multiple spectroscopic modalities, emulating the approach employed by human experts. Additionally, we provide benchmarks for evaluating single-modality tasks such as structure elucidation, predicting the spectra for a target molecule, and functional group predictions. This dataset has the potential automate structure elucidation, streamlining the molecular discovery pipeline from synthesis to structure determination. The dataset and code for the benchmarks can be found at https://rxn4chemistry.github.io/multimodal-spectroscopic-dataset.
A reinforcement learning strategy to automate and accelerate h/p-multigrid solvers
Authors: David Huergo, Laura Alonso, Saumitra Joshi, Adrian Juanicoteca, Gonzalo Rubio, Esteban Ferrer
Date: 2024-07-18 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning

We explore a reinforcement learning strategy to automate and accelerate h/p-multigrid methods in high-order solvers. Multigrid methods are very efficient but require fine-tuning of numerical parameters, such as the number of smoothing sweeps per level and the correction fraction (i.e., proportion of the corrected solution that is transferred from a coarser grid to a finer grid). The objective of this paper is to use a proximal policy optimization algorithm to automatically tune the multigrid parameters and, by doing so, improve stability and efficiency of the h/p-multigrid strategy. Our findings reveal that the proposed reinforcement learning h/p-multigrid approach significantly accelerates and improves the robustness of steady-state simulations for one dimensional advection-diffusion and nonlinear Burgers' equations, when discretized using high-order h/p methods, on uniform and nonuniform grids.
Starting Small: Prioritizing Safety over Efficacy in Randomized Experiments Using the Exact Finite Sample Likelihood
Authors: Neil Christy, A. E. Kowalski
Date: 2024-07-25 00:00:00

We use the exact finite sample likelihood and statistical decision theory to answer questions of ``why?'' and ``what should you have done?'' using data from randomized experiments and a utility function that prioritizes safety over efficacy. We propose a finite sample Bayesian decision rule and a finite sample maximum likelihood decision rule. We show that in finite samples from 2 to 50, it is possible for these rules to achieve better performance according to established maximin and maximum regret criteria than a rule based on the Boole-Frechet-Hoeffding bounds. We also propose a finite sample maximum likelihood criterion. We apply our rules and criterion to an actual clinical trial that yielded a promising estimate of efficacy, and our results point to safety as a reason for why results were mixed in subsequent trials.
Enhanced power enhancements for testing many moment equalities: Beyond the $2$- and $\infty$-norm
Authors: Anders Bredahl Kock, David Preinerstorfer
Date: 2024-07-25 00:00:00

Contemporary testing problems in statistics are increasingly complex, i.e., high-dimensional. Tests based on the $2$- and $\infty$-norm have received considerable attention in such settings, as they are powerful against dense and sparse alternatives, respectively. The power enhancement principle of Fan et al. (2015) combines these two norms to construct improved tests that are powerful against both types of alternatives. In the context of testing whether a candidate parameter satisfies a large number of moment equalities, we construct a test that harnesses the strength of all $p$-norms with $p\in[2, \infty]$. As a result, this test is consistent against strictly more alternatives than any test based on a single $p$-norm. In particular, our test is consistent against more alternatives than tests based on the $2$- and $\infty$-norm, which is what most implementations of the power enhancement principle target. We illustrate the scope of our general results by using them to construct a test that simultaneously dominates the Anderson-Rubin test (based on $p=2$), tests based on the $\infty$-norm and power enhancement based combinations of these in terms of consistency in the linear instrumental variable model with many instruments.
Identification and inference of outcome conditioned partial effects of general interventions
Authors: Zhengyu Zhang, Zequn Jin, Lihua Lin
Date: 2024-07-24 00:00:00
Tasks: counterfactual

This paper proposes a new class of distributional causal quantities, referred to as the \textit{outcome conditioned partial policy effects} (OCPPEs), to measure the \textit{average} effect of a general counterfactual intervention of a target covariate on the individuals in different quantile ranges of the outcome distribution. The OCPPE approach is valuable in several aspects: (i) Unlike the unconditional quantile partial effect (UQPE) that is not $\sqrt{n}$-estimable, an OCPPE is $\sqrt{n}$-estimable. Analysts can use it to capture heterogeneity across the unconditional distribution of $Y$ as well as obtain accurate estimation of the aggregated effect at the upper and lower tails of $Y$. (ii) The semiparametric efficiency bound for an OCPPE is explicitly derived. (iii) We propose an efficient debiased estimator for OCPPE, and provide feasible uniform inference procedures for the OCPPE process. (iv) The efficient doubly robust score for an OCPPE can be used to optimize infinitesimal nudges to a continuous treatment by maximizing a quantile specific Empirical Welfare function. We illustrate the method by analyzing how anti-smoking policies impact low percentiles of live infants' birthweights.
Bayesian modelling of VAR precision matrices using stochastic block networks
Authors: Florian Huber, Gary Koop, Massimiliano Marcellino, Tobias Scheckel
Date: 2024-07-23 00:00:00
Tasks: Stochastic Block Model

Commonly used priors for Vector Autoregressions (VARs) induce shrinkage on the autoregressive coefficients. Introducing shrinkage on the error covariance matrix is sometimes done but, in the vast majority of cases, without considering the network structure of the shocks and by placing the prior on the lower Cholesky factor of the precision matrix. In this paper, we propose a prior on the VAR error precision matrix directly. Our prior, which resembles a standard spike and slab prior, models variable inclusion probabilities through a stochastic block model that clusters shocks into groups. Within groups, the probability of having relations across group members is higher (inducing less sparsity) whereas relations across groups imply a lower probability that members of each group are conditionally related. We show in simulations that our approach recovers the true network structure well. Using a US macroeconomic data set, we illustrate how our approach can be used to cluster shocks together and that this feature leads to improved density forecasts.
Simulation in discrete choice models evaluation: SDCM, a simulation tool for performance evaluation of DCMs
Authors: Amirreza Talebi
Date: 2024-07-24 00:00:00
Tasks: Decision Making, Discrete Choice Models, Experimental Design

Discrete choice models (DCMs) have been widely utilized in various scientific fields, especially economics, for many years. These models consider a stochastic environment influencing each decision maker's choices. Extensive research has shown that the agents' socioeconomic characteristics, the chosen options' properties, and the conditions characterizing the decision-making environment all impact these models. However, the complex interactions between these factors, confidentiality concerns, time constraints, and costs, have made real experimentation impractical and undesirable. To address this, simulations have gained significant popularity among academics, allowing the study of these models in a controlled setting using simulated data. This paper presents multidisciplinary research to bridge the gap between DCMs, experimental design, and simulation. By reviewing related literature, the authors explore these interconnected areas. We then introduce a simulation method integrated with experimental design to generate synthetic data based on behavioral models of agents. A utility function is used to describe the developed simulation tool. The paper investigates the discrepancy between simulated data and real-world data.