Automated Coordination of Distributed Energy Resources using Local Energy Markets and Reinforcement Learning
Creators: May, Daniel Christopher
Publisher: name: University of Alberta Library
Year: 2024

<p>The conventional unidirectional model of the electricity grid operations is no longer sufficient. The continued proliferation of distributed energy resources and the resultant surge in net load variability at the grid edge necessitates deploying adequate demand response methods.<br>
This thesis proposes, investigates, and demonstrates the Autonomous Local Energy eXchange (ALEX), an indirect demand response mechanism grounded in the principles of transactive energy. ALEX operates as a fully automated, decentralized, and economy-driven local energy market with the overarching objective of reducing net load variability on a community level to enhance grid operability.<br>
ALEX strongly distinguishes itself from schedule-based approaches commonly utilized for indirect demand response in how it addresses the challenges of interest alignment and end-user participation.<br>
The alignment of end-user and grid stakeholder interests is achieved through the market mechanism, which incentivizes pricing in relation to the current timestep’s supply/demand ratio.<br>
To facilitate broad end-user participation in the face of such a granular incentive signal, ALEX relies on model-free automation through deep reinforcement learning.<br>
The thesis employs a reductionist approach to navigate the complex dynamics of this interconnected system.<br>
It formulates three primary research goals, addressed through corresponding chapters.</p>

<p>Chapter 2 explores the challenges of economy-driven transactive energy, focusing on designing an appropriate local energy market mechanism.<br>
Through classification driven experiments, a market mechanism is identified that strongly incentivizes pricing in relation to the supply/demand ratio.<br>
This provides an effective solution to the alignment problem between grid stakeholders and electricity end-users.</p>

<p>Chapter 3 develops a benchmarking approach for local energy markets, confirming the central hypothesis of emergent, community-level variability reduction within ALEX.<br>
ALEX significantly outperforms baseline approaches, demonstrating its capability to enable community-wide coordination of distributed energy resources.<br>
The benchmarking process addresses broader research gaps in the current literature related to Local Energy Markets.</p>

<p>Chapter 4 concludes the thesis by training deep reinforcement learning agents to achieve near-optimal performance on ALEX.<br>
An augmented proximal policy optimization algorithm demonstrates the ability to produce a convergent set of policies close to a Nash equilibrium.<br>
The resulting policies reduce community-level variability across several timescales without information sharing between agents and without access to future information.</p>

<p>In summary, this thesis advances the state of the art in indirect demand response by introducing and demonstrating ALEX.<br>
ALEX’s decentralized, autonomous nature positions it as a robust solution to the challenges posed by the growing adoption of distributed energy resources, aligning with the Smart Grid’s principles of intelligent asset integration for efficient and reliable grid operations.</p>