# Microgrid secure pricing and real-time flexible energy market integrated with trust-based demand response: a multi-layer FDRL architecture

Reza Sepehrzad, Hossein Hosseinalibeiki, Mahdieh S Sadabadi, Ahmed Al Durra

► To cite this version:

Reza Sepehrzad, Hossein Hosseinalibeiki, Mahdieh S Sadabadi, Ahmed Al Durra. Microgrid secure pricing and real-time flexible energy market integrated with trust-based demand response: a multi-layer FDRL architecture. International Journal of Electrical Power & Energy Systems, 2026, 174, 10.1016/j.ijepes.2025.111536. hal-05459222

HAL Id: hal-05459222

https://hal.science/hal-05459222v1

Submitted on 15 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

# Microgrid secure pricing and real-time flexible energy market integrated with trust-based demand response: a multi-layer FDRL architecture

Reza Sepehrzad<sup>a, *, †</sup>, Hossein Hosseinalibeiki<sup>b, †</sup>, Mahdieh S. Sadabadi<sup>c, †</sup>, Ahmed al Durra<sup>d</sup>

## ARTICLE INFO

### Keywords:

Cyber-resilient control framework

Demand response

Federated deep reinforcement learning

Microgrid

Real-time flexible energy markets

Secure pricing

## ABSTRACT

This article presents a novel cyber-resilient control strategy for enabling secure, real-time pricing in flexible energy markets of microgrids. As the energy infrastructure gets more digitally connected and more integrated with distributed energy resources, microgrids face increased challenges to their capacity to preserve pricing integrity, system trust, and operational resilience in the face of evolving cyber threats. To address these issues, this paper developed a federated deep reinforcement learning model based on the soft actor-critic algorithm. The system includes long short-term memory-based anomaly detection for early cyberattack detection, trust-aware demand response (DR) functionality to manage participation incentives, and a privacy-preserving coordination scheme to enable decentralized learning without raw data exchange. This multi-layered configuration supports local agents in making optimal joint pricing and control decisions and being responsive to false data injection, denial of service, and hybrid cyberattack scenarios. Simulations in IEEE 33-bus, 69-bus, and 141-bus power systems corroborate that the introduced model maximizes social welfare by up to 11%, reduces coordination failures by over 50%, and maintains DR compliance over 92%. The price volatility index is below 0.02 under attack. These results demonstrate the framework's capability of enhancing economic stability, cyberattack resilience, and scalable coordination in secure energy market settings.

## 1. Introduction and motivation

The digital transformation of power systems has drastically altered the role of microgrids (MGs) in the new energy infrastructure [1]. Microgrids have changed from being independent backup systems to fully interactive, decentralized energy systems [2] with the advent of distributed energy resources (DERs), intelligent edge controllers and real-time market connections. This evolution has brought about the development of real-time flexible energy markets at the microgrid level, where prosumers are engaged dynamically in pricing, control, and demand response functions [3]. In these markets, the development of secure and responsive pricing mechanisms is crucial—not only to ensure economic efficiency and fairness but also to enable localized energy balancing amidst high variability and uncertainty [4].

Since microgrids are increasingly digitized and networked, they are also exposed to a broader range of cybersecurity vulnerabilities [5]. Manipulation of control signals, pricing data, or demand response

coordination through cyberattacks such as false data injection (FDI) or denial of service (DoS) can lead to instability, economic losses, and loss of stakeholder trust [6]. Therefore, implementing and systematically assessing cyber-resilient control mechanisms is no longer optional—it is at the heart of ensuring the autonomy, reliability, and scalability of future microgrids [7]. In addition, trust-based demand response (DR) engagement is paramount in this environment because trust measures directly influence user engagement and system-level responsiveness [8].

In addition to cyberattacks, certain other technical and operational challenges further make secure market operation in MGs complicated [9]. Those include decentralized decision-making nature, user heterogeneity and energy asset heterogeneity, communication latency in distributed controllers, and privacy-preserving coordination mechanisms needed [10]. Central bottlenecks-free scalability, high-dimensional action space stability, and adaptation to varying environmental conditions all necessitate sophisticated learning-based control architectures [11]. With no systematically formulated policies capable

<footnote><sup>a</sup> Department of Electrical and Computer Engineering, University of Canterbury, Christchurch, New Zealand</footnote>

<footnote><sup>b</sup> DISA-MIS Department, University of Salerno, Fisciano 84084, Italy</footnote>

<footnote><sup>c</sup> Department of Electrical and Electronic Engineering, University of Manchester, Manchester, United Kingdom</footnote>

<footnote><sup>d</sup> Advanced Power & Energy Center, EECS Department, Khalifa University, Abu Dhabi, the United Arab Emirates</footnote>

<footnote>* Corresponding author.
E-mail address: Sepehrzad.reza@gmail.com (R. Sepehrzad).</footnote>
of addressing these dynamic, hostile, and distributed natures in tandem, security and efficiency of MG markets remain under threat [12].

To mitigate these challenges and promote more efficient MG market operation, this research applies the capabilities of machine learning-based control systems, namely deep reinforcement learning, combined with federated coordination and secure signal processing. The novel approach introduces a state-of-the-art, cyber-aware market control framework tailored to digitized microgrids with integrated secure pricing systems, trust-aware demand response, and fault-tolerant distributed decision-making agents. This enables microgrids to autonomously detect anomalies, maintain price integrity, and respond to cyberattacks and enhance total market transparency and operational effectiveness.

## 1.1. Literature review

Current advancements in real-time pricing architectures and control structures of microgrids have laid some groundwork for dynamic energy management, but the majority of models are yet to be consolidated with secure pricing, cyber-resiliency, and trust-aware demand response. study [13] proposes a floating real-time pricing mechanism in a local energy market governed by a Stackelberg game, which is effective in cost saving and emission reduction. However, the paper takes into account a safe communication environment and neglects cyberattacks and trust issues of users. Ref [14] discusses adversarial manipulation in electricity markets based on demand response, developing optimization-based attacks by malicious microgrids to control market equilibrium. While the framework is constructed based on EPEC is mathematically sound, it is totally attacker-oriented with no detection or resilience strategy, and overlooks DR behavioral impacts. Ref [15] offers an effective finite-time frequency control solution to provide stability in microgrids under false data injection attacks. Although a master of physical-layer resiliency, it is short of market dynamics, price security, and consumer involvement considerations, limiting its application to responsive flexible market situations. These works collectively indicate a lack of unifying frameworks that are capable of managing real-time market responsiveness, cyberattack countermeasures, and trust-driven DR participation together.

Integration of blockchain, federated learning, and fault-tolerant control structures is a promising direction for secure decentralized market operation, yet current approaches isolate technical functions with no end-to-end coordination between pricing, trust, and attack resilience. Ref [16] offers a blockchain-based P2P market based on federated privacy-preserving learning for frequency regulation. While the framework enhances cybersecurity and decentralization, it does not address the interaction between price and trust measures in DR settings. There is a self-supervised GAN-based method for detecting cyberattacks from market-level RTLMP information only presented by research [17]. While it could be as good as it gets for fast anomaly detection, the framework does not include feedback integration with control or demand-side participation, nor provides pricing recovery. Ref [18] addresses resilient trajectory tracking under cyber-attack with barrier-function-based adaptive sliding mode control. Although possessing high resilience at the physical layer, the model lacks secure market pricing, federated learning coordination, or behavioral modeling. Although technologically potent in isolation, these answers emphasize the need for the right integrated models that combine learning, price security, and responsiveness to user behavior in cyber-vulnerable microgrids.

Cyber-physical threats to electricity markets have increased the focus on economic manipulation threats, yet most research concentrates on attacker modeling or decreased detection without accounting for federated resilience, trust integration of DR, or decentralized secure control. Ref [19] investigates market power abuse via FDI by hacked aggregators and proposes a remedial action scheme to re-clear the market. Though efficient in economic fairness recovery, the approach depends on manual exclusion and lacks built-in detection and user-level

trust feedback. Ref [20] presents a stochastic bi-level FDIA model accounting for load uncertainty and partial grid knowledge with the aim of maximizing attacker profit. Though the model embodies realistic attacker constraints, it lacks countermeasures, user protection strategies, or federated trust schemes. Study [21] examines DoS attacks against LEMs and indicates a very high risk even at low bid loss rates. Macro-level proposals such as delayed clearing and market insurance are proposed by the model but without built-in detection, trust modeling, or coordination control. These papers all indicate that current defenses are coarse-grained or reactive, lacking dynamic, distributed resilience and trust-based secure market control as done in this work.

Machine learning-driven detection paradigms and DR-aware optimization have made anomaly detection more accurate and scheduling more robust, but are likely to be distinct from trust-adaptive DR pricing or federated multi-agent systems. Ref [22] proposes a CNN-driven detector for DR system smart meter fraud. While very accurate, the centralized detection paradigm is not integrated with DR incentive strategies or pricing correction mechanisms. Study [23] introduces a multi-agent DRL model with fuzzy systems to safeguard EV charging against FDI and delay attacks with high detection efficiency and low operation cost. The model does not have federated learning and user trust feedback support, which prevents long-term behavioral flexibility. Study [24] introduces an energy management strategy for home communities through LSTM-based price forecasting to detect anomalies and integrated with an aggregate game for DR coordination. While the hybrid approach is semi-decentralized, it does not federate to deployments or include adaptive pricing based on responses to cyber-attacks. These pieces of work emphasize the growing use of ML in market resilience, but also illustrate the absence of single control-pricing-trust structures.

Privacy-preserving and data-driven anomaly detection approaches have enhanced smart grid applications' resilience, though none of them are integrated with trust-based DR participation, federated microgrid control systems, or dynamic pricing integrity. Study [25] introduces a privacy-conscious equilibrium-based microgrid price-setting optimization framework with boundary data exchange enabled. The model preserves local privacy but lacks consideration of cyberattack scenarios or adaptive trust modeling. Study [26] suggests an ensemble LOF technique to FDIA detection in DR systems with high accuracy for the detection of anomalies. It does not, however, operate under a centralized platform and has no price correction, DR incentives, and multi-agent resilience coordination. Although these studies contribute to the domains of privacy and detection, they fail to cross the divide that connects secure pricing, behavior trust, and scalable cyber-aware learning-based coordination, which this research closes with its federated, trust-synthesized DR and pricing framework.

Recent advances in networked microgrid trading mechanisms have emphasized decentralized coordination, economic optimization, and carbon-aware pricing models. A multi-layered market framework was proposed in [27], combining intra-cluster proportional bargaining with inter-cluster marginal pricing, supported by multi-agent reinforcement learning to dynamically determine bid/ask prices. This approach effectively reduced surplus energy dependency on the utility grid and improved cost efficiency through scalable peer-to-peer energy balancing. In a different direction, [28] introduced a game-theoretic pricing scheme for joint electricity and carbon emission rights (CERs) trading among microgrid prosumers. By coupling Stackelberg and noncooperative games and implementing smart contracts on a blockchain, the model ensured price equilibrium and transparent trading, resulting in both economic and environmental benefits. Similarly, [29] focused on internal trade pricing within a network of microgrids, proposing a dual-layer control system where local energy management optimizes individual microgrids, and a central controller facilitates adaptive power sharing using time-responsive prices. This strategy achieved significant reductions in both power loss and operational costs. While these studies demonstrate notable progress in decentralized
optimization and pricing structures, they share a common challenge in ensuring real-time robustness and resilience of pricing and trading logic under cyber or behavioral uncertainty, which remains an open problem for secure, large-scale deployment. Table 1 presents the primary features of the related reviewed papers and the proposed model.

## 1.2. Research gap

In spite of recent advancements in reinforcement learning and distributed power management, existing literature suggests critical gaps in the secure and resilient operation of real-time energy markets in microgrids. Most of the effort is focused on economic optimization or cyber-resilience, but rarely tries to integrate both in one and the same behavior-aware control scheme. Centralized soft actor-critic (SAC) or distributed proximal policy optimization (PPO)-based methods have shown promise in pricing and demand response automation, but are usually lacking active anomaly detection, trust analysis, or federated coordination. As a result, the models remain vulnerable to cyberattacks, collapse under adversarial uncertainty, and are unable to reflect dynamic interaction between price signals and user trust in decentralized networks.

Besides that, there are very few system-level analyses that evaluate concurrently pricing sincerity, DR compliance, trust actions, and cyber-resilience in microgrid environments. Current solutions make optimistic communication assumptions, ignore trust feedback loops, or unrealistically simplify cyberattack representations, which limits their realistic applicability. While federated learning and LSTM-based detection have emerged in adjacent disciplines, their combined implementation for secure pricing and behavior-aware and real-time flexible microgrid market design is not well explored. This void recognizes the need for a combined framework involving secure signal processing, trust-aware DR control, and federated reinforcement learning to ensure operational continuity, data privacy, and immunity against escalating cyber threats.

## 1.3. Main Contribution

As next-generation MGs are decentralized, cyber-physical systems with real-time market interfaces, secure management of dynamic energy pricing, demand-side coordination, and resilience against cyberattacks is increasingly crucial. Despite a number of control and pricing strategies proposed in the recent literature, very few works have succeeded in

integrating secure pricing, trust-aware DR, and cyber-resilient market operation in one learning-based framework. This work bridges this critical gap by introducing a novel FDRL-SAC framework for real-time flexible energy markets in digitized microgrids. The proposed framework is not only targeted at ensuring accurate, dynamic pricing but also control continuity, system trust, and user participation under cyber-attacks such as FDI, DoS, and hybrid attacks.

The centerpiece of the proposed architecture is a decentralized multi-agent control system based on the SAC algorithm, complemented by federated coordination for preserving data privacy and scalability. The novelty of this research compared to the existing literature is the simultaneous integration of a secure pricing mechanism and a trust evaluation model into the DR scheme. LSTM-based residual filtering and anomaly detection are combined to identify cyber anomalies in real time, and a trust update mechanism regulates participation incentives based on both behavioral reliability and attack resilience. Additionally, the control system is designed to dynamically respond to both cyber and market uncertainty with an adaptive, signal-processing-augmented state space. This combined approach ensures that control agents not only learn optimal pricing and dispatch policies but also adapt them in a privacy-preserving, cyber-aware environment.

The practical usefulness of this work is confirmed via large-scale simulation on three benchmark microgrid networks (IEEE 33-bus, 69-bus, and 141-bus) with various attack scenarios and coordination configurations. The results confirm that the new FDRL-SAC framework significantly outperforms traditional centralized SAC and federated PPO baselines in social welfare (SW), DR compliance, control latency, and pricing stability. It is also resilient to coordinated cyberattacks and enables scalable trust-based coordination of agents. All these cumulatively render the proposed model a comprehensive solution for next-generation secure, resilient, and behaviorally adaptive energy market control in smart microgrids. The main contributions in summary are:

1. The study develops a novel pricing mechanism with secure, coherent real-time market flexibility, dynamic trust evaluation, and incentive-congruent demand response in a federated reinforcement learning environment, providing a strong foundation for effective market coordination in smart microgrids

2. A multi-layer cyber-resilient control architecture is developed, comprising LSTM-based anomaly detection and dynamic trust-based

<table><caption>**Table 1**
Comparison of the proposed approach with related studies.</caption><thead><tr><th>Ref</th><th>Secure pricing</th><th>Trust-based DR</th><th>Cyberattack type</th><th>Control method</th><th>Federated learning</th><th>Privacy preserving</th><th>Real-time market</th><th>Resilient to FDI</th><th>Resilient to DoS</th></tr></thead><tbody><tr><td>[13]</td><td>✓</td><td>✗</td><td>Not addressed</td><td>Stackelberg Game</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>[14]</td><td>✗</td><td>✗</td><td>FDI</td><td>Optimization (EPEC)</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>[15]</td><td>✗</td><td>✗</td><td>FDI</td><td>Finite-Time Control</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>[16]</td><td>✗</td><td>✗</td><td>FDI, DoS</td><td>FL-FORNN</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td></tr><tr><td>[17]</td><td>✗</td><td>✗</td><td>FDI</td><td>self-supervised learning</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>[18]</td><td>✗</td><td>✗</td><td>FDI</td><td>Barrier Function Control</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>[19]</td><td>✗</td><td>✗</td><td>FDI</td><td>Market Re-clearing</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>[20]</td><td>✗</td><td>✗</td><td>FDI</td><td>Stochastic Bi-level</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>[21]</td><td>✗</td><td>✗</td><td>DoS</td><td>Simulation Analysis</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✓</td></tr><tr><td>[22]</td><td>✗</td><td>✗</td><td>FDI</td><td>DR Game</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>[23]</td><td>✗</td><td>✗</td><td>FDI, Delay</td><td>MADRL</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>[24]</td><td>✗</td><td>✓</td><td>FDI</td><td>Game (I-ForB)</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>[25]</td><td>✓</td><td>✗</td><td>✗</td><td>MILP/EPEC</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[26]</td><td>✗</td><td>✗</td><td>FDI</td><td>ELOF</td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>✗</td></tr><tr><td>[27]</td><td>✗</td><td>✗</td><td>Not addressed</td><td>MARL</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[28]</td><td>✓</td><td>✗</td><td>Not addressed</td><td>game-theoretic pricing</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td></tr><tr><td>[29]</td><td>✗</td><td>✗</td><td>Not addressed</td><td>optimization</td><td>✗</td><td>✗</td><td>✓</td><td>✗</td><td>✗</td></tr><tr><td>Proposed method</td><td>✓</td><td>✓</td><td>FDI, DoS, Hybrid</td><td>FDRL-SAC</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr></tbody></table>
participation adjustment, utilizing decentralized SAC, to ensure security and responsiveness while under malicious pressures

3. The approach proposed here realistically offers a privacy-preserving federated coordination approach, which provides scalability while executing learning and control processes with data privacy and is still engaged even during advanced cyberattack stimuli

4. This paper proposes a new merge of federated learning with real-time signal processing and trust-adaptive DR, offering an innovative autonomous learning paradigm where agents simultaneously learn cooperative secure control policies while dynamically removing cyber anomalies and adjusting pricing schemes based on process user trust—allowing for a real shift in intelligent, decentralized market resiliency

The rest of the study is: In section 2, the structure of the proposed approach is presented along with the model for detecting cyberattacks. In section 3, the problem-solving approach is described. In section 4, the results and findings of the proposed approach are structured and finally, in section 5, the conclusion is presented.

## 2. Cyber-resilient real-time flexible energy market

According to Fig. 1, the proposed multi-layer architecture integrates cyber-resilient control, trust-based DR, and secure real-time pricing into a unified framework for decentralized microgrid markets. Layer 1 (Data Acquisition) captures high-fidelity, multi-domain signals, including price, demand behavior, trust scores, and grid conditions, into a structured state vector that feeds all downstream layers. Layer 2 (Cyberattack Simulation) introduces adversarial perturbations (FDI, DoS, hybrid) using Markov-modeled transitions and selective masking to simulate real-world attack surfaces, ensuring robustness in control policies. Layer 3 (Signal Preprocessing) employs variational autoencoders (VAE) for latent feature extraction and Gaussian mixture models (GMM) for operational clustering, enabling denoised context-aware representation of market behavior. Layer 4 (Anomaly Detection) utilizes cluster-specific LSTM predictors with residual thresholding to identify cyber anomalies, flagging deviations from normal operational patterns with statistical precision. Layer 5 (SAC-Based Market Control) implements a soft actuator agent that learns entropy-regularized pricing and dispatch policies, balancing SW, trust, and anomaly penalties. Layer 6 (Secure Pricing and Trust Evaluation) applies anomaly-aware smoothing and dynamically updates trust indices based on DR compliance and attack exposure, feeding into pricing logic. Layer 7 (Federated Learning Coordination) ensures privacy-preserving, scalable model training across distributed agents through federated averaging, maintaining global convergence without raw data sharing. Layer 8 (DR Feedback Loop) closes the system by modeling subscriber elasticity, translating price and trust into actual DR shifts, which in turn refine trust updates and control rewards.

This paper makes several structured assumptions to enable tractable simulation, decentralized optimization, and cyber-resilient control in a federated microgrid context. First, we assume that each microgrid agent has local observability of its own grid-side features (e.g., voltage, frequency, SoC) through SCADA or EMS interfaces, and these measurements are not shared across agents. This preserves data privacy and aligns with the federated architecture but also implies limited global observability, meaning that agents coordinate through price and trust signals rather than raw data exchange. Second, we assume that day-ahead price signals are exogenously declared by system operators or aggregators, while real-time prices are generated endogenously within the SAC agent. This reflects a two-stage market structure but assumes availability and accuracy of day-ahead forecasts, which may be imperfect in real-world deployments.

The proposed modeling approach assumes that cyberattacks manifest as manipulations of the input observation space, rather than structural disruptions to communication networks or physical devices. This

design choice reflects a focus on data-level adversarial risk, which is highly relevant in distributed energy systems where sensor integrity and input tampering are more common than infrastructure-level breaches. By modeling attacks through controlled perturbations such as FDI and DoS within the state vector, the system can learn to detect and respond to adversarial behaviors in real time, without relying on external security infrastructure. Additionally, we assume that agent behavior remains within a bounded rationality framework, meaning agents react to incentive structures rather than behaving adversarially or strategically against each other. This allows the SAC agent to focus on optimizing a multi-objective reward that balances social welfare, DR compliance, pricing stability, and anomaly responsiveness, thereby ensuring that control decisions remain interpretable and economically aligned. These assumptions are not simplifications but deliberate architectural choices that support real-time learning, robustness, and deployment realism within the federated smart grid context.

### 2.1. Layer 1: data acquisition layer

This layer is responsible for collecting high-fidelity, multi-domain data in discrete time steps to enable downstream analysis and control in the real-time flexible energy market [13,14]. This data includes market signals in real-time, how subscribers are bidding, measurements from operators in the grid, and behavioral trust indicators. At each time step $t$, the system constructs a comprehensive state vector $s_t$, defined as:

$$s_t = [\mathbf{p}_t, \mathbf{b}_{i,t}, \mathbf{r}_{i,t}, \mathcal{T}_{i,t}, \mathbf{h}_{i,t}^{DR}, \mathbf{o}_t, C_k, \mathbf{z}_t, a_t] \quad (1)$$

According to (1), $\mathbf{p}_t = [p_t^{RT}, p_t^{DA}, \Delta p_t]$ shows price signals, where $p_t^{RT}$ is real-time electricity market price, $p_t^{DA}$ is day-ahead forecasted price, $\Delta p_t = p_t^{RT} - p_t^{DA}$ is price deviation (used for volatility/trust modeling). $\mathbf{b}_{i,t} = [\hat{b}_{i,t}]$ describes bid price from subscriber $i$, and $\mathbf{r}_{i,t} = [\pi_{i,t}, \lambda_{i,t}]$ shows DR behavior where $\pi_{i,t}$ is proposed demand shift by subscriber $i$, $\lambda_{i,t}$ is accepted demand shift (actual DR commitment). $\mathcal{T}_{i,t}$ is trust score of subscriber $i$, derived from behavior history and anomaly exposure, $\mathbf{h}_{i,t}^{DR}$ shows DR compliance record (e.g., participation frequency, accuracy), $\mathbf{o}_t = [V_t, f_t, \text{SoC}_t]$ is grid-related operational features where $V_t$ is local voltage, $f_t$ is local frequency and SoC$_t$ is state of charge (SoC) of the battery storage system. $C_k$ is cluster ID determined from signal preprocessing (VAE-GMM), $\mathbf{z}_t$ describes latent representation vector extracted from VAE encoding, and $a_t$ is anomaly detection flag (1 if anomaly is detected, 0 otherwise). This multidimensional state vector $s_t$ provides a unified representation of the systems economic, technical, and behavioral conditions. This allows intelligent agents in subsequent layers to make pricing and control decisions that are context aware, and adding trust and anomalies signals allows the system to operate securely even in the presence of adversarial conditions. The day-ahead price is included as a known forecast input, consistent with real-world market practice where it is published in advance by aggregators or grid operators. In contrast, the real-time price is not forecasted or fed into the model, it is dynamically generated by the SAC agent during operation and shaped by ongoing anomaly detection and trust evaluation processes. Regarding grid-related inputs, such as voltage, frequency deviation, and state of charge, these are measured directly and locally by each microgrid controller at every time step. The architecture assumes that such telemetry is readily available through standard SCADA or energy management systems and does not require inter-agent communication. These inputs are processed in downstream layers through clustering and anomaly detection, ensuring that control decisions are informed by real-time, localized, and validated data.

### 2.2. Layer 2: cyberattack simulation and injection layer

This layer defines cyberattack scenarios for injection into the systems input signal stream. These injected simulated attacks stand for
<figure><img src="image_3.png" /><figcaption>Fig. 1. Microgrid secure pricing and real-time flexible energy market approach flowchart.</figcaption></figure>
various conceivable cyberattacks such as FDI, DoS and correlated hybrid attacks that target data integrity and availability on the communication layer [12–16]. To simulate the presence of a cyberattack, we define an observed state vector $\tilde{\mathbf{s}}_t$ as a corrupted version of the clean state vector $\mathbf{s}_t$. The corruption is introduced through an additive perturbation modeled as a product of a binary mask and an error vector.

$$ \tilde{\mathbf{s}}_t = \mathbf{s}_t + \delta_t \qquad (2) $$

where $\tilde{\mathbf{s}}_t \in \mathbb{R}^d$ is attacked (observed) state vector at time $t$, $\mathbf{s}_t \in \mathbb{R}^d$ shows clean state vector from Layer 1, $\delta_t \in \mathbb{R}^d$ describes attack perturbation vector, and $d$ is dimension of the state vector. The perturbation vector $\delta_t$ is generated using an element-wise multiplication of an attack mask and an error vector. This formulation allows selective corruption of specific components within the state vector.

$$ \delta_t = \mathbf{M}_t \cdot \mathbf{E}_t \qquad (3) $$

where $\mathbf{M}_t \in \{0, 1\}^{d \times d}$ is diagonal binary attack mask matrix, $\mathbf{E}_t \in \mathbb{R}^d$ shows attack error vector based on matrix-vector multiplication. To emulate dynamic adversarial strategies, attack types are selected based on a discrete-time Markov chain. This enables probabilistic switching between no attack, FDI, DoS, and hybrid attack modes.

$$ \mathbb{P}(A_{t+1} = a_j | A_t = a_i) = T_{ij} \qquad (4) $$

where $A_t \in \{\text{None, FDI, DoS, Hybrid}\}$ is attack mode at time $t$, and $T_{ij}$ is transition probability from attack type $a_i$ to $a_j$. By implementing attack simulation at the initial phases of the pipeline, this layer offers a controlled test environment to evaluate the resilience of signal processing, decision-making, and market control strategies to a wide variety of potential threat condition scenarios. These formulations allow each attack mode (None, FDI, DoS, Hybrid), selected via the Markov transition model in Eq. (4), to apply targeted corruption to specific components of the state vector $\mathbf{s}_t$ using the binary mask $\mathbf{M}_t$ and error vector $\mathbf{E}_t$ in Eq. (3). All entries in $\mathbf{s}_t$, including grid-side signals, market variables, and behavioral states, may be selectively attacked depending on the active mode, ensuring fine-grained and realistic simulation of evolving cyber threats.

## 2.3. Layer 3: signal preprocessing and clustering layer

This layer uses a two-step signal processing approach to recover the attacked state vector. The first step is to extract low-dimensional latent features using a VAE from the attacked input [30]. The second step is to use a GMM on these latent features to find the clusters that match different modes of operation [31].

### 2.3.1. VAE-based latent feature extraction

A VAE consists of an encoder $q_{\phi}(\mathbf{z}_t | \tilde{\mathbf{s}}_t)$ and a decoder $p_{\theta}(\tilde{\mathbf{s}}_t | \mathbf{z}_t)$ trained to minimize a regularized reconstruction loss.

$$ \mathcal{L}_{\text{VAE}}(\theta, \phi; \tilde{\mathbf{s}}_t) = \mathbb{E}_{q_{\phi}(\mathbf{z}_t | \tilde{\mathbf{s}}_t)} \left[ \log p_{\theta}(\tilde{\mathbf{s}}_t | \mathbf{z}_t) - D_{\text{KL}}(q_{\phi}(\mathbf{z}_t | \tilde{\mathbf{s}}_t) \| p(\mathbf{z}_t)) \right] \qquad (5) $$

where $\mathcal{L}_{\text{VAE}}$ is evidence lower bound (ELBO) loss function, $q_{\phi}(\mathbf{z}_t | \tilde{\mathbf{s}}_t)$ is encoder network (approximate posterior), $p_{\theta}(\tilde{\mathbf{s}}_t | \mathbf{z}_t)$ shows decoder network (likelihood function), $p(\mathbf{z}_t)$ is prior distribution over latent space (typically standard normal), $D_{\text{KL}}(\cdot|\cdot)$ describes the Kullback-Leibler divergence [32], and $\theta, \phi$ is decoder and encoder parameters. After training, the encoder is used to extract the latent feature vector:

$$ \mathbf{z}_t = \mu_{\phi}(\tilde{\mathbf{s}}_t) + \sigma_{\phi}(\tilde{\mathbf{s}}_t) \odot \epsilon, \epsilon \in \sim \mathcal{N}(0, I) \qquad (6) $$

where $\mu_{\phi}(\cdot), \sigma_{\phi}(\cdot)$ shows mean and standard deviation outputs of the encoder, $\odot$ is element-wise multiplication, and $\epsilon$ is Gaussian noise for reparameterization trick.

### 2.3.2. GMM-based cluster assignment

The latent features $\mathbf{z}_t$ are assigned to one of $K$ clusters using a GMM [31]. The GMM assumes the latent space is generated from a mixture of Gaussian components.

$$ p(\mathbf{z}_t) = \sum_{j=1}^{K} \pi_{j-1} \cdot \mathcal{N}(\mathbf{z}_t | \mu_j, \Sigma_j) \qquad (7) $$

where $\pi_{j}$ is mixing coefficient for cluster $j$, with $\sum_{j=1}^{K} \pi_{j} = 1$, $\mu_j$ shows mean of Gaussian component $j$, $\Sigma_j$ is covariance matrix of component $j$. The cluster assignment is made by selecting the most probable Gaussian component:

$$ C_k = \underset{j}{\operatorname{argmax}} \pi_{j-1} \cdot \mathcal{N}(\mathbf{z}_t | \mu_j, \Sigma_j) \qquad (8) $$

This layer reduces the noise on the signal while recognizing behaviors and operational patterns through VAE encoding and GMM clustering. These patterns allow downstream components to adapt to context changes due to load behaviors, DER dynamics, or attacks.

## 2.4. Layer 4: signal forecasting and anomaly detection layer

This layer focuses on recognizing cyber anomalies in the latent signal space. It uses a LSTM network to make temporal forecasts and verifies their predicted accuracy by performing a residual analysis. Anomaly detection is based on a threshold, where an anomaly will be flagged if the current behavior is sufficiently different from the previously predicted behavior.

### 2.4.1. LSTM-based forecasting model

The LSTM learns temporal dependencies in the latent signal sequence to forecast future behavior. For a given cluster $C_k$, the LSTM is trained to minimize the mean squared error (MSE) over a prediction horizon [33].

$$ \begin{gathered} \hat{\mathbf{z}}_t = \text{LSTM}_{C_k}(\mathbf{z}_{t-1}, \dots, \mathbf{z}_{t-n}) \\ \mathcal{L}_{\text{MSE}} = \frac{1}{n} \sum_{i=1}^{n} \| \hat{\mathbf{z}}_{t-i} - \mathbf{z}_{t-i} \|_{2}^{2} \end{gathered} \qquad (9) $$

where $\hat{\mathbf{z}}_t$ is predicted latent vector at time $t$, $\mathcal{L}_{\text{MSE}}$ shows forecasting loss minimized during training, and $n$ is the length of the temporal window. To detect anomalies, we evaluate the residual between predicted and observed latent signals. This residual captures the deviation of the current state from the expected behavior.

$$ \mathbf{r}_t = \hat{\mathbf{z}}_t - \mathbf{z}_t \qquad (10) $$

$$ \mathbf{r}_t = \|\mathbf{r}_t\|_2 = \sqrt{\sum_{j=1}^{d_z} (\hat{\mathbf{z}}_{t,j} - \mathbf{z}_{t,j})^2} \qquad (11) $$

where $\mathbf{r}_t$ is residual vector, $\mathbf{r}_t$ shows Euclidean norm of residual vector (magnitude), and $d_z$ is dimensionality of the latent space.

### 2.4.2. Cluster-specific anomaly thresholding

To adapt to different operating conditions, each cluster $C_k$ has a statistically derived anomaly threshold $\theta_k$. The threshold can be based on training residuals assuming Gaussian behavior [31]:

$$ \theta_k = \mu_k^{(r)} + \eta \cdot \sigma_k^{(r)} \qquad (12) $$

where $\mu_k^{(r)}$ is mean residual norm during training for cluster $C_k$, $\sigma_k^{(r)}$ describes standard deviation of residuals for cluster $C_k$, and $\eta$ is tuning parameter (typically between 2 and 3). The decision rule uses the computed residual magnitude and the threshold to assign a binary
anomaly flag.

$$a_t = \begin{cases} 1, & \text{if } r_t > \theta_k \\ 0, & \text{otherwise} \end{cases} \tag{13}$$

**Theoretical justification for residual-based detection:** Let $z_t \sim \mathcal{N}(\hat{z}_t, \Sigma_k)$ under normal conditions. Then the residual vector $r_t$ follows a multivariate Gaussian distribution, and the squared Mahalanobis distance:

$$D_t^2 = \mathbf{r}_t^T \Sigma_k^{-1} \mathbf{r}_t \tag{14}$$

follows a chi-squared distribution with $d_\sigma$ degrees of freedom.

**Proof:** Since $z_t \sim \mathcal{N}(\hat{z}_t, \Sigma_k)$, the residual $r_t = \hat{z}_t - z_t \sim \mathcal{N}(0, \Sigma_k)$. Thus, the quadratic form $D_t^2 = r_t^T \Sigma_k^{-1} r_t$ follows $\chi^2(d_\sigma)$.

**Remark:** A Mahalanobis-based detector may be used as an alternative to the Euclidean norm to account for correlated noise in latent dimensions.

The layer detects undesirable deviations in latent signal patterns through empirical thresholding and statistical decision theory. It generates real-time anomaly flags that can be used for penalizing risky actions in control layers or triggering defensive pricing strategies. The Following algorithm works in the latent feature space served by the VAE (Layer 3). Each LSTM model is cluster-specific, providing for adaptive anomaly detection based on operational mode.

**Algorithm 1 Anomaly detection via residual forecasting**

1. **Input:** Latent signal history ($z_{t-1}, \dots, z_{t-n}$), cluster ID $C_k$, current latent signal $z_t$

2. **Model:** Trained LSTM $C_k$, threshold $\theta_k$ for cluster $C_k$

3. **Output:** Anomaly flag $a_t$

4. **procedure** Detect anomaly ($\{z_{t-1}, \dots, z_{t-n}\}, z_t, C_k$)

5. $\hat{z}_t \leftarrow \text{LSTM}_{C_k}(z_{t-1}, \dots, z_{t-n})$

6. $r_t \leftarrow \hat{z}_t - z_t$

7. $r_t \leftarrow |r_t|_2$

8. if $r_t > \theta_k$ then

9. $a_t \leftarrow 1$

10. else

11. $a_t \leftarrow 0$

12. end if

13. return $a_t$

14. end procedure

# 3. Secure real-time pricing and problem-solving approach

## 3.1. Layer 5: SAC-based market control layer

This layer manages the pricing and control policy in real time using an SAC reinforcement learning architecture. The SAC agent has been trained to maximize long-term expected returns while being robust against uncertain situations when cyberattacks occur. The environment is modeled as a Markov decision process (MDP) defined by tuple $(\mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{r}, \gamma)$, where $\mathcal{S}$ is set of state vectors $\mathbf{s}_t \in \mathbb{R}^d$, $\mathcal{A}$ is continuous action space (e.g., pricing decisions $p_t$), $\mathcal{P}$ is transition function $\mathcal{P}(\mathbf{s}_{t+1} | \mathbf{s}_t, \mathbf{a}_t)$, $\mathbf{r}(\mathbf{s}_t, \mathbf{a}_t)$ shows reward function at time $t$, and $\gamma \in (0, 1)$ mentions discount factor [34]. The agent uses a stochastic policy $\pi_\phi(\mathbf{a}_t | \mathbf{s}_t)$ parameterized by neural network weights $\phi$, and aims to maximize the entropy-augmented objective:

$$J(\pi_{\phi}) = \sum_{t=0}^{\infty} \mathbb{E}_{\{\mathbf{s}_t, \mathbf{a}_t\} \sim \pi_{\phi}} [\mathbf{r}(\mathbf{s}_t, \mathbf{a}_t) + \alpha \mathcal{H}(\pi_{\phi}(\cdot | \mathbf{s}_t))] \tag{15}$$

where $\rho_\pi$ is state-action visitation distribution, $\mathcal{H}(\pi) = -\mathbb{E}_\pi[\log\pi]$ is policy entropy, and $\alpha$ shows entropy regularization coefficient. SAC maintains two Q-networks $Q_\theta$, $Q_\phi$ and a target value function $V_\psi$. The soft Q-function is updated using the following Bellman backup:

$$Q_\theta(\mathbf{s}_t, \mathbf{a}_t) = \mathbf{r}(\mathbf{s}_t, \mathbf{a}_t) + \gamma \mathbb{E}_{\mathbf{s}_{t+1} \sim \mathcal{P}} [V_\psi(\mathbf{s}_{t+1})] \tag{16}$$

$$V_{\bar{\psi}}(\mathbf{s}_t) = \mathbb{E}_{\mathbf{a}_t \sim \pi_{\phi}} [Q_{\theta}(\mathbf{s}_t, \mathbf{a}_t) - \alpha \log \pi_{\phi}(\mathbf{a}_t | \mathbf{s}_t)] \tag{17}$$

where $Q_\theta$ is action-value function, $V_{\bar{\psi}}$ shows soft value function, and

$\bar{\psi}$ describes target network parameters. The reward function encodes multi-objective criteria including social welfare, trust maximization, and market stability:

$$r(\mathbf{s}_t, \mathbf{a}_t) = (w_1 \cdot SW_t) + (w_2 \cdot \mathcal{T}_{i,t}) - (w_3 \cdot \Delta p_t^2) - (w_4 \cdot a_t) \tag{18}$$

where $SW_t$ is social welfare at time $t$, $\mathcal{T}_{i,t}$ is trust index of subscriber $i$, $\Delta p_t = p_t^{RT} - p_t^{DA}$ shows price deviation, $a_t$ shows anomaly flag (from Layer 4), and $w_1, w_2, w_3, w_4$ describe weighting coefficients for reward shaping.

### 3.1.1. Optimality conditions and stability

This layer will allow adaptive, resilient decision making at the market level while operating under cyber uncertainty. By incorporating trust metrics, along with anomalous flags when generating the SAC reward, the system balances the model towards economic efficiency in terms of monetary reward but, also in fairness and security. Under mild regularity conditions on $r$ and bounded entropy, the SAC policy converges to a soft-optimal solution that maximizes the entropy-augmented return $J(\pi)$ [31]. Let $\pi_\phi^*$ be the optimal policy maximizing (15). Then $\pi_\phi^*$ satisfies the fixed-point condition:

$$\pi_{\phi}^{*}(\mathbf{a}_{t}|\mathbf{s}_{t}) \propto \exp\left(\frac{1}{\alpha} Q_{\theta}(\mathbf{s}_{t}, \mathbf{a}_{t})\right) \tag{19}$$

## 3.2. Layer 6: secure pricing and trust evaluation layer

This layer clarifies the output of the SAC agent to ensure that the real-time pricing signal is reliable and secure. It accomplishes this through anomaly-aware smoothing filters and trust scores that dynamically adjust based on participant actions and exposure to cyberattacks. To avoid price volatility due to cyber-induced noise or anomalies, a smoothing filter is applied to the output price $p_t$ based on historical pricing trends and anomaly flags.

$$\bar{p}_t = \lambda p_t + (1 - \lambda) \bar{p}_{t-1}, \quad \lambda \in (0, 1) \tag{20}$$

where $\bar{p}_t$ is smoothed real-time price, $p_t$ is raw price from SAC agent, and $\lambda$ shows exponential smoothing factor. To account for attack awareness, a correction is introduced when an anomaly is flagged:

$$\bar{p}_t^{\text{corr}} = \bar{p}_t \cdot (1 - \beta a_t) \tag{21}$$

where $\bar{p}_t^{\text{corr}}$ is trust-corrected price, $\beta$ shows anomaly penalty scaling factor, and $a_t$ is binary anomaly flag from Layer 4. The trust score $\mathcal{T}_{i,t}$ of subscriber $i$ is updated based on DR compliance and attack history. The update rule balances recent behavior with long-term trends:

$$\mathcal{T}_{i,t} = \rho \mathcal{T}_{i,t-1} + (1 - \rho) \psi_{i,t} \tag{22}$$

where $\mathcal{T}_{i,t}$ is trust index of subscriber $i$ at time $t$, $\rho$ is smoothing factor for historical memory, $\psi_{i,t}$ is instantaneous trust increment (defined below). The instantaneous score $\psi_{i,t}$ is computed from the accuracy of the subscribers DR action:

$$\psi_{i,t} = 1 - \left| \frac{\lambda_{i,t} - \pi_{i,t}}{\pi_{i,t} + \epsilon} \right| - \delta a_t \tag{23}$$

where $\lambda_{i,t}$ is accepted demand shift, $\pi_{i,t}$ shows proposed demand shift, $\epsilon$ is small constant to avoid division by zero, and $\delta$ is penalty weight for cyber anomaly exposure.

**Theorem:** Assume $\mathcal{T}_{i,t} \in [0, 1]$ evolves under bounded increments $\psi_{i,t} \in [0, 1]$ and constant $\rho \in (0, 1)$. Then the trust score converges to a unique fixed point if subscriber behavior stabilizes.

**Proof:** Since $\mathcal{T}_{i,t} = \rho \mathcal{T}_{i,t-1} + (1 - \rho) \psi_{i,t}$ is a weighted average and both $\mathcal{T}_{i,t-1}$ and $\psi_{i,t} \in [0, 1]$, the sequence is bounded and contractive. Therefore, it converges to a steady-state value under consistent behavior [31].

This layer helps to ensure that market signals are stable and interpretable even given the uncertainty of a cyber disaster. It also builds
participant confidence as the trust factor is transparent in its evaluation, and contributes to fair and reliable energy market prices. The trust index in algorithm 2 dynamically accounts for DR accuracy and exposure to anomalies, and the price adjustment enables economic stability in a cyber-disaster context.

**Algorithm 2 Secure pricing and trust evaluation**

1. **Input:** Raw price $p_t$, smoothed price $\tilde{p}_{t-1}$, anomaly flag $a_t$, trust score $\mathcal{T}_{it-1}$, proposed shift $\pi_{it}$, accepted shift $\lambda_{it}$.

2. **Parameters:** Smoothing factor $\lambda$, anomaly penalty $\beta$, trust smoothing $\rho$, penalty factor $\delta$, small $\epsilon$.

3. **Output:** Corrected price $\tilde{p}_t^{\text{corr}}$, updated trust score $\mathcal{T}_{it}$

4. **procedure** Secure price and trust update {$p_t, \tilde{p}_{t-1}, a_t, \mathcal{T}_{it-1}, \lambda_{it}$}

5. $\tilde{p}_t \leftarrow \lambda p_t + (1 - \lambda) \tilde{p}_{t-1}$

6. $\tilde{p}_t^{\text{corr}} \leftarrow \tilde{p}_t \cdot (1 - \beta a_t)$

7. **if** $\pi_{it} > 0$ **then**

8. $\psi_{it} \leftarrow 1 - \frac{|\lambda_{it} - \pi_{it}|}{|\pi_{it} + \epsilon|} - \delta a_t$

9. **else**

10. $\psi_{it} \leftarrow 1 - \delta a_t$

11. **end if**

12. $\mathcal{T}_{it} \leftarrow \rho \mathcal{T}_{it-1} + (1 - \rho) \psi_{it}$

13. **return** $\tilde{p}_t^{\text{corr}}, \mathcal{T}_{it}$

14. **end procedure**

### 3.3. Layer 7: federated learning and coordination layer

This layer allows decentralized training of SAC agents across various edge nodes (e.g., microgrids or DER controllers). Each agent learns to optimize a local policy involving private data and communication with the global coordinator using a federated averaging update. This process maintains privacy, scalability, and robustness [34–36].

#### 3.3.1. Local policy update at each node

Each edge agent $j \in \{1, \dots, M\}$ trains its own SAC model locally using its private trajectory $\mathcal{J}_j$. Let $\phi_j$ denote the local actor parameters.

$$ \phi_j^{(t+1)} = \phi_j^{(t)} - \eta \nabla_{\phi_j} \mathcal{L}_j^{\text{SAC}} \quad (24) $$

where $\phi_j^{(t)}$ is policy parameters of agent $j$ at iteration $t$, $\mathcal{L}_j^{\text{SAC}}$ is local SAC loss function, $\eta$ is learning rate.

#### 3.3.2. Global aggregation via federated averaging

At synchronization step $T$, the central aggregator computes the weighted average of local models:

$$ \phi^{(T)} = \sum_{j=1}^{M} \frac{n_j}{n_{\text{total}}} \phi_j^{(T)} \quad (25) $$

where $\phi^{(T)}$ is global actor parameters after aggregation, $n_j$ shows number of training samples at node $j$, and $n_{\text{total}} = \sum_{j=1}^{M} n_j$ is total number of samples across all nodes. The global training objective is to minimize the weighted sum of local SAC losses:

$$ \min_{\phi} \sum_{j=1}^{M} \frac{n_j}{n_{\text{total}}} \mathcal{L}_j^{\text{SAC}}(\phi) \quad (26) $$

**Theorem (Privacy-Preserving Guarantee):** Let each local update be performed over disjoint datasets and transmitted model weights be encrypted. Then no raw data from any agent is exposed during training, ensuring local data privacy.

**Proof:** The algorithm exchanges only encrypted weight vectors $\phi_j$, without transmitting raw trajectories $\mathcal{J}_j$. As data remains local and distinct, privacy is preserved by design.

This layer allows for collaborative SAC training among distributed microgrids, while also enabling differentiation in policy identity. The global policy benefits from the aggregate learning from collective knowledge while keeping the decentralized structure for strength

against data breaches and cyberattacks that impact all units at once.

**Algorithm 3 Trust-driven demand response evaluation**

1. **Input:** Trust score $\mathcal{T}_{it-1}$, proposed shift $\pi_{it}$, corrected price $\tilde{p}_t^{\text{corr}}$, updated trust score $\mathcal{T}_{it}$

2. **Parameters:** Trust responsiveness $\eta$, price elasticity $\kappa$, trust smoothing $\rho$.

3. **Output:** Realized shift $\lambda_{it}$, feedback error $\epsilon_{it}^{\text{DR}}$, updated trust score $\mathcal{T}_{it}$.

4. **Procedure:** Evaluate DR response {$\pi_{it}, \mathcal{T}_{it-1}, \tilde{p}_t^{\text{corr}}$}

5. $\mathcal{T}_{it} \leftarrow \mathcal{T}_{it-1}$

6. $\lambda_{it} \leftarrow \pi_{it} (1 + \eta \mathcal{T}_{it} - \kappa \tilde{p}_t^{\text{corr}})$

7. $\epsilon_{it}^{\text{DR}} \leftarrow |\lambda_{it} - \pi_{it}|$

8. **if** $\pi_{it} > 0$ **then**

9. $\psi_{it} \leftarrow 1 - \frac{\epsilon_{it}^{\text{DR}}}{\pi_{it}}$

10. **else**

11. $\psi_{it} \leftarrow 1$

12. **end if**

13. $\mathcal{T}_{it} \leftarrow \rho \mathcal{T}_{it-1} + (1 - \rho) \psi_{it}$

14. **return** $\epsilon_{it}^{\text{DR}}$, $\mathcal{T}_{it}$, $\lambda_{it}$

15. **end procedure**

### 3.4. Layer 8: subscriber demand response and trust feedback layer

According to Algorithm 3, last layer represents the behavioral dynamics of market participants in terms of their responses to real-time price signals and trust signals. It incorporates demand elasticity, checks compliance with DR commitments, and dynamically feeds back trust-adjusted responses to improve future pricing and policy design. For DR activation function, subscribers adjust their actual demand $\lambda_{it}$ based on their proposed response $\pi_{it}$, current trust score $\mathcal{T}_{it}$, and the secure price $\tilde{p}_t^{\text{corr}}$:

$$ \lambda_{it} = \mathcal{I}_i(\pi_{it}, \tilde{p}_t^{\text{corr}}, \mathcal{T}_{it}) \quad (27) $$

Assuming a linear DR elasticity model with trust amplification:

$$ \lambda_{it} = \pi_{it} \cdot (1 + \eta \cdot \mathcal{T}_{it} - \kappa \tilde{p}_t^{\text{corr}}) \quad (28) $$

where $\lambda_{it}$ is actual demand shift executed by subscriber $i$, $\pi_{it}$ shows proposed shift amount, $\tilde{p}_t^{\text{corr}}$ is secure corrected price, $\mathcal{T}_{it}$ is trust index at time $t$, $\eta$ describes trust responsiveness coefficient, and $\kappa$ is price elasticity coefficient. To analysis feedback to market mechanism, the feedback signal evaluates compliance and feeds adjustments to the reward structure and pricing logic:

$$ \epsilon_{it}^{\text{DR}} = |\lambda_{it} - \pi_{it}| \quad (29) $$

where $\epsilon_{it}^{\text{DR}}$ is absolute deviation in DR compliance. This deviation contributes to both trust updates (as in Layer 6) and to potential penalty/incentive updates in market control.

**Theorem (DR Behavior Convergence):** Suppose trust updates are stable and prices converge. Then the DR response function $\lambda_{it}$ converges to a fixed point provided $\eta, \kappa > 0$ and $\pi_{it}$ is stationary.

**Proof:** Given that $\lambda_{it} = \pi_{it} (1 + \eta \mathcal{T}_{it} - \kappa \tilde{p}_t^{\text{corr}})$, if both $\mathcal{T}_{it} \to \mathcal{T}_{\infty}$ and $\tilde{p}_t^{\text{corr}} \to p_{\infty}$, and $\pi_{it} = \pi_i$, then $\lambda_{it} \to \lambda_i^*$ as a deterministic function of time-invariant inputs.

This layer closes the control loop by modeling DR behavior of subscribers and their feedback to the learning aspect of the system. Increasing market fairness and cyber-resilience involves improving trust-driven response dynamics so economic incentives become less misaligned with secure behavior. An example of a complete behavioral feedback loop is in Algorithm 3, where DR compliance affects trust and future participation in the network through adaptive elasticity.

### 3.5. Stability analysis of the proposed system

This section provides theoretical justification for the stability of key components in the proposed cyber-resilient market model: the SAC control policy, the anomaly-aware pricing mechanism, and the trust-driven DR feedback loop [31–33].
**Lemma 1.** Stability of Trust-Filtered price signal

Let $p_t \in \mathbb{R}$ be the raw SAC-generated price and $\bar{p}_t$ the exponentially smoothed price:

$$ \bar{p}_t = \lambda p_t + (1 - \lambda) \bar{p}_{t-1}, \lambda \in (0, 1) \quad 1.1) $$

If the raw SAC output $p_t$ is bounded by $|p_t| \le P_{max}$, then the smoothed price $\bar{p}_t$ satisfies:

$$ |\bar{p}_t| \le P_{max}, \forall t \ge 0 \quad 1.2) $$

**Proof:** The recursion defines a convex combination. Since $|\bar{p}_t| \le \max(|p_t|, |\bar{p}_{t-1}|)$, and initial $\bar{p}_0 \in [-P_{max}, P_{max}]$, the bound is preserved inductively.

**Theorem 1.** Convergence of DR response under stable price and trust

Suppose:

* $\mathcal{T}_{i,t} \to \mathcal{T}_{i,\infty} \in [0, 1]$ as $t \to \infty$.

* $\bar{p}_t^{\text{corr}} \to p_{\infty} \in \mathbb{R}$

* The proposed DR shift $\pi_{i,t} = \pi_i$ is stationary

Then the actual DR response:

$$ \lambda_{i,t} = \pi_i (1 + \eta \mathcal{T}_{i,t} - \kappa \bar{p}_t^{\text{corr}}) \quad 1.1) $$

converges to:

$$ \lambda_i^* = \pi_i (1 + \eta \mathcal{T}_{i,\infty} - \kappa p_{\infty}) \quad 1.2) $$

**Proof:** This follows by applying the limits $\mathcal{T}_{i,t} \to \mathcal{T}_{i,\infty}$ and $\bar{p}_t^{\text{corr}} \to p_{\infty}$ into the linear expression.

**Theorem 2.** Policy stability under soft Actor-Critic

Let the SAC objective be:

$$ J(\pi) = \mathbb{E}_{\pi} \left[ \sum_{t=0}^{\infty} \gamma^t (r(\mathbf{s}_t, \mathbf{a}_t) + \alpha \mathcal{H}(\pi(\cdot|\mathbf{s}_t))) \right] \quad (T2.1) $$

If:

* The reward $r(\mathbf{s}, \mathbf{a})$ is bounded: $|r| \le R_{\max}$

* The policy entropy $\mathcal{H}(\pi) \le H_{\max}$

Then $J(\pi)$ is bounded and the SAC update sequence $\{\pi_k\}$ converges to a fixed point under the soft Bellman backup.

**Proof:** Boundedness of reward and entropy implies convergence of value function and monotonic improvement under SAC's off-policy training. **Proof** follows from SAC convergence guarantees in [34].

These assessments collectively establish that under bounded rewards, price, and trust evolution, the entire cyber-resilient control system is stable in policy, pricing, and demand feedback.

# 4. Result

## 4.1. Simulation requirements

Table 2 provides a consistent DER deployment, cyberattack scenario, and centralized control actions of the selected IEEE test systems. For the sake of relative comparison, the same power generation cost, maintenance level, and base load consumption rate apply to all systems. Table 3 provides the key hyperparameters and operational configurations used in the FDRL-SAC based model. These principles were selected based on empirical tuning and benchmark stability for cyber-resilient distributed control in MGs. Table 4 expands on the quantitative comparison of FDRL-SAC model with Centralized SAC [37,38] and Federated PPO [39,40] by explaining the specific strengths of the FDRL structure. Every row shows how the FDRL-SAC configuration improves learning robustness, cyber-resilience, or demand-side flexibility.

The proposed FDRL-SAC framework was implemented in a hybrid simulation environment combining MATLAB and Python. The physical microgrid models, including distributed energy resources, demand response mechanisms, and IEEE test systems, were developed in MATLAB using the MATPOWER toolbox and custom Simulink blocks. The reinforcement learning architecture, including SAC policy training, LSTM-based anomaly detection, and federated coordination, was implemented in Python using TensorFlow and PyTorch. Equations governing trust dynamics, pricing logic, reward shaping, and anomaly scoring were encoded as vectorized functions, mapped directly from the

<table><caption>**Table 3**
Key hyperparameters and operational configurations of FDRL-SAC.</caption><thead><tr><th>Parameter</th><th>Value</th><th>Description</th></tr></thead><tbody><tr><td>Learning rate (Actor/Critic)</td><td>0.0003 / 0.0003</td><td>Learning rates for actor and critic networks</td></tr><tr><td>Discount factor (γ)</td><td>0.99</td><td>Discount factor for future reward accumulation</td></tr><tr><td>Entropy coefficient (α)</td><td>0.2</td><td>Regularization of policy entropy (exploration strength)</td></tr><tr><td>Batch size</td><td>256</td><td>Mini-batch size per training step</td></tr><tr><td>Replay buffer size</td><td>1,000,000</td><td>Size of experience replay buffer</td></tr><tr><td>Target update rate (τ)</td><td>0.005</td><td>Soft update rate for target networks</td></tr><tr><td>Federated update interval</td><td>every 10 episodes</td><td>Frequency of global model aggregation</td></tr><tr><td>Episode horizon</td><td>96 steps (15-min intervals)</td><td>Max steps per episode</td></tr><tr><td>Total episodes</td><td>75,000</td><td>Total offline training episodes</td></tr><tr><td>Clustering latent dimension</td><td>16</td><td>Latent dimension for VAE/GMM clustering</td></tr><tr><td>Anomaly detection window (LSTM)</td><td>10</td><td>Time window for LSTM anomaly forecasting</td></tr><tr><td>Trust update smoothing (ρ)</td><td>0.85</td><td>Memory factor for trust score averaging</td></tr></tbody></table>

<table><caption>**Table 2**
Topology configuration.</caption><thead><tr><th>Test System</th><th>Voltage / Frequency</th><th>PV Sets</th><th>WT Sets</th><th>ESS Sets</th><th>EV Charging Stations</th><th>Controllable Loads</th></tr></thead><tbody><tr><td>IEEE 33-bus</td><td>12.66 kV / 60 Hz</td><td>2 sets (2.6 MW)</td><td>1 set (2 MW)</td><td>2 sets (2 MWh)</td><td>1 unit (960 kW)</td><td>4 MW</td></tr><tr><td>IEEE 69-bus</td><td>12.66 kV / 60 Hz</td><td>4 sets (5.2 MW)</td><td>3 sets (6 MW)</td><td>4 sets (4 MWh)</td><td>3 units (2.88 MW)</td><td>8 MW</td></tr><tr><td>141-bus</td><td>12.47 kV / 60 Hz</td><td>8 sets (10.4 MW)</td><td>5 sets (10 MW)</td><td>6 sets (6 MWh)</td><td>5 units (4.8 MW)</td><td>12 MW</td></tr><tr><td>Test System</td><td>Control Type</td><td>Cyberattack Scenarios</td><td>Demand Response Strategy</td><td>Generation Cost ($/kWh)</td><td>Maintenance Cost ($/unit/day)</td><td>Base Load Consumption Price ($/kWh)</td></tr><tr><td>IEEE 33-bus</td><td>FDRL-SAC</td><td>FDI, DoS, Hybrid</td><td>Trust-linked linear DR</td><td>PV: 0.02, WT: 0.03, BESS: 0.06</td><td>PV: 3, WT: 5, BESS: 8, EVCS: 9</td><td>0.07–0.18 (dynamic, trust-adjusted)</td></tr><tr><td>IEEE 69-bus</td><td>FDRL-SAC</td><td>FDI, DoS, Hybrid</td><td>Trust-weighted DR with anomaly correction</td><td>PV: 0.02, WT: 0.03, BESS: 0.06</td><td>PV: 3, WT: 5, BESS: 8, EVCS: 9</td><td>0.07–0.18 (dynamic, trust-adjusted)</td></tr><tr><td>141-bus</td><td>FDRL-SAC</td><td>FDI, DoS, Hybrid</td><td>Hierarchical trust-DR with feedback loop</td><td>PV: 0.02, WT: 0.03, BESS: 0.06</td><td>PV: 3, WT: 5, BESS: 8, EVCS: 9</td><td>0.07–0.18 (dynamic, trust-adjusted)</td></tr></tbody></table>
<table><caption>**Table 4**
Comparison of operational process of the FDRL-SAC with other methods.</caption><thead><tr><th>Parameter</th><th>Proposed FDRL-SAC</th><th>Centralized SAC</th><th>Federated PPO</th><th>FDRL-SAC advantage description</th></tr></thead><tbody><tr><td>Total training episodes</td><td>75,000</td><td>45,000</td><td>60,000</td><td>Longest training enables better generalization to cyberattack scenarios</td></tr><tr><td>Learning rate</td><td>0.0003</td><td>0.0003</td><td>0.0005</td><td>Equal to SAC, more stable than PPO; not a major differentiator</td></tr><tr><td>Entropy coefficient (α)</td><td>0.2</td><td>0.05</td><td>0.1</td><td>Higher entropy encourages exploratory behavior in adversarial conditions</td></tr><tr><td>Batch size</td><td>256</td><td>128</td><td>128</td><td>Larger batch improves learning stability in federated multi-agent systems</td></tr><tr><td>Replay buffer size</td><td>1,000,000</td><td>500,000</td><td>800,000</td><td>Largest buffer retains more experience for rare-event and attack replay</td></tr><tr><td>Target update rate (τ)</td><td>0.005</td><td>0.01</td><td>0.01</td><td>More conservative update improves value network stability</td></tr><tr><td>Federated update interval</td><td>Every 10 episodes</td><td>Not applicable</td><td>Every 25 episodes</td><td>Frequent sync ensures responsive coordination across agents</td></tr><tr><td>Anomaly detection</td><td>Yes (LSTM, window = 10)</td><td>No</td><td>Partial (reward penalty only)</td><td>Only model with LSTM-based anomaly detection for cyber resilience</td></tr><tr><td>Trust integration</td><td>Yes (Dynamic, p = 0.85)</td><td>No</td><td>Fixed score</td><td>Only model with dynamic trust feedback integrated into decision-making</td></tr></tbody></table>

mathematical formulations presented in Sections 2 and 3. Data exchange between MATLAB and Python was managed through inter-process communication at each simulation timestep, enabling closed-loop coordination and learning. Simulations were executed on a personal computer with an Intel® Core™ i7-1355U processor (1.7 GHz up to 5.0 GHz, 12 MB L3 Cache), supporting real-time interaction between system dynamics and federated agent learning across all tested scenarios. This configuration ensured accurate, scalable, and cyber-resilient validation of the proposed control model.

## 4.2. Policy learning convergence and stability

This part discusses the rate of convergence and policy stabilization of the new FDRL-SAC model proposed in this study compared to Centralized SAC and Federated PPO in normal and adversarial training scenarios. Fig. 2 displays the average reward per episode over 75,000 training episodes. FDRL-SAC is quicker converging and more rewarding with a stabilization at episode 14,500, compared to slower-converging SAC and PPO that achieve lower reward levels. Particularly, for FDI and Hybrid attacks, the FDRL-SAC experiences smooth learning trends and recovers from disturbances easily as well. Compared to that, the centralized SAC approach experiences high reward fluctuations and Federated PPO experiences extended instability of initial and mid-level learning. All these trends validate the effectiveness of federated training and reward stabilization methods employed in FDRL-SAC for MGs market control problems. Fig. 3 provides a comparison of stability of the

output signal based on price standard deviation and trust index variance among the three models. FDRL-SAC has the lowest variance for both signals, indicating higher anomaly robustness and more stable market response. This is due to the integration of LSTM-based anomaly filtering and trust-based DR methods.

Table 5 presents a comparative analysis of the policy learning performance and output stability for the proposed FDRL-SAC framework against two baselines, Centralized SAC and Federated PPO, across three standard test systems of increasing complexity. The FDRL-SAC model consistently achieves the highest final reward in all cases, with faster convergence and significantly lower price volatility. In the IEEE 33-bus system, convergence is achieved after approximately 14,500 episodes with a stable price standard deviation of 0.018. As system size scales, FDRL-SAC shows graceful degradation, maintaining reward stability and robustness even in the 141-bus case, where centralized SAC and federated PPO both exhibit longer training times and higher variance in price output. These results reflect the proposed model's ability to balance multi-agent coordination, trust updates, and anomaly handling, resulting in faster policy convergence and stable market signals even under cyber-induced disturbances. Notably, the Centralized SAC baseline becomes increasingly unstable as the network grows, and Federated PPO, while more scalable, fails to match FDRL-SAC in reward optimization or price consistency. These all point towards the fact that the proposed model not only converges faster but also provides more stable and consistent market operation in the presence of cyberattacks.

According to the mathematical relations presented in Eqs. (9) to (11)

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Reward convergence over episodes.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>Fig. 3. Signal stability across models.</figcaption></figure>

<table><caption>**Table 5**
Policy learning convergence and stability comparison.</caption><thead><tr><th>Test System</th><th>Model</th><th>Avg. Final Reward</th><th>Time to Convergence (Episodes)</th><th>Price Std. Dev.</th></tr></thead><tbody><tr><td rowspan="3">IEEE 33-bus</td><td>FDRL-SAC</td><td>542.3</td><td>14,500</td><td>0.018</td></tr><tr><td>Centralized SAC</td><td>481.9</td><td>21,000</td><td>0.043</td></tr><tr><td>Federated PPO</td><td>458.6</td><td>25,500</td><td>0.038</td></tr><tr><td rowspan="3">IEEE 69-bus</td><td>FDRL-SAC</td><td>537.1</td><td>15,200</td><td>0.021</td></tr><tr><td>Centralized SAC</td><td>465.4</td><td>23,500</td><td>0.046</td></tr><tr><td>Federated PPO</td><td>442.2</td><td>26,900</td><td>0.040</td></tr><tr><td rowspan="3">141-bus</td><td>FDRL-SAC</td><td>528.4</td><td>16,400</td><td>0.023</td></tr><tr><td>Centralized SAC</td><td>453.8</td><td>24,800</td><td>0.051</td></tr><tr><td>Federated PPO</td><td>438.3</td><td>27,600</td><td>0.045</td></tr></tbody></table>

and also Eqs. (12) to (14), the process of detecting and classifying dynamic and time-varying data can be defined and implemented with high accuracy. In Eq. (11), the Euclidean norm is used to calculate the size of the residual vector; this vector represents the difference between the predicted value by the LSTM model and the actual value observed in the latent feature space and is used as the main indicator of deviation from natural behavior. On the other hand, in Eq. (12), an anomaly threshold based on the mean and standard deviation of the residual vectors during the training period is determined cluster-specifically. The final decision is made in Eq. (13): if the residual vector norm exceeds this threshold,

the data in question is marked as a sample suspected of cyberattack and having anomalous behavior; Otherwise, the data is considered as normal and undisturbed input. This process is graphically shown in Fig. 4, and the model performance is analyzed at three time points of the training process, namely episodes 5000, 10000, and 15,000 (time to convergence). Especially in episode 15000, the model was able to provide a clearer separation between normal data and data from cyberattacks, indicating the maturity of the prediction structure and the effectiveness of statistical thresholding in identifying hidden threats in temporal data.

For a more accurate assessment of stability, refer to Fig. 5. In this figure, the root locus diagram based on four components in various bases of system stability evaluation and behavior is presented. In this assessment, the components of entropy coefficient ($\alpha$), trust smoothing factor ($\rho$), target update rate ($\tau$), and discount factor ($\gamma$) are considered. The range of changes presented for each component in Fig. 5 indicates the stability interval due to each variable. According to this figure, with respect to the bases presented, the system poles are located to the left of the imaginary axis (thick black axis) and show the stability of the proposed approach.

### 4.3. Cyberattack resilience performance

This section evaluates the control and price signal stability of the FDRL-SAC model subject to three types of cyberattacks: FDI, DoS, and Hybrid attacks. The experimental results compare the FDRL-SAC against Centralized SAC and Federated PPO models. Fig. 6 plots the bar chart of the price volatility index (PVI) across models in IEEE 33 bus power system. As shown, the FDRL-SAC displays much lower volatility due to incorporating LSTM-based anomaly detection and dynamic trust-

<figure><img src="image_6.png" /><figcaption>Fig. 4. Detect and separate normal data and cyber attacks.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Fig. 5. Assessing the stability of the proposed approach based on variables affecting stability.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6. PVI under cyberattacks in IEEE 33 bus.</figcaption></figure>

weighted pricing mechanisms. Centralized SAC shows sudden oscillations in attacks, while Federated PPO also shows greater instability resulting from lagged policy aggregation. Fig. 7 shows the DR compliance rate, which shows how consistently each model maintains in-control decisions under attack conditions. FDRL-SAC again has the highest DR compliance rate, leveraging adaptive demand shaping and trust-based DR. Federated PPO is relatively good, while SAC cannot have stable control in DoS and hybrid attack scenarios. Table 6 extends the

cyberattack resilience evaluation across all three benchmark test systems, capturing the robustness of each model under three threat types: FDI, DoS, and Hybrid attacks. In every scenario and system size, the FDRL-SAC framework exhibits the lowest Price Volatility Index (PVI) and the highest DR compliance rate, reflecting its superior ability to adapt to cyber-induced distortions through anomaly-aware filtering and trust-based DR correction. Notably, in the 141-bus system, where coordination complexity and data volatility are higher, FDRL-SAC
<figure><img src="image_9.png" /><figcaption>Fig. 7. DR compliance under cyberattacks in IEEE 33 bus.</figcaption></figure>

<table><caption>**Table 6**

Cyberattack resilience performance.</caption><thead><tr><th>Test System</th><th>Model</th><th>Attack Type</th><th>PVI</th><th>DR Compliance Rate (%)</th></tr></thead><tbody><tr><td rowspan="12">IEEE 33-bus</td><td rowspan="3">FDRL-SAC</td><td>FDI</td><td>0.017</td><td>94.2</td></tr><tr><td>DoS</td><td>0.019</td><td>92.5</td></tr><tr><td>Hybrid</td><td>0.023</td><td>89.8</td></tr><tr><td rowspan="2">Centralized SAC</td><td>FDI</td><td>0.042</td><td>84.7</td></tr><tr><td>DoS</td><td>0.048</td><td>81.2</td></tr><tr><td rowspan="3">Federated PPO</td><td>Hybrid</td><td>0.055</td><td>78.5</td></tr><tr><td>FDI</td><td>0.036</td><td>86.3</td></tr><tr><td>DoS</td><td>0.041</td><td>83.1</td></tr><tr><td rowspan="3">FDRL-SAC</td><td>Hybrid</td><td>0.047</td><td>80.4</td></tr><tr><td>FDI</td><td>0.018</td><td>92.9</td></tr><tr><td>DoS</td><td>0.021</td><td>90.7</td></tr><tr><td rowspan="12">IEEE 69-bus</td><td rowspan="3">Centralized SAC</td><td>Hybrid</td><td>0.025</td><td>87.4</td></tr><tr><td>FDI</td><td>0.045</td><td>81.5</td></tr><tr><td>DoS</td><td>0.051</td><td>78.2</td></tr><tr><td rowspan="2">Federated PPO</td><td>Hybrid</td><td>0.058</td><td>75.0</td></tr><tr><td>FDI</td><td>0.038</td><td>84.9</td></tr><tr><td rowspan="2">Centralized SAC</td><td>DoS</td><td>0.043</td><td>81.6</td></tr><tr><td>Hybrid</td><td>0.049</td><td>78.9</td></tr><tr><td rowspan="3">FDRL-SAC</td><td>Hybrid</td><td>0.020</td><td>91.3</td></tr><tr><td>FDI</td><td>0.023</td><td>88.6</td></tr><tr><td>DoS</td><td>0.027</td><td>85.1</td></tr><tr><td rowspan="3">Centralized SAC</td><td>Hybrid</td><td>0.049</td><td>78.3</td></tr><tr><td>FDI</td><td>0.054</td><td>74.5</td></tr><tr><td>DoS</td><td>0.060</td><td>71.2</td></tr><tr><td rowspan="3">Federated PPO</td><td>Hybrid</td><td>0.041</td><td>83.5</td></tr><tr><td>FDI</td><td>0.046</td><td>79.9</td></tr><tr><td>DoS</td><td>0.052</td><td>76.5</td></tr></tbody></table>

maintains a DR compliance rate above 85 % even under hybrid attacks, whereas Centralized SAC degrades sharply across all metrics, especially under compound disruptions. Federated PPO performs moderately but still lags behind in stability and compliance. These results underscore the FDRL-SAC model's scalable cyber-resilience and its ability to preserve control and trust even in large, adversarially stressed microgrid environments.

For a deeper understanding of the performance accuracy of the proposed approach in the process of detecting and countering cyber-attacks, one can refer to the performance accuracy roulette of the proposed approach in Fig. 8. According to this figure, it can be seen that the proposed approach has an accuracy between 0.9 and 1. While the Federated PPO and Centralized SAC approaches have an accuracy between 0.7 and 0.8 and also 0.4 and 0.6, respectively. This result also shows the remarkable performance of the proposed approach and its

<figure><img src="image_10.png" /><figcaption>Fig. 8. Performance accuracy roulette in detecting cyberattacks in IEEE 33 bus.</figcaption></figure>

high accuracy in the process of detecting and countering cyberattacks.

### 4.4. Market-level analytical results

Fig. 9 illustrates the profile of SW over a 24-hour horizon in FDI attack situations. FDRL-SAC not only reflects superior median SW but also lower variability, which reflects robust control of demand-side forces. SAC with centralized control has the highest spread of welfare, reflecting sensitivity to system perturbations. Federated PPO is acceptable but lacks consistency with FDRL-SAC. Fig. 10 shows a variance heatmap of trust score among three cyberattack scenarios (FDI, DoS, and hybrid). FDRL-SAC is always lower in variance across all scenarios with the LSTM-based anomaly filtering. Centralized SAC has the highest variance, especially with hybrid attacks, and follows intermediate stability with Federated PPO.
<figure><img src="image_11.png" /><figcaption>Fig. 9. SW distribution under FDI attack.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 10. Trust score variance heatmap.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fig. 11. PVI vs. DR participation.</figcaption></figure>
Fig. 11 shows a two-axis plot comparison of PVI (left axis) and DR compliance rate (right axis). FDRL-SAC achieves best trade-off: low PVI with high DR participation. This reflects its effective trust-weighted DR mechanism. Centralized SAC performs poorly on both metrics, and Federated PPO performs better but still falls behind the model of this work. Fig. 12 depicts the revenue allocated to three consumer classes: residential, commercial, and EVCS. FDRL-SAC provides highest overall revenue with a balanced allocation to all user classes. For contrast, the revenue allocation of SAC is asymmetrical and derives less revenue from EVCS participants because of unstable pricing. PPO attains small profits but without price dynamic adjustment. Fig. 13 is a normalized performance heatmap of all three models for five attributes: SW, trust variance, DR compliance rate, market fairness, and revenue. Each is normalized on a scale of 0 to 1 for comparison. FDRL-SAC performs best consistently in all measures, especially in trust stability and fairness. Centralized SAC performs far behind, whereas PPO provides an inbetween balance.

Table 7 presents the aggregated resilience scores for each control model across all test systems, calculated based on performance under FDI, DoS, and Hybrid attack scenarios. The FDRL-SAC model consistently achieves the highest resilience across all network sizes, maintaining stable pricing and high DR compliance despite increasing system complexity and adversarial pressure. In contrast, both Centralized SAC and Federated PPO exhibit sharper performance degradation as the system scales, with the Centralized model particularly vulnerable in the 141-bus case. These results demonstrate the superior adaptability and robustness of the proposed FDRL-SAC architecture in sustaining market operations under coordinated cyber threats.

### 4.5. Scalability comparison across networks

Table 8 shows a comparative analysis of performance at each model on three ascending levels of network complexity, measured through three IEEE benchmark systems. Each model is rated against six fundamental factors: average SW, PVI, Trust Score standard deviation, DR compliance rate, control latency, and coordination failures.

These six metrics reflect not only economic efficiency but system reliability, resilience to cyber stress, and coordination scalability. The FDRL-SAC model is highly scalable in all networks. Its SW consistently high, and its very small drop from the 33-bus to the 141-bus system shows wonderful economic performance irrespective of system size. Its PVI and trust variance are all low in all networks, verifying its ability to perform stable prices and behavioral trust even for greater load and agent coordination demands. Besides, FDRL-SAC has maximum

compliance rates for DR even in the biggest system and has low latency with nearly zero coordination failure. This is an indicator of its high efficiency in distributed optimization and federated control design. Centralized SAC, however, deteriorates much with system size. Its volatility and trust instability degrade, and DR compliance decreases significantly. The rise in coordination failures signifies centralized bottlenecks and scale sensitivity. Federated PPO works better with average scalability but still has lesser trust management and DR participation than FDRL-SAC. All these comparative results reflect the superiority of the FDRL-SAC model to deliver cyber-resilient, economically optimal, and behaviorally stable control in increasingly complex MGs environments.

### 4.6. Component-wise ablation study

This section gives an ablation analysis for approximating the critical impact of each part in the constructed FDRL-SAC model. Approximation is done by partially deactivating one key feature at a time, such as LSTM filtering, trust feedback, federated updates, anomaly detection, or dynamic DR strategy, while keeping the rest of the architecture as usual. Comparison measures are SW, PVI, trust score standard deviation, DR compliance rate, and coordination failures. numerical outcomes are in Table 9. The overall model achieves maximum SW and minimum instability across all the indicators. Deactivating the LSTM filtering results in a drastic increase in PVI and coordination failures as the system is left vulnerable to anomalies without being filtered. Removing the trust feedback module critically weakens DR compliance and increases trust variance, affirming its pivotal role in promoting stable prosumer behavior. Removing federated updates causes greater coordination overhead and systemic inconsistency, especially for larger networks. Removing the anomaly detector or replacing adaptive DR with static DR heuristics, on the other hand, provides lower welfare and reliability. These findings establish interdependence as well as non-redundancy of each module in the FDRL-SAC system. Total integration of federated coordination, signal processing, and trust-aware learning needs to be performed in order to ensure cyber-resilient, scalable, and economically resilient smart grid operation.

### 4.7. Sensitivity analysis

Sensitivity analysis is conducted to evaluate the effect of the most significant hyperparameters on the control performance and robustness of the FDRL-SAC model. The parameters selected are the entropy coefficient ($\alpha$), trust smoothing factor ($\rho$), target update rate ($\tau$), and

<figure><img src="image_14.png" /><figcaption>Fig. 12. Revenue distribution by load class.</figcaption></figure>
<figure><img src="image_15.png" /><figcaption>**Fig. 13.** Normalized performance heatmap.</figcaption></figure>

<table><caption>**Table 7**

Resilience score across models and test systems.</caption><thead><tr><th>Test System</th><th>Model</th><th>Resilience Score (FDI + DoS + Hybrid)</th></tr></thead><tbody><tr><td rowspan="3">IEEE 33-bus</td><td>FDRL-SAC</td><td>91.5</td></tr><tr><td>Centralized SAC</td><td>81.4</td></tr><tr><td>Federated PPO</td><td>85.6</td></tr><tr><td rowspan="3">IEEE 69-bus</td><td>FDRL-SAC</td><td>89.7</td></tr><tr><td>Centralized SAC</td><td>77.8</td></tr><tr><td>Federated PPO</td><td>83.5</td></tr><tr><td rowspan="3">IEEE 141-bus</td><td>FDRL-SAC</td><td>88.0</td></tr><tr><td>Centralized SAC</td><td>74.1</td></tr><tr><td>Federated PPO</td><td>80.2</td></tr></tbody></table>

discount factor ($\gamma$). Each parameter was varied marginally in a reasonable working range, and all other parameters were fixed. The effects on SW and DR compliance rate were quantified and are given in Table 10. Among these, trust smoothing ($\rho$) was most sensitive. Altering $\rho$ directly affected behavioral stability and trust-based participation, leading to performance changes of up to 7.2% in DR compliance. The parameter $\alpha$ of entropy moderately affected both economic and behavior metrics because it governs the intensity of exploration. On the other hand, the target update rate ($\tau$) was low in sensitivity and thus marginal delays in policy updates have negligible effects on macro-performance. The discount factor ($\gamma$) contributed very little to aggregate variance but even suboptimal tuning could threaten temporal credit assignment. Overall, this treatment asserts careful parameter tuning of trust and exploration is necessary. Perfect calibration ensures that the model does not deviate under varying grid conditions, user behavior trends, and potential attack trends.

### 4.8. Computational and processing performance analysis

Fig. 14 provides a comparative heatmap that ranks the suggested

<table><caption>**Table 9**

Ablation analysis.</caption><thead><tr><th>Model Variant</th><th>Avg. SW ($)</th><th>PVI</th><th>Trust score Std. Dev</th><th>DR compliance rate (%)</th><th>Coordination failures (%)</th></tr></thead><tbody><tr><td>FDRL-SAC (Full Model)</td><td>421.3</td><td>0.018</td><td>0.006</td><td>92.3</td><td>0.2</td></tr><tr><td>No LSTM Filtering</td><td>404.2</td><td>0.031</td><td>0.011</td><td>85.2</td><td>0.7</td></tr><tr><td>No Trust Feedback</td><td>397.8</td><td>0.027</td><td>0.014</td><td>81.4</td><td>1.5</td></tr><tr><td>No Federated Update</td><td>390.1</td><td>0.034</td><td>0.012</td><td>80.6</td><td>2.3</td></tr><tr><td>No Anomaly Detector</td><td>399.3</td><td>0.029</td><td>0.013</td><td>83.1</td><td>1.8</td></tr><tr><td>Static DR Strategy</td><td>393.0</td><td>0.032</td><td>0.01</td><td>82.9</td><td>1.2</td></tr></tbody></table>

<table><caption>**Table 10**

Sensitivity analysis.</caption><thead><tr><th>Parameter</th><th>Baseline value</th><th>Perturbation range</th><th>Impact on SW (%)</th><th>Impact on DR compliance (%)</th><th>Stability risk</th></tr></thead><tbody><tr><td>α</td><td>0.2</td><td>[0.1, 0.3]</td><td>-4.8 to + 2.1</td><td>-5.1 to + 2.3</td><td>Medium</td></tr><tr><td>ρ</td><td>0.85</td><td>[0.7, 0.95]</td><td>-6.3 to + 3.5</td><td>-7.2 to + 3.9</td><td>High</td></tr><tr><td>τ</td><td>0.005</td><td>[0.002, 0.01]</td><td>-2.9 to + 1.4</td><td>-2.1 to + 1.5</td><td>Low</td></tr><tr><td>γ</td><td>0.99</td><td>[0.95, 0.995]</td><td>-3.2 to + 0.9</td><td>-3.5 to + 1.1</td><td>Medium</td></tr></tbody></table>

<table><caption>**Table 8**

Analytical comparison of model performance across increasing network complexity.</caption><thead><tr><th>Test System</th><th>Model</th><th>Avg. SW ($)</th><th>PVI</th><th>Trust Score Std. Dev</th><th>DR compliance rate (%)</th><th>Control latency (ms)</th><th>Coordination failures (%)</th></tr></thead><tbody><tr><td>IEEE 33-bus</td><td>FDRL-SAC</td><td>428.6</td><td>0.017</td><td>0.005</td><td>94.5</td><td>21</td><td>0.0</td></tr><tr><td>IEEE 69-bus</td><td>FDRL-SAC</td><td>421.3</td><td>0.018</td><td>0.006</td><td>92.3</td><td>26</td><td>0.2</td></tr><tr><td>141-bus</td><td>FDRL-SAC</td><td>416.7</td><td>0.02</td><td>0.008</td><td>90.1</td><td>34</td><td>0.4</td></tr><tr><td>IEEE 33-bus</td><td>Centralized SAC</td><td>392.5</td><td>0.039</td><td>0.01</td><td>84.6</td><td>18</td><td>1.8</td></tr><tr><td>IEEE 69-bus</td><td>Centralized SAC</td><td>384.8</td><td>0.043</td><td>0.012</td><td>82.1</td><td>23</td><td>2.5</td></tr><tr><td>141-bus</td><td>Centralized SAC</td><td>375.9</td><td>0.051</td><td>0.014</td><td>77.8</td><td>30</td><td>3.6</td></tr><tr><td>IEEE 33-bus</td><td>Federated PPO</td><td>400.2</td><td>0.035</td><td>0.008</td><td>86.7</td><td>24</td><td>0.8</td></tr><tr><td>IEEE 69-bus</td><td>Federated PPO</td><td>395.6</td><td>0.038</td><td>0.009</td><td>84.2</td><td>28</td><td>1.1</td></tr><tr><td>141-bus</td><td>Federated PPO</td><td>388.1</td><td>0.045</td><td>0.011</td><td>81.0</td><td>36</td><td>2.3</td></tr></tbody></table>
<figure><img src="image_16.png" /><figcaption>Fig. 14. Computational and processing metrics analysis.</figcaption></figure>

FDRL-SAC model against Centralized SAC and Federated PPO on six computational and processing metrics: Computational Burden, Processing Time, Processing Speed, Processing Accuracy, Latency, and Computational Variance. The values are normalized to be in the range [0, 1], where higher values indicate better performance. The FDRL-SAC performs better than the baseline models for all the given metrics. It is high in computational burden (0.85) and processing time (0.88), indicating its highly scalable training and inference pipeline with system complexity. It also leads in processing speed (0.92) and accuracy (0.95), indicating its highly optimized federated architecture and successful reward-driven control learning. Low variance (0.91) and low latency (0.93) also testify to its utilization for real-time market operations for cyber-exposed smart grids. On the other hand, Centralized SAC performs the worst with decreased scores across the board, particularly in Processing Time (0.55) and Latency (0.60), due to its excessive dependence on a centralized controller that is a bottleneck under distributed loads and larger grid sizes. Federated PPO has mid-level performance, as a benefit of decentralization, but lacking the anomaly-resilient control architecture and trust feedback of FDRL-SAC. This comparison confirms the computational advantage of the new model, and hence it is an improved option for large-scale, robust, and real-time control of smart energy markets.

## 5. Conclusion

This study introduced a multi-layered federated learning framework, FDRL-SAC, for resilient energy market coordination in interconnected microgrids operating under cyber and behavioral uncertainties. The model was designed to overcome limitations of centralized or purely federated control by embedding anomaly detection, trust dynamics, and DR compliance into the agent-level learning pipeline. Built around SAC policies, the architecture integrates three key innovations: (i) latent anomaly detection using VAE-GMM and LSTM forecasting, (ii) trust-aware demand response gating based on behavioral feedback and cyber exposure, and (iii) secure, adaptive real-time pricing that responds to both local conditions and network-wide learning via federated aggregation.

These layers were structured as operational algorithms (Algorithms 1 to 3), each mapped to a functional block in the system pipeline (Fig. 1), and jointly enabled agents to detect stealthy attacks, filter unreliable input, and adjust participation in market-based DR without requiring raw data exchange. The solution was trained and tested across three standard testbeds (IEEE 33-bus, IEEE 69-bus, and 141-bus systems), with extensive variation in scale, attack intensity, and control complexity.

Modeling assumptions, including agent-level observability, bounded behavioral rationality, and attack vectors targeting input-level states, were carefully chosen to reflect realistic deployment settings in distributed smart grids, while ensuring the tractability and generalizability of the model.

The performance of the proposed model has been extensively verified through detailed simulations on IEEE 33-bus, IEEE 69-bus, and 141-bus (Caracas) power grids. The simulations were conducted for different attack scenarios and with baseline comparisons against centralized SAC and federated PPO control systems. The results confirm that the model can provide high operation continuity rates and economic efficiency under attack. The performance metrics are at most 11 % increase in mean social welfare, over 50 % reduction in coordination failure, and PVI is always less than 0.02. Compliance with DR was always higher than 92 %, reflecting the effectiveness of trust-weighted participation mechanisms to provide system stability and prosumer engagement. The federated learning architecture of the platform also successfully ensured agent-level data privacy with global policy convergence. An ablation study further confirmed that all the building blocks, LSTM filtering, trust adaptation, and federated coordination, were necessary and mutually complementary to model performance. Such a confirmation is a testament to the proposed framework's robustness, flexibility, and generality across different system scales and attack surfaces. Compared to other models based on centralized detection or static heuristics, our model shows that both trust-aware control and secure pricing can be achieved cooperatively and in parallel in distributed systems.

The core insight derived from this work is that cyber-resilience must be co-designed with learning, not added as a post-processing step or hard constraint. By embedding trust and anomaly feedback directly into the reward structure and participation logic, the agents become inherently more adaptive and robust to both cyber and behavioral uncertainty. The federated structure further supports scalability and privacy preservation without sacrificing coordination. This modular integration of detection, decision-making, and incentive alignment provides a pathway for real-time, secure energy market coordination in next-generation power systems.

Looking ahead, this architecture offers a foundation for several promising extensions. Future work may integrate adversarial reinforcement learning to simulate strategic attackers and better stress-test policy robustness. The model can also be extended to multi-energy systems involving gas, heat, and storage coordination, or hybrid frameworks that incorporate blockchain-based transaction layers for verifiable DR contracts. Ultimately, the FDRL-SAC model contributes not only a novel technical framework but also a replicable and forward-
looking approach to aligning intelligent control with security, economics, and decentralization in modern power systems.

## CRediT authorship contribution statement

**Reza Sepehrzad:** Writing – original draft, Software, Resources, Methodology, Investigation, Formal analysis, Conceptualization. **Hosein Hosseinalibeiki:** Writing – review & editing, Validation, Software, Formal analysis, Data curation. **Mahdieh S. Sadabadi:** Writing – review & editing, Visualization, Supervision, Project administration, Methodology. **Ahmed al Durra:** Methodology, Project administration, Supervision, Validation, Writing – review & editing.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgments

This work was supported by Khalifa University of Science and Technology (KUST), Abu Dhabi, UAE, under Award CIRA-2021-063.

## Data availability

Data will be made available on request.

## References

[1] Sepehrzad R, et al. Two-Stage experimental intelligent dynamic energy management of microgrid in smart cities based on demand response programs and energy storage system participation. Int J Electr Power Energy Syst 2024;15:109613.

[2] Fazlhashemi SS, et al. Risk-averse operation of microgrids in distribution systems with price-responsive demands. Int J Electr Power Energy Syst 2025;168:110581.

[3] Hedayatnia A, et al. Two-stage data-driven optimal energy management and dynamic real-time operation in networked microgrid based on a deep reinforcement learning approach. Int J Electr Power Energy Syst 2024;160:110142.

[4] Sepehrzad R, et al. Demand response-based multi-layer peer-to-peer energy trading strategy for renewable-powered microgrids with electric vehicles. Energy 2025; 320:135206.

[5] Reza S, et al. Multilevel energy management and optimal control system in smart cities based on deep machine learning. In: Applications of Deep Machine Learning in Future Energy Systems. Elsevier; 2024. p. 265–314.

[6] Chaudhary AK, et al. Adaptive cyber-tolerant finite-time frequency control framework for renewable-integrated power system under deception and periodic denial-of-service attacks. Energy 2024;302:131809.

[7] Rouhani SH, et al. Cyber resilience in renewable microgrids: a review of standards, challenges, and solutions. Energy 2024;309:133081.

[8] Dai J, et al. Cyber-resilience enhancement with cross-domain software-defined network for cyber-physical microgrids against denial of service attacks. IEEE Trans Ind Cyber-Phys Syst 2025.

[9] Asghari M, et al. An optimal cyber-physical attack strategy on DC microgrids. Int J Electr Power Energy Syst 2024;157:109900.

[10] Li X, et al. Resilient distributed economic dispatch for cyber-physical power systems considering carbon emissions trading and false data injection attacks. Energy 2025:136881.

[11] Khodadadi A, et al. Data-Driven hierarchical energy management in multi-integrated energy systems considering integrated demand response programs and energy storage system participation based on MADRL approach. Sustain Cities Soc 2024;103:105264.

[12] Sepehrzad R, et al. Enhancing cyber-resilience in electric vehicle charging stations: a multi-agent deep reinforcement learning approach. IEEE Trans Intell Transp Syst 2024.

[13] Wang D, et al. Research on floating real-time pricing strategy for microgrid operator in local energy market considering shared energy storage leasing. Appl Energy 2024;368:123412.

[14] Melendez KA, Matamala Y. Adversarial attacks in demand-side electricity markets. Appl Energy 2025;377:124615.

[15] Rouhani SH, Mobayen S, Chun-Lien Su. Robust finite-time frequency control of grid connected microgrids for real-time electricity market against cyber-attacks. IEEE Trans Power Syst 2025.

[16] Veerasamy V, et al. Blockchain-enabled peer-to-peer energy trading and resilient control of microgrids. Appl Energy 2024;353:122107.

[17] Zhang Z, et al. Market-level integrated detection against cyberattacks in real-time market operations by self-supervised learning. IEEE Trans Smart Grid 2024;15(4): 4128–42.

[18] Rouhani SH, et al. Barrier-function adaptive finite-time trajectory tracking controls for cyber resilience in smart grids under an electricity market environment. IEEE Trans Smart Grid 2024.

[19] Naderi E, Asrari A. A remedial action scheme to mitigate market power caused by cyberattacks targeting a smart distribution system. IEEE Trans Ind Inf 2023;20(3): 3197–208.

[20] Farahani A, et al. A new bi-level model for the false data injection attack on real-time electricity market considering uncertainties. Comput Electr Eng 2024;118: 109468.

[21] Dedrick J, et al. Assessing cyberattacks on local electricity markets using simulation analysis: impacts and possible mitigations. Sustainable Energy Grids Networks 2023;34:100993.

[22] Tang D, Fang Y-P, Zio E. Vulnerability analysis of demand-response with renewable energy integration in smart grids to cyberattacks and online detection methods. Reliab Eng Syst Saf 2023;235:109212.

[23] Sepehrzad R, et al. A multi-agent deep reinforcement learning paradigm to improve the robustness and resilience of grid connected electric vehicle charging stations against the destructive effects of cyber-attacks. Energy 2024;307:132669.

[24] Zhou Z, et al. Resilient energy management strategy in smart residential buildings considering price attack: an aggregative game perspective. Energ Buildings 2024; 320:114593.

[25] Tostado-Véliz M, et al. An Interval-based privacy-aware optimization framework for electricity price setting in isolated microgrid clusters. Appl Energy 2023;340: 121041.

[26] Tirulo A, Chauhan S, Issac B. Ensemble LOF-based detection of false data injection in smart grid demand response system. Comput Electr Eng 2024;116:109188.

[27] Ahsan SM, Gholizadeh N, Musilek P. Multi-agent systems in networked microgrids: reinforcement learning and strategic pricing mechanisms. Renew Energy 2025: 123678.

[28] Xu W, et al. Game-based pricing for joint carbon and electricity trading in microgrids. IEEE Internet Things J 2024;11(16):27732–43.

[29] Ahsan SM, et al. Adaptive pricing-based optimized resource utilization in networked microgrids. IEEE Access 2025.

[30] Pinheiro Cinelli L, et al. Variational autoencoder. In: Variational Methods for Machine Learning with Applications to Deep Networks. Cham: Springer International Publishing; 2021. p. 111–49.

[31] Gogebakan M. A novel approach for Gaussian mixture model clustering based on soft computing method. IEEE Access 2021;9:159987–60003.

[32] Ji S, et al. Kullback-Leibler divergence metric learning. IEEE Trans Cybern 2020;52 (4):2047–58.

[33] Wang JQ, Yu Du, Wang J. LSTM based long-term energy consumption prediction with periodicity. Energy 2020;197:117197.

[34] Haarnoja T, et al. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. International Conference on Machine Learning. 2018.

[35] Han M, et al. Actor-critic reinforcement learning for control with stability guarantee. IEEE Rob Autom Lett 2020;5(4):6217–24.

[36] Wang S, et al. On multi-event co-calibration of dynamic model parameters using soft actor-critic. IEEE Trans Power Syst 2020;36(1):521–4.

[37] Hu Y, Fu J, Wen G. Graph soft actor-critic reinforcement learning for large-scale distributed multirobot coordination, IEEE Transactions on Neural Networks and Learning Systems, 2023.

[38] Gao J, et al. Learning end-to-end visual servoing using an improved soft actor-critic approach with centralized novelty measurement. IEEE Trans Instrum Meas 2023; 72:1–12.

[39] Ghane S, et al. Federated proximal policy optimization with action masking: application in collective heating systems. Energy AI 2025;20:100506.

[40] Yu T, et al. Multi-agent proximal policy optimization-based dynamic client selection for federated AI in 6G-oriented internet of vehicles. IEEE Trans Veh Technol 2024;73(9):13611–24.