# Enhanced cyber-resilience in flexible energy markets for microgrids: A trust-aware federated deep reinforcement learning framework

Reza Sepehrzad, Hossein Hosseinalibeiki, Neda Taghinezhad, Nima Khosravi, Ahmed Al Durra, Mahdieh S Sadabadi

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Reza Sepehrzad, Hossein Hosseinalibeiki, Neda Taghinezhad, Nima Khosravi, Ahmed Al Durra, et al.: Enhanced cyber-resilience in flexible energy markets for microgrids: A trust-aware federated deep reinforcement learning framework. Applied Energy, 2026, 406, 10.1016/j.apenergy.2025.127282. hal-05459256

HAL Id: hal-05459256

https://hal.science/hal-05459256v1

Submitted on 15 Jan 2026

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# Enhanced cyber-resilience in flexible energy markets for microgrids: A trust-aware federated deep reinforcement learning framework

Reza Sepehrzad<sup>a, *,</sup>, Hossein Hosseinalibeiki<sup>b</sup>, Neda Taghinezhad<sup>c</sup>, Nima Khosravi<sup>d</sup>, Ahmed al Durra<sup>e</sup>, Mahdieh S. Sadabadi<sup>f</sup>

## HIGHLIGHTS

* Trust-sensitive filtering and robust market model integrating demand response.

* Secure model of day-ahead and real-time pricing against the cyber threats.

* Attack-aware policy verification in future flexible energy markets of microgrid.

## ARTICLE INFO

### Key words::

Microgrid

Flexible energy markets

Cyberattack resilience

Secure energy market

Trust-aware federated deep reinforcement

learning

## ABSTRACT

This paper introduces a trust-aware, cyber-resilient approach to defend against adversarial cyberattacks against microgrid flexible energy markets. The new method applies federated day-ahead and real-time pricing controlled through a trust-aware federated deep reinforcement learning (TAFDRL) framework. The TAFDRL framework leverages encrypted updates, where each prosumer agent identifies a local soft actor-critic policy, along with a local policy that was updated based on Bayesian trust scores and tested for policy anomaly (latent policy anomaly model). Malicious updates are detected using the Mahalanobis distance from learned embedding space, which enables a trust-weighted average, which allows for secure model aggregation of individual learning. The TAFDRL framework was tested under numerous scenarios for false data injection (FDI), denial-of-service (DoS), and hybrid attack. The TAFDRL framework demonstrated a 94% cumulative reward rate (with 92% stability) and very high accuracy for detecting cyberattacks (< 2% error tolerance using 200 operational points), which maximized the reliable identification of suspicious policies that possess a persistent, resilient ability despite cyber threats. These results confirm that TAFDRL effectively balances cyber defense, aggressive learning performance, while maintaining economic stability. The research provides a robust framework for operating energy markets securely against an increasing number of cyber threats to energy distribution systems.

## 1. Introduction

The increase in distributed energy resources (DERs), prosumer participation, and two-way power flows has rendered modern power systems considerably dynamic and data-driven paradigms [1]. As a countermeasure, flexible energy markets have been suggested to enable real-time trading of energy, adaptive demand response (DR) schemes,

and price-based coordination. Such markets heavily rely on automated control, multi-agent decision-making, and iterative data exchange via cyber-physical interfaces [2]. While this flexibility improves system responsiveness and efficiency, it simultaneously expands the attack surface and poses significant cybersecurity risks. In particular, false data injection (FDI), denial-of-service (DoS), and adversarial control policies pose growing risks to market stability, DR performance, and price

<footnote><sup>a</sup> Department of Electrical and Computer Engineering, University of Canterbury, Christchurch, New Zealand</footnote>

<footnote><sup>b</sup> DISA-MIS Department, University of Salerno, Fisciano 84084, Italy</footnote>

<footnote><sup>c</sup> Faculty of Electrical and Computer Engineering, Shiraz University, Shiraz, Iran</footnote>

<footnote><sup>d</sup> Centre of Research Impact and Outcome, Chitkara University, Institute of Engineering and Technology, Chitkara University, Rajpura 140401, Punjab, India</footnote>

<footnote><sup>e</sup> Advanced Power & Energy Center, EECS Department, Khalifa University, Abu Dhabi, United Arab Emirates</footnote>

<footnote><sup>f</sup> Department of Electrical and Electronic Engineering, University of Manchester, Manchester, United Kingdom</footnote>

<footnote>* Corresponding author.
E-mail address: Sepehrzad.reza@gmail.com (R. Sepehrzad).</footnote>
integrity [3].

The growing complexity of modern microgrids in the future requires an architecture capable of providing the safe, reliable distributive and coordinating functions and which will still be accountable for the emerging spectrum of cybersecurity threats. The emergence of decentralized and data-based elements such as DERs and multi-agent decision making means that safeguarding the market is of primary importance [4]. Failing to secure energy markets from cyber threats such as data manipulation or unauthorized access can end in major disruptions, which can include outages in energy supply, or worse, a cascading effect. Therefore, securing this part of the architecture as an important security framework is essential [5]. The ability to evaluate real-time potential cyberattacks, as well as assessing the designed resilience of the existing systems against adversarial actions, are crucial to maintaining the trust and stability of the energy market [6]. Solidifying the security provisions will not only improve the system's ability to detect and protect against future threats but also to promote the overall reliability and transparency of the market, ensuring secure and efficient energy transactions [7].

Recent work has advanced combating these challenges through federated learning, trust-aware coordination, and multi-agent reinforcement learning [8]. Federated deep reinforcement learning (FDRL), specifically, has been noted to have received attention for allowing decentralized policy training without compromising data privacy. Yet, most FDRL implementations in energy systems rely on the assumption of a benign environment and do not incorporate strong mechanisms for handling compromised agents or unreliable policy updates [9]. Furthermore, legacy DR architecture conventionally resolves day-ahead and real-time coordination as distinct layers without considering interdependencies between short-term flexibility and long-term planning. Further, trust values are hardly being directly applied within the reinforcement learning process or used for policy contribution filtering in federated aggregation [10].

## 1.1. Motivation and main idea

To alleviate these challenges, this paper proposes a trust-aware, cyber-resilient market structure integrating federated deep reinforcement learning with secure pricing and DR coordination in a microgrid. The system architecture includes a hybrid day-ahead and real-time model run by a central aggregator, whereby prosumers train local SAC-based DRL models. A Bayesian trust engine evaluates the behavior of agents through multi-dimensional cyber-physical metrics, influencing both market engagement and contribution to learning. To oppose adversarial updates, a neural encoder-decoder model maps policy embeddings to a latent space and employs trust-sensitive anomaly scoring by Mahalanobis distance. Moreover, the paper introduces formal mathematical models of FDI and DoS attacks on the observation, communication, and policy levels to enable rigorous analysis of the resilience of the framework. This combined architecture ensures decentralized coordination, and attack-aware policy verification in future flexible energy markets of microgrid.

## 1.2. Related works

Cyber and FDI attacks have a significant negative impact on the financial and operational stability of the energy markets, but the research is lacking in literature and actionable plans to mitigate their impacts in a comprehensive and timely manner. An FDI attack can interfere with state estimation capabilities and locational marginal price (LMP), which can affect the potential for cascading outages and market price manipulation. Ref [11] investigates revenue adequacy risks for ISOs under FDI but stops short of implementing agent-level filtering/trust-based protections that would block the attacks from the system. Ref [12] and [13] use bilevel optimization to model the attitude of stealthy attacks, but only used static system topologies without including or

accounting for cyber-layer dynamics that would allow the system to adapt for resilience. Alternatively, Ref [14] extends S- and MM-estimators to protect estimation quality under attacks, but it does not consider market adjustments or adversarial activities learned by models. While Ref [14] suggests the use of weighted least squares (WLS) in federated control, it neglects to take into account the contributions of the models when managed with historic behavioral data or adversarial detection. Study [15] suggested a reconfiguration strategy to curb the exposure of vulnerabilities, but did not give consideration to the ability of the system to respond to a changing operational state or trust over time. Therefore, many existing methods are static, reactive, and poorly model the cyber-physical feedback that is crucial to the process. The consequence for federated and decentralized systems is high exposure to risks such model poisoning, unsafe convergence and systemic instability. The techniques in studies [12] and [14] use optimization or estimation methods that have no explicit measures of risk for poisoned models or corrupted data during training, while [13] models a set of FDI attacks towards market-clearing mechanisms without modelling the learning algorithms themselves, or their security. These research gaps highlight an immediate need for monitors or models that are dynamic, trust-aware, and equipped with feeds for detecting adversarial impacts across the energy market systems.

Although the body of work concerning trust functions has advanced significantly, the literature continues to overlook the full impact of trust functions with regard to energy trading, behavioral learning, and market participation. For example, while study [16] employs game-theoretic models to investigate the strategic approach to interacting with consumers, it does not provide a formal methodology for trust computation or sanctioning opportunistic agents. Without a trust framework the market will be blind to malicious behavior. Similarly, study [17] presents a blockchain-based transactive energy system with the goal of preventing the alteration of transaction authenticity, but the system assumes that all participants are honest and neglects to assess the risks of policy manipulation or adversarial actors acting against the energy market. Study [18] proposes a local market structure based on IoT device-based nodes and attributes cyber-resilience to the decentralized operation of the system. While this may offer additional security, no trust metric is utilized to assess the reliability of agents over time, leaving a potential weakness to unreliable or malicious agents. Both studies [17] and [18] show DR at the prosumer level but fail to implement a real-time process for responding to an attack or filtering out malicious policy updates from the DR process. Ref [19] proposes a probabilistic model to assess the cost of cyberattacks, but utterly omits long-term feedback and trust metrics based on agent behavior. There is no dynamic trust estimation or filtering mechanisms, allowing agents who have not been verified, or are malicious, to continue impacting system learning and decision-making based on the agent classification; therefore, delaying the reliability and subsequent cyber resilience of the energy market.

The literature recognizes a gap in the literature reviewing DR for modern energy systems that integrates DR in the cyber-resilient market architecture. For example, [20] examined the forces of digitalization to execute DR benefits from cost savings and efficiencies, but did not consider cyberattack incidents that may be used to manipulate DR metrics. Meanwhile, [21] establishes a process, "corrective process" that relies on the DR agency but does not establish DR as an affordance and as a coordinated tactic for dynamic pricing based exclusively on agent trust. Similar issues can be found on [22] where they proposed self-supervised learning to abstract out real-time market anomalies, yet there was no logic associated with control of DR, or coordination with decentralized agents, where coordination of many decentralized energy systems is either uncoordinated across time-scales or exploitable through adversarial strategies. In addition, even the federated and decentralized forms of learning have been addressed in regard to power systems, they often do not utilize crucial security measures and trust-control applications to protect the contributing models to distort. In
[22], for instance, the main study finds anomalies in market data but does not provide any means of preventing unsafe policy updates or filtering the unnecessary or unsafe contributions to learning. Thus, none of these publications provide a full framework/approach to protecting and modelling DR in a decentralized, cyber-resilient energy market efficiently.

In microgrids, a primary challenge to the integrity and resilience of local electricity markets (LEMs) emerges from the increasing complexity of distributed energy resources and decentralized energy trading arrangements. Ref [23] elaborately examined the impact of cyberattacks on LEMs from the perspective of cyberattack tactics and behavior, specifically focusing on DoS attacks. The research utilized simulation approaches to determine the effects of data lost through DoS attacks. The conclusions drawn from the information loss, if not verified, were substantial, even as little as a 3% loss of data led to significant market distortions and system instability. The study emphasized the necessity of resilient mitigation strategies to reduce LEM's vulnerabilities to an attack. Another study on P2P energy trading found in microgrids focused on modelling, and eventually leads to the use of federated learning-based fractional order recurrent neural network (FL-FORNN) adaptive controller to regulate frequency in the system. Importantly, the FLFORNN is paired with local differential privacy (LDP) in order to counteract false data injection attacks, thus facilitating secure trading for renewable energy consumers. The results showed the proposed controller was capable of securing the microgrid's P2P trading system [24]. In Ref [25], the authors examine and compare some of the prominent double-sided auction mechanisms for P2P transactions taking place with LEMs, using distributed ledger technology. Their research concludes that there is not one auction model which clearly dominates the others, and that there are considerable challenges surrounding the efficient use of distributed ledger technology for local energy trading. One major challenge that each of these studies points to is the need for holistic approaches, which tie together the areas of cyber-resiliency, trust management, and secure communications protocols, in a way that can both accommodate risk and maximize market performance, during a period of evolving risk in flexible energy systems.

While the field of literature on cyber resilience in energy systems is expanding, there is no study in the literature that combines a way of integrating key aspects of cyber resilience, trust management, DR, and market learning at different levels, even though several studies either look at individual aspects like trust-aware energy market models or DR eligibility. However, there are no studies combining the aspects in an integrated way via a market architecture for microgrids. There is a dearth of studies presenting a framework integrating critical components such as policy-level anomaly detection, encrypted federated

aggregation and modeling of cyber-physical attacks. Much existing literature provides the static role of agents, assumes benign participation, or considers sequential mitigation after the fact to deal with cyber threats. Many of these models do not account for the dynamic process of energy markets, in which the adaption for day-ahead and real-time pricing requires the cooperation of these time horizons to ensure system stability and viability. Additionally, the current models do not address the complexities of the trust-informed learning process nor the role of flexible participation and usage in conjunction with trust-accountable energy markets. As such, these models lack the assurance of a secure, reliable, and inclusive model of participation for an energy market of the future, requiring a more integrated, layered model of engagement. Table 1 compares related studies and research.

## 1.3. Main contribution

The primary contribution of this research is the contribution of a new cyber-aware, resilient, and trust-aware energy market architecture for microgrids that incorporates the state-of-the-art TAFDRL, secure demand response coordination, real-time mitigation measures against attacks, and a hybrid day-ahead and real-time market. This synthesis addresses critical weak points that exist in established microgrid energy market models, particularly weak points that are associated with adversarial disruptive cyberattacks, such as FDI, DoS, and hybrid attacks. The framework includes a new Bayesian trust engine that continuously evaluates the behavior of agents with respect to multidimensional cyber-physical indicators. This allows for the real-time evaluation of both the present and trusted future actions of agents, as well as their contributions to federated learning processes in the wider market. The framework includes an encrypted model sharing, which gives agents the opportunity to collaborate while also protecting sensitive information. Trust-sensitive policy aggregation adds additional value by ensuring that only reliable contributions are aggregated for learning and overall improvement of the energy market.

To protect against adversarial threats, the framework implements neural policy anomaly detection using Mahalanobis distance that can actively identify and reject malicious updates in real-time, thus improving the security and reliability of the framework under persistent cyberattacks. The framework also formalizes, and simulates attack combinations, as well as the FDI, DoS, and hybrid threats, where the attackers can impact the observation, communication, and learning layers of the market. This way, the multiple layers of attack can be studied regarding the resilience of the market against other cyber threats, enabling adaptive decentralized coordination to dynamically varying contexts. The proposed framework incorporates advanced

<table><caption>**Table 1**
Comprehensive comparison of related research</caption><thead><tr><th>References</th><th>Cybersecurity</th><th>Trust-aware Management</th><th>Federated Learning</th><th>Demand Response</th><th>Attack Detection and Mitigation</th><th>Resilience</th><th>Market Flexibility</th><th>Data Privacy</th><th>Real-Time Market Coordination</th></tr></thead><tbody><tr><td>[11]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>-</td></tr><tr><td>[12]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>-</td></tr><tr><td>[13]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>-</td></tr><tr><td>[14]</td><td>*</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td>[15]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>-</td></tr><tr><td>[16]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td>[17]</td><td>*</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td>[18]</td><td>*</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td>[19]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>-</td></tr><tr><td>[20]</td><td>*</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td>[21]</td><td>*</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>-</td><td>*</td><td>*</td></tr><tr><td>[22]</td><td>*</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td>[23]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>*</td><td>*</td><td>-</td></tr><tr><td>[24]</td><td>*</td><td>-</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>-</td></tr><tr><td>[25]</td><td>*</td><td>-</td><td>-</td><td>-</td><td>*</td><td>*</td><td>-</td><td>*</td><td>-</td></tr><tr><td>Proposed method</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td><td>*</td></tr></tbody></table>
developments in federated learning, trust management, and attack mitigation to preserve fairness, security, and functionality in microgrid energy markets. It is clear that the framework is highly adaptable and responsive, but it goes beyond current cyber threats to prepare for the future threats we will face in the increasingly decentralized energy flow. Nevertheless, this paper is a considerable development in building secure, adaptive, and sustainable energy markets, which maximize reliable and efficient energy distribution in the presence of sophisticated adversaries, moreover, in capturing their operational continuity.

In all, the key contributions of this research are:

1. The paper advances a hybrid and robust market model integrating demand response coordination and TAFDRL under cyberattacks in microgrid.

2. This work presents a multi-layer trust engine that filters market participation and learning contributions based on Bayesian inference, behavioral profiling, and anomaly detection using neural policy embeddings.

3. Suggested framework guards against FDI, DoS, and hybrid attacks using encrypted model aggregation, trust-sensitive filtering, and resilience-aware optimization on both microgrid's market and cyber layers.

The rest of this paper is organized as follows: the problem formulation is presented in section 2, The problem-solving space and policy filtration are presented in section 3, configuration of the results in section 4 and conclusion in section 5 are presented.

## 2. Problem formulation

The distribution grid architecture, shown in Fig. 1, is implemented over a radial microgrid power network based on the IEEE 69-bus test system and includes the following distributed energy resources (DERs): PV panels; wind turbines (WTs); batteries as battery energy storage systems (BESS); diesel generators (DG); and EV charging stations (EVCS). EVs are considered flexible demand response participants and can either flex their load or provide energy to the grid via V2G methods. The operation is based on intelligent prosumer agents (consumers and businesses and EV stations), who are permitted to participate in energy trading. The operation enables simulation of secure price development, trust-aware coordination, and cyber-resilient operation across multiple circumstances. The market structure can be described at two time scales. The first is a day-ahead layer where prosumers are offered an advanced schedule of participation in the market, and the second is a real-time

<figure><img src="image_5.png" /><figcaption>Fig. 1. Microgrid architecture in IEEE 69-bus distribution grid</figcaption></figure>
layer which is dynamically responsive to events. In the day-ahead market, prosumers make a forecast of their energy interaction with the market, while in the real-time market, system prices and demand response signals are updated every few minutes to respond to the variability of DERs or cyber-attack indicators and deviations from established schedules.

A TAFDRL mechanism supports privacy-preserving, decentralized learning and coordination across agents. This mechanized two-layer market provides flexibility to respond to sudden changes and leverages trust scores for both short- and long-term decision-making. The mechanism, which is tied into communication and cybersecurity, deals with encrypted communications for agents, DER controllers, and the aggregator. For the architecture, cyber-related threats will consider several different scenarios testing hybrid attacks (FDI, DoS, and hybrid attacks) and mitigating these through trust filtering, anomaly detection, and policy verification. There are three primary agent role types: market aggregator, prosumers, and trust evaluation engine, collaborating to manage prices, deviations (DR events), and secure integration of policies. Trust scores provide a way of dynamically managing participation rights, which provide advantages to agent access in the market and consideration of weight in the learning process; hence, reliable agents contribute most to the behavior of the system, while having an emphasis on inclusiveness and resilience.

## 2.1. Hybrid multi-timescale market aggregator

The market aggregator runs simultaneous hybrid market activities that enable them to optimize both day-ahead and real-time decisions, manage DR, publish dynamic price signals, and execute trust-aware federated learning [26]-[28]. This formulation enables greater consistency, resilience, and learning interdependence across temporal layers using a multi-task learning algorithm based on multi-objectives function (MTL-MOF) algorithm [29].

### 2.1.1. Hybrid market objective

This sub-section formulates a coordinated optimization objective for day-ahead and real-time markets for energy. It captures energy prices, DR incentives, and adaptive pricing under trust-based aggregation. The hybrid setup allows the system to respond adaptively to uncertainties and dynamic behavior across various time layers. The objective function in (1) maximizes the aggregate cost of energy transactions and rewarded DR, including day-ahead commitments and real-time adjustments [14]-[16]. This shared formulation enables the aggregator to take flexibility and trust into a single decision structure.

$$ \min_{E_i^{\text{DA}}, \Delta_i^{\text{DA}}, e_i^{\text{RT}}, \delta_i^{\text{RT}}} \sum_{i=1}^{N} [C_i^{\text{DA}}(E_i^{\text{DA}}) - R_i^{\text{DA}}(\Delta_i^{\text{DA}}) + p_i^{\text{RT}} e_i^{\text{RT}} - r_i^{\text{RT}} \delta_i^{\text{RT}}] \quad (1) $$

where $E_i^{\text{DA}}$ is day-ahead energy scheduled for prosumer i. $\Delta_i^{\text{DA}}$ shows day-ahead DR flexibility bid from prosumer i. $C_i^{\text{DA}}(\cdot)$ is cost function of day-ahead scheduled energy. $R_i^{\text{DA}}(\cdot)$ describes incentive function for DR capacity commitment. $e_i^{\text{RT}}$ is real-time energy deviation from schedule. $\delta_i^{\text{RT}}$ shows actual DR delivered in real time. $p_i^{\text{RT}}$ is real-time unit energy price. $r_i^{\text{RT}}$ is real-time DR reward rate. Market constraints define the fundamental operational constraints for energy balance and demand response activation. It ensures that planned and actual energy schedules align with system limitations, while also enforcing DR availability caps. These constraints provide a safety envelope around the optimization objectives.

$$ \sum_{i=1}^{N} E_i^{\text{DA}} = D^{\text{DA}}, \sum_{i=1}^{N} (e_i^{\text{RT}} + \delta_i^{\text{RT}}) = D^{\text{RT}} - D^{\text{DA}} \quad (2) $$

$$ 0 \le \Delta_i^{\text{DA}} \le \Delta_i^{\text{max}}, |\delta_i^{\text{RT}}| \le \Delta_i^{\text{DA}} \quad (3) $$

where $D_i^{\text{DA}}$ describes total day-ahead forecasted demand. $D_i^{\text{RT}}$ is realized real-time demand. $\Delta_i^{\text{max}}$ shows maximum DR flexibility from prosumer i.

### 2.1.2. Trust-based DR eligibility and federated model aggregation

The structure articulated below specifies a filtering mechanism based on agent trust levels. In this way, it restricts the participation of agents in DR and learning to agents who meet a certain minimum set of behavioral standards. This mechanism also avoids undesired influence upon the critical functions of the system by agents that are unstable or malicious [7]-[9].

$$ \mathcal{T} = \{i \in \{1, \dots, N\} : T_i \ge \tau\}, \delta_i^{\text{RT}} = 0 \text{ if } i \notin \mathcal{T} \quad (4) $$

where $T_i$ is trust score of agent i. $\tau$ shows minimum trust threshold. $\mathcal{T}$ describes set of DR-eligible agents. The following equation describes a federated learning approach whereby the market aggregator will aggregate the encrypted model updates. The trust-weighted average will ensure the only trustworthy agents can affect global learning which enhances robustness against poisoning and harmful models.

$$ \theta^{\text{agg}} = \frac{1}{\sum_{j \in \mathcal{T}} T_j} \sum_{i \in \mathcal{T}} T_i \cdot \theta_i \quad (5) $$

where $\theta_i$ shows local DRL model parameters from agent i. $\theta^{\text{agg}}$ describes trust-weighted aggregated global policy vector.

### 2.1.3. Prosumer agent modeling

Every prosumer agent is an independent, intelligent decision-maker that can modify energy consumption, generation, and DR participation based on market signals and trust-based incentives. The following section formulates the internal objective, DRL framework, and privacy-preserving learning contribution of every agent. Eq. (6) formulates the individual decision-making objective of every prosumer agent [26], [27]. The objective is to maximize cumulative expected rewards over time through adaptation to price signals, trust incentives, and penalties. It forms the fundamental rationale for reinforcement learning optimization [28]-[29]. Each agent $j \in \{1, 2, \dots, N\}$ aims to maximize its local return by responding to energy prices and DR incentives while preserving energy balance and local constraints [30], [31]. The cumulative reward is defined as:

$$ \max_{\pi_j} \mathbb{E}_{\pi_j} \left[ \sum_{t=0}^{T} \gamma^t \mathcal{R}_j^t \right] \quad (6) $$

where $\pi_j$ is local policy of agent j. $\mathcal{R}_j^t$ shows trust-aware reward at time t. $\gamma \in [0, 1)$ is discount factor and T describes horizon length. Eq. (7) explains how each agent trains its local policy using DRL methods. The agent optimizes its Q-value function based on private interactions, maintaining autonomy and privacy [2], [5]. Each agent uses a continuous-action DRL algorithm, such as SAC, to learn an optimal policy $\pi_j$. Training is based solely on local interactions:

$$ \pi_j^* = \underset{\pi_j}{\operatorname{argmax}} Q_j^\gamma(\mathbf{s}_j, \mathbf{a}_j) \quad (7) $$

where $Q_j^\gamma$ is action-value function under policy $\pi_j$. $\mathbf{s}_j$ shows agent's local state and $\mathbf{a}_j$ is selected action. Eqs. (8) and (9) define the inputs and outputs of the DRL agent: state vectors include load, DER status, price, and trust; actions involve flexible dispatch; rewards include incentives and penalties. At each time t, the agent observes a local state vector and selects an action:

$$ \mathbf{s}_j^t = [\text{load forecast}, \text{DER status}, \text{price signal}, \text{trust score}] $$

$$ \mathbf{a}_j^t = [\text{DR level}, \text{charging rate}, \text{storage dispatch}] \quad (8) $$

$$ \mathcal{R}_j^t = - \alpha_j c_j^t + \beta_j r_j^t - \lambda_j v_j^t \quad (9) $$
where $c_j^t$ shows energy cost incurred by agent j. $r_j^t$ is DR reward received, $v_j^t$ describes penalty for violating trust-aligned constraints and $\alpha_j, \beta_j, \lambda_j$ are agent-specific scaling weights.

### 2.1.4. Encrypted model sharing

Eq. (10) characterizes encrypted policy updates to enable reliable model sharing in federated learning. Agents transmitted policy updates that were encrypted by the agents only once the trust score threshold has been met established the trustworthiness of the data, while also preserving privacy [2], [5]. After each local training round, agent j encrypts and transmits its model parameters $\theta_j$ to the aggregator:

$$ \mathcal{E}(\theta_j) \rightarrow \text{Aggregator, if } T_j \ge \tau \quad (10) $$

where $\theta_j$ is updated policy parameters, $\mathcal{E}(\cdot)$ shows encryption function, $T_j$ is agent's current trust score and $\tau$ describes trust threshold for upload eligibility. This structure facilitates privacy-protecting participation in the federated learning process while addressing that only reliable agents contribute to the global policy.

### 2.2. Adversarial threat modeling

In this section, we model adversarial behaviors with matrix representation, which provides structural clarity, and computational scalability. We addressed two major classes, FDI with DoS, both of which compromise the system's learning, coordination, and trustworthiness. The FDI attacks are modeled as structured matrix perturbations applied to local observation vectors [9], [10]. Let $\mathbf{X}_j \in \mathbb{R}^{T \times d}$ denote the true observation matrix of agent j over T time steps and d features. The FDI-corrupted matrix is:

$$ \tilde{\mathbf{X}}_j = \mathbf{X}_j + \mathbf{M}_j \odot \mathbf{B}_j \quad (11) $$

where $\tilde{\mathbf{X}}_j$ is falsified observation matrix, $\mathbf{M}_j \in \{0, 1\}^{T \times d}$ shows binary attack mask matrix, $\mathbf{B}_j \in \mathbb{R}^{T \times d}$ describes perturbation magnitude matrix and $\odot$ is Hadamard (element-wise) product. Eq. (11) enables dynamic, time-varying multi-channel deception where different elements of the observation matrix are corrupted at different times. DoS attacks are modeled as temporal dropouts in the communication matrix. Let $\mathbf{\Phi}_j \in \{0, 1\}^{T \times 1}$ be the binary transmission vector for agent j, and let $\mathbf{\Theta}_j \in \mathbb{R}^{T \times d}$ be its model or message matrix:

$$ \mathbf{\Theta}_j^{\text{recv}} = \text{diag}(\mathbf{\Phi}_j) \cdot \mathbf{\Theta}_j \quad (12) $$

where $\mathbf{\Theta}_j^{\text{recv}}$ shows masked message matrix received by aggregator, and diag($\mathbf{\Phi}_j$) describes diagonal matrix formed from the transmission success vector. This blocks all communication rows ( timestamps ) where $\phi_j^t = 0$, simulating full or partial communication outage during attacks.

### 2.2.1. Combined impact on aggregator

Let $\mathbf{W}^t = \text{diag}(\mathbf{w}^t \odot \phi^t)$ be the trust-weighted availability matrix at time t, and stack all encrypted models $\mathbf{\Theta}^t = [\mathcal{E}(\theta_1)^T, \dots, \mathcal{E}(\theta_N)^T]^T$. This matrix-based formulation makes it easier to model adversarial manipulation across time and agents, enabling future resilience strategies using linear algebra and anomaly detection [11], [12]. The aggregated global model becomes:

$$ \mathbf{\Theta}^{\text{agg}} = (\mathbf{1}^T \mathbf{W}^t \mathbf{1})^{-1} \cdot \mathbf{W}^t \cdot \mathbf{\Theta}^t \quad (13) $$

where $\mathbf{\Theta}^{\text{agg}}$ is trust-weighted aggregated global policy vector, $\mathbf{w}^t$ shows vector of trust weights, $\phi^t$ mentions binary communication success vector, $\mathbf{W}^t$ is trust and availability weighted contribution matrix, and $\mathbf{\Theta}^t$ is stacked model updates from eligible agents.

### 2.2.2. Adversarial objective

The adversarial objective is designed to evaluate the potential impact of targeted disruptions within the learning process. The adversarial objective is formulated to maximize disruption by manipulating the model parameters $\mathbf{M}_j, \mathbf{B}_j$ and $\mathbf{\Phi}_j$ over the prediction horizon. The core metric is the squared Euclidean distance between the aggregated adversarial model $\mathbf{\Theta}^{\text{agg}}$ and the benign reference model $\mathbf{\Theta}^{\text{benign}}$, capturing the extent of policy deviation. This formulation offers an empirical basis for measures of adversarial resilience and its influence on learning outcomes.

$$ \max_{\mathbf{M}_j, \mathbf{B}_j, \mathbf{\Phi}_j} \|\mathbf{\Theta}^{\text{agg}} - \mathbf{\Theta}^{\text{benign}}\|_2^2 \quad (14) $$

where $\mathbf{\Theta}^{\text{benign}}$ is aggregated model under clean, untampered conditions and $\|\cdot\|_2$ shows Euclidean norm, quantifying policy deviation. This final objective frames adversarial resilience in measurable terms and links directly to learning outcomes.

### 2.3. Trust modeling and cyber-resilience

This section formalizes a sophisticated mechanism for evaluating trust that incorporates Bayesian inference, time-based modeling, and cyber-resilient policy enforcement. The trust values are being estimated from multivariate behavior profiles from the agents, and they evolve in time. The trust scores directly influence participation/recommendations for demand response, access to federated learning, and anomaly filtering to support dynamic adjustment of trust levels. Eq. (15) establishes a trust scoring mechanism providing a moving weighted exponential Bayesian update. The behavior vectors provide the adaptive structure for the trust levels and allow for timely time-based feedback on the reliability of the agent. Let $\mathbf{B}_i^t \in \mathbb{R}^d$ be the behavior vector of agent i at time t with d features. Trust score $T_i^t$ is updated by:

$$ T_i^t = \frac{\exp(\mathbf{B}_i^t \cdot \omega_i^t) \cdot T_i^{t-1}}{\int_{\mathbb{B}} \exp(\mathbf{B} \cdot \omega_i^t) d\mathbf{B}} \quad (15) $$

where $\mathbf{B}_i^t$ is normalized behavior vector (e.g., DR compliance, deviation index, communication reliability), $\omega_i^t$ is credibility weights for agent i behavior dimensions and $T_i^{t-1}$ shows prior trust score. This soft Bayesian update and the similar process amplify assent observed behavior while dampening deviations. It reintroduces nonlinearity to the process and allows for sensitivity to deviations in standard behavioral context. Below, a recursive estimator process for trust assessment based on a Bayesian filtering and Kalman gain approach [30] is formulated. This process accumulates the trust evidence across time, correcting for observation noise and uncertainty, adding to their robustness against temporary aberrations. Let $\hat{T}_i^t$ be the posterior trust estimate and $z_i^t$ the trust evidence. The update is defined as:

$$ \hat{T}_i^t = \hat{T}_i^{t-1} + K_i^t (z_i^t - H_i^t \hat{T}_i^{t-1}) \quad (16) $$

$$ K_i^t = P_i^{t-1} H_i^T (H_i^T P_i^{t-1} H_i^T + R_i^t)^{-1} \quad (17) $$

$$ P_i^t = (I - K_i^t H_i^T) P_i^{t-1} \quad (18) $$

where $\hat{T}_i^t$ is estimated trust score, $K_i^t$ shows Kalman gain matrix, $P_i^t$ describes error covariance, $H_i^t$ mentions observation matrix, and $R_i^t$ is observation noise covariance. This filter smooths noisy evidence across time, enabling adaptive and probabilistic trust estimation under uncertainty. The following equations embed agent policies into a latent space and evaluates their proximity to a trusted policy manifold. Deviations exceeding a trust-sensitive threshold are flagged. This detects adversarial or misaligned behavior at the policy level. Let $\pi_i$ be the policy of agent i and $\mathbf{\Phi}(\pi_i)$ its embedded representation in latent space:
$$ \epsilon_i^t = \| \Phi(\pi_i) - \mathcal{P}_{\mathcal{R}_T} |\Phi(\pi_i)| \|_2 \quad (19) $$

$$ \|_{\text{valid}} = \| [\epsilon_i^t \le \xi (T_i^t)] \|_2 \quad (20) $$

where $\mathcal{R}_T$ is trusted policy manifold, $\Phi(\pi_i)$ shows latent representation of policy, $\epsilon_i^t$ is policy projection residual (Euclidean residual), $\xi(T_i^t)$ describes adaptive acceptance radius, $\|_{\text{valid}}$ is binary indicator whether policy is accepted, and $\mathcal{P}_{\mathcal{R}_T}[(\cdot)]$ is projection onto the manifold. These equations filter unsafe policies by comparing embeddings with known-good policies. The threshold is trust-sensitive.

### 2.3.1. Trust entropy for quarantine

A statistical rule that uses entropy to flag agents with erratic trust histories is described, below. High entropy indicates unpredictable, or manipulated, behavior. Agents with high entropy are quarantined from participating in DR or learning. Let $T_i^{(t-k)}$ be the trust history of agent i. Define its normalized entropy as:

$$ \mathcal{H}_i^t = - \sum_{j=0}^{k} \rho_j^t \log \rho_j^t, \quad \rho_j^t = \frac{T_i^{t-j}}{\sum_{l=0}^{k} T_i^{t-l}} \quad (21) $$

$$ \|_{\text{quarantine}} = \| [\mathcal{H}_i^t > \bar{H}] \|_2 \quad (22) $$

where $\mathcal{H}_i^t$ is trust entropy of agent i, $\rho_j^t$ shows trust proportion at offset j, and $\bar{H}$ is entropy-based quarantine threshold. Agents with high entropy are deemed unstable and flagged for isolation. This adds statistical interpretability to anomaly detection.

## 3. TAFDRL and policy filtering

This section describes the TAFDRL architecture with SAC learning, trust-weighted ensemble of models, and neural policy validation. Decentralized training and cyber-resilient policy execution is facilitated by considering embedded anomaly detection and trust-adaptive filtering. Each agent learns its local policy separately from an entropy-regularized reinforcement learning objective in a way to control exploration and exploitation. The SAC architecture enhances learning performance by making not only the cumulative reward but also the policy distribution's entropy as efficient as possible, thus providing more stable convergence under uncertainty [31], [31]. The critic network approximates the Q-function and is updated using a temporal-difference error. In conjunction, the actor network formulates a stochastic policy via gradient-based updates that increase action probabilities that correspond to better Q-values. The SAC framework encompasses agents that learn in a local privacy-preserving setting, even while tolerating noise or partial observability [33], [34], [35]. This framework facilitates decentralized intelligence to exist in the federated architecture, as well as provide an opportunity for secure model aggregation across agents. Each agent j optimizes a stochastic policy $\pi_j(\mathbf{a}|\mathbf{s})$ using entropy-regularized reinforcement learning. The SAC objective maximizes the expected reward and entropy [22]:

$$ J(\pi_j) = \sum_{t=0}^{T} \mathbb{E}_{(\mathbf{s}_t, \mathbf{a}_t) \sim \rho_{\pi_j}} \left[ \mathbf{r}_j(\mathbf{s}_t, \mathbf{a}_t) + \alpha \mathcal{H}(\pi_j(\cdot|\mathbf{s}_t)) \right] \quad (23) $$

where $\mathbf{r}_j(\cdot)$ is local reward function for agent j, $\alpha$ shows temperature coefficient balancing exploration and reward and $\mathcal{H}(\pi_j) = - \mathbb{E}_{(\mathbf{a}_t, \pi_j)}[\log \pi_j(\mathbf{a}_t|\mathbf{s}_t)]$ shows entropy. The critic network $Q_{\theta_j}$ is updated by minimizing the Bellman error:

$$ \mathcal{L}_{\theta_j} = \mathbb{E}_{(\mathbf{s}_t, \mathbf{a}_t)} \left[ \left( Q_{\theta_j}(\mathbf{s}_t, \mathbf{a}_t) - y_j \right)^2 \right], $$

$$ y_j = \mathbf{r}_j + \gamma \mathbb{E}_{(\mathbf{a}_{t+1} \sim \pi_j)} \left[ Q_{\theta_j}(\mathbf{s}_{t+1}, \mathbf{a}_{t+1}) - \alpha \log \pi_j(\mathbf{a}_{t+1} | \mathbf{s}_{t+1}) \right] \quad (24) $$

where $\mathcal{L}_{\theta_j}$ is loss function for updating the critic network $Q_{\theta_j}$ of agent j, $\mathbb{E}_{(\mathbf{s}_t, \mathbf{a}_t)}$ shows expectation over the sampled state-action pairs from the replay buffer, $Q_{\theta_j}(\mathbf{s}_t, \mathbf{a}_t)$ is Q-value estimated by the critic network with parameters $\theta_j$ for agent j, $y_j$ describes target Q-value used to compute the loss, combining reward and bootstrapped future return, $\gamma$ is discount factor, $\mathbb{E}_{(\mathbf{a}_{t+1} \sim \pi_j)}$ shows expectation over the next action $\mathbf{a}_{t+1}$ sampled from the policy $\pi_j$, and $\log \pi_j(\mathbf{a}_{t+1} | \mathbf{s}_{t+1})$ is log-probability of the next action under policy $\pi_j$, promoting exploration. The actor network is updated via policy gradient:

$$ \nabla_{\phi_j} J_{\pi_j} = \mathbb{E}_{(\mathbf{s}_t, \mathcal{S}_j)} \left[ \nabla_{\phi_j} \log \pi_{\phi_j}(\mathbf{a}_t | \mathbf{s}_t) \left( \alpha \log \pi_{\phi_j}(\mathbf{a}_t | \mathbf{s}_t) - Q_{\theta_j}(\mathbf{s}_t, \mathbf{a}_t) \right) \right] \quad (25) $$

where $\phi_j$ is gradient of the objective function $J_{\pi_j}$ with respect to the policy parameters $\phi_j$ of agent j, $\mathbb{E}_{(\mathbf{s}_t, \mathcal{S}_j)}$ is expectation over states $\mathbf{s}_t$ sampled from the replay buffer $\mathcal{S}_j$ of agent j, and $\phi_j$ is parameters of the actor (policy) network for agent j.

### 3.1. Federated aggregation with trust filtering

Here, we introduce the trust-weighted global model update rule. Locally learned SAC parameters by individual agents are encrypted and disclosed only if its current trust value is greater than a threshold value. In this way, the system will not incorporate contributions from untrusted or possibly malicious agents. The trust-weighted weighting scheme further boosts the contribution of previously compliant and cyber-secure agents towards the global model. The aggregator aggregates such encrypted updates to produce a global SAC model while maintaining privacy and resilience. Such trust-aware technique minimizes poisoning attacks and corruption hazards on the data by eliminating dynamically agents displaying erratic behavior or suspicious divergence. Therefore, federated design remains scalable, privacy-preserving, and resistant to cyber and behavioral anomalies [2], [5]. Model updates $\theta_i$ are encrypted and transmitted if $T_i^t \ge \tau$:

$$ \theta_{\text{agg}} = \frac{1}{\sum_{i \in \mathcal{S}} T_i^t} \sum_{j \in \mathcal{S}} T_j^t \cdot \mathcal{E}(\theta_j) \quad (26) $$

where $\theta_j$ shows local SAC model of agent j, $T_j^t$ is trust score at time t, and $\mathcal{E}(\cdot)$ is encryption function.

### 3.2. Neural policy embedding and anomaly detection

To filter and validate incoming policy updates, a neural embedding mechanism is proposed. The encrypted SAC policy of each agent is passed through a neural encoder to get a latent representation. The Mahalanobis distance to the centroid of the previously accepted, trusted policy embeddings is calculated as a geometric anomaly score. This distance quantifies the dissimilarity of a novel policy to known safe policies. The covariance matrix of the embedding space encodes variation in legitimate behavior such that the anomaly detector is sensitive yet not overly rigid [32], [34], [35]. This method effectively identifies adversarial or unstable policies, including ones that would initially pass trust-based verification. It introduces a learned, adaptive, and high-dimensional defense layer. Let $\mathbf{z}_j = \Phi_{\psi}(\theta_j)$ be the encoded representation. We define the normalized Mahalanobis distance from the centroid of trusted policy embeddings $\mathbf{z}_c$:

$$ \delta_j = \sqrt{(\mathbf{z}_j - \mathbf{z}_c)^T \Sigma^{-1} (\mathbf{z}_j - \mathbf{z}_c)} \quad (27) $$

where $\Sigma$ is covariance of the trusted embedding cluster, and $\mathbf{z}_c$ is
centroid of trusted policies.

### 3.2.1. Adaptive filtering rule

Here, an adaptive acceptance rule integrating the geometric anomaly score with the agent's runtime trust value is introduced. The system is more lenient with trusted agents, yet continues to monitor low-trust contributors strictly through dynamic adaptation of the anomaly threshold using the trust score of the agent. Both false positives (unwarranted rejections) and false negatives (malevolent acceptances) are thus avoided in the formulation. The trust-weighted tolerance mechanism ensures fairness and robustness, especially in the event of uncertainty or model drift. The rule as a whole combines statistical anomaly detection with trust-informed decision boundaries to present a strong filtering structure for high-assurance cyber-physical systems. Policy acceptance is based on:

$$ \|^{\text{accept}}_j = \| [\delta_j \le \tau_{\text{max}} - \lambda(1 - T_j^t)] \quad (28) $$

where $τ_{max}$ is max acceptance threshold and $λ$ is sensitivity to low trust. The acceptance rule combines geometric anomaly detection with trust modulation, enabling robust protection against poisoned or misaligned updates. The structure of the proposed approach is shown in Algorithm 1 and Fig. 2.

**Algorithm 1.** TAFDRL with latent policy filtering

threshold. Only updates that satisfy this dual-filtering mechanism, both behavioral and structural, are transmitted to the aggregator. Fig. 2 complements Algorithm 1 by visually illustrating this multi-phase execution pipeline, highlighting the key loops, decision checkpoints, and inter-agent coordination logic. Together, the algorithm and its flowchart convey the modular and resilient configuration of TAFDRL, emphasizing its ability to dynamically adapt, filter adversarial agents, and securely coordinate distributed learning across microgrids.

## 4. Simulation requirements and results

The simulation requirements are presented in Table 2. This table presents the distribution grid topology, market structure, EVCS details, and simulation details. Table 3 describes the operational process of the TAFDRL approach. In Table 4, the structure and configuration of cyberattacks are presented. To test the resilience of the framework under the three scenarios of cyberattack, they were simulated: FDI, DoS, and hybrid attacks. For the FDI scenario, noise (±15%) was injected into load forecasts and DER status to bias agent decision inputs. For simulating the DoS attack, random disruption of communication or policy updates for 10% to 30% of the agents with blackout durations between 5 and 30 minutes was simulated. The hybrid scenario combined FDI and DoS effects on intersecting sets of agents, representing a more realistic and complex attack state. Attack impact was varied at low (10%), medium (20%), and high (30%) compromise levels. System performance

<table><caption>**Algorithm 1 TAFDRL with latent policy filtering**</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>1.</td><td><b>Phase 1: Initialization</b><br>Trust scores $T_i^0$ for all agents i, global policy parameters $\theta^0$</td></tr><tr><td>2.</td><td><b>Phase 2: Federated Learning Loop</b><br>for each round $t=1$ to $T$ do</td></tr><tr><td>3.</td><td><b>Agent-Level Parallel Loop</b><br>for each agent i in parallel do.</td></tr><tr><td>4.</td><td><b>Local Observation and Action</b><br>Observe local state $s_i^t$ (load forecast, DER status, local price).</td></tr><tr><td>5.</td><td>Select action $s_i^t \sim \pi_{\theta^t}(\mathbf{a}|\mathbf{s})$ and apply in environment.</td></tr><tr><td>6.</td><td><b>Experience Collection</b><br>Store $(s_i^t, a_i^t, r_i^t, s_i^{t+1})$ in local replay buffer $\mathcal{D}_i$.</td></tr><tr><td>7.</td><td>Update local SAC parameters $\theta_i^t$ using sampled batches from $\mathcal{D}_i$.</td></tr><tr><td>8.</td><td><b>Trust Score Computation</b><br>Compute trust score $T_i^t$ based on behavior (DR compliance, comm. reliability).</td></tr><tr><td>9.</td><td><b>Latent Encoding and Anomaly Filtering</b><br>Encode $\theta_i^t$ to latent vector $z_i^t = \Phi(\theta_i^t)$</td></tr><tr><td>10.</td><td>Compute Mahalanobis distance $\delta_i^t$ from trusted policy cluster.</td></tr><tr><td>11.</td><td><b>if</b> $\delta_i^t < \lambda (T_i^t)$ then</td></tr><tr><td>12.</td><td><b>Send</b> $\theta_i^t$ to aggregator.</td></tr><tr><td>13.</td><td><b>else</b></td></tr><tr><td>14.</td><td><b>Reject</b> update (possible adversarial agent).</td></tr><tr><td>15.</td><td><b>end if</b></td></tr><tr><td>16.</td><td><b>end for</b></td></tr><tr><td>17.</td><td><b>Phase 3: Aggregation and Distribution</b><br>Aggregator collects updates $\{\theta_i^t\}$ from accepted agents.</td></tr><tr><td>18.</td><td>Aggregate with trust-weighted averaging: $\theta^{\text{agg}} = (1^T \mathbf{W}^t \mathbf{1})^{-1} \cdot \mathbf{W}^t \cdot \theta^t$.</td></tr><tr><td>19.</td><td><b>Broadcast</b> updated global parameters $\theta^t$ to all agents.</td></tr><tr><td>20.</td><td><b>end for</b></td></tr></tbody></table>

Algorithm 1 presents the phased structure of the proposed TAFDRL framework, which integrates trust-aware filtering, policy-level anomaly detection, and secure aggregation in a federated learning setting. The algorithm is organized into three main phases: initialization, parallel agent-side processing, and centralized aggregation. In each round, agents independently observe local states, update their SAC policies, and compute dynamic trust scores based on behavior indicators. Policy updates are encoded into a latent space, and their proximity to a trusted policy manifold is assessed using a trust-sensitive Mahalanobis

was measured using market-level indicators such as price deviation, DR effectiveness, and trust score deterioration. Together, detection performance was also assessed using false acceptance/rejection rate monitoring, rejection delay measurement, and policy anomaly score measurement.

### 4.1. Policy filtering performance

To empirically validate the efficacy of the proposed trust-integrated
<figure><img src="image_6.png" /><figcaption>Fig. 2. TAFDRL structure flowchart</figcaption></figure>

policy filtering mechanism, we pay attention to federated training rounds with adversarial and benign policy updates. The local policy vectors are mapped to a latent representation using a neural encoder. Within this representation, we calculate the Mahalanobis distance to the trusted policy centroid. The distance is then used jointly with real-time trust scores to determine whether the update will be accepted into the aggregation. Fig. 3 shows a sample t-SNE projection of the space where trusted policies cluster very tightly and malware updates are strong outliers. Secondly, within a spherical representation space, we can visualize the classification of safe versus suspicious policies, as shown in Fig. 4. In particular, if the definition of a hyperspherical region is according to Eq. (27) where the radius is defined by the Mahalanobis distance, then the spatial locations of policies can be meaningfully displayed.

In this representation, policies located within the spherical boundary exhibit statistical conformity with the trusted distribution, thereby qualifying as safe policies under the condition formalized in Eq. (28). Conversely, policies that deviate significantly from the center of the distribution fall outside this spherical boundary and are therefore considered suspicious. Formally, the acceptance criterion is expressed as follows: if the inequality $\delta_i^t < \lambda (T_i^t)$ holds, where $\delta_i^t$ denotes the

<table><caption>**Table 2**
Simulation input specifications.</caption><thead><tr><th>Category</th><th>Parameter</th><th>Description</th></tr></thead><tbody><tr><td rowspan="10">Grid topology</td><td>Topologies</td><td>IEEE 69-bus; 141-bus (Caracas), IEEE 342-bus</td></tr><tr><td>Voltage levels</td><td>12.66 kV (69-bus), 12.47 kV (141-bus), 13.2 kV (342-bus)</td></tr><tr><td>DER units and locations</td><td>PV: 5 × 1 MW, Wind: 4 × 2 MW, BESS: 5 × 900 kWh, Diesel: 3 × 2.5 MW.</td></tr><tr><td>DER placement</td><td>Strategically placed based on load and voltage sensitivity.</td></tr><tr><td>EVCS</td><td>69-bus: 6 stations, 141-bus and 342-bus: 12 stations; Each station has 5 × 120 kW chargers (600 kW total/station) equipped with Wi-Fi technology</td></tr><tr><td>DR market design</td><td>Hybrid day-ahead + real-time pricing.</td></tr><tr><td>DR participation</td><td>Trust-based tiered access and incentives</td></tr><tr><td>Price update interval</td><td>Real-time pricing every 15 minutes.</td></tr><tr><td>Market Structure</td><td>DR event duration</td><td>30 minutes per event.</td></tr><tr><td rowspan="3">Simulation Details</td><td>Power generation prices</td><td>PV: $0.045/kWh, Wind: $0.035/kWh, BESS: $0.07/kWh, Diesel: $0.18/kWh.</td></tr><tr><td>Service & maintenance costs</td><td>PV: $2.5/kW/month, Wind: $1.8/kW/month, BESS: $4.0/kWh/month, Diesel: $5.0/kW/month.</td></tr><tr><td>Time horizon Comparison models</td><td>24 hours (96 intervals × 15 min). Centralized SAC, Federated A2C, Static Filtered DRL.</td></tr></tbody></table>

<table><caption>**Table 3**
TAFDRL setup.</caption><thead><tr><th>Parameter</th><th>Description</th></tr></thead><tbody><tr><td>Policy type</td><td>Stochastic (Gaussian)</td></tr><tr><td>Entropy regularization</td><td>Target entropy: -|A| (per SAC), adaptive temperature tuning</td></tr><tr><td>Learning rate (Actor& Critic)</td><td>0.0003</td></tr><tr><td>Learning rate (Alpha)</td><td>0.0001</td></tr><tr><td>Replay buffer size</td><td>1,000,000 transitions per agent</td></tr><tr><td>Batch size</td><td>256 transitions per update</td></tr><tr><td>Discount factor (γ)</td><td>0.99</td></tr><tr><td>Soft target update rate (τ)</td><td>0.005</td></tr><tr><td>Local training episodes</td><td>65,000 episodes per agent</td></tr><tr><td>Update interval</td><td>Every 100 episodes (federated round)</td></tr><tr><td>Convergence criteria</td><td>Stabilized cumulative reward and policy entropy &lt; 0.05</td></tr><tr><td>Number of layers (Actor & Critic)</td><td>2-3 fully connected layers with 256-512 neurons (ReLU activation)</td></tr><tr><td>Optimizer (Actor & Critic)</td><td>Adam optimizer (β₁ = 0.9, β₂ = 0.999, ε = 10⁻⁵)</td></tr><tr><td>Temperature tuning</td><td>Adaptive tuning based on target entropy; update every 1000 steps</td></tr></tbody></table>

<table><caption>**Table 4**
Cyberattack scenarios.</caption><thead><tr><th>Parameter</th><th>Description</th></tr></thead><tbody><tr><td>FDI</td><td>Alters load forecasts and DER data using ±15% Gaussian noise</td></tr><tr><td>DoS</td><td>Blocks agent communication and policy updates (10–30% randomly)</td></tr><tr><td>Hybrid</td><td>Simultaneous FDI and DoS</td></tr><tr><td>Agent compromise rates</td><td>Low (10%), Medium (20%), High (30%)</td></tr><tr><td>Duration</td><td>5–30 min random communication dropout</td></tr><tr><td>System metrics</td><td>Price deviation, DR effectiveness, trust decay</td></tr><tr><td>Detection metrics</td><td>False acceptance/rejection, detection delay, rejection rate</td></tr></tbody></table>

instantaneous Mahalanobis distance of policy *i* at iteration *t*, and $\lambda (T_i^t)$ is the adaptive trust threshold parameterized by contextual state $T_i^t$, then the local update $\theta_i^t$ is transmitted to the central aggregator. Otherwise, the update is flagged as potentially adversarial and is excluded from the
<figure><img src="image_7.png" /><figcaption>Fig. 3. t-SNE projection of the policy embedding space.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 4. Detect and separate suspicious and safe policies.</figcaption></figure>

<table><caption>**Table 5**
Policy filtering performance comparison.</caption><thead><tr><th>Method</th><th>AUC</th><th>Precision</th><th>Recall</th><th>FAR (%)</th><th>FRR (%)</th></tr></thead><tbody><tr><td>Proposed model</td><td>0.94</td><td>0.91</td><td>0.88</td><td>4.7</td><td>5.5</td></tr><tr><td>Static Mahalanobis</td><td>0.86</td><td>0.85</td><td>0.73</td><td>7.2</td><td>9.6</td></tr><tr><td>Autoencoder Only</td><td>0.81</td><td>0.78</td><td>0.69</td><td>10.3</td><td>13.1</td></tr></tbody></table>

aggregation process. This adaptive mechanism will only allow updates that are consistent with the statistical and trust boundaries to be accepted, further strengthening the federated aggregation process's capacity for robustness. Therefore, as indicated in Fig. 4, the distinct geometric division between safe policies and suspicious policies is a natural by-product of the thresholding mechanism; safe policies group within the spherical decision boundary, whereas suspicious policies

disperse outside the safe zone. This division affords not only real-time auditing of incoming updates but also serves as a proxy for the system's ability to continuously adapt its tolerance boundaries based on contextually relevant trust measures. Ultimately, this geometric filtering process contributes to a federated learning system that is more resilient and trustworthy, free from adversarial contamination, while also converging at a reasonable pace.

Agent trust levels control the adaptive filtering boundary that allows higher freedom for trusted agents but requires hard rejection against low-trusted contributors. Testing of the introduced filter's performance is compared with two baselines: static Mahalanobis thresholding and autoencoder-only filtering. As can be seen in Table 5, the approach achieves an AUC of 0.94, better than static (0.86) and autoencoder (0.81) approaches. It also achieves higher precision (0.91) and recall (0.88) with fewer false acceptances and rejections. This confirms that the incorporation of trust dynamics together with latent anomaly scoring enables rigorous policy validation without inducing extreme disturbance to benign agents. The filter effectively thwarts policy poisoning attacks in federated learning without reducing convergence speed or penalizing honest nodes.

## 4.2. Cyberattack resilience

According to Fig. 5, the dynamic process of detecting cyberattacks and distinguishing between secure and suspicious data across different episodes can be observed through the evolution of the cyber resilience index. At the initial stage, raw data streams are fed directly into the proposed model without any prior filtering, ensuring that the framework is exposed to both authentic operational inputs and potentially compromised signals. The detection mechanism is then activated by evaluating the condition $\delta_i^t < \lambda (T_i^t)$, where $\delta_i^t$ denotes the deviation of observed data from expected policy trajectories at time $t$ (instantaneous Mahalanobis distance), and $\lambda (T_i^t)$ represents a dynamic threshold function derived from the cyber resilience index (adaptive trust threshold).

A key insight highlighted in Fig. 5 is the system's ability to adaptively refine its detection accuracy over time. Specifically, by episode 15,000, the proposed framework demonstrates a stable separation of secure operational data and policies from those contaminated by cyberattacks.
<figure><img src="image_9.png" /><figcaption>Fig. 5. Separating safe and suspicious data.</figcaption></figure>

This indicates not only the robustness of the resilience index in long-term training cycles but also the capability of the model to maintain consistent detection under evolving adversarial conditions. The clear demarcation observed at this stage underscores the framework's effectiveness in mitigating cyber-induced risks while ensuring uninterrupted functionality of the underlying energy market or microgrid environment. Such adaptive separation is crucial for preserving decision integrity and preventing cascading failures that may arise from undetected malicious inputs.

Table 6 presents a comprehensive performance comparison of the proposed TAFDRL model against three baseline methods (Static Filtered DRL, Federated A2C, and Centralized SAC) under varying cyber-attack scenarios across three different distribution network configurations: IEEE 69-bus, 141-bus (Caracas), and the larger-scale IEEE 342-node network. Each model was evaluated under FDI, DoS, and hybrid attacks, using four key performance metrics: price deviation, DR fulfillment, reward drop, and malicious agent rejection rate. Across all network scales and attack types, the TAFDRL model consistently achieved superior performance, maintaining minimal price deviation (as low as 2.4–4.3%), high DR fulfillment rates (>86%), and significantly lower reward losses compared to baseline approaches. Moreover, it demonstrated strong resilience by rejecting over 80% of malicious agents, even in larger networks and hybrid threat settings.

The comparative results underscore the limitations of traditional and static models in dynamically adversarial environments. As the network size increases from 69 to 342 nodes, baseline methods exhibited performance degradation, particularly under hybrid attacks, where reward drops exceeded 12% and rejection rates fell below 50% in the worst

cases (Centralized SAC). In contrast, TAFDRL maintained robust performance even at the 1,500-agent scale in the IEEE 342-node system, reinforcing the importance of trust-aware filtering and adaptive anomaly detection. These findings highlight the necessity of integrating decentralized trust mechanisms and intelligent threat response capabilities in modern DR frameworks. The results validate TAFDRL not only as an effective anomaly mitigation strategy but also as a scalable, security-conscious solution for future microgrid resilience and operational reliability.

### 4.2.1. Scalability and computational feasibility analysis

To assess the scalability and computational feasibility of the proposed TAFDRL framework, we conducted a comparative analysis across three increasingly complex benchmark networks: IEEE 69-bus (500 agents), 141-bus-Caracas (900 agents), and IEEE 342-bus (1,500 agents). Table 7 summarizes the computational performance metrics measured across these deployments. The results confirm that each module of the system (including local SAC training, Bayesian trust updates, latent-space anomaly filtering, and secure global aggregation) scales efficiently with agent count. Average SAC training time per round increases modestly with agent population, from 5.2 s at 500 agents to 13.6 s at 1,500 agents. Trust updates and anomaly detection remain highly efficient, requiring less than 0.6 s per round even at the largest scale. Importantly, the global aggregation process remains tractable due to trust-based filtering, which excludes unverified agents from the final policy merge.

System-level resource usage also scales in a manageable fashion. Peak memory requirements rise from 890 MB to 1,630 MB, while CPU
<table><caption>**Table 6**
Proposed model consistently outperformed baseline methods across all threat scenarios</caption><thead><tr><th>Model</th><th>Test Network</th><th>Attack Type</th><th>Target Buses</th><th>Price Deviation (%)</th><th>DR Fulfillment (%)</th><th>Reward Drop (%)</th><th>Malicious Agent Rejection (%)</th></tr></thead><tbody><tr><td rowspan="10">Proposed Model</td><td rowspan="4">IEEE 69-bus</td><td>FDI</td><td>18, 31, 45</td><td>2.4</td><td>94.2</td><td>3.5</td><td>91.3</td></tr><tr><td>DoS</td><td>12, 28, 62</td><td>2.8</td><td>92.7</td><td>4.1</td><td>88.9</td></tr><tr><td>Hybrid</td><td>17, 34, 47, 54</td><td>3.6</td><td>90.5</td><td>5.4</td><td>86.7</td></tr><tr><td>FDI</td><td>23, 89, 124</td><td>2.9</td><td>92.8</td><td>4.2</td><td>89.6</td></tr><tr><td rowspan="4">141-bus<br>(Caracas)</td><td>DoS</td><td>19, 76, 133</td><td>3.3</td><td>91.0</td><td>4.8</td><td>86.2</td></tr><tr><td>Hybrid</td><td>42, 77, 108, 135</td><td>4.1</td><td>88.6</td><td>4.9</td><td>83.5</td></tr><tr><td>FDI</td><td>102, 144, 198</td><td>3.1</td><td>91.5</td><td>5.2</td><td>87.9</td></tr><tr><td>DoS</td><td>95, 186, 224</td><td>3.4</td><td>89.8</td><td>5.6</td><td>85.1</td></tr><tr><td rowspan="4">IEEE 342-node</td><td>Hybrid</td><td>112, 190, 243, 267</td><td>4.3</td><td>86.2</td><td>6.3</td><td>81.2</td></tr><tr><td>FDI</td><td>18, 31, 45</td><td>4.0</td><td>88.4</td><td>6.2</td><td>79.4</td></tr><tr><td>DoS</td><td>12, 28, 62</td><td>4.3</td><td>86.5</td><td>6.8</td><td>74.1</td></tr><tr><td>Hybrid</td><td>17, 34, 47, 54</td><td>4.6</td><td>85.7</td><td>7.1</td><td>75.9</td></tr><tr><td rowspan="10">Static Filtered DRL</td><td rowspan="4">IEEE 69-bus</td><td>FDI</td><td>23, 89, 124</td><td>4.5</td><td>87.2</td><td>6.0</td><td>76.3</td></tr><tr><td>DoS</td><td>19, 76, 133</td><td>4.7</td><td>85.6</td><td>6.5</td><td>72.5</td></tr><tr><td>Hybrid</td><td>42, 77, 108, 135</td><td>5.0</td><td>84.1</td><td>7.3</td><td>73.2</td></tr><tr><td>FDI</td><td>102, 144, 198</td><td>4.6</td><td>84.9</td><td>7.0</td><td>72.4</td></tr><tr><td rowspan="4">IEEE 342-node</td><td>DoS</td><td>95, 186, 224</td><td>5.0</td><td>83.0</td><td>7.4</td><td>69.7</td></tr><tr><td>Hybrid</td><td>112, 190, 243, 267</td><td>5.5</td><td>81.3</td><td>7.9</td><td>66.5</td></tr><tr><td>FDI</td><td>18, 31, 45</td><td>5.1</td><td>83.2</td><td>7.2</td><td>71.4</td></tr><tr><td>DoS</td><td>12, 28, 62</td><td>5.3</td><td>81.7</td><td>8.0</td><td>68.9</td></tr><tr><td rowspan="10">Federated A2C</td><td rowspan="4">IEEE 69-bus</td><td>Hybrid</td><td>17, 34, 47, 54</td><td>5.1</td><td>83.2</td><td>8.4</td><td>69.3</td></tr><tr><td>FDI</td><td>23, 89, 124</td><td>5.6</td><td>82.9</td><td>7.4</td><td>70.1</td></tr><tr><td>DoS</td><td>19, 76, 133</td><td>5.9</td><td>80.4</td><td>7.8</td><td>67.3</td></tr><tr><td>Hybrid</td><td>42, 77, 108, 135</td><td>6.0</td><td>79.8</td><td>8.2</td><td>66.5</td></tr><tr><td rowspan="4">IEEE 342-node</td><td>FDI</td><td>102, 144, 198</td><td>6.1</td><td>78.3</td><td>8.6</td><td>65.4</td></tr><tr><td>DoS</td><td>95, 186, 224</td><td>6.4</td><td>76.2</td><td>9.1</td><td>62.1</td></tr><tr><td>Hybrid</td><td>112, 190, 243, 267</td><td>6.7</td><td>74.0</td><td>9.7</td><td>59.3</td></tr><tr><td>FDI</td><td>18, 31, 45</td><td>5.7</td><td>79.5</td><td>9.6</td><td>63.8</td></tr><tr><td rowspan="4">IEEE 69-bus</td><td>DoS</td><td>12, 28, 62</td><td>6.2</td><td>77.3</td><td>10.2</td><td>60.4</td></tr><tr><td>Hybrid</td><td>17, 34, 47, 54</td><td>5.7</td><td>79.5</td><td>9.6</td><td>63.8</td></tr><tr><td>FDI</td><td>23, 89, 124</td><td>6.5</td><td>78.1</td><td>9.9</td><td>59.7</td></tr><tr><td>DoS</td><td>19, 76, 133</td><td>6.8</td><td>75.8</td><td>10.3</td><td>57.1</td></tr><tr><td rowspan="10">Centralized SAC</td><td rowspan="4">IEEE 342-node</td><td>Hybrid</td><td>42, 77, 108, 135</td><td>6.9</td><td>74.2</td><td>10.7</td><td>55.3</td></tr><tr><td>FDI</td><td>102, 144, 198</td><td>7.1</td><td>72.6</td><td>11.1</td><td>51.2</td></tr><tr><td>DoS</td><td>95, 186, 224</td><td>7.5</td><td>70.1</td><td>11.7</td><td>47.9</td></tr><tr><td>Hybrid</td><td>112, 190, 243, 267</td><td>7.9</td><td>68.0</td><td>12.3</td><td>44.1</td></tr><tr><td>No. of agents</td><td>500</td><td>900</td><td>1500</td><td></td><td></td><td></td></tr><tr><td>Avg. SAC training time / round (s)</td><td>5.2</td><td>8.7</td><td>13.6</td><td></td><td></td><td></td></tr><tr><td>Avg. trust update time / round (s)</td><td>0.14</td><td>0.24</td><td>0.38</td><td></td><td></td><td></td></tr><tr><td>Anomaly detection time / round (s)</td><td>0.19</td><td>0.33</td><td>0.52</td><td></td><td></td><td></td></tr><tr><td>Global aggregation time / round (s)</td><td>0.31</td><td>0.56</td><td>0.91</td><td></td><td></td><td></td></tr><tr><td>Total federated rounds</td><td>60</td><td>65</td><td>68</td><td></td><td></td><td></td></tr><tr><td>Peak memory usage (MB)</td><td>890</td><td>1210</td><td>1630</td><td></td><td></td><td></td></tr><tr><td>Peak CPU utilization (%)</td><td>78</td><td>85</td><td>91</td><td></td><td></td><td></td></tr><tr><td>GPU (if used) avg load (%)</td><td>52</td><td>57</td><td>61</td><td></td><td></td><td></td></tr></tbody></table>

<table><caption>**Table 7**
Comparison table of computational evaluation in different test power systems.</caption><thead><tr><th>Metric</th><th>IEEE 69-bus</th><th>141-bus (Caracas)</th><th>IEEE 342-bus</th></tr></thead><tbody><tr><td>No. of agents</td><td>500</td><td>900</td><td>1500</td></tr><tr><td>Avg. SAC training time / round (s)</td><td>5.2</td><td>8.7</td><td>13.6</td></tr><tr><td>Avg. trust update time / round (s)</td><td>0.14</td><td>0.24</td><td>0.38</td></tr><tr><td>Anomaly detection time / round (s)</td><td>0.19</td><td>0.33</td><td>0.52</td></tr><tr><td>Global aggregation time / round (s)</td><td>0.31</td><td>0.56</td><td>0.91</td></tr><tr><td>Total federated rounds</td><td>60</td><td>65</td><td>68</td></tr><tr><td>Peak memory usage (MB)</td><td>890</td><td>1210</td><td>1630</td></tr><tr><td>Peak CPU utilization (%)</td><td>78</td><td>85</td><td>91</td></tr><tr><td>GPU (if used) avg load (%)</td><td>52</td><td>57</td><td>61</td></tr></tbody></table>

utilization grows from 78% to 91% across the three testbeds. GPU load increases only moderately, remaining below 65% in all cases. These findings demonstrate the framework's capability to support secure, decentralized coordination in microgrids with over a thousand active consumers. Furthermore, all modules are designed to operate in parallel across agents, and encryption overhead is lightweight due to symmetric key usage. The observed linear growth in compute time confirms that the system is well-suited for real-time deployment in medium to large-scale microgrids and could be further extended via clustered federators or edge-based policy aggregation.

<table><caption>**Table 8**
Evaluate the learning performance and resilience.</caption><thead><tr><th>Model</th><th>Cumulative reward</th><th>Convergence speed (rounds)</th><th>Policy divergence</th><th>Reward stability (%)</th></tr></thead><tbody><tr><td>Proposed method</td><td>94</td><td>61</td><td>0.12</td><td>92</td></tr><tr><td>Static Filtered DRL</td><td>89</td><td>73</td><td>0.15</td><td>87</td></tr><tr><td>Federated A2C</td><td>87</td><td>78</td><td>0.19</td><td>81</td></tr><tr><td>Centralized SAC</td><td>81</td><td>82</td><td>0.25</td><td>72</td></tr></tbody></table>

<table><caption>**Table 9**
Stability assessment.</caption><thead><tr><th>Attack type</th><th>System stability (reward stability %)</th><th>Policy stability (policy deviation %)</th><th>Convergence time (iterations)</th></tr></thead><tbody><tr><td>FDI (Low)</td><td>94.0</td><td>2.4</td><td>61</td></tr><tr><td>FDI (High)</td><td>92.0</td><td>3.2</td><td>63</td></tr><tr><td>DoS (Low)</td><td>92.7</td><td>3.0</td><td>62</td></tr><tr><td>DoS (High)</td><td>92.2</td><td>3.6</td><td>65</td></tr><tr><td>Hybrid (Low)</td><td>92.1</td><td>3.5</td><td>68</td></tr><tr><td>Hybrid (High)</td><td>92.0</td><td>4.0</td><td>71</td></tr></tbody></table>
<figure><img src="image_10.png" /><figcaption>Fig. 6. Comparison of training and convergence processes.</figcaption></figure>

### 4.3. Federated learning effectiveness and stability assessment

For the comparison of the learning performance and robustness of the proposed model, four models were compared: the proposed TAFDRL, Static Filtered DRL, Federated A2C, and Centralized SAC. Training of the models was conducted in an adversarial environment with policy poisoning and communication noise. As evident in Table 8, the proposed model obtained the highest cumulative reward (94) and reward stability (92%) and minimum policy divergence (0.12). It also required fewer iterations of training (61) to converge than all of the baselines. The stability assessment of the proposed model under multiple attack scenarios (low) FDI, (high) FDI, DoS, and Hybrid are provided in Table 9. Reward stability is strong, between 92% and 94% with reward for FDI (low) scenario 94% and FDI (high) scenario 92%. DoS and hybrid attack scenarios Los a small amount of stability but remain stable from 92.7% to 92.0%. The average policy stability demonstrated some minor deviations, from 2.4% under high reward FDI (low) to 4.0% under hybrid attacks (high). Convergence time changed based on attack scenario from 61 to 71 iterations; FDI (low) was the shortest convergence time and hybrid attacks (high) was the longest compared to the other scenarios. Overall, the proposed model maintained performance and stability under high, undefined, extreme situations with reasonable and efficient convergence across multiple scenarios.

Fig. 6 illustrates the cumulative reward convergence across training rounds for four DRL-based models: the proposed TAFDRL, Static Filtered DRL, Federated A2C, and Centralized SAC. The x-axis denotes the number of federated training rounds (0–100), while the y-axis captures the cumulative reward achieved at each round. The proposed TAFDRL model demonstrates a significantly faster and more stable convergence trajectory, reaching near-optimal performance around the 60th round, consistent with the convergence data shown in Table 8. The 100-round limit was selected to reflect the practical convergence interval observed during model training. As stated in the paper, local agents are trained for 65,000 episodes, with federated aggregation occurring every 100 episodes, leading to 650 total communication rounds. The 100-round convergence plot is designed to illustrate the early-to-mid stage convergence phase, which is the most critical window for comparing convergence speed and learning dynamics. Showing this resolution allows readers to observe early improvements, model differentiation, and plateau points, which would be obscured in longer plots with compressed x-axes. The quantitative results shown in Table 8 confirm the trends visualized in the convergence plot. Specifically, TAFDRL achieves: highest cumulative reward (94), fastest convergence (61 rounds), lowest policy divergence (0.12), and strong reward stability (92%). These numerical metrics directly align with the plotted trends: TAFDRL rises rapidly within the first 50–60 rounds and stabilizes near optimal performance, while other baselines show slower convergence and lower asymptotic reward values. The visual plot thus complements Table 8, offering an intuitive and transparent view of how quickly and reliably

<figure><img src="image_11.png" /><figcaption>Fig. 7. Evaluating stability against various cyber attacks.</figcaption></figure>
each model learns under adversarial training conditions. Together, Fig. 6 and Table 8 offer a comprehensive justification for the claim that TAFDRL not only outperforms baselines but does so efficiently and robustly across training iterations.

Fig. 7 confirms the results shown in Table 9 as it shows the graphical indication for validating the stability assessment for the proposed model using the root-locus technique, using the Smith chart to identify where the poles of the model will be. In the figure, we can see the responses of the systems with changes in severity for different cyber-attacks (FDI, DoS and combined) in low and high domains. In the figure, the degree of severity for the various cyber-attacks will be assessed, and poles of the system will be plotted onto the Smith chart throughout the assessment. We see that behavior appears stable for the plot of the poles, even as we assess cyber-attacks from both minor to large severity types. The poles of the system remain strictly to the left on the imaginary axis on the Smith chart, indicating that the systems were stable. The conclusions from the model and trends from the Smith chart show that the proposed approach is stable across the range of attack patterns that were assessed. Although the poles of the model respond to the defined low and high severity domains and can move, on the Smith chart, the poles always remain stable (or left of the imaginary axis), which demonstrates the strength of this model across the various threat scenarios we investigated.

This stability is important because it illustrates that the methodology is reliable in all components, even while the system is being attacked. Regardless of whether the system is exposed to FDI, DoS, or joint attack designs, the system demonstrated strong resilience as it was only marginally affected. Energy systems must be defended in such a way that their continuous and secure operation is not endangered, especially since threats to the integrity of the system are becoming more sophisticated. The root locus method's application is also remarkable in this case because it demonstrates how the poles of the system change throughout the timing of the cyberattacks, which allows for a more comprehensive perception of stability. Mapping these behaviors in a clear way, as observed in Fig. 7, would indicate what future cyber-resilient systems may look like, as determining how a system would respond to not only prospective attack modes but also future uncommon attacks map a clear objective to the design. This capability is not only supportive of the validity of the current proposal, but it also shows its potential application to wider critical infrastructure systems that prioritize strong stability and resilience.

Table 10 presents a comparative analysis of the impact of false positive rejections of benign agents across three benchmark networks (IEEE 69-bus, 141-bus (Caracas), and IEEE 342-bus) and evaluates four models: the proposed TAFDRL, Static Filtered DRL, Federated A2C, and Centralized SAC. The metrics focus on benign rejection rate, reward loss due to false rejections, and associated DR contribution loss. Across all networks, TAFDRL consistently demonstrated the lowest benign rejection rates (1.8–3.2%) and minimal economic disruption, with reward losses and DR contribution reductions kept well below 2.5%. In contrast, other methods exhibited higher false rejection rates and substantially greater operational penalties, particularly in the larger networks. These

<table><caption>**Table 11**
Multidimensional economic comparison.</caption><thead><tr><th>Model</th><th>Total DR cost ($)</th><th>Welfare index ($)</th><th>Pricing spread ($/kWh)</th><th>Welfare loss (%)</th></tr></thead><tbody><tr><td>Proposed method</td><td>1250</td><td>4820</td><td>0.024</td><td>6.1</td></tr><tr><td>Static Filtered DRL</td><td>1390</td><td>4390</td><td>0.032</td><td>10.4</td></tr><tr><td>Federated A2C</td><td>1475</td><td>4175</td><td>0.039</td><td>13.9</td></tr><tr><td>Centralized SAC</td><td>1580</td><td>3930</td><td>0.048</td><td>17.6</td></tr></tbody></table>

results affirm that TAFDRL achieves a robust balance between security enforcement and operational fairness, maintaining system performance while minimizing unnecessary isolation of trustworthy agents. The consistent performance advantage across network scales further reinforces the scalability and practical viability of the proposed framework.

### 4.4. Market-level economic performance

The TAFDRL model was evaluated in terms of its economic performance under hybrid cyberattack scenarios for DR cost savings, social welfare, and price stability. As evident in Table 11, the model proposed resulted in the minimum compensation cost for DR (1250$) and maximum welfare index (4820$) among all of the tested approaches. In addition, it also provided the lowest price spread (0.024$/kWh), indicating maximum fairness in the distribution of energy costs among prosumers and buses during concurrent FDI and DoS attacks. Baseline models demonstrated varying degrees of economic degradation. The worst hit was the Centralized SAC model with 17.6% loss of social welfare and maximum price volatility (0.048$/kWh). Federated A2C and Static Filtered DRL performed average improvement but fell short of the reward efficiency and cost stability of the proposed method. These experiments validate that the collaboration of trust filtering, fault-tolerant federated reinforcement learning, and anomaly-aware policy aggregation supports not only secure coordination but also cost-effective energy market operation in challenging situations.

Fig. 8 presents the adaption rates of the participant load profile, as a function of the variable rate of the DR program. The DR program, as a dynamic and adaptive method, enabled participants to engage with a deviant load profile. In this case, the change rate across the limit of 1 pu and 1.1 pu indicates the system adapts to short-term and long-term change. As the analysis shows participants have enabled curve flattening by aligning their peak with the dynamic nature of the DR program. This adapter may lead to a more uniform consumption of electricity over time, which reduces unwanted peaks.

Changing consumption patterns not only reduces fluctuations in power demand but in turn, resulting in lower energy production and supply costs. By changing energy consumption patterns, we can supply

<table><caption>**Table 10**
Impact of false benign agent rejections on reward loss and DR contribution across test networks.</caption><thead><tr><th>Test Network</th><th>Model</th><th>Benign Rejection Rate (%)</th><th>Reward Loss from False Rejection (%)</th><th>DR Contribution Loss (%)</th></tr></thead><tbody><tr><td rowspan="5">IEEE 69-bus</td><td>TAFDRL</td><td>1.8</td><td>0.9</td><td>1.1</td></tr><tr><td>Static Filtered DRL</td><td>4.1</td><td>2.1</td><td>2.4</td></tr><tr><td>Federated A2C</td><td>5.2</td><td>2.9</td><td>3.1</td></tr><tr><td>Centralized SAC</td><td>5.6</td><td>3.8</td><td>3.9</td></tr><tr><td>TAFDRL</td><td>2.6</td><td>1.4</td><td>1.7</td></tr><tr><td rowspan="5">141-bus (Caracas)</td><td>Static Filtered DRL</td><td>4.6</td><td>2.6</td><td>2.8</td></tr><tr><td>Federated A2C</td><td>5.4</td><td>3.2</td><td>3.5</td></tr><tr><td>Centralized SAC</td><td>6.2</td><td>4.4</td><td>4.7</td></tr><tr><td>TAFDRL</td><td>3.2</td><td>2.1</td><td>2.4</td></tr><tr><td>Static Filtered DRL</td><td>4.9</td><td>2.9</td><td>3.2</td></tr><tr><td rowspan="2">IEEE 342-bus</td><td>Federated A2C</td><td>5.8</td><td>3.7</td><td>4.1</td></tr><tr><td>Centralized SAC</td><td>6.5</td><td>4.9</td><td>5.2</td></tr></tbody></table>
<figure><img src="image_12.png" /><figcaption>Fig. 8. Adaptability rate of participants in the DR program.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fig. 9. Tracking DR program target points and error rates.</figcaption></figure>

customers with less energy because it stops the introduction of more expensive power plants that are activated during peak consumption periods. Not only do these changes bring economic benefits to customers by reducing consumption in peak times, and also increase efficiency. Ultimately, these changes reduce operational and maintenance rates as managing energy and load consumption optimally reduces the burden on the grid infrastructure. Overall, this is a more efficient and economically sustainable way of managing energy systems, particularly in situations where the network is stressed, making this both an

economically and technically efficient method.

Fig. 9 shows not only how well the proposed method performed relative to the goals for tracking each of the target points in the DR program, but also the error rate between the target values and actual values [36]. From Fig. 9, it is clear that the proposed method replicates the DR targets with high accuracy. This suggests that the proposed method is very effective in reducing energy consumption following the dynamic targets of the DR program. Additionally, the figure shows that the error rate for the target DR values and actual values were below 2% error. This suggests that the proposed method was both robust and accurate in achieving the DR targets. Error rates this low are very valuable in energy management as they demonstrate the ability of the system to provide a consistent and dependable service despite changing conditions. Apart from ensuring energy grid stability, the ability to accurately track DR targets is foundational to reducing costs and optimizing energy consumption, which improves overall efficiency. Furthermore, the performance of the proposed approach, with an error rate of less than 2%, demonstrates the ability to manage load shifting and energy consumption effectively in dynamic/running scenarios. The impressive accuracy of this approach demonstrates that this is a definite direction for reliable provision of the demand response program. Programs such as this can provide a demand supply while not overloading the grid and having to turn on expensive extra sources of energy. The accuracy necessary for forward progress on smart grid developments is derived from these demonstrations and implementations. Accurate delivery in their timely delivery will be a key enabler for future directions towards efficiency and sustainability. Further performance verifies the success of

<table><caption>**Table 12**

Economic sensitivity analysis.</caption><thead><tr><th>Parameter</th><th>Price deviation (%)</th><th>DR fulfillment (%)</th><th>Reward drop (%)</th></tr></thead><tbody><tr><td>Agent trust threshold (Low)</td><td>2.5</td><td>93.5</td><td>2.2</td></tr><tr><td>Agent trust threshold (Medium)</td><td>3.1</td><td>92.8</td><td>3.1</td></tr><tr><td>Agent trust threshold (High)</td><td>3.9</td><td>91.2</td><td>3.6</td></tr><tr><td>DER availability (Low)</td><td>2.3</td><td>93.3</td><td>2.3</td></tr><tr><td>DER availability (Medium)</td><td>2.9</td><td>92.6</td><td>3.1</td></tr><tr><td>DER availability (High)</td><td>3.8</td><td>91.5</td><td>3.9</td></tr><tr><td>Market price fluctuation (Low)</td><td>2.5</td><td>94.0</td><td>2.1</td></tr><tr><td>Market price fluctuation (Medium)</td><td>3.2</td><td>91.7</td><td>3.5</td></tr><tr><td>Market price fluctuation (High)</td><td>4.3</td><td>91.1</td><td>4.1</td></tr></tbody></table>
<figure><img src="image_14.png" /><figcaption>**Fig. 10.** Sensitivity analysis of threshold parameters.</figcaption></figure>

integrating a highly adaptive system architecture into the greater reality of the future energy market where a highly coordinated and precisely timed real-time operation will be critical.

## 4.5. Operational and economic sensitivity analysis

According to Table 12, the economic sensitivity analysis provides numerical conclusions on how the entirety of the system works while changing some of the parameters. For agent trust threshold, changing the trust requirements from low (2.5%) to high (3.9%) decreases DR fulfillment from 93.5% to 91.2% and increases the reward drop from 2.2% to 3.6%, demonstrating that higher thresholds make the market exclusive and exacerbate penalties. In the case of DER availability, increasing from low (2.3%) to high (3.8%) lowers DR fulfillment just slightly from 93.3% to 91.5% and increases the reward drop from 2.3%

to 3.9%, indicating stability of resource is preferred over agility. For market price fluctuations, increasing volatility from low (2.5%) to high (4.3%) produces a drop in DR fulfillment from 94.0% to 91.1% and increase in reward drop from 2.1% to 4.1%, indicating the system is sensitive to price instability.

### 4.5.1. Sensitivity analysis of threshold parameters

To evaluate the impact of threshold parameters on system performance, we conducted a sensitivity analysis on the trust eligibility threshold $\tau$ and the entropy-based quarantine threshold $\bar{H}$. These parameters govern which agents are allowed to contribute to federated learning and demand response based on behavior stability and historical trust variability. We systematically varied $\tau$ from 0.1 to 0.9 and $\bar{H}$ from 0.5 to $\log(k+1)$, observing their joint effect on DR fulfillment and reward drop. Fig. 10 (a,b) provides a two-panel heatmap showing that optimal DR performance is maintained when $\tau \in [0.3,0.4]$ and $\bar{H} \in [0.85,0.95] \cdot \log(k+1)$. Outside this zone, DR effectiveness begins to degrade due to either overly permissive or overly strict filtering. The reward drop heatmap confirms that system performance remains robust even with moderate changes to either threshold. As trust thresholds increase beyond 0.5, more agents are excluded, slightly reducing collaborative learning and reward convergence. Conversely, setting $\bar{H}$ too low results in unnecessary quarantining of fluctuating but benign agents. The heatmaps demonstrate the interaction dynamics between security strictness and system efficiency, offering actionable guidance for parameter tuning. These visualizations validate that the proposed framework maintains high reward and DR performance across a wide range of configurations.

## 4.6. Evaluating the accuracy of cyberattack tracking

Fig. 11 shows the accuracy and performance for the detection of cyberattacks using the proposed approach, which was evaluated over 200 different operational points. The results indicate a high level of detection accuracy (less than 2% error) associated with identifying policies contaminated with either malicious manipulations or suspicious data. This robustness highlights the detection scheme's ability to discriminate legitimate policies against policies affected by manipulations, and within differing operational conditions. Furthermore, the ability to confidently track and separate suspicious data streams and policies, identified across multiple exploitation points, suggests a potential opportunities for more resilient cyber-physical energy systems and environments.

## 4.7. Ablation analysis

Table 13 presents a qualitative ablation study across three test networks to assess the relative contribution of the trust engine, anomaly detection, and encryption modules within the proposed TAFDRL framework. The results indicate that the trust engine consistently exerts the highest impact on system resilience, with its absence leading to severe degradation, especially in the larger IEEE 342-bus system. Anomaly

<figure><img src="image_15.png" /><figcaption>**Fig. 11.** Tracking suspicious policies and data.</figcaption></figure>
<table><caption>**Table 13**
Ablation study across test networks.</caption><thead><tr><td>Test network</td><td>Trust Engine impact</td><td>Anomaly Detection impact</td><td>Encryption impact</td><td>Overall degradation (without modules)</td></tr></thead><tbody><tr><td>IEEE 69-bus</td><td>High</td><td>Moderate</td><td>Low</td><td>Significant</td></tr><tr><td>141-bus (Caracas)</td><td>High</td><td>Moderate</td><td>Low</td><td>Severe</td></tr><tr><td>IEEE 342-node</td><td>Very High</td><td>Moderate</td><td>Low</td><td>Severe</td></tr></tbody></table>

<figure><img src="image_16.png" /><figcaption>**Fig. 12.** Experimental framework.</figcaption></figure>

detection provides moderate but consistent benefits across all scales by capturing structural misalignments in agent behavior that are not evident from trust history alone. Encryption, while important for protocol security, has minimal direct influence on performance metrics under simulated attack conditions. The overall degradation severity illustrates that removing any of these modules compromises the system's ability to maintain stability, coordination, and threat isolation under adversarial pressure, with the trust engine playing the most critical role in sustaining resilience.

### 4.8. *Experimental framework and implementation architecture*

According to Fig. 12, the experimental framework for evaluating the proposed TAFDRL approach is designed with both software and hardware layers to enable multi-level simulation, real-time interaction, and cyber-physical emulation. The structure follows a hybrid testbed methodology, combining high-fidelity simulation environments with real-time co-simulation infrastructure to validate the model's performance under dynamic and adversarial conditions. At the software layer,
the architecture comprises two integrated simulation environments: the MATLAB/Python co-simulation environment and the OPAL-RT's RT-Lab platform. In the first stage, the physical distribution network, including bus topology, load profiles, DER configurations, and power flow dynamics, is implemented in MATLAB using a steady-state and quasi-dynamic modeling framework. This setup supports economic dispatch modeling, voltage stability analysis, and power quality evaluation. In parallel, the federated deep reinforcement learning logic, including agent-level SAC modules, trust modeling, and anomaly detection filters, is configured in Python using modular libraries (e.g., PyTorch, TensorFlow Federated). The Python-based RL structure interfaces with the MATLAB power simulation using middleware APIs and shared data sockets to ensure bidirectional communication between physical system states and agent decision variables.

To support real-time testing and cyber-physical integration, the compiled MATLAB/Python co-simulation framework is deployed onto the RT-Lab simulation environment. This transfer is achieved through a Human-Machine Interface (HMI), which acts as a control and communication middleware layer. The HMI is developed in compliance with IEC 61850 communication protocols, enabling secure and synchronized communication across distributed nodes and control agents. Moreover, the simulation architecture is integrated with the EXata Cyber-Physical System (CPS) emulator. EXata-CPS plays a critical role in modeling the networked cyber-layer, including latency, jitter, packet loss, and the injection of realistic cyberattack scenarios such as FDI, DoS, and hybrid attacks. The inclusion of EXata allows for granular emulation of how adversarial behaviors affect control signals, trust scores, and agent collaboration, thereby enhancing the fidelity of the testbed. In the hardware layer, the compiled RT-Lab simulation codes are uploaded onto real-time digital simulators. The core of this setup is based on the OPAL-RT OP5707XG real-time simulator, which enables deterministic execution of electrical system dynamics with microsecond-level resolution. The hardware testbed is equipped with a real-time code generator, signal conditioning units, and a data acquisition system to monitor and record high-frequency signals such as voltage, frequency deviations, agent rewards, and communication delays. This hardware layer supports loop-in-the-loop control experiments, where the learning agents' outputs are used to drive physical signal injections or simulations of load switching and DER behavior.

Furthermore, the architecture includes an array of sensing and measurement devices that provide fine-grained observability over both the physical and cyber layers. These measurements are used to validate trust inference accuracy, reward convergence, anomaly detection precision, and system-level resilience metrics under stress-testing conditions. The layered integration of software and hardware ensures that the proposed model is evaluated not only in idealized conditions but also in emulated real-world environments, thereby increasing the practical relevance, generalizability, and reproducibility of the findings.

### 4.8.1. Experimental evaluation results

Table 14 presents the hardware-in-the-loop experimental results for the proposed TAFDRL framework under various cyber-attack scenarios across three microgrid test networks. Compared to the simulation results

previously reported in Table 6, these experimental values reflect realistic deviations due to implementation in a real-time federated architecture using OPAL-RT, EXata-CPS, and HMI-based coordination. Differences of approximately ±1–2% are observed across DR fulfillment, reward drop, and malicious agent rejection rates. These deviations are attributed to time-synchronization noise, hardware-induced delays, and real-time jitter in trust communication and reward aggregation.

This experimental table was selected to emphasize the model's robustness under operational conditions beyond ideal simulations. It validates the TAFDRL architecture's practicality, demonstrating that the performance advantage seen in software environments translates into physically realizable systems. The consistency of behavior across attacks (FDI, DoS, Hybrid) and scalability across network sizes (69-bus, 141-bus, 342-bus) supports the claim that TAFDRL is suitable for deployment in real microgrid contexts where trust adaptation, anomaly filtering, and DR control must co-exist under uncertainty.

### 4.9. Discussion

The proposed TAFDRL framework integrates trust as a core driver for federated control and anomaly mitigation. Rather than relying on static or binary indicators, trust evolves through multiple behavioral signals over time, which are analyzed collectively to adjust agent eligibility in both demand response execution and federated learning aggregation. This adaptability is essential in adversarial environments, where agent reliability must be assessed probabilistically and adjusted dynamically to reflect current conditions rather than historic labels alone. The structure not only supports accurate filtering of malicious behavior but also contributes to system-wide resilience by continuously recalibrating trust based on observable actions.

In deploying the system, new agents are introduced with a neutral trust level that avoids favoring or penalizing them at entry. This ensures that they are allowed to participate initially, but their continued involvement is earned through consistent interaction quality. As agents begin contributing, their actions are analyzed and incorporated into trust calculations, which evolve over time. The update process is responsive to both recent behavior and long-term trends, enabling rapid adaptation if patterns shift. In this way, the system encourages fair integration of newcomers while still enforcing robustness in the presence of misbehavior.

To avoid the dominance of long-standing participants, several mechanisms are incorporated. First, the decision logic determining whether an agent's policy is accepted is not absolute but allows for close approximations to the trusted set. This means newer agents with aligned behavior are able to contribute even if they do not have a long performance history. Second, the system continually re-evaluates agents based on their behavioral consistency. This allows trusted agents to lose standing if their performance degrades or becomes erratic. Together, these design choices ensure that trust remains dynamic and equitable, reducing the risk of exclusionary effects over time.

The system's ability to avoid biased outcomes is further evidenced by experimental results. Across multiple test networks, the proposed method consistently achieved low levels of false rejection and minimal

<table><caption>**Table 14**
Experimental results of TAFDRL.</caption><thead><tr><th>Test network</th><th>Attack type</th><th>Target buses</th><th>Price deviation (%)</th><th>DR fulfillment (%)</th><th>Reward drop (%)</th><th>Malicious agent rejection (%)</th></tr></thead><tbody><tr><td rowspan="3">IEEE 69-bus</td><td>FDI</td><td>18, 31, 45</td><td>2.5</td><td>93.6</td><td>3.6</td><td>90.4</td></tr><tr><td>DoS</td><td>12, 28, 62</td><td>2.9</td><td>91.9</td><td>4.2</td><td>88.3</td></tr><tr><td>Hybrid</td><td>17, 34, 47, 54</td><td>3.7</td><td>89.8</td><td>5.6</td><td>86.1</td></tr><tr><td rowspan="3">141-bus (Caracas)</td><td>FDI</td><td>23, 89, 124</td><td>3.0</td><td>92.1</td><td>4.3</td><td>88.8</td></tr><tr><td>DoS</td><td>19, 76, 133</td><td>3.4</td><td>90.5</td><td>4.9</td><td>85.5</td></tr><tr><td>Hybrid</td><td>42, 77, 108, 135</td><td>4.3</td><td>87.8</td><td>5.2</td><td>82.1</td></tr><tr><td rowspan="3">IEEE 342-bus</td><td>FDI</td><td>102, 144, 198</td><td>3.2</td><td>89.6</td><td>5.5</td><td>85.6</td></tr><tr><td>DoS</td><td>95, 186, 224</td><td>3.6</td><td>88.1</td><td>5.9</td><td>83.2</td></tr><tr><td>Hybrid</td><td>112, 190, 243, 267</td><td>4.5</td><td>85.7</td><td>6.7</td><td>79.8</td></tr></tbody></table>
losses due to mistakenly filtered participants. These results were maintained even in large-scale environments, showing that the system performs well under practical deployment conditions. Additionally, parameter sensitivity studies confirmed that the framework remains stable and effective across a wide range of settings, which reflects strong design robustness and flexibility in handling various operational scenarios.

The trust model embedded within TAFDRL is designed to be adaptive, fair, and resistant to systemic biases. It supports secure operation without penalizing honest but inexperienced agents and protects against sophisticated attacks without relying on rigid rule-based filters. This balance enables the proposed method to function effectively across dynamic and adversarial environments, ensuring a resilient and inclusive architecture for federated coordination in modern energy systems.

## 5. Conclusion

This paper presents a trust-aware and cyber-resilient energy market framework for coordinating day-ahead and real-time pricing under adversarial conditions within a microgrid. The proposed model is based on TAFDRL and extends the TAFDRL framework by adding various sophisticated components to enhance the security of the market and enable decentralized learning. A special component is a Bayesian trust score used to continuously evaluate the reliability of agents based on cyber-physical indicators. The system is further complemented by a latent-space policy validation so that the system can evaluate the legitimacy of policies with respect to neural network embeddings to reject malicious policy updates. The capabilities of secure model sharing and privacy across decoupled agents are further made possible with the encrypted aggregation of the SAC models. Altogether, these capabilities give the framework the ability to appropriately adapt and securely coordinate DR actions in case of cyber conditions such as FDI, DoS or combined cyberattacks. The proposed framework contains formal trust modeling and adaptive DR coordination to maintain operational stability and efficiency of the market while allowing dynamic responses to real-time scenarios. This paper advances beyond the use of established components by tightly integrating trust modeling, anomaly detection, and federated reinforcement learning into a unified TAFDRL architecture. This synergy enables real-time adaptivity, adversarial resilience, and fairness in decentralized energy systems, capabilities that isolated approaches cannot achieve alone. The proposed framework thus sets a foundation for secure, trust-aware coordination in future smart grids and distributed energy markets.

The results of the simulation confirm that the proposed methodology is effective in the real world. The model has a cumulative reward of 94% and reward stability of 92% with welfare loss below 6.5% under hybrid

attacks. This indicates a substantial improvement in performance compared to the baseline models based on static DRL, Federated A2C, and centralized SAC, where average performance ranged from 9.3% to 15.7% lower than the model generated in this study. These findings also illustrate the model's ability to sustain its peak performance even when faced with adversarial measures or cyberattacks, which provides an improved robustness against cyberattacks. The stability analysis shows that the system is also capable of maintaining excellent reward stability and negligible policy shifts despite being subjected to extreme attack scenarios, which demonstrates the resilience of the model. The sensitivity analysis illustrates the trade-off relationship between security, available resources, and capacity to be in the open market. Greater trust thresholds, more available DERs, or more volatile prices all result in less DR fulfillment, leading to an increased reward loss. This finding demonstrates the equilibrium between maximizing system security and effective market results, providing guidelines for future design and policy decisions in the energy market framework.

Future work will consist of augmenting this framework to support non-cooperative agent ranged environments through adversarial game theory to protect against smart or colluding adversaries. Future work will also include the addition of blockchain based audit trails to verify the update trail, as well as utilizing edge-device functionality to allow for on-premise policy verification.

### CRediT authorship contribution statement

**Reza Sepehrzad:** Writing – original draft, Validation, Software, Resources, Methodology, Investigation, Data curation, Conceptualization. **Hossein Hosseinalibeiki:** Writing – review & editing, Software, Resources, Investigation, Formal analysis. **Neda Taghinezhad:** Writing – review & editing, Validation, Software, Resources, Formal analysis. **Nima Khosravi:** Data curation, Formal analysis, Validation, Writing – review & editing. **Ahmed al Durra:** Funding acquisition, Investigation, Project administration, Supervision, Validation. **Mahdieh S. Sadabadi:** Funding acquisition, Investigation, Resources, Supervision, Validation.

### Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

### Acknowledgment

This work was supported by Khalifa University of Science and Technology (KUST), Abu Dhabi, UAE, under Award CIRA-2021-063.

## Appendix A. : Proof of equations 15 to 20

**Theorem 1 (Simplified Exponential Trust Scaling):** Let $B_i^t \in \mathbb{R}^d$ be the normalized behavior vector, and $\omega_i^t$ the credibility weights. If we approximate Eq. (15) by absorbing the integral into a proportionality constant, then:

$$T_i^t \approx \exp(B_i^t \cdot \omega_i^t) \cdot T_i^{t-1}$$

This yields an exponential scaling rule, where aligned behavior boosts trust multiplicatively.

**Proof:** The exponential term acts as a scaling factor $\eta_i^t \in (0, \eta_{\max}]$, giving:

$$T_i^t \approx \eta_i^t \cdot T_i^{t-1}, \text{ with } \eta_i^t = \exp(B_i^t \cdot \omega_i^t)$$

Thus, when behavior aligns with credibility ($B_i^t \cdot \omega_i^t > 0$), trust increases; otherwise, it diminishes. This update ensures nonlinearity and adaptive learning from behavior history.

**Theorem 2 (Kalman Update as Discounted Error Correction):** The recursive trust estimator:

$$\hat{T}_i^t = \hat{T}_i^{t-1} + K_i (z_i^t - H_i \hat{T}_i^{t-1})$$

can be interpreted as a gain-modulated correction:
$$ \hat{T}_i^t = \hat{T}_i^{t-1} + \alpha_i^t \cdot \epsilon_i^t, \text{ with } \epsilon_i^t = z_i^t - H_i^t \hat{T}_i^{t-1} $$

**Proof:** Kalman gain $K_i^t$ is equivalent to a dynamic learning rate $\alpha_i^t$, which downweights updates when uncertainty is high (large $R_i^t$). The filter thus smooths trust signals while remaining sensitive to sustained behavior patterns. $z_i^t$ is trust evidence (observation) for agent i at time t.

**Theorem 3 (Latent Policy Filtering as Soft Clipping):** Let $\Phi(\pi_i)$ be the embedding of agent i's policy, and $P_{M_i}[\Phi(\pi_i)]$ its projection onto the trusted manifold. Then:

$$ \epsilon_i^t = \|\Phi(\pi_i) - P_{M_i}[\Phi(\pi_i)]\|_2 $$

If $P_{M_i}[\Phi(\pi_i)] \approx \bar{\Phi}$ (centroid of trusted policies), we get:

$$ \epsilon_i^t \approx \|\Phi(\pi_i) - \bar{\Phi}\|_2 $$

and acceptance indicator:

$$ \|\text{valid}_i\| = \|[\epsilon_i^t \le \xi(T_i^t)]\|_2 $$

**Proof:** This forms a geometric filter: if the agents policy embedding lies within a trust-scaled hypersphere around the centroid, it is accepted. The threshold $\xi(T_i^t)$ shrinks as trust declines, ensuring only compliant updates are allowed.

## Appendix B. : Proof of equations 21 to 22

**Theorem (Trust Entropy as Anomaly Indicator):** Let $T_i^{t-j}$ be the historical trust scores of agent i over j = 0 to k. Define:

$$ \rho_j^t = \frac{T_i^{t-j}}{\sum_{l=0}^{k} T_i^{t-l}} \mathcal{H}_i^t = - \sum_{j=0}^{k} \rho_j^t \log \rho_j^t \quad (21) $$

and let:

$$ \|\text{quarantine}_i\| = \|[\mathcal{H}_i^t > \bar{\mathcal{H}}]\| \quad (22) $$

Assume $T_i^{t-j} > 0$ for all j. Then $\mathcal{H}_i^t$ satisfies:

$$ 0 \le \mathcal{H}_i^t \le \log(k+1) $$

and achieves its maximum when all trust values are equal, i.e.,

$$ T_i^{t-j} = c > 0 \Rightarrow \rho_j^t = \frac{1}{k+1}, \mathcal{H}_i^t = \log(k+1) $$

**Proof:** From the definition of Shannon entropy:

$$ \mathcal{H}_i^t = - \sum_{j=0}^{k} \rho_j^t \log \rho_j^t $$

the function is maximized when all probabilities are uniform. That is, when:

$$ \rho_j^t = \frac{1}{k+1} \Rightarrow \mathcal{H}_i^t = - \sum_{j=0}^{k} \frac{1}{k+1} \log \left( \frac{1}{k+1} \right) = \log(k+1) $$

Conversely, if trust is concentrated at one time point, e.g., $\rho_0^t = 1$, then $\mathcal{H}_i^t = 0$. Hence, entropy measures unpredictability:

$$ \mathcal{H}_i^t \uparrow \Rightarrow \text{agent is less stable and } T_i^{\text{quarantine}} = 1 \text{ if } \mathcal{H}_i^t > \bar{\mathcal{H}} $$

Above rule detects trust volatility reliably.

## Data availability

Data will be made available on request.

## References

[1] Ahmad Jameel, et al. Cybersecurity in smart microgrids using blockchain-federated learning and quantum-safe approaches: A comprehensive review. Appl Energy 2025;393:126118.

[2] Sepehrzad Reza, et al. Enhancing cyber-resilience in electric vehicle charging stations: a multi-agent deep reinforcement learning approach. IEEE Trans Intell Transp Syst 2024;25(11):18049-62.

[3] Solat Amirhossein, et al. On the control of microgrids against cyber-attacks: A review of methods and applications. Appl Energy 2024;353:122037.

[4] Melendez Kevin A, Matamala Yolanda. Adversarial attacks in demand-side electricity markets. Appl Energy 2025;377:124615.

[5] Khodadadi Amin, et al. Data-Driven hierarchical energy management in multi-integrated energy systems considering integrated demand response programs and energy storage system participation based on MADRL approach. Sustain Cities Soc 2024;103:105264.

[6] Hassanzadeh Mohammad Esmaeil, et al. Hierarchical optimal allocation of BESS using APT-FPSO based on stochastic programming model considering voltage sensitivity and eigenvalues analyses. Int J Electr Power Energy Syst 2023;153: 109291.

[7] Sepehrzad Reza, et al. A multi-agent deep reinforcement learning paradigm to improve the robustness and resilience of grid connected electric vehicle charging stations against the destructive effects of cyber-attacks. Energy 2024;307:132669.

[8] Zhang Yiwen, et al. Federated deep reinforcement learning for varying-scale multi-energy microgrids energy management considering comprehensive security. Appl Energy 2025;380:125072.

[9] Khalatbarisoltani Arash, Boulon Loic, Xiaosong Hu. Integrating model predictive control with federated reinforcement learning for decentralized energy management of fuel cell vehicles. IEEE Trans Intell Transp Syst 2023;24(12): 13639-53.

[10] Yang Ting, et al. Cooperative optimal dispatch of multi-microgrids for low carbon economy based on personalized federated reinforcement learning. Appl Energy 2025;378:124641.

[11] Zhang Q, Li F, Wang X. Cyber-Impact Analysis for ISO Revenue Adequacy Considering FDIA in Real-Time Market Operations. IEEE Trans Power Syst 2023;38 (5):4042-53.

[12] Jena PK, Koley E, Ghosh S. An Optimal Scheme for Installation of PMUs and IEDs to Reinforce Electricity Market Immunity Against Data Attacks in Smart Grid. IEEE J Emerg Selected Topics Indus Electronics 2023;4(2):603-13.

[13] Seshasai B, Koley E, Jena PK, Ghosh S. Design of Real-Time False Data Injection Attack on Electricity Market With Limited Sensor Accessibility. IEEE Syst J 2024;18 (4):1999-2009.
[14] Miles AG, Chakhchoukh Y, Alam SMS. Robust Distribution State Estimation for Reliable Locational Marginal Pricing Under Cyber-Attacks. IEEE Trans Power Syst 2024;39(3):4963-74.

[15] M. Asghari, A. Ameli, M. Ghafouri and M. N. Uddin, "Optimal Data Aggregation Reconfiguration Scheme to Mitigate Stealthy False Data Injection Attacks in Energy Management Systems," in IEEE Transactions on Smart Grid, vol. 16, no. 4, pp. 3269-3281, July 2025.

[16] Badia L, Borgo M, Principe B, Spina L, Crosara L, Gindullina E. Attack Strategies Among Prosumers in Smart Grids: A Game Theoretic Analysis. IEEE Trans Ind Appl 2025;61(1):1068-79.

[17] Mishra DK, Wang J, Li L, Zhang J, Hossain MJ. Resilience-Driven Scheme in Multiple Microgrids With Secure Transactive Energy System Framework. IEEE Trans Ind Appl 2024;60(2):2277-89.

[18] Andriopoulos N, Kanakaris N, Birbas A, Papalexopoulos A, Birbas M. Cyber-Resilient Operation of IoT-Enabled Power Grid: A Nodal Local Energy Market Approach. IEEE Transact Indus Cyber-Phy Syst 2025;3:27-38.

[19] Li Y, Mamon R. The Price Tag of Cyber Risk: A Signal-Processing Approach. IEEE Access 2023;11:44294-318.

[20] Bellizio F, et al. Transition to Digitalized Paradigms for Security Control and Decentralized Electricity Market. Proc IEEE 2023;111(7):744-61.

[21] Naderi E, Asrari A. A Remedial Action Scheme to Mitigate Market Power Caused by Cyberattacks Targeting a Smart Distribution System. IEEE Transact Indus Informatics 2024;20(3):3197-208.

[22] Zhang Z, Bu S, Zhang Y, Han Z. Market-Level Integrated Detection Against Cyber Attacks in Real-Time Market Operations by Self-Supervised Learning. IEEE Transact Smart Grid 2024;15(4):4128-42.

[23] Dedrick Jason, et al. Assessing cyber attacks on local electricity markets using simulation analysis: Impacts and possible mitigations. Sustain Energy Grids Networks 2023;34:100993.

[24] Veerasamy Veerapandiyan, et al. Blockchain-enabled peer-to-peer energy trading and resilient control of microgrids. Appl Energy 2024;353:122107.

[25] Galici Marco, et al. Comparative techno-economic analysis of market models for peer-to-peer energy trading on a distributed platform. Appl Energy 2025;380: 125005.

[26] Sepehrzad Reza, et al. Demand response-based multi-layer peer-to-peer energy trading strategy for renewable-powered microgrids with electric vehicles. Energy 2025;320:135206.

[27] Gao Y, Li S, Dong W, Xiao Y, Lu B. Intelligent Building Energy Management Considering Uncertainties and Variation of Day-Ahead and Real-Time Electricity Prices. IEEE Internet Things J 2024;11(23):37785-98.

[28] Sepehrzad Reza, et al. Short-term and probability scenario-oriented energy management of integrated energy distribution systems with considering energy market interactions and end-user participation. Energy 2025;322:135691.

[29] Sener Ozan, Koltun Vladlen. Multi-task learning as multi-objective optimization. Adv Neural Inf Proces Syst 2018;31.

[30] Dehghannasiri R, Esfahani MS, Dougherty ER. Intrinsically Bayesian Robust Kalman Filter: An Innovation Process Approach. IEEE Trans Signal Process 2017;65 (10):2531-46.

[31] Sepehrzad Reza, et al. An applied deep reinforcement learning approach to control active networked microgrids in smart cities with multi-level participation of battery energy storage system and electric vehicles. Sustain Cities Soc 2024;107: 105352.

[32] Maji S, Banerjee U, Chandrakasan AP. Leaky Nets: Recovering Embedded Neural Network Models and Inputs Through Simple Power and Timing Side-Channels—Attacks and Defenses. IEEE Internet Things J 2021;8(15):12079-92.

[33] Sepehrzad Reza, et al. Two-Stage experimental intelligent dynamic energy management of microgrid in smart cities based on demand response programs and energy storage system participation. Int J Electr Power Energy Syst 2024;155: 109613.

[34] Sepehrzad Reza, et al. Intelligent hierarchical energy and power management to control the voltage and frequency of micro-grids based on power uncertainties and communication latency. Electr Power Syst Res 2022;202:107567.

[35] Hedayatnia Atefeh, et al. Two-stage data-driven optimal energy management and dynamic real-time operation in networked microgrid based on a deep reinforcement learning approach. Int J Electr Power Energy Syst 2024;160:110142.

[36] Sepehrzad, Reza, et al. "A critical overview of local energy communities: State-of-the-art, real-life applications & challenges and tackling the academia-industry gap." *Renewable and Sustainable Energy Reviews* 226 (2026): 116165.