# Voltage Control for DC Microgrids: A Review and Comparative Evaluation of Deep Reinforcement Learning

Sharafadeen Muhammad, Hussein Obeid, Abdelilah Hammou, Melika Hinaje, Hamid Gualous

► To cite this version:

Sharafadeen Muhammad, Hussein Obeid, Abdelilah Hammou, Melika Hinaje, Hamid Gualous. Voltage Control for DC Microgrids: A Review and Comparative Evaluation of Deep Reinforcement Learning. Energies, 2025, 18 (21), pp.5706. 10.3390/en18215706. hal-05359562

HAL Id: hal-05359562

https://hal.science/hal-05359562v1

Submitted on 23 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

Review

# Voltage Control for DC Microgrids: A Review and Comparative Evaluation of Deep Reinforcement Learning

Sharafadeen Muhammad ¹ID, Hussein Obeid <fnref n="2" />*ID, Abdelilah Hammou ¹ID, Melika Hinaje ³ID and Hamid Gualous <fnref n="1" />

## Abstract

Voltage stability in DC microgrids (DC MG) is crucial for ensuring reliable operation and component safety. This paper surveys voltage control techniques for DC MG, classifying them into model-based, model-free, and hybrid approaches. It analyzes their fundamental principles and evaluates their strengths and limitations. In addition to the survey, the study investigates the voltage control problem in a critical scenario involving a DC/DC buck converter with an input LC filter. Two model-free deep reinforcement learning (DRL) control strategies are proposed: twin-delayed deep deterministic policy gradient (TD3) and proximal policy optimization (PPO) agents. Bayesian optimization (BO) is employed to enhance the performance of the agents by tuning their critical hyperparameters. Simulation results demonstrate the effectiveness of the DRL-based approaches: compared to benchmark methods, BO-TD3 achieves the lowest error metrics, reducing root mean square error (RMSE) by up to 5.6%, and mean absolute percentage error (MAPE) by 7.8%. Lastly, the study outlines future research directions for DRL-based voltage control aimed at improving voltage stability in DC MG.

**Keywords:** DC/DC converters; DC microgrid; deep reinforcement learning; Bayesian optimization; renewable energy; voltage control

## 1. Introduction

The 21st century has witnessed a rapid increase in distributed energy resources (DER), driven by advances in power electronics, the widespread adoption of renewable energy sources (RES), and the growing use of energy storage devices (ESD) [1]. RES such as solar photovoltaic (PV), fuel cells, wind, and tidal energy systems are increasingly favored for their sustainability, abundance, and environmental benefits. They are widely used in residential, industrial, and remote applications where access to conventional grid power is limited or economically impractical. This transition supports global efforts to mitigate climate change and ensure clean, sustainable energy for future generations [2], forming the foundation for the evolution from conventional centralized grids to distributed microgrid (MG) architectures.

MG can be defined as an electrically bounded area of a distributed network that integrates local DER, ESD, and loads to form a self-sufficient energy system [3]. MG are generally classified into three types, based on the current in their main bus: AC, DC, and

<figure><img src="image_3.png" /></figure>

check for
updates

Academic Editor: José Matas

Received: 24 September 2025

Revised: 23 October 2025

Accepted: 27 October 2025

Published: 30 October 2025

**Citation:** Muhammad, S.; Obeid, H.; Hammou, A.; Hinaje, M.; Gualous, H. Voltage Control for DC Microgrids: A Review and Comparative Evaluation of Deep Reinforcement Learning. *Energies* **2025**, *18*, 5706. https://doi.org/10.3390/en18215706

**Copyright:** © 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

<footnote n="1">¹ LUSAC Laboratory, University of Caen Normandy, 50130 Cherbourg-en-Cotentin, France; sharafadeen.muhammad@etu.unicaen.fr (S.M.); abdelilah.hammou@unicaen.fr (A.H.); hamid.gualous@unicaen.fr (H.G.)</footnote>

<footnote n="2">² Department of Mechanical and Industrial Engineering, College of Engineering, Sultan Qaboos University, Muscat 123, Oman</footnote>

<footnote n="3">³ LRGP, CNRS, University of Lorraine, 54000 Nancy, France; melika.hinaje@univ-lorraine.fr</footnote>

<footnote>* Correspondence: h.obeid@squ.edu.om</footnote>
hybrid. AC MG are widely used to reduce dependence on the main grid and supply power to target loads [4]. Despite challenges, such as grid synchronization, power sharing among coupled DER, and frequency control, AC MG remain prevalent due to their grid compatibility and the maturity of their control and protection technologies [5].

Over the past few decades, AC MG have received primary attention due to the dominance of AC networks. However, the growing integration of RES, ESD, and DC loads has shifted interest toward DC MG [6]. In particular, DC MG offers a simpler structure with zero reactive power flow and provides reduced power losses with improved efficiency. Moreover, they exhibit superior compatibility with the increasing number of modern DC-based consumer devices [7].

Despite numerous advantages, voltage instability remains a significant challenge that affects the performance of DC MG [8]. Voltage and power fluctuations, often exacerbated by the intermittent nature of RES and other uncertainties, can degrade power quality and overall efficiency. Prolonged instability may even cause equipment malfunction or system failure. Consequently, voltage stability has remained a critical area of research, with extensive efforts devoted to developing various model-based, model-free, and hybrid control techniques.

In model-based control, a mathematical model of the system is required to design the control algorithms. These methods rely on accurate system parameters and are commonly applied in optimal and predictive control [9], with examples including sliding mode control, predictive control, passivity-based control, and the classical proportional-integral (PI) controller. They offer analytical stability guarantees and can generate control actions with high accuracy [10], but are often sensitive to parameter variations and unmodeled dynamics. In contrast, model-free techniques do not require an explicit system model and are typically experience-based or rely on data-driven approaches [11], such as fuzzy logic control, artificial neural networks, and reinforcement learning. These methods offer flexibility and adaptability, especially in complex systems. However, their reliance on data can be challenging, particularly when system dynamics change rapidly or training data is insufficient [11]. On the other hand, hybrid control techniques combine aspects of both model-based and model-free schemes, examples include physics informed neural networks and metaheuristic-optimized control techniques.

Several studies have reviewed voltage control methods for DC MG applications. For example, ref. [12] discusses various topologies and control strategies, focusing on centralized, decentralized, and hierarchical control structures. Similarly, ref. [13] analyzes the strengths and weaknesses of various control topologies, with particular emphasis on droop control. In [14], the performance of advanced control technologies is evaluated, specifically for their application in bidirectional DC/DC converters to address instability issues caused by constant power loads (CPL). Another survey, ref. [15], explores voltage control techniques that incorporate different levels of communication for standalone MG applications.

This study provides a comprehensive survey of various advanced voltage control techniques, which are categorized into model-based, model-free, and hybrid approaches. In addition to this, the paper proposes model-free DRL-based control strategies for a DC/DC converter with an input LC filter: value-based TD3 and policy-based PPO. The proposed agents are designed, trained, and tuned using Bayesian optimization (BO) to address instability issues arising in the cascaded system. Furthermore, the proposed approaches are benchmarked against other control strategies through simulations under different operating conditions. Lastly, future research directions for voltage control in DC MG are identified. The main contributions of this work are summarized as follows:
* It provides a survey of various state-of-the-art model-based, model-free, and hybrid voltage control techniques.

* It proposes BO-DRL-based solutions for voltage control of a DC/DC buck converter with an input LC filter.

* It proposes recommendations for future research, particularly those that employ machine learning and data-driven control algorithms.

The organization of the rest of the paper is as follows: Section 2 provides an overview of DC MG control. Detailed reviews of model-based, model-free, and hybrid control techniques are discussed in Section 3, Section 4, and Section 5, respectively. Section 6 presents the design and analysis of a DRL control strategy based on TD3 and PPO, incorporating BO for optimal tuning of the agents' hyperparameters. Section 7 outlines future research directions, and finally, the conclusion is given in Section 8.

## 2. Background of the Study

DC MG has gained significant attention for enabling more reliable and sustainable energy systems. In general, their operation can be classified into two modes: grid-connected or standalone (autonomous) [12]. When the DC MG exchanges power with the main grid, it operates in grid-connected mode. On the other hand, when it operates independently, relying on local generation to meet load demand, it is in stand-alone mode. In this mode, ESD must absorb or release the power difference between the generation and the load.

Depending on the application, a DC MG incorporates various types of power electronic converters to integrate distributed energy sources, energy storage systems, and loads. The converters are typically connected to a common DC bus, which serves as an interface for achieving high efficiency and straightforward control. Notably, DC MG has seen widespread adoption across industrial, residential, and military sectors, including electric vehicles, data centers, naval ships, submarines, spacecraft, and net-zero energy buildings. A typical architecture of DC MG is shown in Figure 1.

<figure><img src="image_4.png" /><figcaption>Figure 1. Typical configuration of DC microgrid.</figcaption></figure>

### 2.1. *Cause of Voltage Instability*

The pressing operational challenges of DC MG, particularly in maintaining voltage stability, arise from several key factors, including the following:
* Intermittent generation: The fluctuating nature of RES causes mismatch between generation and demand, leading to voltage instability [16]. For example, PV output depends on weather conditions like temperature and solar irradiance.

* CPL are nonlinear loads characterized by negative incremental impedance [17]. They maintain constant power consumption despite DC bus voltage fluctuates. DC load regulated by DC/DC converters often exhibits CPL behavior, which can destabilize the system. Voltage instability caused by CPL has been extensively studied [18,19]. Figure 2 illustrates this negative incremental impedance behavior.

* Pulse power loads (PPL) draw large current in short durations, potentially causing voltage instability due to their high-power characteristics [20]. They are common in onboard MG of electric ships, particularly for systems like sonar and radar.

* Faults and aging can alter system dynamics, posing the risk of instability and poor performance. Additional challenges related to the operation, control, and protection of DC MGs are reported in [21].

* Filters, such as commonly used LC types, improve power quality, but can reduce the damping ratio of DC MG, increasing the risk of instability [22].

<figure><img src="image_5.png" /><figcaption>Figure 2. Negative incremental impedance characteristics of CPL.</figcaption></figure>

## 2.2. DC Microgrid Control

To ensure stable and efficient operation, DC MG commonly employs a hierarchical control structure comprising three levels: primary, secondary, and tertiary [23]. The primary controller acts locally to regulate the DC bus voltage and maintain power sharing at the lowest level. It operates at the fastest timescale, and voltage deviations at this level are corrected by the slower secondary controller [24]. The tertiary controller, operating at the highest and slowest level, optimizes the overall system performance and power flow among units. This study specifically focuses on recent advances in primary control, emphasizing model-based, model-free, and hybrid approaches.

# 3. Model-Based Techniques

Model-based control, which relies on knowledge of the system model, forms the foundation of control theory for designing effective controllers. In DC MG, model-based voltage control employs both conventional and advanced controllers. Traditional linear techniques, such as PI and LQR [25], have been successfully implemented but often struggle with system uncertainties. A review of advanced control techniques is presented in the following sub-sections.
## 3.1. Sliding Mode Control

Sliding mode control (SMC) is a nonlinear technique well suited for systems with uncertainties and disturbances [26,27]. It operates by driving system states onto a predefined sliding surface and maintaining them there. In DC MG, SMC is widely used for voltage regulation [28,29]. However, it can suffer from chattering, a phenomenon that, if not properly addressed, may cause excessive wear of power components.

Various sliding surfaces have been proposed to reduce chattering. The authors in [30] introduce an approach incorporating a hysteresis band in the switching function. To improve the voltage stability of a boost converter, ref. [31] proposes a fully decentralized second-order SM (SOSM) control capable of handling unknown loads without a state observer. A distinguishing feature of this method is the inclusion of an auxiliary integral control using an appropriate sliding function.

The investigation in [32] presents a particle swarm optimization (PSO)-tuned SMC. The evolutionary algorithm optimizes controller parameters based on stability conditions. To further reduce chattering and provide an adjustable sliding coefficient, ref. [33] introduces an adaptive SMC technique. In [34], a global fast terminal sliding mode controller with hysteresis modulation (HM-GFTSMC) is proposed to enhance voltage stability in a buck converter. The study in [35] proposes a robust higher-order PID sliding mode controller (PID-HOSMC) using a double power reaching law. The PID-HOSMC stabilizes the bus voltage via battery control signals. To achieve proportional load power sharing and DC bus voltage regulation, ref. [36] proposes a decentralized control approach combining SMC based on a higher-order finite-time observer (HOFTO) with droop control.

Moreover, SMC has been combined with a radial basis function neural network (RBFNN) to enhance performance. RBFNNs are three-layer feedforward neural networks with a single hidden layer that can approximate continuous functions with arbitrary precision [37,38]. In [39], an RBFNN adaptively tunes the switching gain of a sliding mode observer, ensuring asymptotic error convergence in the state of charge (SOC) estimation of a lithium-polymer battery. Similarly, ref. [40] employs an adaptive SMC with RBFNN estimation and backstepping control to improve the performance of a proton exchange membrane fuel cell (PEMFC).

Although SMC has been successfully applied for voltage regulation, its implementation faces several challenges. Increasing system complexity can make designing effective SMC difficult. Moreover, its performance strongly depends on model accuracy, sliding surface selection, and control law formulation. Despite efforts to reduce chattering, it remains a persistent challenge in many applications.

## 3.2. Adaptive Droop Control

Unlike conventional droop control with fixed coefficients, adaptive droop control adapts to variations in power, enabling more precise control [41]. Several droop-based methods have been developed to enhance voltage stability and power sharing accuracy. Reference [42] proposes a hierarchical two-stage controller. The first stage involves an adaptive droop control with voltage feedback compensation, while the second provides supervisory control. The study in [43] presents an adaptive droop and voltage shifting technique for balanced current sharing. Additionally, ref. [44] presents a unified control strategy for supercapacitor systems. The strategy rejects disturbances and regulates terminal voltage, demonstrating satisfactory plug-and-play capability using local measurements.

Another study in [45] presents a nonlinear droop control algorithm for a hybrid islanded MG. The method achieves good tracking performance through the integration of SMC. To further enhance voltage stability, ref. [46] introduces a resiliency-driven droop index control strategy, in which the droop coefficient is tuned using PSO.
Overall, adaptive droop control presents a promising solution for improving both voltage stability and power-sharing accuracy, although its performance may degrade in complex and fast-dynamic systems.

### 3.3. *Model Predictive Control*

Model Predictive Control (MPC) uses a system model to predict future behavior and optimize control actions over a finite time horizon [47]. At each sampling instant, an optimization problem is evaluated using a defined cost function while explicitly considering constraints [48].

MPC approaches are generally categorized as continuous control set (CCS-MPC) and finite control set (FCS-MPC) [49]. In CCS-MPC, converter switching is governed by PWM at fixed frequency, whereas FCS-MPC operates with a variable switching frequency [50]. It is worth noting that FCS-MPC offers simple implementation and high dynamic response [51]. However, its online computational burden increases with prediction horizon length. A comprehensive analysis of MPC is discussed in [52].

Recent studies have introduced different schemes of MPC-based control strategies. In [53], an MPC-based scheme for hybrid energy storage systems (HESS) is developed to enhance MG resilience under diverse disturbances. The authors adopt FCS-MPC to control a superconducting magnetic energy storage unit and a backup battery. Reference [54] proposes a fast-distributed MPC method that improves steady-state performance using the alternating-direction method of multipliers. To strengthen dynamic stability, ref. [55] investigates a hybrid MPC capable of operating in both continuous and discontinuous current modes. The simulation results showed improvements in voltage stability and faster response compared to CCS. Moreover, ref. [56] introduces an FCS-MPC algorithm integrated with a Kalman observer. In this framework, the FCS-MPC tracks the reference current, while the Kalman observer provides feedforward compensation.

Despite its advantages, MPC performance remains dependent on model accuracy, and its computational requirements increase with prediction horizon, necessitating a powerful processor.

### 3.4. *Passivity-Based Control*

Passivity-based control (PBC) ensures system stability by modeling the system as an energy-conserving (passive) structure [57]. It relies on the concept of passivity to shape the energy of the system and stabilize it [58]. In DC MG, PBC is employed to regulate bus voltage, manage power flows, and maintain stability under varying operating conditions. This approach is particularly effective for controlling DC/DC converters with ESDs, where energy management is critical. The main advantage of PBC lies in its robustness and inherent stability; however, its performance can be limited when dealing with unmodeled disturbances.

Several studies have proposed PBC-based control strategies. For example, ref. [10] presents a passivity-based approach using a Krasovskii-type storage function. In [59], a passivity-based design is introduced for a non-affine, nonlinear system. The study proposes a low-gain controller using a dynamical model described by three inputs, three outputs non-affine system with six states. To improve voltage stability in an isolated DC MG with hydrogen energy storage, ref. [60] proposes an interconnection and damping assignment passivity-based controller (IDA-PBC). A sliding mode reference conditioning (SMRC) loop is complemented to prevent overvoltage. Additionally, ref. [61] proposes an adaptive IDA-PBC for a buck-boost converter supplying unknown CPLs. The authors incorporated an online parameter estimator to track load power, which is challenging to measure in practical applications.
### 3.5. Active Disturbance Control

Active Disturbance Rejection Control (ADRC) is a model-assisted technique that estimates and compensates for internal and external disturbances in real time [62]. It typically employs an extended state observer to reject disturbances. This makes it effective in situations where the system model is not completely known [63]. Its primary advantage is its ability to handle uncertainties and disturbances. However, the performance depends on the accuracy of the model and the observer.

In recent years, ADRC has been widely applied across various fields. Reference [64] implements ADRC for a flywheel energy storage system. The results showed superior disturbance rejection and dynamic performance compared with PI. Reference [65] proposes a time-scale voltage control technique based on ADRC. This approach offers improved robustness against system uncertainties and external disturbances. By developing a reduced-order model, the authors simplified the controller design and achieved better performance.

The study in [66] evaluates the performance of various ADRC techniques (linear, nonlinear, higher order, and generalized predictive integral). The study also proposes a modified model-assisted ADRC that uses pole information. Results show that incorporating system dynamics (pole information) as a known disturbance enhances stability compared with other model-assisted techniques.

In summary, ADRC offers distinct advantages in handling model uncertainties and suppressing disturbances and can be combined with other control methods to enhance stability. Its design, however, assumes knowledge of the system's state-space model and order, and its tuning complexity can make implementation challenging, particularly for higher-order systems.

### 3.6. H-Infinity Control

H-infinity ($H_\infty$) control is used to design controllers that handle external disturbances and model uncertainties by minimizing the worst-case system gain, ensuring stability under adverse conditions. The control problem is formulated as a mathematical optimization task, and the controller is synthesized through a repetitive approach [67].

Reference [68] presents an $H_\infty$ control technique for DC MG stabilization. A robust $H_\infty$-based approach is proposed in [69] to improve voltage regulation under uncertainties. The authors uses Lebesque-measurable matrix to determine the upper and lower bounds of the uncertainties. In [70], a decentralized $H_\infty$ loop-shaping design control is proposed for a small, isolated DC MG. The parametric uncertainties are modeled using an upper linear fractional transformation. This technique shows good dynamic performance; however, it has not been benchmarked against other robust controllers. A summary of model-based voltage control strategies is provided in Table A1 in Appendix A.

## 4. Model-Free Techniques

While model-based control strategies can achieve high performance when accurate system dynamics are available, their effectiveness is often limited by modeling complexity, parameter uncertainties, and unmodeled nonlinearities. In many practical DC MG applications, deriving an exact model is frequently difficult or impractical. These challenges have motivated the development of model-free techniques, which regulate system behavior without relying on explicit models. This section reviews representative model-free methods, with a focus on their operating principles, practical applications, and inherent challenges.

### 4.1. Fuzzy Logic Control

Fuzzy Logic Control is a rule-based mathematical concept that extends Boolean logic to a multi-valued framework [71]. It is widely applied in DC MG due to its simple design
and strong robustness [72]. An FLC typically comprises three main blocks: fuzzification, rule base lookup table, and defuzzification [73]. The two most commonly used inference engines are Mamdani and Takagi-Sugeno. The unique difference between these models lies in the consequent parts of their rule base, aggregation, and defuzzification procedures [74].

Several studies have applied FLC for voltage regulation. Reference [75] proposes an FLC that determines the reference current of an ultracapacitor converter. An FLC is proposed in the outer voltage control loop, and a K-type regulator is used in the inner current control loop. To improve voltage stability in a DC/DC converter feeding CPLs, ref. [76] presents an intelligence-based, Type II FLC. The proposed method provides a powerful tool for handling systems with uncertainties. To stabilize the voltage in a full-electric ferry boat, ref. [77] proposes an intelligent single-input interval Type-II fuzzy PI controller (iSIT2-FPI). The proposed technique combines SMC with a model-free iSIT2-FPI to compensate for destructive impedance instability.

The study in [78] proposes a voltage stabilization technique using a Fuzzy-PI dual-mode controller. The PI ensures good steady-state response, and the FLC improves transient response. In [79], the stability of an isolated MG is enhanced by tuning fuzzy membership functions using PSO and Cuckoo search algorithms. Simulation results have shown that energy management using PSO outperformed the Cuckoo search algorithm. Another study in [80] applies an adaptive neuro-fuzzy inference system (ANFIS) to mitigate input voltage and load current perturbations. It is worth noting that ANFIS combines the learning ability of artificial neural networks and the structured intuition of FLC. The conclusion drawn from this survey is that its performance is sensitive to membership function selection, input/output tuning, and fuzzy rule base design, which often lacks rigorous mathematical rules.

## 4.2. Data-Driven Control

Model-free data-driven control (DDC) relies entirely on input/output (I/O) data for training and tuning, eliminating the need for explicit system models. This section surveys recent studies on the application of DDC for voltage regulation in DC MG.

### 4.2.1. Artificial Neural Network

An artificial neural network (ANN) uses data to learn complex relationships between system parameters and voltage fluctuations, enabling adaptation to changing conditions and improved stability [81]. Neural network-based control is widely adopted due to its ability to learn control strategies without an explicit system model [82]. However, its performance depends on training data and proper selection of network structures and activation functions.

Neural network architectures use distinctive principles to determine rules for diverse applications. Common types include feedforward, convolutional, radial basis function, recurrent, generative adversarial, and hybrid neural networks [83]. Backpropagation is popularly used training algorithm that optimize ANN parameters by minimizing the error between the desired and actual outputs [84].

The research community has proposed numerous ANN-based control strategies. For example, ref. [85] proposes a neural network predictive controller (NNPC) for voltage control in a buck converter. The proposed method combines the advantages of MPC with the high estimation power of ANN. A similar approach was employed in [86] to enhance boost converter stability, where MPC is applied for data generation, and a feedforward network is subsequently trained to generate the converter's duty cycle. Moreover, ref. [87] integrates CS algorithm with ANN to regulate voltage in a system with a composite energy storage system (CESS).
To regulate voltage in a low-voltage DC MG, ref. [88] proposes a hybrid Bat search and ANN (HBSANN) approach. Similarly, ref. [89] introduces a supervised deep-learning controller that eliminates the need for sensor data. Another study in [90] presents an ANN-based control approach using approximate dynamic programming. The ANN is trained offline to minimize a cost function over a long-time horizon.

Based on the analysis of ANN-based voltage control, several conclusions can be drawn. The application of ANN improves stability in DC MG even with inaccurate model parameters. However, performance depends on the quantity and quality of the training data. Incomplete or inaccurate training data can lead to biased or unpredictable behavior. Furthermore, as system complexity increases, ANN become more susceptible to overfitting, reducing their ability to adapt to real-world disturbances and leading to inaccurate predictions and suboptimal performance.

### 4.2.2. Local Model Networks

Local model networks (LMNs) are a neural network architecture that combines the advantages of both local and global modeling approaches. They are particularly effective for handling complex, high-dimensional data, which can be challenging for conventional neural networks [91]. The LMN concept involves creating local models (LM) that collectively cover the entire operating range of the controlled process. These LM are then integrated using validity or weighting functions to form a non-linear global model of the plant [92]. Each LM within LMN is responsible for capturing the system dynamics within a specific operating regime [93].

Based on this concept, the study in [94] presents two methods for designing LM structures: the linearization-based approach and the LM-based approach. In the linearization-based method, the LM network is linearized around an initial operating point, and a linear controller is designed accordingly. In contrast, the LM-based approach considers a separate controller for each LM, with the overall control output determined by interpolating the responses of the local controllers according to the current operating point.

To improve the performance of conventional NN, ref. [91] proposes a data-driven strategy based on a LMN for the identification and voltage control of DC/DC converters. The LMN consists of local linear models (LLM), each responsible for modeling the dynamics of the converter within a specific operating regime, as determined by its validity function. The structure of the LMN is determined directly from the input-output measurements using a hierarchical binary tree learning algorithm. The identified LMN, employed in the local linear controller (LLC), is designed based on inverse error dynamics control. The results show that the identified model captures the actual dynamic behavior of the converters.

### 4.2.3. Model-Free Adaptive Control

Model-free adaptive control (MFAC) is a data-driven control approach for a class of unknown nonaffine, nonlinear, discrete-time systems [95]. This approach only relies on system I/O data, eliminating the need for explicit physical models and traditional stability theory. The controller establishes a dynamic linearization data model of the controlled system at each operating point using pseudo partial derivative, pseudo gradient, or pseudo-Jacobian matrix [96]. It is one of the most promising DDC for its computational efficiency and ease of operation [97]. The application of MFAC has been explored in various areas, including power distribution systems, automobiles, robotics, and MG. Notably, MFAC demonstrates strong potential for maintaining voltage stability in a system where the exact model is unavailable.

Recent studies show that MFAC provides strong control performance with high robustness and effective disturbance rejection. For example, ref. [98] presents an MFAC scheme
for active stabilization in DC MG. The control approach based on compact form dynamic linearization shows a satisfactory result. To realize a robust control, ref. [99] presents an intelligent model-free SMC for regulating charging/discharging of ESD. Similarly, ref. [96] proposes a control approach for an ultracapacitor-based three-phase interleaved bidirectional DC/DC converter. The results demonstrate fast transient response and enhanced steady-state performance.

In [100], a model-free composite disturbance rejection control (MFC) is developed to regulate the voltage of a dynamic wireless charging system. A switching-gain-based MFC law is introduced to generate the desired duty cycle without requiring prior model knowledge. An online parameter identification law is then developed to recover the unknown control input gain. To improve the voltage stability, ref. [101] proposes a data-driven model-free disturbance rejection control for a DC/DC converter. In this study, the authors considered a data-driven adaptive extended observer (DAESO) for voltage regulation in the presence of disturbances. DAESO is designed by incorporating a data memory stack and a parameter learning law to estimate the unknown control input gain and the lumped disturbance.

The literature shows that the stability, error convergence, and internal stability of systems controlled by MFAC approaches are theoretically guaranteed through contraction mapping analysis. Design assumptions, such as bounded system states, are typically considered; however, these assumptions may introduce bias into the learning process. Moreover, the performance can be influenced by the quality of the data used for learning.

### 4.2.4. Deep Reinforcement Learning

DRL is an emerging machine learning technique where an agent learns to perform actions through trial-and-error interaction with a controlled system [24]. In the context of DRL, the controlled system is referred to as an environment. It finds applications in DC MG and many other fields for control and optimization. The study in [102] identified three major factors influencing the growing adoption of DRL: data, computing resources, and learning algorithms. A schematic representation of a typical DRL agent is shown in Figure 3.

<figure><img src="image_6.png" /><figcaption>Figure 3. Architecture of DRL controller.</figcaption></figure>

Unlike other classes of machine learning, DRL involves sequential decision-making [103], where an agent learns to navigate a potentially stochastic environment, balancing exploration and exploitation. This balancing is crucial, particularly in a system where the agent must decide between exploiting known rewards and exploring new possibilities
to maximize long-term reward. A comprehensive review of the applications of DRL is discussed in [92].

DRL models are classified into two main types: model-based and model-free DRL. In this context, the model refers to the agent's perception of its environment. Model-based DRL explicitly requires a model of system dynamics to find a near-optimal policy [104]. Here, the agent has a clear understanding of the environment in which it operates. The environment can be either partially observable or fully observable. In contrast, a model-free agent has no prior knowledge of the environment it interacts with. The agent learning process mainly relies upon experience.

The Markov decision process (MDP) provides a mathematical framework that describes environment-agent interaction at each sequence of time steps [105]. MDP is used for modeling decisions in uncertain conditions using a trajectory shown in Figure 4 [106].

<figure><img src="image_7.png" /><figcaption>Figure 4. Typical MDP trajectory.</figcaption></figure>

After formulating voltage control problem as an MDP, it can be trained and solved using various model-free DRL algorithms, either on-policy or off-policy. On-policy agents attempt to enhance the policy they are using to make decisions. In contrast, off-policy agents improve a policy that can be different from the one they use to make decisions. The basic classification of model-free DRL algorithms is shown in Table 1.

DRL agent optimization is either value, policy, or a combination of both policy and value (using actor and critic). In a value-based approach, a value function is initially selected for each state, and a new value function is then estimated using learning data. This process is iterated until the optimal value function is achieved. While the policy-based method directly optimizes the policy function, mapping states to actions without explicitly estimating value functions [107].

Recent studies have investigated the effectiveness of DRL for voltage control in DC MG. Reference [108] proposes an approach using PID and DRL algorithm. The PID serves as the base controller, while a DDPG agent is employed as an adaptive compensator to improve the system's dynamic response. In [109], the application of DQN for voltage control of buck converter feeding CPLs is investigated. To overcome the limited exploration capability of DQN, ref. [110] adopts an improved version based on DDPG. The study in [111] presents a nonlinear-based scheme for voltage stabilization of a boost converter. The authors use a regression-based optimization algorithm to improve system efficiency. To overcome the overestimation of value functions that often occurs in DDPG, ref. [112] proposes a TD3 algorithm that uses two separate critic networks and delayed policy updates. Similarly, ref. [113] presents a TD3-based algorithm for a buck converter feeding CPL. The distinctive feature of this study is its framework, which is designed for direct control of the power switches. Moreover, the study in [114] modified and trained a TD3 agent to improve the dynamic stability in a boost converter.

In [115], a DDPG-PI based on a sliding mode observer is proposed to address the effects of unknown and non-ideal CPLs. The proposed scheme eliminates dependence on an accurate model, as the agent learns to accommodate unmodeled dynamics. In [116], an adaptive disturbance rejection controller employing DDPG and Internet of Things (IoT) is proposed for smart grid systems. The DDPG agent offers online tuning through neural
network learning. To address the challenges of offline training, ref. [117] presents an integral DRL-based voltage control approach for an interleaved DC/DC boost converter that autonomously approximates the controller gains online.

Overall, DRL-based control algorithms demonstrate strong adaptability in a completely unknown environment; however, they require significant computational effort for agent training. Additionally, sparse rewards and excessive exploration may lead to unstable control actions oscillations. A summary of the reviewed model-free strategies is provided in Table A2 in Appendix A.

<table><caption>**Table 1.** Classification of model-free DRL agents.</caption><thead><tr><th>Learning Policy</th><th>Key Features</th><th>Agent</th><th>Type</th><th>Action</th></tr></thead><tbody><tr><td rowspan="5">On-policy</td><td rowspan="5">Learning is based on the current policy. Exploration is limited to the current policy. Suitable when the environment is relatively stable and the agent can explore safely.</td><td>State-action-reward-state-action (SARSA)</td><td>value-based</td><td>discrete</td></tr><tr><td>Policy gradient (PG)</td><td>policy-based</td><td>discrete or continuous</td></tr><tr><td>Actor critic (AC)</td><td>actor-critic</td><td>discrete or continuous</td></tr><tr><td>Trust region policy optimization (TRPO)</td><td>actor-critic</td><td>discrete or continuous</td></tr><tr><td>Proximal Policy optimization (PPO)</td><td>actor-critic</td><td>discrete or continuous</td></tr><tr><td rowspan="10">Off-policy</td><td rowspan="10">Learn from different policies. Explore more broadly through behavior policy. Suitable for complex environment</td><td>Q-learning</td><td>value-based</td><td>discrete</td></tr><tr><td>Deep Q-network (DQN)</td><td>value-based</td><td>discrete</td></tr><tr><td>Double DQN (DDQN)</td><td>value-based</td><td>discrete</td></tr><tr><td>Deep deterministic policy gradient (DDPG)</td><td>actor-critic</td><td>continuous</td></tr><tr><td>Twin-delay deep deterministic policy gradient (TD3)</td><td>actor-critic</td><td>continuous</td></tr><tr><td>Soft actor-critic (SAC)</td><td>actor-critic</td><td>continuous</td></tr><tr><td>Asynchronous advantage actor-critic (A3C)</td><td>actor-critic</td><td>discrete or continuous</td></tr></tbody></table>

# 5. Hybrid Control Techniques

Hybrid control techniques combine aspects of both model-based and model-free control schemes. This section examines the recent advances in hybrid control techniques for voltage control in DC MG.

## 5.1. Metaheuristic Optimization Algorithms

Metaheuristic optimization algorithms (MOA) are nature-inspired algorithms designed to find optimal solutions to complex optimization problems. The main advantages of MOAs are their versatility and flexibility [118], as they can be easily modified to meet the specific requirements of a given problem. These algorithms are widely used to optimize the parameters of both classical and advanced control techniques. For example, ref. [119] proposes an intelligent binary and real-code genetic algorithm tuned PID controller for regulating fuel cell voltage. The binary genetic algorithm (GA) is a well-established optimization method for identifying near-optimal solutions. It consists of binary-encoded chromosomes that undergo roulette wheel selection, multipoint crossover, and uniform mutation to produce offspring [119]. Each chromosome in the offspring population is evaluated using a defined objective function.

To improve stability in a buck-boost converter, ref. [120] proposes a PSO-tuned PI controller. The PSO algorithm, inspired by the social behavior of bird flocks searching for food, iteratively adjusts the positions of particles (potential solutions) based on their individual best-known positions and the global best-known position of the swarm. The study
in [121] introduces a quasi-oppositional Archimedes optimization algorithm (QOAOA) for tuning a fractional-order PID (FOPID) controller to regulate the output voltage of a cascaded DC/DC boost converter. To enhance voltage stability in PV-powered MGs, ref. [122] combines direct SMC with a hybrid salp swarm algorithm and particle swarm optimization (SSA-PSO). Moreover, ref. [123] proposes a control strategy for a buck converter by optimizing PID parameters using an improved sine-cosine algorithm.

Overall, MOAs offer a powerful framework for controller parameter optimization. They have demonstrated significant improvements in performance across both linear and nonlinear control applications. Nonetheless, their effectiveness may be constrained by sensitivity to local minima and the need for careful parameter tuning.

## 5.2. *Physics-Informed Neural Networks*

Physics-Informed Neural Networks (PINN) have recently gained attention as a promising approach for accelerating computations in nonlinear dynamical systems. PINN integrate physics-informed laws (model-based) into neural network methodologies (model-free) to learn complex physical behaviors [124]. This approach ensures that the learned solutions adhere to the underlying physics of the systems. Unlike the conventional neural network paradigm, PINN uses scientific knowledge or physics laws to guide the optimization, design, and implementation of deep neural networks [125]. The four core paradigms of PINN, as outlined in [126], include physics-informed loss functions, physics-informed initialization, physics-informed design of architecture, and hybrid physics-deep learning models.

In recent years, the adoption of PINN in power systems has expanded to applications such as state/parameter estimation, optimal power flow, and dynamics analysis. Reference [127] presents a PINN approach for estimating the state of health (SOH) of lithium-ion batteries. The authors model the attributes influencing battery degradation through empirical degradation and state-space equations and use neural networks to capture system dynamics. The proposed network comprises two components: a solution NN $F(.)$ that builds feature-to-SOH mapping and a NN $G(.)$ that models battery degradation dynamics.

To enhance adaptability and minimize reliance on extensive quality datasets and high computational power, ref. [128] proposes a PINN-based control framework for improving the stability of a buck converter. The framework employs a parallel architecture integrating data-driven, physics, control, and loss function modules. The proposed method performed well in identifying and rejecting unknown load disturbances and internal circuit parameters. Notably, PINN can achieve high accuracy and strong generalization compared to traditional neural networks due to their integration of physical laws. However, they often require high-quality supervised data for training and validation, which can limit their applications. Table A3 in Appendix A summarises recent research on hybrid voltage control techniques.

The voltage control techniques reviewed in Sections 3–5 exhibit distinct characteristics in terms of modeling requirements and performance. A summarized comparison is provided in Table 2. It is important to note that hybrid methods are not included in this comparative summary to maintain a clear distinction between model-based and model-free control techniques. These methods typically combine model knowledge with optimization-based (e.g., MOA) or data-driven (e.g., PINN) adaptation and often function as tuning or augmentation layers rather than standalone controllers. As a result, their performance largely depends on the main controller (model-based or model-free) and may inherit the strengths and limitations of the underlying control strategy.
<table><caption>**Table 2.** Comparative summary of voltage control techniques.</caption><thead><tr><th>Category</th><th>Strengths</th><th>Limitations</th></tr></thead><tbody><tr><td>Model-based</td><td>High accuracy and fast response when the system model is known. Stability can be guaranteed analytically. Facilitates optimal and predictive control.</td><td>Requires system model and parameters. Limited adaptability to uncertainties. Requires re-modeling in the event of system changes.</td></tr><tr><td>Model-free</td><td>Does not require an explicit system model. High adaptability and flexibility. Robust to parameter variations and uncertainties.</td><td>Lacks analytical stability guarantees. High computational cost during training. Requires high-quality data.</td></tr></tbody></table>

## 6. A Case Study of DC/DC Converter with LC Filter

The preceding sections critically reviewed voltage control strategies, highlighting their respective strengths and limitations. While this survey provides a comprehensive view of state-of-the-art, it is equally important to evaluate their effectiveness in a representative and challenging scenario. To this end, this study presents a case study of a DC/DC buck converter cascaded with an input LC. This configuration is particularly relevant, as it captures one of the critical instability issues in DC MG: the adverse interaction between the LC filters and converters, which substantially reduces system damping and compromises voltage stability.

In practical DC MG, as shown in Figure 1, an input LC filter is commonly included at the DC/DC converter stage to suppress harmonics injected into the DC bus and to mitigate instantaneous impulse voltages and electromagnetic interference [129]. However, its inclusion can lower the damping ratio of the system and exacerbate instability risks through resonance with the converter dynamics [22]. When this effect is combined with the negative incremental impedance of CPL, the resulting system stability is severely degraded.

To address such challenges, researchers have proposed a wide range of strategies, including linear feedback control [130], passivity-based control [131], and feed-forward virtual impedance combined with proportional-resonant control [129]. More recently, however, attention has shifted toward model-free approaches, whose flexibility and adaptability to disturbances and parameter variations make them attractive for DC MGs without the need for continuous model updates [95].

The present case study is designed with two objectives: (i) to benchmark conventional and advanced controllers within a unified framework, and (ii) to demonstrate the potential of DRL strategies enhanced through BO for achieving robust voltage regulation under uncertain and nonlinear operating conditions.

It should be noted that hybrid approaches are not included in this comparative study. This decision is motivated by two considerations: first, hybrid methods such as MOA and PINN generally serve as tuning or augmentation layers for existing controllers rather than as standalone real-time control schemes; second, incorporating them would require extensive data or embedding additional optimization modules in the control loop, which is beyond the scope of this benchmark investigation. Instead, this work focuses on a fair comparison between widely adopted conventional controllers and the proposed DRL-based strategies, while recognizing hybrid approaches as a promising direction for future research.
## 6.1. System Model

To formalize this benchmark scenario, Figure 5 illustrates the studied system. It consists of a buck converter feeding a CPL through an input LC filter connected to a common DC bus. The DC bus voltage and the converter’s output voltage are defined as $E$ and $v_c$, respectively. Similarly, $L_f$, $L$, $C_f$, and $C$ represent the inductances and capacitances of the LC filter and the converter. Additionally, $r_f$ and $r$ are the series resistances of $L_f$ and $L$, respectively, while $S$ denotes a controllable switch. These parameters are selected such that the converter operates in continuous conduction mode.

<figure><img src="image_8.png" /><figcaption>Figure 5. Schematic of buck converter with LC filter.</figcaption></figure>

The system’s model is represented by the following differential equations:

$$L_f \frac{di_f(t)}{dt} = E - i_f(t)r_f - v_f(t) \quad (1)$$

$$C_f \frac{dv_f(t)}{dt} = i_f(t) - u(t)i_L(t) \quad (2)$$

$$L \frac{di_L(t)}{dt} = u(t)v_f(t) - i_L(t)r_L - v_c(t) \quad (3)$$

$$C \frac{dv_c(t)}{dt} = i_L(t) - i_{CPL}(t) \quad (4)$$

The control signal $u(t)$ takes the value of ‘1’ for the switch ‘ON’ position and ‘0’ for the ‘OFF’ position.

The complete model is a four-state model with a state vector $x(t) = [i_f, v_f, i_L, v_c]^T$. The state-space average model is given by

$$\begin{aligned} \dot{x}(t) &= Ax(t) + Bu(t) \\ y(t) &= Cx(t) \end{aligned} \quad (5)$$

where

$$A = \begin{bmatrix} -\frac{r_f}{L_f} & -\frac{1}{L_f} & 0 & 0 \\ \frac{1}{C_f} & 0 & -\frac{S}{C_f} & 0 \\ 0 & \frac{S}{L} & -\frac{r}{L} & -\frac{1}{L} \\ 0 & 0 & \frac{1}{C} & -\frac{1}{RC} \end{bmatrix}, \quad B = \begin{bmatrix} 1 \\ 0 \\ 0 \\ 0 \end{bmatrix} \\ C = \begin{bmatrix} 0 & 0 & 0 & 1 \end{bmatrix}$$

## 6.2. DRL Algorithms

Since the objective is to ensure that the output voltage remains fully regulated even under supply and load perturbations, this study proposes a model-free control strategy
based on DRL and BO. To provide a balanced evaluation, the article adopts two representative DRL agents: PPO, an on-policy algorithm known for its stable policy updates and robustness across a range of control problems, and TD3, an off-policy algorithm that addresses overestimation bias often found in DDPG.

### 6.2.1. PPO

PPO is an on-policy DRL algorithm that uses a clipped surrogate function to prevent large policy updates and ensure stable learning [132]. It directly optimizes the policy through interactions with the environment, aiming to maximize the expected cumulative long-term reward. By constraining policy updates from deviating significantly from the current policy, PPO establishes a robust framework for optimizing control policies in a complex dynamic environment [133]. It can be applied in an environment with either continuous or discrete control action [134]. To estimate the policy and value function, PPO is particularly implemented using actor-critic function approximators [135]. The actor estimates the conditional probability of taking an action based on the current state, while the critic evaluates the chosen action and returns the corresponding expectation of the discounted long-term reward.

Policy optimization algorithms maximize the expected return $G(\theta)$ of a policy $\pi_{\theta}(a | s)$, parametrized by $\theta$, over MDP trajectories:

$$G(\theta) = \mathbb{E}_{\tau \sim \pi_{\theta}} \left[ \sum_{t=0}^{\infty} \gamma^t r_t \right] \qquad (6)$$

Applying policy gradient theorem, the gradient of (6) is given by:

$$\nabla_{\theta} G(\theta) = \mathbb{E}_{s_t, a_t \sim \pi_{\theta}} [\nabla_{\theta} \log \pi_{\theta}(a_t | s_t) A_t] \qquad (7)$$

where $A_t$ represents an advantage function, defined as the discounted sum of temporal difference error.

In general, PPO optimizes a clipped surrogate objective function using stochastic gradient descent [135]:

$$L^{\text{CLIP}}(\theta) = \hat{\mathbb{E}}_{t} [\min(r_{t}(\theta)\hat{A}_{t}, \text{clip}(r_{t}(\theta), 1-\epsilon, 1+\epsilon)\hat{A}_{t})] \qquad (8)$$

where $r_t(\theta)$ is a probability ratio between the new policy $\pi_{\theta}$ and old policy $\pi_{\theta_{old}}$, clip(·) restricts $r_t(\theta)$ to the range [1 - $\epsilon$, 1 + $\epsilon$], and $\epsilon$ is a hyperparameter controlling the allowed policy change.

To encourage exploration and prevent premature convergence of the learned policy, PPO often adds an entropy loss term:

$$L^{\text{ENTROPY}}(\theta) = \mathbb{E}[H(\pi_{\theta}(\cdot|s_t))] \qquad (9)$$

Therefore, the final loss is given in (10):

$$L_t(\theta) = \mathbb{E}_t \left[ L_t^{\text{CLIP}}(\theta) - c_1 L_t^{\text{VF}}(\theta) + c_2 L_t^{\text{ENTROPY}}(\theta) \right] \qquad (10)$$

where $L_t^{VF}$ is a value function loss ($V(s_t) - V_t^{\text{target}}$)$^2$, $c_1$ and $c_2$ are weighting coefficients.

### 6.2.2. TD3

TD3 is an off-policy actor-critic DRL agent that employs deterministic policies for continuous tasks [136]. It addresses the Q-value overestimation issue commonly observed in DDPG by introducing clipped double Q-learning, delayed policy updates, and target
policy smoothing [137]. To estimate the policy and value functions, TD3 uses an actor network with a parametrized policy. The actor takes observation as input and outputs a control action that maximizes long-term reward. TD3 uses two identical critic networks to estimate the value of the state-action pairs.

In general, TD3 trains two critic networks $Q_{\phi_1}(s, a)$, $Q_{\phi_2}(s, a)$, and the minimum of the two values is used by the target. The target for the Q-value is obtained using (11) [137]:

$$y = r + \gamma \min_{k=1,2} Q_{\phi_k}(s', \tilde{a}') \quad (11)$$

where $r$ is the immediate reward, $\gamma$ is the discount factor, $Q_{\phi_k}$ is the target Q-function, and $\tilde{a}' = \pi_{\theta'}(s') + \epsilon$ represents the target action with added noise. It is important to note that, if $s'$ is a terminal state, the target Q-value is set to $r$ only.

$$\epsilon \sim \text{clip}(\mathcal{N}(0, \sigma), -c, c) \quad (12)$$

where $\sigma$ is the noise standard deviation, and $c$ is the clipping range.

After every training step, the parameters of each critic are updated by minimizing the loss $L_k$ across all sampled experiences:

$$L_k = \frac{1}{2M} \sum_{i=1}^{M} (y_i - Q_{\phi_k}(s_i, a_i))^2 \quad (13)$$

Similarly, the actor parameters are updated using the following sampled policy gradient:

$$\nabla_{\theta} J = \frac{1}{M} \sum_{i=1}^{M} G_{\pi_i} G_{a_i} \quad (14)$$

where $G_{\pi_i}$ is the gradient of the actor output with respect to the actor parameters, $G_{a_i}$ is the gradient of the minimum critic output with respect to the action estimated by the actor network.

TD3 employs a target actor ($\theta'$) and two target critics ($\phi'_k$). The target networks are updated periodically to avoid too fast convergence using a soft update rule as follows [112]:

$$\begin{aligned} \theta' &= \tau\theta + (1 - \tau)\theta' \\ \phi'_k &= \tau\phi_k + (1 - \tau)\phi'_k \end{aligned} \quad (15)$$

where $\tau \ll 1$ is the smoothing factor.

## 6.3. DRL Controller Design

The complete block diagram illustrating the interaction between the proposed approach and the system is shown in Figure 6. The design of the state space, action space, reward function, and hyperparameters optimization is described in the following steps:

### 6.3.1. State Space

The state space contains sufficient information from the environment for the agent to make control decisions. For the objective of voltage regulation, it includes the voltage tracking error, the integral of the error, the delayed output voltage, and the reference voltage. The random variation of the reference voltage during training necessitates its inclusion to facilitate the learning of dynamic control policies. Thus, the state space is defined as

$$s_t = \left[ e_v(t), \int e_v(t) dt, v_o(t-1), V_{\text{ref}} \right] \quad (16)$$
where $e_v(t) = V_{ref} - v_o$.

<figure><img src="image_9.png" /><figcaption>Figure 6. Block diagram of the BO-DRL framework.</figcaption></figure>

### 6.3.2. Action Space

The action space represents the set of all valid control actions that the agent can execute within the environment. In this study, the converter’s duty cycle serves as the control signal generated by the agent to regulate the voltage. Therefore, the action space is defined as

$$a_t = d \in [0.0, 0.99] \qquad (17)$$

### 6.3.3. Reward Function

The reward function is structured to encourage the agent for favorable actions and penalize it for poor decisions. The reward function used in this study is based on tracking error and is defined in (18).

$$R_t = r_1 + r_2 + r_3 \qquad (18)$$

$$r_1 = -(e_v)^2, \quad r_2 = \begin{cases} 1 & \text{if } |e_v| \le 0.01 \\ 0.1 & \text{if } 0.1 < |e_v| < 0.1 \\ 0 & \text{otherwise} \end{cases} \\ r_3 = \begin{cases} -100 & \text{if } v_o \ge 1.3V_{ref} \text{ or } v_o \le 0.7V_{ref} \\ 0 & \text{otherwise} \end{cases}$$

where $r_1$ denotes a parametric reward proportional to the negative squared voltage error, $r_2$ is a non-parametric discrete reward that incentivizes favorable actions, and $r_3$ penalizes violations of voltage limits. The scaling gains of the reward function are tuned using a trial-and-error approach.
### 6.3.4. Hyperparameter Optimization

DRL algorithms are making significant strides in the field of control engineering; however, their performance is challenged by hyperparameter sensitivity. These hyperparameters, such as discount factor, learning rate, minibatch size, etc., affect the learning process and the learned policy [138]. A popular tuning method involves trial-and-error or manual search. In this method, different hyperparameter values are used, and their impact on the performance is observed. It is noteworthy that manual search requires intuition and sufficient expertise to identify good hyperparameter sets. Furthermore, as the number and range of the hyperparameters increase, optimization becomes increasingly tedious, making it unlikely to find the best possible set.

To overcome the limitations of manual tuning, automatic search methods, such as grid search and random search, have been proposed. Grid search performs an exhaustive search of the entire space [139]. It trains an agent by trying all possible combinations of the hyperparameters. This method is straightforward to implement and widely adopted; however, it suffers from the curse of dimensionality and can be time-consuming, especially as the number and range of hyperparameters increase [139]. To address these drawbacks, random search selects hyperparameters randomly from a defined distributed search space [140]. While this approach reduces training time, it may not fully cover the parameter space.

To reduce computational effort and improve control performance, researchers employ BO to optimize the hyperparameters of DRL agents. BO attempts to minimize a scalar objective function, $f(\lambda)$ for all $\lambda \in \Lambda$ [141]. It iteratively builds Gaussian process model of the function based on the observed data and uses the model to predict performance. Hyperparameter optimization with BO relies on three key components: a Gaussian model of the objective function, a Bayesian update that modifies the model with each new evaluation of $f(\lambda)$, and an acquisition function [142].

In this study, BO is employed to optimize the hyperparameters by minimizing the cost function defined in (19).

$$f(\lambda) = -\frac{1}{N} \sum_{i=1}^{N} \sum_{t=1}^{T_i} r(\lambda)_{i,t} \qquad (19)$$

where $r(\lambda)_{i,t}$ is the reward received with candidate hyperparameters $\lambda$, at time step $t$ in episode $i$, $N$ is the number of evaluation episodes, and $T_i$ is the maximum number of time steps in the $i$-th evaluation episode.

The process involves computing four evaluations consecutively after every 20 training episodes. The mean value of the evaluation rewards used in the objective function provides insight into the agent's performance in the environment, thereby facilitating more effective hyperparameter tuning.

Following the guidelines in [143], the hyperparameters most critical to learning performance, exploration, and stability were selected for the optimization. These hyperparameters and their respective bounds are provided in Table 3 and the complete algorithm is presented in pseudocode in Algorithm 1.
<table><caption>**Table 3.** Agents' hyperparameter bounds.</caption><thead><tr><th>Hyperparameter</th><th>PPO</th><th>TD3</th></tr></thead><tbody><tr><td>Minibatch size (m)</td><td>[50, 400]</td><td>[50, 400]</td></tr><tr><td>Discount factor (γ)</td><td>[0.9, 1]</td><td>[0.9, 1]</td></tr><tr><td>Actor learning rate (α<sup>a</sup>)</td><td>[1 × 10<sup>-6</sup>, 1 × 10<sup>-2</sup>]</td><td>[1 × 10<sup>-6</sup>, 1 × 10<sup>-2</sup>]</td></tr><tr><td>Critic learning rate (α<sup>c</sup>)</td><td>[1 × 10<sup>-6</sup>, 1 × 10<sup>-2</sup>]</td><td>[1 × 10<sup>-6</sup>, 1 × 10<sup>-2</sup>]</td></tr><tr><td>Number of epochs (k)</td><td>[1, 10]</td><td>[1, 10]</td></tr><tr><td>Target smooth model standard deviation (σ<sub>τ</sub>)</td><td>—</td><td>[0.1, 0.5]</td></tr><tr><td>Entropy loss function (w)</td><td>[0.01, 0.1]</td><td>—</td></tr></tbody></table>

**Algorithm 1 Bayesian optimization**

**Input:** Objective function $f(\lambda)$, maximum number of iterations $n_{max} = 30$, maximum number of training episodes $p_{max} = 300$, hyperparameter sets $\lambda_{TD3} = [m, \gamma, \alpha^a, \alpha^c, k, \tau_{sd}]$, $\lambda_{PPO} = [m, \gamma, \alpha^a, \alpha^c, k, \omega]$, bounds $\Lambda$.

**Initialize:** Train the agent using randomly sampled hyperparameters $\lambda_0 \in \Lambda$. Collect initial dataset $D_0 = \{\lambda_0, f(\lambda_0)\}$.

**for** $n = 1$ to $n_{max}$ **do**

Fit Gaussian model with observed data $D_{n-1}$.

Select new hyperparameter set $\lambda_n \in \Lambda$ by optimizing acquisition function.

**for** $p = 1$ to $p_{max}$ **do**

Train the agent and store mean evaluation rewards after every $N$ episodes; $r_i$.

**end for**

Evaluate $f(\lambda_n) = -\frac{1}{N} \sum_{i=1}^{N} r_i$.

Augment dataset $D_n \leftarrow D_{n-1} \cup \{\lambda_n, f(\lambda_n)\}$.

**end for**

**Return:** Best solution $\lambda^*$ in observed dataset $D$.

## 6.4. Simulation Results

Simulations in MATLAB/Simulink (R2024a) are carried out to verify the performance of the proposed methods. A comparative evaluation with model-based (PI, SMC) and model-free FLC controllers is also presented.

### 6.4.1. Training and Optimization Results

The control agents' hyperparameters were optimized through multiple training experiments to minimize the objective function defined in (19). During each experiment, the agent interacted with the system model to track a series of randomly generated reference voltages. Thirty (30) evaluations of BO were performed to identify the optimal hyperparameter combination within the defined search space (Table 3). Each evaluation involved 300 training episodes. It is important to note that high computational power was required only during the training and optimization phases. Once trained, the agent maps system states to corresponding control actions according to the learned policy with negligible computational demand.

The results of hyperparameter optimization are presented in Figure 7. Figure 7a,b illustrate the objective function minimization over 30 iterations for the PPO and TD3 agents, respectively. The results show that the objective function attained its global minimum at the 4th evaluation for the TD3 agent, whereas the PPO agent reached its minimum at the 9th iteration. This observation highlights the faster convergence of TD3 compared to PPO. Moreover, the trajectories demonstrate that the estimated minimum objective closely corresponds to the observed minimum, further validating the effectiveness of the BO process.
<figure><img src="image_10.png" /><figcaption>Figure 7. Bayesian optimization results: (a) evaluation progress for PPO (b) evaluation progress for TD3.</figcaption></figure>

Figure 8 illustrates the trend of the episodic and average rewards for the two agents during training. The results were obtained over 1000 training episodes using two hyperparameter configurations: the optimized set obtained through BO and a manually tuned baseline. Figure 8a,b show that the agents trained with optimized hyperparameters converge faster and achieve higher rewards compared to the baseline agents.

<figure><img src="image_11.png" /><figcaption>Figure 8. Agents learning curves: (a) Episodic; (b) Average.</figcaption></figure>

### 6.4.2. Comparative Evaluation Under Dynamic Conditions

To assess the performance of the proposed methods, simulations were performed under different dynamic operating conditions. Three scenarios were considered: (i) varying reference voltage, (ii) supply disturbances, and (iii) load disturbances.

#### Scenario 1—Varying Reference Voltage

In this scenario, the reference voltage was varied as shown in Figure 9a. Based on the output voltage waveforms in Figure 9b, a detailed transient performance analysis was conducted and summarized in Table 4. It can be seen that all the control methods maintained global stability, with the proposed BO-TD3 demonstrating the most balanced response, achieving a fast rise time of 0.0016 s and the lowest settling time of 0.0032 s. Although the BO-PPO achieved the fastest rise time (0.0012 s), it exhibited a relatively high overshoot compared to BO-TD3. It can also be seen that the SMC achieved the lowest
overshoot, but exhibited slower response compared to the proposed methods. Furthermore, steady-state performance analysis in terms of root mean square error (RMSE), mean of absolute error (MAE), mean absolute percentage error (MAPE), and integral absolute error (IAE) is presented in Table 5. This analysis clearly indicates that the BO-TD3 consistently achieves the lowest steady-state error.

<figure><img src="image_12.png" /><figcaption>**Figure 9.** Simulation results with varying reference voltage (a) reference voltage waveform (b) output voltage waveforms.</figcaption></figure>

<table><caption>**Table 4.** Transient performance metrics.</caption><thead><tr><th>Metrics</th><th>PI</th><th>FLC</th><th>SMC</th><th>BO-PPO</th><th>BO-TD3</th></tr></thead><tbody><tr><td>Rise time (s)</td><td>0.0016</td><td>0.0021</td><td>0.0090</td><td>0.0012</td><td>0.0016</td></tr><tr><td>Settling time (s)</td><td>0.0070</td><td>0.1398</td><td>0.0161</td><td>0.0033</td><td>0.0032</td></tr><tr><td>Overshoot (%)</td><td>11.27</td><td>5.41</td><td>0.03</td><td>7.42</td><td>5.95</td></tr></tbody></table>

<table><caption>**Table 5.** Comparison of control methods based on error metrics.</caption><thead><tr><th>Metrics</th><th>PI</th><th>FLC</th><th>SMC</th><th>BO-PPO</th><th>BO-TD3</th></tr></thead><tbody><tr><td>RMSE</td><td>0.0826</td><td>0.3462</td><td>0.1754</td><td>0.2798</td><td>0.0780</td></tr><tr><td>MAE</td><td>0.0706</td><td>0.2907</td><td>0.0889</td><td>0.2707</td><td>0.0650</td></tr><tr><td>MAPE</td><td>0.147</td><td>0.6057</td><td>0.1853</td><td>0.5639</td><td>0.1355</td></tr><tr><td>IAE</td><td>0.0095</td><td>0.0392</td><td>0.0120</td><td>0.0365</td><td>0.0088</td></tr></tbody></table>

## Scenario 2—Supply Disturbance

Figure 10 represents a scenario with perturbations in supply voltage as illustrated in Figure 10a. The corresponding results, shown in Figure 10b, compare the responses of the proposed and benchmark methods. The results demonstrate that all the methods achieved global stability despite disturbances in the supply. From the analysis presented in Table 6, it can be concluded that the BO-TD3 achieved the lowest steady state error metrics.

<table><caption>**Table 6.** Comparison of control methods based on error metrics.</caption><thead><tr><th>Metrics</th><th>PI</th><th>FLC</th><th>SMC</th><th>BO-PPO</th><th>BO-TD3</th></tr></thead><tbody><tr><td>RMSE</td><td>0.8363</td><td>0.9832</td><td>1.2592</td><td>0.8676</td><td>0.7775</td></tr><tr><td>MAE</td><td>0.1039</td><td>0.3408</td><td>0.1038</td><td>0.4680</td><td>0.0750</td></tr><tr><td>MAPE</td><td>0.2165</td><td>0.7100</td><td>0.2163</td><td>0.9749</td><td>0.1562</td></tr><tr><td>IAE</td><td>0.3117</td><td>1.0223</td><td>0.3115</td><td>1.4039</td><td>0.2248</td></tr></tbody></table>
<figure><img src="image_13.png" /><figcaption>**Figure 10.** Simulation results with varying input voltage (a) filter voltage waveform (b) output voltage waveforms.</figcaption></figure>

### Scenario 3–Load Disturbance

Figure 11 shows the system response when the reference voltage is set to 48 V and the load power varies. Based on the output voltage regulation shown in Figure 11b, a detailed performance analysis is summarized in Table 7. The results indicate that, among the five control methods, the BO-TD3 achieves the lowest error metrics.

<figure><img src="image_14.png" /><figcaption>**Figure 11.** Simulation results with varying load (a) load power waveform (b) output voltage waveforms.</figcaption></figure>

<table><caption>**Table 7.** Comparison of control methods based on error metrics.</caption><thead><tr><th>Metrics</th><th>PI</th><th>FLC</th><th>SMC</th><th>BO-PPO</th><th>BO-TD3</th></tr></thead><tbody><tr><td>RMSE</td><td>0.8437</td><td>0.9671</td><td>1.2679</td><td>0.7627</td><td>0.7942</td></tr><tr><td>MAE</td><td>0.1013</td><td>0.2980</td><td>0.1249</td><td>0.2356</td><td>0.0620</td></tr><tr><td>MAPE</td><td>0.2111</td><td>0.6208</td><td>0.2603</td><td>0.4909</td><td>0.1292</td></tr><tr><td>IAE</td><td>0.3040</td><td>0.8939</td><td>0.3748</td><td>0.7069</td><td>0.1860</td></tr></tbody></table>

## 7. Challenges and Future Works

Although the existing literature has focused on developing various advanced voltage control techniques, several challenges still need to be further considered to improve adaptability and robustness. On the one hand, system complexity increases with the integration of more components and loads, along with the potential interaction between DC MG, which
amplifies uncertainties and external disturbances. On the other hand, the degradation of ESD needs to be considered. Additionally, validating and implementing the proposed approaches in real-world scenarios, such as using realistic test benches, remain crucial. Therefore, future research directions are summarized below:

* Traditional control approaches often struggle under system disturbances, such as the integration of new DER/ESD or system reconfiguration. These events frequently alter DC MG dynamics. Advanced control strategies like DRL can be trained to continuously adapt their control actions, maintaining stability under high perturbations.

* Data-driven control methods, while offering flexibility and reduced model dependency, often lack interpretability due to their black-box nature. Hybrid control strategies that combine data-driven and physics-based models—such as Model-based reinforcement learning (MBRL) and Physics-informed reinforcement learning (PIRL) should be further explored to balance flexibility, interpretability, and explainability while enhancing voltage stability under uncertainty.

* As MG continue to increase in scale and complexity (multiple DER, diverse loads and interconnection with other MG), the need for more adaptable and flexible control schemes becomes apparent. It is recommended to investigate and develop Multi-agent deep reinforcement learning (MADRL) frameworks. This will enable scalable and coordinated control across large-scale MG systems, allowing for robust autonomous decision-making and enhanced operational resilience.

* To ensure the practical applicability and reliability of the proposed control algorithms, future research should incorporate real-time experimental validation. This could be achieved by establishing Hardware-in-the-loop (HIL) test environments, for initial system integration and controller testing, and Power hardware-in-the-loop (PHIL) to assess performance with actual power-level components.

## 8. Conclusions

This study conducted a comprehensive review of voltage control techniques in DC MG. The reviewed methods were categorized based on their dependence on explicit models for controller design, resulting in a structured analysis of model-based, model-free, and hybrid strategies. Each category was critically examined, highlighting its respective strengths and limitations.

Based on the survey findings, DRL-based control frameworks using the BO-TD3 and BO-PPO algorithms were proposed for voltage regulation in a buck converter with an input LC filter. Three simulation scenarios were evaluated: reference voltage variation, input voltage disturbance, and load fluctuation. The BO-TD3 achieved the lowest error metrics, with a reduction of up to 5.6% in root mean square error (RMSE), and 7.8% in mean absolute percentage error (MAPE) compared to benchmark methods. These findings highlight the potential of DRL-based methods to enhance the voltage stability of DC MGs, with the main limitation being their high computational cost during training.

Future research should focus on extending the control methods to complex and large-scale DC MG and ensure their practical applicability and reliability through real-time experimental validation.

**Author Contributions:** Conceptualization, S.M., H.O., A.H., M.H. and H.G.; methodology, S.M., H.O., A.H., M.H. and H.G.; software, S.M.; validation, S.M., H.O., A.H., M.H. and H.G.; formal analysis, S.M.; investigation, S.M.; writing—original draft preparation, S.M.; writing—review and editing, H.O., A.H., M.H. and H.G.; visualization, S.M., H.O. and A.H.; supervision, H.O., A.H., M.H. and H.G. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.
**Data Availability Statement:** The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

## Abbreviations

The following abbreviations are used in this manuscript:

<table><tr><td>ADRC</td><td>Active Disturbance Rejection Control</td></tr><tr><td>ANFIS</td><td>Adaptive Neuro Fuzzy Inference System</td></tr><tr><td>ANN</td><td>Artificial Neural Network</td></tr><tr><td>BO</td><td>Bayesian Optimization</td></tr><tr><td>CPL</td><td>Constant Power Load</td></tr><tr><td>DDC</td><td>Data Driven Control</td></tr><tr><td>DDPG</td><td>Deep Deterministic Policy Gradient</td></tr><tr><td>DER</td><td>Distributed Energy Resources</td></tr><tr><td>DQN</td><td>Deep Q Network</td></tr><tr><td>DRL</td><td>Deep Reinforcement Learning</td></tr><tr><td>ESD</td><td>Energy Storage Devices</td></tr><tr><td>HESS</td><td>Hybrid Energy Storage Systems</td></tr><tr><td>HIL</td><td>Hardware-in-the-loop</td></tr><tr><td>IoT</td><td>Internet of Things</td></tr><tr><td>LLC</td><td>Local Linear Controller</td></tr><tr><td>LLM</td><td>Local Linear Model</td></tr><tr><td>MADRL</td><td>Multi Agent Deep Reinforcement Learning</td></tr><tr><td>MDP</td><td>Markov Decision Process</td></tr><tr><td>MFAC</td><td>Model Free Adaptive Control</td></tr><tr><td>MOA</td><td>Metaheuristic Optimization Algorithm</td></tr><tr><td>MPC</td><td>Model Predictive Control</td></tr><tr><td>PBC</td><td>Passivity Based Control</td></tr><tr><td>PEMFC</td><td>Proton Exchange Membrane Fuel Cell</td></tr><tr><td>PHIL</td><td>Power Hardware-in-the-loop</td></tr><tr><td>PI</td><td>Proportional Integral controller</td></tr><tr><td>PIRL</td><td>Physics Informed Reinforcement Learning</td></tr><tr><td>PPL</td><td>Pulse Power Load</td></tr><tr><td>PPO</td><td>Proximal Policy Optimization</td></tr><tr><td>PSO</td><td>Particle Swarm Optimization</td></tr><tr><td>RES</td><td>Renewable Energy Sources</td></tr><tr><td>SMC</td><td>Sliding Mode Control</td></tr><tr><td>SOC</td><td>State of Charge</td></tr><tr><td>TRPO</td><td>Trust Region Policy Optimization</td></tr></table>

## Appendix A. Additional Tables

<table><caption>Table A1. Summarized review of model-based voltage control techniques.</caption><thead><tr><th>Control Method</th><th>Reference</th><th>Year</th><th>Proposed Method</th><th>Main Contribution</th><th>Limitation</th></tr></thead><tbody><tr><td rowspan="4">SMC</td><td>[33]</td><td>2023</td><td>Adaptive SMC</td><td>Improving voltage stability in a buck converter feeding CPLs</td><td>Complexity in design</td></tr><tr><td>[34]</td><td>2023</td><td>HM-GFTSMC</td><td>Improving voltage stability in a buck converter</td><td>Proving stability can be challenging in a complex MG scenario.</td></tr><tr><td>[35]</td><td>2024</td><td>HOSMC-PID</td><td>Improving large signal stability in a DC MG</td><td>Proving stability can be challenging in a complex MG scenario.</td></tr><tr><td>[40]</td><td>2022</td><td>RBFNN estimation-based Adaptive SMC</td><td>Improving voltage stability in a PEMFC.</td><td>Increased complexity.</td></tr></tbody></table>
<table><caption>Table A1. Cont.</caption><thead><tr><th>Control Method</th><th>Reference</th><th>Year</th><th>Proposed Method</th><th>Main Contribution</th><th>Limitation</th></tr></thead><tbody><tr><td rowspan="4">Adaptive Droop Control</td><td>[42]</td><td>2019</td><td>Hierarchical adaptive droop and supervisory control</td><td>Improving voltage stability and load power sharing in a DC MG with multi-energy storage devices.</td><td>The effectiveness of the proposed technique has not been validated against existing approaches.</td></tr><tr><td>[43]</td><td>2020</td><td>Adaptive distributed droop</td><td>Improving DC bus voltage stability</td><td>Stability challenges in a large-scale system</td></tr><tr><td>[44]</td><td>2023</td><td>Adaptive droop + consensus control</td><td>DC MG power smoothing and voltage control</td><td>Difficulty in tuning parameters.</td></tr><tr><td>[46]</td><td>2022</td><td>Droop index control</td><td>Improving voltage stability.</td><td>Performance may be sensitive to droop index.</td></tr><tr><td rowspan="4">MPC</td><td>[53]</td><td>2021</td><td>FCS-MPC</td><td>Voltage control and power allocation optimization for DC MG with HESS</td><td>Prediction at each control cycle can be computationally intensive.</td></tr><tr><td>[54]</td><td>2020</td><td>Fast distributed MPC</td><td>Improving voltage stability</td><td>High computational cost.</td></tr><tr><td>[55]</td><td>2021</td><td>Hybrid MPC</td><td>Improving voltage stability of a boost converter interfaced with CPLs</td><td>Prediction at each control cycle can be computationally intensive.</td></tr><tr><td>[56]</td><td>2022</td><td>MPC combined with Kalman Observer</td><td>Enhancing voltage stability of an interleaved boost converter</td><td>Increased sensitivity to model accuracy.</td></tr><tr><td rowspan="4">PBC</td><td>[59]</td><td>2019</td><td>PBC</td><td>DC MG voltage regulation</td><td>Performance under varying load conditions has not been investigated.</td></tr><tr><td>[10]</td><td>2019</td><td>Decentralized PBC</td><td>Improving voltage stability</td><td>Performance depends on model accuracy</td></tr><tr><td>[60]</td><td>2024</td><td>IDA-PBC + SMRC</td><td>Improving voltage stability</td><td>Parameter uncertainties have not been considered.</td></tr><tr><td>[61]</td><td>2021</td><td>Adaptive PBC</td><td>Voltage regulation in a buck-boost converter</td><td>Performance depends on the accuracy of the system model.</td></tr><tr><td rowspan="3">ADRC</td><td>[64]</td><td>2015</td><td>ADRC</td><td>Improving performance of a flywheel energy storage system.</td><td>Not specified</td></tr><tr><td>[65]</td><td>2017</td><td>Time-scale droop control based on ADRC</td><td>Time-scale voltage droop control robust to uncertainties and external disturbances.</td><td>Performance depends on model accuracy.</td></tr><tr><td>[66]</td><td>2019</td><td>Modified ADRC</td><td>Comparison of ADRC techniques for suppressing disturbances in a boost converter.</td><td>Evaluation is based on average model.</td></tr><tr><td rowspan="2">H<sub>∞</sub></td><td>[68]</td><td>2019</td><td>H<sub>∞</sub></td><td>Enhancing voltage stability.</td><td>Choosing appropriate weighting functions is challenging.</td></tr><tr><td>[70]</td><td>2023</td><td>Loop-shaping H<sub>∞</sub></td><td>Robust voltage control of DC MG.</td><td>Performance not validated against other methods.</td></tr></tbody></table>

<table><caption>Table A2. Summarized Review of Model-Free Voltage Control Techniques.</caption><thead><tr><th>Control Method</th><th>Reference</th><th>Year</th><th>Proposed Method</th><th>Main Contribution</th><th>Limitation</th></tr></thead><tbody><tr><td rowspan="4">FLC</td><td>[76]</td><td>2020</td><td>SCA-HS tuned Type II Fuzzy</td><td>Enhancing voltage stability in a boost converter feeding CPLs.</td><td>Performance is often sensitive to the choice of optimization parameters and fuzzy rule base.</td></tr><tr><td>[77]</td><td>2020</td><td>iSIT2-FPI + SMC</td><td>The authors proposed an SMC-based model-free FLC.</td><td>Relatively complex to implement.</td></tr><tr><td>[78]</td><td>2020</td><td>Fuzzy-PI dual mode</td><td>The authors combined FLC with PI to enhance dynamic response and restrain fluctuations of the bus voltage.</td><td>Tuning scaling gains is necessary whenever the system dynamic changes.</td></tr><tr><td>[80]</td><td>2019</td><td>ANFIS</td><td>Improving transient and steady-state responses of a flyback converter using FLC and neural network</td><td>Training ANFIS requires high-quality data. High computational cost</td></tr></tbody></table>
<table><caption>**Table A2. Cont.**</caption><thead><tr><th>Control Method</th><th>Reference</th><th>Year</th><th>Proposed Method</th><th>Main Contribution</th><th>Limitation</th></tr></thead><tbody><tr><td rowspan="4">ANN</td><td>[87]</td><td>2022</td><td>CCSNN</td><td>The authors proposed an EMS to enhance power sharing among CESS, as well as maintain bus voltage stability.</td><td>It is computationally intensive to tune CC hyperparameters and train a neural network.</td></tr><tr><td>[88]</td><td>2020</td><td>HBSANN</td><td>Proposed an HBSANN-based power management strategy. Improving the voltage regulation of a DC MG</td><td>It requires high quality training data</td></tr><tr><td>[89]</td><td>2021</td><td>DNN</td><td>Proposed a supervised deep learning aided-sensorless controller</td><td>Risk of overfitting</td></tr><tr><td>[90]</td><td>2021</td><td>ANN-approximate dynamic programming</td><td>Improving voltage stability under variable load and input voltage conditions.</td><td>Requires high-quality data</td></tr><tr><td>LMN</td><td>[91]</td><td>2019</td><td>LMN + LLC</td><td>Identification of a DC/DC converter's dynamics directly from measured data. Developed a voltage controller based on the identified model.</td><td>Performance was not evaluated against robust control methods.</td></tr><tr><td rowspan="2">MFAC</td><td>[96]</td><td>2023</td><td>MFAC</td><td>Design a pseudo-gradient estimation algorithm based on I/O data. Improving voltage stability in a BDC.</td><td>Pseudo-gradient estimation methods may introduce systematic errors due to the approximation process.</td></tr><tr><td>[99]</td><td>2021</td><td>Model-free iSIT2-FPI</td><td>Improving voltage regulation in a stand-alone shipboard DC MG.</td><td>Increased complexity.</td></tr><tr><td rowspan="5">DRL</td><td>[108]</td><td>2023</td><td>PID+DDPG</td><td>Enhancing the voltage stability of a buck converter</td><td>Performance is partially dependent on model accuracy.</td></tr><tr><td>[109]</td><td>2022</td><td>DQN</td><td>Improving the voltage stability of a buck converter</td><td>Handles only discrete actions.</td></tr><tr><td>[114]</td><td>2024</td><td>TD3</td><td>Optimizing parameters of a PI controller.</td><td>Performance was tested under light load conditions.</td></tr><tr><td>[116]</td><td>2020</td><td>DDPG</td><td>Voltage stabilization of IoT-based buck converter feeding CPLs.</td><td>The paper does not discuss training or simulation results.</td></tr><tr><td>[117]</td><td>2023</td><td>Integral RL</td><td>Improving voltage stability in an interleaved boost converter.</td><td>The paper does not discuss training or simulation results.</td></tr></tbody></table>

<table><caption>**Table A3. Summarized Review of Hybrid Control Strategies.**</caption><thead><tr><th>Control Method</th><th>Reference</th><th>Year</th><th>Proposed Method</th><th>Main Contribution</th><th>Limitation</th></tr></thead><tbody><tr><td rowspan="4">MOAs</td><td>[119]</td><td>2020</td><td>GA-tuned PID</td><td>Improving voltage stability and performance of a fuel cell</td><td>Poor parameter tuning can influence effectiveness of the algorithm.</td></tr><tr><td>[120]</td><td>2021</td><td>PSO-tuned PI</td><td>Improving voltage stability in a buck-boost converter.</td><td>Performance in the presence of disturbances has not been evaluated.</td></tr><tr><td>[121]</td><td>2023</td><td>QAOA</td><td>Improving efficiency of a cascaded boost converter.</td><td>Performance with varying load conditions is not discussed.</td></tr><tr><td>[122]</td><td>2023</td><td>SSA-PSO</td><td>Improving voltage stability in a PV-powered MG.</td><td>The proposed method is not compared with other established methods.</td></tr><tr><td rowspan="2">PINNs</td><td>[127]</td><td>2024</td><td>PINN</td><td>Estimating SOH of a lithium-ion battery.</td><td>Difficulty in handling high-dimensional nonlinear models.</td></tr><tr><td>[128]</td><td>2024</td><td>PINN</td><td>Enhancing stability in a buck converter.</td><td>Difficulty in handling high-dimensional converter dynamics.</td></tr></tbody></table>

## References

1. Arshad, R.; Mininni, G.M.; De Rosa, R.; Khan, H.A. Enhancing climate resilience of vulnerable women in the Global South through power sharing in DC microgrids. *Renew. Energy* **2024**, *237*, 121495. [CrossRef]

2. Xu, X.; Xia, J.; Hong, C.; Sun, P.; Xi, P.; Li, J. Optimization of cooperative operation of multiple microgrids considering green certificates and carbon trading. *Energies* **2025**, *18*, 4083. [CrossRef]

3. Cagnano, A.; De Tuglie, E.; Mancarella, P. Microgrids: Overview and guidelines for practical implementations and operation. *Appl. Energy* **2020**, *258*, 114039. [CrossRef]

4. e Ammara, U.; Zehra, S.S.; Nazir, S.; Ahmad, I. Artificial neural network-based nonlinear control and modeling of a DC microgrid incorporating regenerative FC/HPEV and energy storage system. *Renew. Energy Focus* **2024**, *49*, 100565. [CrossRef]

5. Ojo, K.E.; Saha, A.K.; Srivastava, V.M. Microgrids' control strategies and real-time monitoring systems: A comprehensive review. *Energies* **2025**, *18*, 3576. [CrossRef]

6. Eyimaya, S.E.; Altin, N.; Nasiri, A. Optimization of photovoltaic and battery storage sizing in a DC microgrid using LSTM networks based on load forecasting. *Energies* **2025**, *18*, 3676. [CrossRef]
7. Derakhshan, S.; Shafiee-Rad, M.; Shafiee, Q.; Jahed-Motlagh, M.R.; Sahoo, S.; Blaabjerg, F. Decentralized voltage control of autonomous DC microgrids with robust performance approach. *IEEE J. Emerg. Sel. Top. Power Electron.* **2021**, *9*, 5508–5520.
[CrossRef]

8. Eydi, M.; Ghazi, R.; Buygi, M.O. A decentralized control method for proportional current-sharing, voltage restoration, and SOCs balancing of widespread DC microgrids. *Int. J. Electr. Power Energy Syst.* **2024**, *155*, 109645. [CrossRef]

9. Rizk, H.; Chaibet, A.; Kribèche, A. Model-based control and model-free control techniques for autonomous vehicles: A technical survey. *Appl. Sci.* **2023**, *13*, 6700. [CrossRef]

10. Cucuzzella, M.; Lazzari, R.; Kawano, Y.; Kosaraju, K.C.; Scherpen, J.M.A. Robust passivity-based control of boost converters in DC microgrids. In Proceedings of the 2019 IEEE 58th Conference on Decision and Control (CDC), Nice, France, 11–13 December 2019; pp. 8435–8440. [CrossRef]

11. Santoni, C.; Zhang, Z.; Sotiropoulos, F.; Khosronejad, A. A data-driven machine learning approach for yaw control applications of wind farms. *Theor. Appl. Mech. Lett.* **2023**, *13*, 100471. [CrossRef]

12. Modu, B.; Abdullah, M.P.; Sanusi, M.A.; Hamza, M.F. DC-based microgrid: Topologies, control schemes, and implementations. *Alex. Eng. J.* **2023**, *70*, 61–92. [CrossRef]

13. Ashok Kumar, A.; Amutha Prabha, N. A comprehensive review of DC microgrid in market segments and control technique. *Heliyon* **2022**, *8*, e11694. [CrossRef]

14. Xu, Q.; Vafamand, N.; Chen, L.; Dragicevic, T.; Xie, L.; Blaabjerg, F. Review on advanced control technologies for bidirectional DC/DC converters in DC microgrids. *IEEE J. Emerg. Sel. Top. Power Electron.* **2021**, *9*, 1205–1221. [CrossRef]

15. Ekanayake, U.N.; Navaratne, U.S. A survey on microgrid control techniques in islanded mode. *J. Electr. Comput. Eng.* **2020**, *2020*, 6275460. [CrossRef]

16. Bukar, A.L.; Modu, B.; Abdullah, M.P.; Hamza, M.F.; Almutairi, S.Z. Peer-to-peer energy trading framework for an autonomous DC microgrid using game theoretic approach. *Renew. Energy Focus* **2024**, *51*, 100636. [CrossRef]

17. Patel, R.; Chudamani, R. Stability analysis of the main converter supplying a constant power load in a multi-converter system considering various parasitic elements. *Eng. Sci. Technol. Int. J.* **2020**, *23*, 1118–1125. [CrossRef]

18. Rahimian, M.M.; Mohammadi, H.R.; Guerrero, J.M. Constant power load issue in DC/DC multi-converter systems: Past studies and recent trends. *Electr. Power Syst. Res.* **2024**, *235*, 110851. [CrossRef]

19. Gheisarnejad, M.; Akhbari, A.; Rahimi, M.; Andresen, B.; Khooban, M.H. Reducing impact of constant power loads on DC energy systems by artificial intelligence. *IEEE Trans. Circuits Syst. II Express Briefs* **2022**, *69*, 4974–4978. [CrossRef]

20. Liu, Z.; Liu, Y.; Yu, Y.; Yang, R. Coordinated control and optimal flow of shipboard MVDC system for adapting to large pulsed power load. *Electr. Power Syst. Res.* **2023**, *221*, 109354. [CrossRef]

21. Kumar, K.; Kumar, P.; Kar, S. A review of microgrid protection for addressing challenges and solutions. *Renew. Energy Focus* **2024**, *49*, 100572. [CrossRef]

22. Tavagnutti, A.A.; Bosich, D.; Pastore, S.; Sulligoi, G. A reduced order model for the stable LC-filter design on shipboard DC microgrids. In Proceedings of the 2023 IEEE International Conference on Electrical Systems for Aircraft, Railway, Ship Propulsion and Road Vehicles & International Transportation Electrification Conference (ESARS-ITEC), Venice, Italy, 29–31 March 2023; pp. 1–6. [CrossRef]

23. Al-Ismail, F.S. A critical review on DC microgrids voltage control and power management. *IEEE Access* **2024**, *12*, 30345–30361. [CrossRef]

24. Li, F.; Tu, W.; Zhou, Y.; Li, H.; Zhou, F.; Liu, W.; Hu, C. Distributed secondary control for DC microgrids using two-stage multi-agent reinforcement learning. *Int. J. Electr. Power Energy Syst.* **2025**, *164*, 110335. [CrossRef]

25. Saleem, O.; Rizwan, M. Performance optimization of LQR-based PID controller for DC-DC buck converter via iterative-learning-tuning of state-weighting matrix. *Int. J. Numer. Model. Electron. Netw. Devices Fields* **2019**, *32*, e2572. [CrossRef]

26. Sheikhi Jouybary, H.; Arab Khaburi, D.; El Hajjaji, A.; Mpanda Mabwe, A. Optimal sliding mode control of modular multilevel converters considering control input constraints. *Energies* **2025**, *18*, 2757. [CrossRef]

27. Peng, C.; Xie, C.; Zou, J.; Jiang, X.; Zhu, Y. A feedback linearization sliding mode decoupling and fuzzy anti-surge compensation based coordinated control approach for PEMFC air supply system. *Renew. Energy* **2024**, *237*, 121760. [CrossRef]

28. Ullah, Q.; Busarello, T.D.C.; Brandao, D.I.; Simões, M.G. Design and performance evaluation of SMC-based DC–DC converters for microgrid applications. *Energies* **2023**, *16*, 4212. [CrossRef]

29. Obeid, H.; Petrone, R.; Chaoui, H.; Gualous, H. Higher order sliding-mode observers for state-of-charge and state-of-health estimation of lithium-ion batteries. *IEEE Trans. Veh. Technol.* **2023**, *72*, 4482–4492. [CrossRef]

30. Muhammad, R.; Muhammad, A.; Bhatti, A.I.; Minhas, D.M.; Ahmed, B.A. Mathematical modeling and stability analysis of DC microgrid using SM hysteresis controller. *Int. J. Electr. Power Energy Syst.* **2018**, *95*, 507–522. [CrossRef]

31. Cucuzzella, M.; Lazzari, R.; Trip, S.; Rosti, S.; Sandroni, C.; Ferrara, A. Sliding mode voltage control of boost converters in DC microgrids. *Control Eng. Pract.* **2018**, *73*, 161–170. [CrossRef]
32. Mathew, K.K.; Abraham, D.M. Particle swarm optimization based sliding mode controllers for electric vehicle onboard charger. *Comput. Electr. Eng.* **2021**, *96*, 107502. [CrossRef]

33. Mustafa, G.; Ahmad, F.; Zhang, R.; Haq, E.U.; Hussain, M. Adaptive sliding mode control of buck converter feeding resistive and constant power load in DC microgrid. *Energy Rep.* **2023**, *9*, 1026–1035. [CrossRef]

34. Balta, G.; Güler, N.; Altin, N. Global fast terminal sliding mode control with fixed switching frequency for voltage control of DC–DC buck converters. *ISA Trans.* **2023**, *143*, 582–595. [CrossRef]

35. Roy, T.K.; Oo, A.M.T.; Ghosh, S.K. Designing a high-order sliding mode controller for photovoltaic- and battery energy storage system-based DC microgrids with ANN-MPPT. *Energies* **2024**, *17*, 532. [CrossRef]

36. Li, X.; Wang, M.; Dong, C.; Jiang, W.; Xu, Z.; Wu, X.; Jia, H. A robust autonomous sliding-mode control of renewable DC microgrids for decentralized power sharing considering large-signal stability. *Appl. Energy* **2023**, *339*, 121019. [CrossRef]

37. Derakhshannia, M.; Moosapour, S.S. RBFNN based fixed time sliding mode control for PEMFC air supply system with input delay. *Renew. Energy* **2024**, *237*, 121772. [CrossRef]

38. Zhang, L.; Chen, K.; Chi, S.; Lyu, L.; Ma, H.; Wang, K. The bidirectional DC/DC converter operation mode control algorithm based on RBF neural network. In Proceedings of the 2019 IEEE PES International Conference on Innovative Smart Grid Technologies Asia (ISGT Asia), Chengdu, China, 21–24 May 2019. [CrossRef]

39. Chen, X.; Shen, W.; Dai, M.; Cao, Z.; Jin, J.; Kapoor, A. Robust adaptive sliding-mode observer using RBF neural network for lithium-ion battery state of charge estimation in electric vehicles. *IEEE Trans. Veh. Technol.* **2016**, *65*, 1936–1947. [CrossRef]

40. Xiao, X.; Lv, J.; Chang, Y.; Chen, J.; He, H. Adaptive sliding mode control integrating with RBFNN for proton exchange membrane fuel cell power conditioning. *Appl. Sci.* **2022**, *12*, 3132. [CrossRef]

41. Zhang, H.; Sinha, R.; Golmohamadi, H.; Chaudhary, S.K.; Bak-Jensen, B. Autonomous control of electric vehicles using voltage droop. *Energies* **2025**, *18*, 2824. [CrossRef]

42. Yuan, M.; Fu, Y.; Mi, Y.; Li, Z.; Wang, C. Hierarchical control of DC microgrid with dynamical load power sharing. *Appl. Energy* **2019**, *239*, 1–11. [CrossRef]

43. Kumar, R.; Pathak, M.K. Distributed droop control of DC microgrid for improved voltage regulation and current sharing. *IET Renew. Power Gener.* **2020**, *14*, 2499–2506. [CrossRef]

44. Li, X.; Li, P.; Ge, L.; Wang, X.; Li, Z.; Zhu, L.; Guo, L.; Wang, C. A unified control of super-capacitor system based on bi-directional DC-DC converter for power smoothing in DC microgrid. *J. Mod. Power Syst. Clean Energy* **2023**, *11*, 938–949. [CrossRef]

45. Rehmat, A.; Alam, F.; Nasir, M.; Zaidi, S.S. Robust hierarchical non-linear droop control design for the PV based islanded microgrid. In Proceedings of the 19th International Bhurban Conference on Applied Sciences and Technology (IBCAST), Islamabad, Pakistan, 11–15 January 2022; pp. 620–628. [CrossRef]

46. Thogaru, R.B.; Naware, D.; Mitra, A.; Chaudhary, J. Resiliency-driven approach of DC microgrid voltage regulation based on droop index control for high step-up DC-DC converter. *Int. Trans. Electr. Energy Syst.* **2022**, *2022*, 3676438. [CrossRef]

47. Guo, Z.; Zuo, D.; Liu, X.; Zhang, Z.; Ma, J.; Meng, F.; Fang, Y. Dual tracking model-free predictive control for three-level neutral-point clamped inverters. *Electr. Power Syst. Res.* **2025**, *249*, 112054. [CrossRef]

48. Murillo-Yarce, D.; Riffo, S.; Restrepo, C.; González-Castaño, C.; Garcés, A. Model predictive control for stabilization of DC microgrids in island mode operation. *Mathematics* **2022**, *10*, 3384. [CrossRef]

49. Chen, L.; Zhou, J.; Zhai, J.; Yang, L.; Qian, X.; Tao, Z. Continuous-control-set model predictive control strategy for MMC-UPQC under non-ideal conditions. *Energies* **2025**, *18*, 2946. [CrossRef]

50. Babqi, A.J.; Alamri, B. A comprehensive comparison between finite control set model predictive control and classical proportional-integral control for grid-tied power electronics devices. *Acta Polytech. Hung.* **2021**, *18*, 67–87. [CrossRef]

51. Diaz Franco, F.; Vu, T.V.; Gonsulin, D.; Vahedi, H.; Edrington, C.S. Enhanced performance of PV power control using model predictive control. *Sol. Energy* **2017**, *158*, 679–686. [CrossRef]

52. Villalón, A.; Rivera, M.; Salgueiro, Y.; Muñoz, J.; Dragičević, T.; Blaabjerg, F. Predictive control for microgrid applications: A review study. *Energies* **2020**, *13*, 2454. [CrossRef]

53. Ni, F.; Zheng, Z.; Xie, Q.; Xiao, X.; Zong, Y.; Huang, C. Enhancing resilience of DC microgrids with model predictive control based hybrid energy storage system. *Int. J. Electr. Power Energy Syst.* **2021**, *128*, 106738. [CrossRef]

54. Marepalli, L.K.; Gajula, K.; Herrera, L. Fast distributed model predictive control for DC microgrids. In Proceedings of the 21st IEEE Workshop on Control and Modeling for Power Electronics (COMPEL 2020), Aalborg, Denmark, 9–12 November 2020. [CrossRef]

55. Karami, Z.; Shafiee, Q.; Sahoo, S.; Yaribeygi, M.; Bevrani, H.; Dragičević, T. Hybrid model predictive control of DC-DC boost converters with constant power load. *IEEE Trans. Energy Convers.* **2021**, *36*, 1347–1356. [CrossRef]

56. Tan, B.; Li, H.; Zhao, D.; Liang, Z.; Ma, R.; Huangfu, Y. Finite-control-set model predictive control of interleaved DC-DC boost converter based on Kalman observer. *eTransportation* **2022**, *11*, 100158. [CrossRef]

57. Kao, C.Y.; Khong, S.Z.; van der Schaft, A. On the converse passivity theorems for LTI systems. In Proceedings of the IFAC World Congress, IFAC-PapersOnLine, Berlin, Germany, 11–17 July 2020; pp. 6422–6427. [CrossRef]
58. Acevedo, D.M.; Parraguez-Garrido, I.; Gil-Gonzalez, W.; Montoya, O.D.; Gonzalez-Castano, C. Adaptive passivity-based control for DC motor speed regulation in DC-DC converter-fed systems. *IEEE Access* **2025**, *13*, 131957–131966. [CrossRef]

59. Sun, J.; Lin, W.; Hong, M.; Loparo, K.A. Voltage regulation of DC-microgrid with PV and battery: A passivity method. In Proceedings of the IFAC Workshop on Control of Smart Grid and Renewable Energy Systems (CSGRES), IFAC-PapersOnLine, Jeju, Republic of Korea, 10–12 June 2019; pp. 753–758. [CrossRef]

60. Martínez, L.; Fernández, D.; Mantz, R. Passivity-based control for an isolated DC microgrid with hydrogen energy storage system. *Int. J. Hydrogen Energy* **2024**, *67*, 1262–1269. [CrossRef]

61. Soriano-Rangel, C.A.; He, W.; Mancilla-David, F.; Ortega, R. Voltage regulation in buck-boost converters feeding an unknown constant power load: An adaptive passivity-based control. *IEEE Trans. Control Syst. Technol.* **2021**, *29*, 395–402. [CrossRef]

62. Han, J. From PID to active disturbance rejection control. *IEEE Trans. Ind. Electron.* **2009**, *56*, 900–906. [CrossRef]

63. Hao, F.; Guo, J.; Yu, Z.; Ye, J. Output voltage control of LLC resonant converter based on improved linear active disturbance rejection control. *IEEE J. Emerg. Sel. Top. Power Electron.* **2025**, *13*, 3555–3564. [CrossRef]

64. Chang, X.; Li, Y.; Zhang, W.; Wang, N.; Xue, W. Active disturbance rejection control for a flywheel energy storage system. *IEEE Trans. Ind. Electron.* **2015**, *62*, 991–1001. [CrossRef]

65. Yang, N.; Gao, F.; Paire, D.; Miraoui, A.; Liu, W. Distributed control of multi-time scale DC microgrid based on ADRC. *IET Power Electron.* **2017**, *10*, 329–337. [CrossRef]

66. Ahmad, S.; Ali, A. Active disturbance rejection control of DC–DC boost converter: A review with modifications for improved performance. *IET Power Electron.* **2019**, *12*, 2095–2107. [CrossRef]

67. Zhao, Y.; Yang, X.; Zhang, Y.; Zhang, Q. Event-triggered H-infinity pitch control for floating offshore wind turbines. *IEEE Trans. Sustain. Energy* **2025**, *16*, 1329–1339. [CrossRef]

68. Rigatos, G.; Zervos, N.; Siano, P.; Abbaszadeh, M.; Wira, P.; Onose, B. Nonlinear optimal control for DC industrial microgrids. *Cyber-Phys. Syst.* **2019**, *5*, 231–253. [CrossRef]

69. Mehdi, M.; Jamali, S.Z.; Khan, M.O.; Baloch, S.; Kim, C.H. Robust control of a DC microgrid under parametric uncertainty and disturbances. *Electr. Power Syst. Res.* **2020**, *179*, 106074. [CrossRef]

70. Ruchi, S.; Avirup, M.; Shyam, K. Robust control of an islanded DC microgrid using H-infinity loop-shaping design considering parametric uncertainties. In Proceedings of the TENCON 2023—IEEE Region 10 Conference, Chiang Mai, Thailand, 31 October–3 November 2023; pp. 1082–1087. [CrossRef]

71. Al Sumarmad, K.A.; Sulaiman, N.; Wahab, N.I.A.; Hizam, H. Energy management and voltage control in microgrids using artificial neural networks, PID, and fuzzy logic controllers. *Energies* **2022**, *15*, 303. [CrossRef]

72. Chandrasekaran, S.; Durairaj, S.; Padmavathi, S. A performance evaluation of a fuzzy logic controller-based photovoltaic-fed multi-level inverter for a three-phase induction motor. *J. Frankl. Inst.* **2021**, *358*, 7394–7412. [CrossRef]

73. Dumitrescu, C.; Ciotirnae, P.; Vizitiu, C. Fuzzy logic for intelligent control system using soft computing applications. *Sensors* **2021**, *21*, 2617. [CrossRef]

74. Belman-Flores, J.M.; Rodríguez-Valderrama, D.A.; Ledesma, S.; García-Pabón, J.J.; Hernández, D.; Pardo-Cely, D.M. A review on applications of fuzzy logic control for refrigeration systems. *Appl. Sci.* **2022**, *12*, 1302. [CrossRef]

75. Bhosale, R.; Agarwal, V. Fuzzy logic control of the ultracapacitor interface for enhanced transient response and voltage stability of a DC microgrid. *IEEE Trans. Ind. Appl.* **2019**, *55*, 712–720. [CrossRef]

76. Farsizadeh, H.; Gheisarnejad, M.; Mosayebi, M.; Rafiei, M.; Khooban, M.H. An intelligent and fast controller for DC/DC converter feeding CPL in a DC microgrid. *IEEE Trans. Circuits Syst. II Express Briefs* **2020**, *67*, 1104–1108. [CrossRef]

77. Khooban, M.H.; Gheisarnejad, M.; Farsizadeh, H.; Masoudian, A.; Boudjadar, J. A new intelligent hybrid control approach for DC-DC converters in zero-emission ferry ships. *IEEE Trans. Power Electron.* **2020**, *35*, 5832–5841. [CrossRef]

78. Zhang, Y.; Wei, S.; Wang, J.; Zhang, L. Bus voltage stabilization control of photovoltaic DC microgrid based on fuzzy-PI dual-mode controller. *J. Electr. Comput. Eng.* **2020**, 2683052. [CrossRef]

79. Rodriguez, M.; Arcos-Aviles, D.; Martinez, W. Fuzzy logic-based energy management for isolated microgrid using meta-heuristic optimization algorithms. *Appl. Energy* **2023**, *335*, 120771. [CrossRef]

80. Shahid, M.A.; Abbas, G.; Hussain, M.R.; Asad, M.U.; Farooq, U.; Gu, J.; Balas, V.E.; Uzair, M.; Awan, A.B.; Yazdan, T. Artificial intelligence-based controller for DC-Dc flyback converter. *Appl. Sci.* **2019**, *9*, 5108. [CrossRef]

81. Al-Hitmi, M.A.; Islam, S.; Muyeen, S.M.; Iqbal, A.; Thomas, K.; Abdullah, A.K.M.; Ben-brahim, L. An ANN-based distributed secondary controller used to ensure accurate current sharing in DC microgrid. *AEU Int. J. Electron. Commun.* **2025**, *201*, 156002. [CrossRef]

82. Zhao, S.; Blaabjerg, F.; Wang, H. An overview of artificial intelligence applications for power electronics. *IEEE Trans. Power Electron.* **2021**, *36*, 4633–4658. [CrossRef]

83. Han, Y.; Liao, Y.; Ma, X.; Guo, X.; Li, C.; Liu, X. Analysis and prediction of the penetration of renewable energy in power systems using artificial neural network. *Renew. Energy* **2023**, *215*, 118914. [CrossRef]
84. Lopez-Garcia, T.B.; Coronado-Mendoza, A.; Domínguez-Navarro, J.A. Artificial neural networks in microgrids: A review. *Eng. Appl. Artif. Intell.* **2020**, *95*, 103894. [CrossRef]

85. Saadatmand, S.; Shamsi, P.; Ferdowsi, M. The voltage regulation of a buck converter using a neural network predictive controller. In Proceedings of the 2020 IEEE Texas Power and Energy Conference (TPEC), College Station, TX, USA, 6–7 February 2020; pp. 1–6. [CrossRef]

86. Khan, H.S.; Mohamed, I.S.; Kauhaniemi, K.; Liu, L. Artificial neural network-based voltage control of DC/DC converter for DC microgrid applications. In Proceedings of the 2021 6th IEEE Workshop on the Electronic Grid (eGRID), New Orleans, LA, USA, 8–10 November 2021; pp. 1–6. [CrossRef]

87. Singh, P.; Anwer, N.; Lather, J.S. Energy management and control for direct current microgrid with composite energy storage system using combined cuckoo search algorithm and neural network. *J. Energy Storage* **2022**, *55*, 105689. [CrossRef]

88. Singh, P.; Lather, J.S. Dynamic power management and control for low voltage DC microgrid with hybrid energy storage system using hybrid bat search algorithm and artificial neural network. *J. Energy Storage* **2020**, *32*, 101974. [CrossRef]

89. Akpolat, A.N.; Dursun, E.; Kuzucuoglu, A.E. Deep learning-aided sensorless control approach for PV converters in DC nanogrids. *IEEE Access* **2021**, *9*, 106641–106654. [CrossRef]

90. Dong, W.; Li, S.; Fu, X.; Li, Z.; Fairbank, M.; Gao, Y. Control of a buck DC/DC converter using approximate dynamic programming and artificial neural networks. *IEEE Trans. Circuits Syst. I Regul. Pap.* **2021**, *68*, 1760–1768. [CrossRef]

91. Rouzbehi, K.; Miranian, A.; Escaño, J.M.; Rakhshani, E.; Shariati, N.; Pouresmaeil, E. A data-driven based voltage control strategy for DC-DC converters: Application to DC microgrid. *Electronics* **2019**, *8*, 493. [CrossRef]

92. She, B.; Li, F.; Cui, H.; Zhang, J.; Bo, R. Fusion of model-free reinforcement learning with microgrid control: Review and vision. *IEEE Trans. Smart Grid* **2023**, *14*, 3232–3245. [CrossRef]

93. Hartmann, B.; Nelle, O. On the smoothness in local model networks. In Proceedings of the American Control Conference, St. Louis, MO, USA, 10–12 June 2009; pp. 3573–3578. [CrossRef]

94. Novak, J.; Chalupa, P.; Bobal, V. Local model networks for modelling and predictive control of nonlinear systems. In Proceedings of the 23rd European Conference on Modelling and Simulation (ECMS 2009), Madrid, Spain, 9–12 June 2009; European Council for Modelling and Simulation: Kingston upon Thames, UK, 2009; pp. 557–562. [CrossRef]

95. Hou, Z.; Xiong, S. On model-free adaptive control and its stability analysis. *IEEE Trans. Autom. Control* **2019**, *64*, 4555–4569. [CrossRef]

96. Wang, Z.; Wang, D.; Peng, Z.; Liu, L. Model-free adaptive control for ultracapacitor based three-phase interleaved bidirectional DC–DC converter. *IET Power Electron.* **2023**, *16*, 2696–2707. [CrossRef]

97. Yu, W.; Wang, R.; Bu, X.; Hou, Z. Model free adaptive control for a class of nonlinear systems with fading measurements. *J. Frankl. Inst.* **2020**, *357*, 7743–7760. [CrossRef]

98. Saeid, A.H.; Babak, N.-M. Active stabilization of a microgrid using model free adaptive control. In Proceedings of the 2017 IEEE Industry Applications Society Annual Meeting, Cincinnati, OH, USA, 1–5 October 2017; pp. 1–8. [CrossRef]

99. Mosayebi, M.; Sadeghzadeh, S.M.; Gheisarnejad, M.; Khooban, M.H. Intelligent and fast model-free sliding mode control for shipboard DC microgrids. *IEEE Trans. Transp. Electrif.* **2021**, *7*, 1662–1671. [CrossRef]

100. Yue, J.; Liu, Z.; Su, H. Model-free composite disturbance rejection control for dynamic wireless charging system based on online parameter identification. *IEEE Trans. Ind. Electron.* **2024**, *71*, 7777–7785. [CrossRef]

101. Yue, J.; Liu, Z.; Su, H. Data-driven adaptive extended state observer-based model-free disturbance rejection control for DC–DC converters. *IEEE Trans. Ind. Electron.* **2024**, *71*, 7745–7755. [CrossRef]

102. Cai, Q.; Luo, X.Q.; Wang, P.; Gao, C.; Zhao, P. Hybrid model-driven and data-driven control method based on machine learning algorithm in energy hub and application. *Appl. Energy* **2022**, *305*, 117913. [CrossRef]

103. Dan, Y.; Zhong, H.; Wang, C.; Wang, J.; Fei, Y.; Yu, L. A graph deep reinforcement learning-based fault restoration method for active distribution networks. *Energies* **2025**, *18*, 4420. [CrossRef]

104. Wolgast, T.; Niese, A. Approximating energy market clearing and bidding with model-based reinforcement learning. *IEEE Access* **2024**, *12*, 145106–145117. [CrossRef]

105. Bachiri, K.; Yahyaouy, A.; Gualous, H.; Malek, M.; Bennani, Y.; Makany, P.; Rogovschi, N. Multi-agent DDPG based electric vehicles charging station recommendation. *Energies* **2023**, *16*, 6067. [CrossRef]

106. Zandi, O.; Poshtan, J. Voltage control of a quasi Z-source converter under constant power load condition using reinforcement learning. *Control Eng. Pract.* **2023**, *135*, 105499. [CrossRef]

107. Shakya, A.K.; Pillai, G.; Chakrabarty, S. Reinforcement learning algorithms: A brief survey. *Expert Syst. Appl.* **2023**, *231*, 120495. [CrossRef]

108. Hu, K.; Zhang, X.; Ma, H. A novel proportion-integral-differential controller based on deep reinforcement learning for DC/DC power buck converters. In Proceedings of the 2021 IEEE 1st International Power Electronics and Application Symposium (PEAS), Shanghai, China, 13–15 November 2021. [CrossRef]
109. Cui, C.; Yan, N.; Huangfu, B.; Yang, T.; Zhang, C. Voltage regulation of DC-DC buck converters feeding CPLs via deep reinforcement learning. *IEEE Trans. Circuits Syst. II Express Briefs* **2022**, *69*, 1777-1781. [CrossRef]

110. Shi, X.; Chen, N.; Wei, T.; Wu, J.; Xiao, P. A reinforcement learning-based online-training AI controller for DC-DC switching converters. In Proceedings of the 2021 6th International Conference on Integrated Circuits and Microsystems (ICICM 2021), Nanjing, China, 22-24 October 2021; pp. 435-438. [CrossRef]

111. Marahatta, A.; Rajbhandari, Y.; Shrestha, A.; Phuyal, S.; Thapa, A.; Korba, P. Model predictive control of DC/DC boost converter with reinforcement learning. *Heliyon* **2022**, *8*, e11416. [CrossRef]

112. Ye, J.; Guo, H.; Zhao, D.; Wang, B.; Zhang, X. TD3 algorithm based reinforcement learning control for multiple-input multiple-output DC-DC converters. *IEEE Trans. Power Electron.* **2024**, *39*, 12729-12742. [CrossRef]

113. Rajamallaiah, A.; Karri, S.P.K.; Shankar, Y.R. Deep reinforcement learning based control strategy for voltage regulation of DC-DC buck converter feeding CPLs in DC microgrid. *IEEE Access* **2024**, *12*, 17419-17430. [CrossRef]

114. Muktiadji, R.F.; Ramli, M.A.M.; Milyani, A.H. Twin-delayed deep deterministic policy gradient algorithm to control a boost converter in a DC microgrid. *Electronics* **2024**, *13*, 433. [CrossRef]

115. Gheisarnejad, M.; Farsizadeh, H.; Khooban, M.H. A novel nonlinear deep reinforcement learning controller for DC-DC power buck converters. *IEEE Trans. Ind. Electron.* **2021**, *68*, 6849-6858. [CrossRef]

116. Gheisarnejad, M.; Khooban, M.H. IoT-based DC/DC deep learning power converter control: Real-time implementation. *IEEE Trans. Power Electron.* **2020**, *35*, 13621-13630. [CrossRef]

117. Qie, T.; Zhang, X.; Xiang, C.; Yu, Y.; Iu, H.H.C.; Fernando, T. A new robust integral reinforcement learning based control algorithm for interleaved DC/DC boost converter. *IEEE Trans. Ind. Electron.* **2023**, *70*, 3729-3739. [CrossRef]

118. Tomar, V.; Bansal, M.; Singh, P. Metaheuristic algorithms for optimization: A brief review. *Eng. Proc.* **2023**, *59*, 238. [CrossRef]

119. Kumar, S.; Krishnasamy, V.; Neeli, S.; Kaur, R. Artificial intelligence power controller of fuel cell based DC nanogrid. *Renew. Energy Focus* **2020**, *34*, 120-128. [CrossRef]

120. Vadi, S.; Gurbuz, F.B.; Bayindir, R.; Sagiroglu, S. Optimization of PI based buck-boost converter by particle swarm optimization algorithm. In Proceedings of the 9th International Conference on Smart Grid (icSmartGrid), Setubal, Portugal, 29 June-1 July 2021; pp. 295-301. [CrossRef]

121. Hema, S.; Sukhi, Y. Deep learning-based FOPID controller for cascaded DC-DC converters. *Comput. Syst. Sci. Eng.* **2023**, *46*, 1503-1519. [CrossRef]

122. AL-Wesabi, I.; Fang, Z.; Farh, H.M.H.; Dagal, I.; Al-Shamma'a, A.A.; Al-Shaalan, A.M.; Yang, K. Hybrid SSA-PSO based intelligent direct sliding-mode control for extracting maximum photovoltaic output power and regulating the DC-bus voltage. *Int. J. Hydrogen Energy* **2024**, *51*, 348-370. [CrossRef]

123. Nanyan, N.F.; Ahmad, M.A.; Hekimoğlu, B. Optimal PID controller for the DC-DC buck converter using the improved sine cosine algorithm. *Results Control Optim.* **2024**, *14*, 100352. [CrossRef]

124. Farea, A.; Yli-Harja, O.; Emmert-Streib, F. Understanding physics-informed neural networks: Techniques, applications, trends, and challenges. *AI* **2024**, *5*, 1534-1557. [CrossRef]

125. Stiasny, J.; Chatzivasileiadis, S. Physics-informed neural networks for time-domain simulations: Accuracy, computational cost, and flexibility. *Electr. Power Syst. Res.* **2023**, *224*, 109748. [CrossRef]

126. Huang, B.; Wang, J. Applications of physics-informed neural networks in power systems—A review. *IEEE Trans. Power Syst.* **2023**, *38*, 572-588. [CrossRef]

127. Wang, F.; Zhai, Z.; Zhao, Z.; Di, Y.; Chen, X. Physics-informed neural network for lithium-ion battery degradation stable modeling and prognosis. *Nat. Commun.* **2024**, *15*, 48779. [CrossRef]

128. Hui, P.; Cui, C.; Lin, P.; Ghias, A.M.Y.M.; Niu, X.; Zhang, C. On physics-informed neural network control for power electronics. *arXiv* **2024**, arXiv:2406.15787. [CrossRef]

129. Lee, J.S.; Lee, G.Y.; Park, S.S.; Kim, R.Y. Impedance-based modeling and common bus stability enhancement control algorithm in DC microgrid. *IEEE Access* **2020**, *8*, 211224-211234. [CrossRef]

130. Hao, W.; Han, H.; Liu, Z.; Sun, Y.; Su, M.; Hou, X.; Yang, P. A stabilization method of LC input filter in DC microgrids feeding constant power loads. In Proceedings of the 2017 IEEE Energy Conversion Congress and Exposition (ECCE), Cincinnati, OH, USA, 1-5 October 2017.

131. Pang, S.; Nahid-Mobarakeh, B.; Pierfederici, S.; Huangfu, Y.; Luo, G.; Gao, F. Research on LC filter cascaded with buck converter supplying constant power load based on IDA-passivity-based control. In Proceedings of the IECON 2018-44th Annual Conference of the IEEE Industrial Electronics Society, Washington, DC, USA, 21-23 October 2018; pp. 4992-4997. [CrossRef]

132. Firdous, N.; Din, N.M.U.; Assad, A. An imbalanced classification approach for establishment of cause-effect relationship between heart-failure and pulmonary embolism using deep reinforcement learning. *Eng. Appl. Artif. Intell.* **2023**, *126*, 107004. [CrossRef]

133. Rehman, A.U.; Ullah, Z.; Qazi, H.S.; Hasanien, H.M.; Khalid, H.M. Reinforcement learning-driven proximal policy optimization-based voltage control for PV and WT integrated power system. *Renew. Energy* **2024**, *227*, 120590. [CrossRef]
134. Del Rio, A.; Jimenez, D.; Serrano, J. Comparative analysis of A3C and PPO algorithms in reinforcement learning: A survey on general environments. *IEEE Access* **2024**, *12*, 146795–146806. [CrossRef]

135. Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; Klimov, O. Proximal policy optimization algorithms. *arXiv* **2017**, arXiv:1707.06347. [CrossRef]

136. Zhang, F.; Li, J.; Li, Z. A TD3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment. *Neurocomputing* **2020**, *411*, 206–215. [CrossRef]

137. Fujimoto, S.; van Hoof, H.; Meger, D. Addressing function approximation error in actor-critic methods. *arXiv* **2018**, arXiv:1802.09477. [CrossRef]

138. Wang, S.; Duan, J.; Shi, D.; Xu, C.; Li, H.; Diao, R.; Wang, Z. A data-driven multi-agent autonomous voltage control framework using deep reinforcement learning. *IEEE Trans. Power Syst.* **2020**, *35*, 4644–4654. [CrossRef]

139. Liessner, R.; Schmitt, J.; Dietermann, A.; Bäker, B. Hyperparameter optimization for deep reinforcement learning in vehicle energy management. In Proceedings of the ICAART 2019-Proceedings of the 11th International Conference on Agents and Artificial Intelligence, Prague, Czech Republic, 19–21 February 2019; SciTePress: Setúbal, Portugal, 2019; pp. 134–144. [CrossRef]

140. Victoria, A.H.; Maragatham, G. Automatic tuning of hyperparameters using Bayesian optimization. *Evolving Syst.* **2021**, *12*, 217–223. [CrossRef]

141. Wu, J.; Chen, X.Y.; Zhang, H.; Xiong, L.D.; Lei, H.; Deng, S.H. Hyperparameter optimization for machine learning models based on Bayesian optimization. *J. Electron. Sci. Technol.* **2019**, *17*, 26–40.

142. MohammadNoor, I.; Fadi, S.; Ali Bou, N.; Aleksander, E.; Abdallah, S. Bayesian optimization with machine learning algorithms towards anomaly detection. In Proceedings of the IEEE Global Communications Conference, Abu Dhabi, United Arab Emirates, 9–13 December 2018; p. 90. [CrossRef]

143. Eimer, T.; Lindauer, M.; Raileanu, R. Hyperparameters in Reinforcement Learning and How to Tune Them. *arXiv* **2023**, arXiv:2306.01324. [CrossRef]

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.