# Hybrid Fuzzy-DDPG Approach for Efficient MPPT in Partially Shaded Photovoltaic Panels
## 2 Materials and Methods
### Reinforcement Learning Fuzzy-DDPG
If the actor's learning rate is sufficiently small (ensuring stable updates), the actor converges to a local optimum \(\pi^{*}(s)\), where

\[Q(\phi(s),a;\theta_{Q}) \tag{14}\]7Hence, \(\pi_{\theta^{*}}(s)\) is a near-optimal deterministic policy that maximizes the expected return. Conclusion. Putting all steps together, we conclude:

1. _Fuzzy Q-iteration_ converges to a fixed point \(\hat{Q}^{*}\) that closely approximates \(Q^{*}(s,a)\) due to the contraction property of \(\mathcal{T}\). 2. The _DDPG critic_ converges to the optimal Q-function \(Q^{*}(s,a)\) by minimizing the Bellman error. 3. The _DDPG actor_ converges to a near-optimal deterministic policy \(\pi^{*}(s)\) by following the gradient derived from the critic's Q-function. Therefore, under reasonable assumptions on the learning rates, neural network capacity, fuzzy partitioning, and exploration strategy, the hybrid Fuzzy-DDPG algorithm converges to a near-optimal policy. This theorem guarantees convergence toward a near-optimal policy by exploiting the contraction property of the Bellman operator, even under complex fuzzy approximations. Such convergence is instrumental in MPPT under partial shading conditions in PV panels, as it systematically guides the search toward the global maximum rather than local optima. By accommodating continuous variations in solar irradiance, the proposed approach ensures robust policy updates that adaptively track the global peak under dynamically shifting conditions. ### Theoretical Justification of Critic Convergence

In order to provide a deeper theoretical foundation for the convergence properties of the critic network in our proposed Fuzzy-DDPG framework, we now state a lemma that formalizes the convergence of the critic under fuzzy approximation. This result builds upon classical results in reinforcement learning and function approximation, and it is instrumental in validating the effectiveness of our approach under the following standard assumptions. **Lemma 1** (Critic Convergence under Fuzzy Approximation). : _Let \(S\subseteq\mathbb{R}^{n}\) and \(A\subseteq\mathbb{R}^{m}\) be the continuous state and action spaces, respectively, and let \(Q^{*}:S\times A\to\mathbb{R}\) denote the unique fixed point of the Bellman operator_

\[Q(\phi(s),a;\theta_{Q}) \tag{14}\]8

_with \(s^{\prime}=f(s,a)\), where \(f\) represents the deterministic state transition function and \(\gamma\in[0,1)\) is the discount factor. Assume the following:_

1. _The Bellman operator_ \(T\) _is a contraction mapping, i.e.,_ \[Q(\phi(s),a;\theta_{Q}) \tag{14}\]9 (29)
2. _The critic network_ \(Q_{\theta_{c}}(s,a)\)_, parameterized by_ \(\theta_{c}\)_, is a universal approximator on_ \(S\times A\)_. In other words, for every_ \(\epsilon>0\)_, there exists a parameter set_ \(\theta_{c}\) _such that_ \[y_{i}=r_{i}+\gamma Q^{\prime}(\phi(s_{i+1}),\mu^{\prime}(\phi(s_{i+1})|\theta^{ \mu^{\prime}})|\theta^{Q^{\prime}})\]0 (30)
3. _The learning rates_ \(\{\alpha_{t}\}_{t\geq 1}\) _satisfy the standard Robbins-Monro conditions:_\[y_{i}=r_{i}+\gamma Q^{\prime}(\phi(s_{i+1}),\mu^{\prime}(\phi(s_{i+1})|\theta^{ \mu^{\prime}})|\theta^{Q^{\prime}})\]1
4. _The target network parameters are updated using a slow update factor_ \(\tau\in(0,1)\)_, ensuring that the target network evolves sufficiently slowly._
5.