# Soft Fuzzy Reinforcement Neural Network Proportional-Derivative Controller
## 4 Results and Discussion  
### Experimental Setup  
Our algorithm was tested on two systems: the pendulum and the Continuous Stirred Tank Reactor (CSTR). The pendulum was chosen because it is a classic nonlinear system with unstable dynamics, making it a standard benchmark for control algorithms. The CSTR was selected due to its large control input space, requiring extensive exploration, and its strong nonlinear characteristics, which pose significant control challenges. These two systems ensured our algorithm was tested in both mechanical and process control domains. The SFPD controller used in the simulations was implemented in Python 3.9. The pendulum system leveraged the gym-pendulum-v1 environment from Python's Gym library, while the CSTR model was built using MATLAB/Simulink R2023a. The simulations were performed on a personal computer with an Intel(R) Core(TM) i7-12700H processor (2.30 GHz), 64 GB of RAM, and a 64-bit operating system. The proposed controller exhibited low memory usage (peak \(<\) 0.4 MB), demonstrating its memory efficiency. While edge deployment was not the focus of this study, previous works (e.g., [PERSON] et al. [38]) have demonstrated the feasibility of similar lightweight controllers on embedded hardware such as FPGAs.  
The commonly used metrics IAE (Integral of Absolute Error) and ISE (Integral of Squared Error) were adopted to evaluate the performance of the control system [39]. These metrics help quantify the deviation between the desired output (setpoint) and the actual system response. The definitions of IAE and ISE are shown in (17).  
\[IAE=\sum_{k=0}^{n}|e(k)| \tag{17}\] \[ISE=\sum_{k=0}^{n}e^{2}(k)\]  
where \(n\) denotes the total number of steps, and \(e(k)\) represents the system error at step \(k\).  
The fuzzy neural network utilized seven fuzzy memberships [40] for each input, leading to seven means and standard deviations for both the error and the error rate, resulting in a total of 28 trainable parameters in the membership layer. The choice of seven memberships ensured a balance between accuracy and computational efficiency, providing sufficient granularity to capture nonlinear relationships while avoiding excessive complexity. Seven memberships were sufficient to capture the underlying nonlinear relationships in the data, offering adequate granularity without introducing excessive model complexity. This ensured the network could generalize well while keeping computational costs manageable. Additionally, a fully combinatorial fuzzy rule design (also known as the Rule Cartesian Product) was adopted, where each of the seven input membership functions for the two input variables was paired to form \(7\times 7=49\) fuzzy rules. These rules were directly mapped to 49 corresponding weights in the defuzzification layer, enhancing the network's generalization ability while keeping computational costs low. To assess the robustness of the method, we initialized the centers (means) of the Gaussian membership functions uniformly within their respective value ranges, while the variances were fixed at 0.5, ensuring that the initialization was adaptable across different systems. Table 1 summarizes the hyperparameter settings.  
The comparison methods in this paper included SAC and traditional PD algorithms. SAC was chosen as a baseline since the proposed method built upon it, enabling a direct comparison to highlight improvements in convergence and exploration efficiency in large action spaces. The traditional PD algorithm served as a benchmark to demonstrate the benefits of dynamic PD parameter adjustment, which enhances adaptability and robustness while eliminating the need for manual tuning [15].