## 3 Traditional Controllers: PID and MPC Closed-Loop Control Strategies
### Advanced Reinforcement Learning Using Deep Learning Neural Network Control Strategies  
This subsection presents the simulation results of the advanced reinforcement learning deep learning neural networks (RL DLNNs) developed in Section 4, conducted on the MATLAB Simulink R2024 software programming platform.  
Figure 14: MPC MATLAB Simulink simulations resulted in a closed loop for input (u1 = Ucom, u2 = uEXV, u3 = Trr) and output (y1 = Tch-sp, y2 = Level) responses against internal plant using mpcDesigner application based on the input-output measurement dataset for the MIMO CCS extended state model: (**a**) actuator control effort; (**b**) temperature disturbance; (**c**) chilled water temperature in evaporator; (**d**) liquid refrigerant level in condenser.  
The Section 5.2.1 refers to the Simulink simulation results for the RL DLNN that generated the reward function from an MPC dataset extracted from the MIMO CCS simplified model represented in state-space, given in Equation (7). Section 5.2.2 presents the simulations result for the RL DLNN that generated the reward function from two step responses' block specifications for the same MIMO CCS simplified model in state-space representation.  
2.1 Reinforcement Learning Deep Learning Neural Network Control Strategies-Generate Reward Function from MPC of MIMO Centrifugal Chiller Simplified Model in State-Space Representation  
The Simulink simulations results are shown in Figure 16a-c. Results for the RL DLNN control of evaporator chilled water temperature are displayed in Figure 16a, and Figure 16b represents the liquid refrigerant level in the condenser, both versus their corresponding MPC step responses on the same graphs. The Figure 16c illustrates the separate RL DLL MPC control efforts.  
2.2 Reinforcement Learning Deep Learning Neural Network Control Strategies--Generate Reward Function from Step Response Specifications of a MIMO Centrifugal Chiller Simplified Model in State-Space Representation  
The simulation results for RL DLNN control of evaporator chilled water temperature are depicted in Figure 17a, and Figure 17b illustrated the liquid refrigerant levels versus digital PID control of MIMO ANFIS model responses.  
Figure 16: RL DLNN—Generate reward function based on MPC specifications: (**a**) RL DLNN MPC evaporator temperature control versus MPC; (**b**) RL DLNN MPC condenser level control versus MPC; (**c**) RL DLNN MPC actuator efforts.  
Figure 17: MATLAB Simulink simulation results of RL DLNN versus digital PID MIMO ANFIS model generate reward function from the step response specifications of the MIMO Centrifugal Chiller simplified model in state-space representation. (**a**) RL DLNN of chilled water temperature inside the evaporator versus digital PID controller using ANFIS CCS models; (**b**) RL DLNN control of liquid refrigerant level within the condenser versus digital PID controller using ANFIS CCS models; (**c**) compressor actuator control effort, RL DLNN; (**d**) expansion valve opening actuator control effort, RL DLNN; (**e**) compressor actuator control effort, digital PID CCS ANFIS model; (**f**) expansion valve opening actuator control effort, digital PID CCS ANFIS model.  
The RL DLNN actuators control efforts for the compressor and for the expansion valve opening is illustrated in Figure 17c,d, respectively, while the control efforts of digital PID control CCS ANFIS models are shown in Figure 17e for the compressor actuator and Figure 17f for expansion valve opening actuator.