## 3 Traditional Controllers: PID and MPC Closed-Loop Control Strategies
### Deep Learning Neural Network
#### 4.3.1 Generate Reward Function from the Cost and Constraint Specifications Defined in an MPC Object
A set of agent options is specifiedto train the agent from an experience buffer of maximum capacity 1 \(\times\) 10\({}^{6}\) by randomly selecting mini-batches of size 256. It has been reported that a discount factor of 0.995 favors long-term rewards [20; 21]. The optimizer options are specified for the actor and critic functions: a learning rate of 1 \(\times\) 10\({}^{-3}\) and a gradient threshold of 1, as set out in [20; 21]. During training, the agent explores the action space using a Gaussian noise model of action. The standard deviation and decay rate of the noise are set by using an ExplorationModel property, as is shown in [20; 21]. The RL Agent was trained using _train_ function, as depicted in Figure 9k; after 13 epochs, it is produced the best tracking performance for chilled water temperature inside the evaporator and for liquid refrigerant level within the condenser. Figure 9l also illustrates the results of the training process after 262 epochs, when the tracking performance of the liquid refrigerant level in condenser was at its best, while that for the chilled water temperature inside the evaporator was slightly attenuated compared with the first result obtained, as shown in Figure 9k. A snapshot of the main steps of the MPC RL DLNN Algorithm A1 is presented in Appendix B.1. The layer graphs of criticNet and actorNet are displayed in Figures 10a,b, respectively. 3.2 Reinforcement Learning Deep Learning Neural Network Control Strategy--Generate Reward Function from a Step Response Specifications of MIMO Centrifugal Chiller Simplified Model in State-Space Representation  
Both The digital PID and RL DLNN closed-loop control strategies are depicted in Figure 11a in a compact Simulink diagram architecture. A description in state space of MIMO CCS model is provided in Figure 11b. Figure 11c shows the block parameters of RL Agent1, and Figure 11d details the observations block as a component of the same compact Simulink diagram. The compact Simulink diagram also shows how to automatically generate two reward functions--rewardfunctionVfb1 and rewardfunctionVfb2, shown in Figure 11e,g from the performance requirements defined in the Simulink Design Optimization model verification blocks provided in MATLAB R2024a Simulink Toolbox for evaporator temperature control (chosen as the first agent, RL Agent1) and condenser liquid refrigerant level control (the second agent, RL Agent2), respectively. Also in the compact Simulink diagram appear two Boolean logic termination blocks, which stop the training of the RL Agent1 and RL Agent2 when the evaporator temperature and the condenser level reach good accuracy performance, as shown in Figure 11f,h. The rewardfunctionVfb1 and rewardfunctionVfb2 are used to train both reinforcement learning RL Agent1 and RL Agent2, following the stated steps and the MATLAB Reinforcement Learning subroutine procedure similar to those developed in [20; 21] applied for MIMO plants. The training of both agents, RL Agent1 and RL Agent2, is performed similarly to the MPC RL DLNN  
\[y_{min}=[0\;0],\;y_{max}=[10\;52],\;mv_{min}=0,\;mv_{max}=1.1,\;mv_{ ratemin}=-1000,\;\text{and}\;mv_{ratemax}=\text{Inf}\]9 Layer graphs for (**a**) criticNet; (**b**) actorNet. procedure described above in Section 4.3.1. Compared with MPC RL DLNN, both RL agents are decentralized with two distinct paths. The untrained simulation results for evaporator temperature control and condenser liquid refrigerant level control are presented in Figure 11i, respectively. Figure 11k,l presents the training simulation results for the evaporator temperature and liquid refrigerant level inside of the condenser after 200/200 epochs. As shown in Figure 11m, the training process of both RL Agent1 and RL Agent2 finishes after the agents reach the criteria to stop training set out in the training options MATLAB subroutine presented in the print screen snapshot of Algorithm A2 from Appendix B.1. The penalty function weight is specified nonnegative and set to the value 2.