## 3 Traditional Controllers: PID and MPC Closed-Loop Control Strategies
### Deep Learning Neural Network
#### 4.3.1 Generate Reward Function from the Cost and Constraint Specifications Defined in an MPC Object  
This intelligent control strategy and the training phase results were implemented in MATLAB Simulink in Figure 9.  
Its compact and detailed architecture is presented in Figure 9a-d and is of practical interest. It automatically generates the reward function from the cost and constraint specifications defined in a Simulink model predictive controller (MPC) object, as shown in Figure 9e. It is connected in a closed loop with the MIMO centrifugal cooling plant model given by Equations (6) and (7) in Section 2.2 to control the chilled water temperature Tchw-sp in the evaporator subsystem and the level of liquid refrigerant in the second condenser subsystem, depicted in the state-space representation shown in Figure 9g. Implementing this control strategy follows the same procedure that is well described in the updated MathWorks documentation for MIMO Centrifugal Chiller System (CCS) model [18,19]. The MPC object appearing at the top of Figure 9b,c with its parameters block depicted in Figure 9e was created based on the CCS MIMO model, using an inline MATLAB code procedure. For the CCS MIMO plant model under investigation, the sampling time value is set to Ts = 0.1 s, the prediction horizon was assumed to be p = 10 steps, and the control horizon m = 2 steps. The following specifications of the MPC object are helpful for RL DLNN MPC strategy design:  
* Standard linear bounds for output variables (OVs) and manipulated variables (MVs): \[y_{min}=[0\;0],\;y_{max}=[10\;52],\;mv_{min}=0,\;mv_{max}=1.1,\;mv_{ ratemin}=-1000,\;\text{and}\;mv_{ratemax}=\text{Inf}\]
* Scale factors as specified for OVs and MVs: \[S_{y}=[10\;52],\;S_{mv}=[1.1\;1]\]
* Standard cost weights: \[Q_{y} =[0.1\;0.1],\;Q_{mv}=[0\;0],\;\text{and}\;Q_{nrate}=[0.1\;0.1]\] The reward function generated based on the MPC object specifications was the starting point for the reward design. The reward function is used to train the RL Agent, as shown in Figure 9f. The implementer can modify the reward function with different penalty function options and adjust the weights: The reward function requires the following two components to be computed: 1. The cost component, calculated according to the following equations: \[dy=(refy(:)-y(:))./{S_{y}}^{\text{T}}\] (29) \[dmv=(refmv(:)-mv(:))./{S_{mv}}^{\text{T}}\] (30) \[dmvrate=(mv(:)-lastmv(:))./{S_{mv}}^{\text{T}}\] (31) \[J_{y}=dy^{\text{T}}diag\big{(}Q_{y}.\hat{\ }2\big{)}dy\] (32) \[J_{mv}=dmv^{\text{T}}diag(Q_{mv}.\hat{\ }2)dmv\] (33) \[J_{nrate}=dmvrate^{\text{T}}diag(Q_{nrate\cdot}\hat{\ }2)dmvrate\] (34) \[y_{min}=[0\;0],\;y_{max}=[10\;52],\;mv_{min}=0,\;mv_{max}=1.1,\;mv_{ ratemin}=-1000,\;\text{and}\;mv_{ratemax}=\text{Inf}\]0 (35) 2. The penalty component for violation of linear bound constraints, with the following components:
* Penalty function weight (specify nonnegative): \[y_{min}=[0\;0],\;y_{max}=[10\;52],\;mv_{min}=0,\;mv_{max}=1.1,\;mv_{ ratemin}=-1000,\;\text{and}\;mv_{ratemax}=\text{Inf}\]1 (36) \[y_{min}=[0\;0],\;y_{max}=[10\;52],\;mv_{min}=0,\;mv_{max}=1.1,\;mv_{ ratemin}=-1000,\;\text{and}\;mv_{ratemax}=\text{Inf}\]2 Choose the **step*
* or **quadratic*
* penalty method to calculate the **exteriorPenalty**; Set the Pmv value to 0 if the RL Agent action specification has appropriate \"_LowerLimit_\" and \"_UpperLimit_\" values.