## 2 Methods
### Setting of Indoor Temperature and Humidity
achieve the control effect of comfort and energy-saving; therefore, the integrated reward is set as the sum of energy consumption and thermal comfort reward, the comforting reward was set by combining _PMV-BPD_ and _PMV-PPD_ models, and the _PMV-PPD_ relationship is shown in Figure 3.  
The ASHRAE Standard 55-2020 [6] sets the human comfort range to \(-0.5<PMV<0.5\), but there is still a 20% thermal dissatisfaction rate due to the difficulty of achieving a 90% thermal satisfaction rate in actual studies so that the thermal comfort value for the user is between [\(-1\), \(+1\)] as much as possible. Design thermal comfort reward for the group thermal dissatisfaction rate brings the reward value, for the comforting reward, as far as possible to make the indoor group thermal dissatisfaction rate value close to the thermal comfort for [\(-1\), \(+1\)] corresponding to the thermal dissatisfaction rate value of 26%, and the closer the group thermal dissatisfaction rate is to 26%, the greater the reward value, otherwise the system will get the penalty value and thermal comfort reward as shown in Equation (10). \[R_{comfort}=\left\{ \begin{array}{l}-BPD_{t},BPD_{t}>26\%\\ 26\%-BPD_{t},BPD_{t}<26\%\end{array} \right., \tag{10}\]  
where \(R_{comfort}\) is the reward value of the environmental state for \(S_{t}\) at moment \(t\). In the indoor thermal environment regulation model of comfort and energy-saving, the default comfort and energy-saving are equally important. The energy-saving effect reward referred to the reward brought by the room temperature setting value. To make the output value of the deep deterministic strategy gradient learning system more reasonable and reliable and reduce the number of controller learning, the rule of room temperature setting value change was added, when \(BPD_{t}>26\%\), it indicated that the user feels colder or hotter in the current indoor thermal environment, and the indoor temperature and humidity should be adjusted reasonably at this time. In summer conditions, the higher the room temperature setting value and the smaller the cooling load within the indoor comfort zone, the lower the energy consumption of the air conditioning system, and in winter conditions, the lower the room temperature setting value and the smaller the heating load within the indoor comfort zone, the lower the energy consumption of the air conditioning system. Therefore, to make the adjusted room temperature setting value as close as possible to the current moment room temperature, the greater the bonus value obtained by the system, the energy saving bonus as shown in Equation (11). \[R_{\text{energy}}\,=\,\left\{ \begin{array}{l}1-\left|T_{\text{set}}\,-\,T_{ r}\right|,\left|T_{\text{set}}\,-\,T_{r}\right|\leq 1\\ -\left|T_{\text{set}}\,-\,T_{r}\right|,\left|T_{\text{set}}\,-\,T_{r}\right|>1 \end{array} \right.. \tag{11}\]  
Figure 3: _PMV-PPD_ relationship curve. Based on the above analysis, it is determined that the total reward of the DDPG-based indoor thermal environment regulation method is the sum of the comfort reward and the energy saving reward, and the design comfort and energy saving are equally important in this study, and the calculation formula is shown in Equation (12). \[R_{sum}=R_{comfort}+R_{energy}. \tag{12}\]
3. Cost minimization: The goal of indoor thermal environment regulation is to balance the relationship between thermal comfort and energy consumption, get the maximum reward value in return, and obtain the optimal setting value of indoor temperature and humidity. The expression is shown in Equation (13). \[\max_{i}\sum_{t^{\prime}=0}^{\infty}\gamma^{t^{\prime}}R_{t+t^{\prime}}(S_{t+t ^{\prime}},A_{t+t^{\prime}}). \tag{13}\]