# Decision-Time Learning and Planning Integrated Control for the Mild Hyperbaric Chamber
## 5 Performance Evaluation
### Control Planning Evaluation  
We evaluate the control planning results by comparing the algorithms in terms of average score and sample efficiency. The compared algorithms include: (1) PID: the proportional-integral-derivative control algorithm; (2) CEM-SEP: model-based decision-time planning algorithm using cross-entropy method with separated uncertainty measurement; (3) TD-MPC: extension of CEM-SEP with temporal difference model predictive control; and (4) VN-DTP (Value Network guided Decision-Time Planning): our further proposed approach that utilizes value network and online policy to guide decision-time planning. The hyperparameter settings of are summarized in Table 2. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Discount factor \((\gamma)\) & 0.99 \\ Initial steps & 10 \\ Planning horizon & 30 \\ Initial parameters \((\mu^{0},\sigma^{0})\) & (0, 2) \\ Population size & 128 \\ Elite ratio & 16 \\ Iterations & 6 \\ Shrink ratio & 5\% \\ Number of particles & 1 \\ Momentum coefficient & 0.1 \\ Learning rate & \(3\times 10^{-4}\) \\ Optimizer \((\theta)\) & Adam \((\beta_{1}=0.9,\beta_{2}=0.999)\) \\ Temporal coefficient \((\lambda)\) & 0.5 \\ Exploration rate \((\epsilon)\) & \(0.5\to 0.05\) \\ Planning horizon scheduling & \(1\to 5\) \\ Batch size & 128 \\ Momentum coefficient & 0.99 \\ Gradient updates per step & 1 \\ \(\theta\) update frequency & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameter settings for TD-MPC and VN-DTP. Figure 8: Performance profile of different model outputs. Figure 9 presents the average scores of the algorithms. From the metrics, it is evident that applying decision-time planning algorithms results in significant performance improvements compared to the PID control algorithm. Specifically, CEM-SEP outperforms PID by approximately 7.8%. TD-MPC further improves upon CEM-SEP by an additional 2.3% through the incorporation of a value network, which enhances modeling of long-term returns and facilitates better trade-offs between short-term and long-term rewards. VN-DTP achieves the best performance, surpassing TD-MPC by a further 0.4%, attributed to its integration of an online policy during planning. Ultimately, VN-DTP achieved a performance improvement exceeding 10.5% compared to the commercial PID algorithm applied in MHBC. Figure 10 compares the asymptotic and convergence performance of different decision-time planning algorithms. It can be observed that, compared to the baseline CEM-SEP planning algorithm, incorporating value networks leads to significant improvements in both progression and convergence. The VN-DTP algorithm exhibits a notably faster asymptotic rate within the first 4000 timesteps, followed by a steady convergence in the later stages of training. We also design experiments to evaluate the algorithm's performance when the system is facing abnormal conditions. We consider three types of abnormal scenarios:  
1. Valve calibration offset: Simulated by introducing small random disturbances into the valve calibration values. Figure 10: Asymptotic and convergence performance of different control methods. Figure 9: VN-DTP outperforms other methods in average score with 95% CI of different control algorithms. Value failure: Simulated by either setting the valve completely closed or fixing it at a specific opening, with the valve potentially reporting correct or incorrect feedback to the control system. 3. Chamber leakage: Simulated by introducing unexpected additional valve openings. More specifically, the above abnormal conditions are categorized into the following six scenarios: 1. Valve calibration offset: Originally, the valve opens at a set-point of 5%; due to prolonged usage, it now opens only when the input reaches 6%. 2. Valve permanently closed with correct feedback: Valve remains fully closed regardless of inputs, correctly reporting this closed status. 3.