# Decision-Time Learning and Planning Integrated Control for the Mild Hyperbaric Chamber
## 5 Performance Evaluation
### Model Learning Evaluation  
First, we evaluate the effectiveness of state context representation learning in our model. Regarding the neural network architecture, we use Linear \((M,N)\) to denote linear layer with \(M\) input features and \(N\) output features, ReLU\(()\) to represent the rectified linear unit activation function, Tanh () to indicate the hyperbolic tangent activation function, and Softplus () to denote smoothed rectified linear unit activation function with additive bias. As described in Section 5.1, the system has a state dimension of \(D_{S}=2\), and an action dimension of \(D_{A}=8\). We define the latent state dimension in GRU as \(D_{Z}=20\), and a state context representation dimension as \(D_{C}=20\). The proposed neural network architecture is as follows:  
State encoder:  
\([\text{Linear}(D_{S},50);\text{ReLU}();\text{Linear}(50,D_{Z});\text{Tanh}()]\)  
State decoder:  
\([\text{Linear}(D_{Z},50);\text{ReLU}();\text{Linear}(50,D_{S})]\)  
Action encoder for GRU:  
\([\text{Linear}(D_{A},50);\text{ReLU}();\text{Linear}(50,D_{Z});\text{ReLU}()]\)  
Context representation transformation from the past three states and actions information:  
\([\text{Linear}(3\times(D_{S}+D_{A}),50);\text{ReLU}();\text{Linear}(50,D_{C}); \text{ReLU}()]\)  
Context representation encoder for GRU:  
\([\text{Linear}(D_{C},50);\text{ReLU}();\text{Linear}(50,D_{Z});\text{ReLU}()]\)  
We focus on algorithm research rather than investigating variations in the internal structure of neural networks. Therefore, the basic neural network structure remains unchanged across different algorithmic comparisons. The architectures of the Multilayer Perceptron (MLP) network and Long Short-Term Memory (LSTM) network used for comparison are as follows:  
MLP:  
\([\text{Linear}(D_{S}+D_{A},50);\text{ReLU}();\text{Linear}(50,D_{Z});\text{ ReLU}();\text{Linear}(D_{Z},50);\)  
ReLU(); Linear\((50,D_{S})\)]  
Input and Output of the LSTM:  
\([\text{Linear}(D_{S}+D_{A},50);\text{ReLU}();\text{Linear}(50,D_{Z});\text{ ReLU}();\text{Linear}(D_{Z},50);\)  
ReLU(); Linear\((50,D_{S})\)]  
Recursion of the LSTM:  
\([\text{Linear}(D_{Z},50);\text{ReLU}();\text{Linear}(50,D_{Z})]\)  
Under these neural network architectures, we compare our proposed state context representation (denoted as SCR) method with models learned using MLP and LSTM. The evaluation is conducted on average score and sample efficiency [28]. For the average score, we assess the performance using the following four metrics: (1) Median: The middle value of the performance scores across multiple runs. (2) Interquartile Mean (IQM): The mean score calculated after discarding the lowest 25% and highest 25% of results, retaining only the middle 50%. (3) Mean: The arithmetic average of all performance scores. (4) Optimality Gap: The difference between the algorithm's performance and the predefined optimal performance threshold. The optimal performance threshold is established based on a predefined criterion for satisfactory control performance, beyond which further improvements yield diminishing effects. Specifically, we define this threshold as pressure control error below 5 Pa. For sample efficiency, we analyze how the objective function value changes during the training process, providing insights into the algorithm's data requirements. Figure 5 presents the average scores with 95% Confidence Interval (CI) of different model predictions. From the metrics, it can be observed that after incorporating LSTM and SCR, the average score of the algorithms are improved by approximately 6.8% and 9.2%, respectively, compared to MLP. This improvement is attributed to the ability of recursive structures to capture long-term dependencies in temporal sequences. By effectively retaining hidden state information in partially observable Markov decision processes, these methods enable the model to make more accurate inferences about the current state based on historical observations.