# Decision-Time Learning and Planning Integrated Control for the Mild Hyperbaric Chamber
## 3 Model Learning
### Model Output Optimization with Separated Uncertainties  
Since the actual operation of MHBC exhibits stochastic characteristics, the environment model should be formulated as a stochastic dynamic system to accurately capture the inherent uncertainties within the process. The uncertainties normally include aleatoric uncertainty and epistemic uncertainty. Aleatoric uncertainty arises from the inherent randomness of the environment, reflecting the system's intrinsic stochastic nature. Epistemic uncertainty, on the other hand, arises from a lack of information and an insufficient ability to generalize beyond observed data, particularly in regions where data is sparse. The Probabilistic Ensembles with Trajectory Sampling (PETS) [23] algorithm is a prominent method for incorporating model uncertainty into outputs. PETS leverages ensemble models to quantify both types of uncertainty simultaneously. In this study, we separately express aleatoric uncertainty and epistemic uncertainty, and finally integrate them into the planning process. We consider using a Bayesian Neural Network (BNN) to capture aleatoric uncertainty. BNNs are based on Bayesian statistics and are trained using prior distribution knowledge [24]. We integrate an \(a\)-divergence minimization method to effectively capture aleatoric uncertainty in the BNN function. The state transition model defined in Equation (1) can be expressed as follows:  
\[y_{t}=f(x_{t};\mathcal{W})+\epsilon_{t} \tag{10}\]  
where \(x_{t}\) is the system's input value, \(y_{t}\) is the output value, \(f(\cdot;\mathcal{W})\) represents the neural network function parameterized by weight matrix \(\mathcal{W}\), and \(\epsilon_{t}\) is an additive noise term. To enhance the model's ability to capture aleatoric uncertainty, we introduce input perturbation in the neural network:  
\[y_{t}=f(x_{t},z_{t};\mathcal{W})+\epsilon_{t} \tag{11}\]  
where \(z_{t}\sim\mathcal{N}(0,\gamma)\) represents random perturbation drawn from a normal distribution. The role of \(z_{t}\) is to capture aleatoric uncertainty in the network's output. The neural network consists of \(L\) layers, where the \(l\)-th layer contains \(M_{l}\) hidden units. The weight matrices \(\mathcal{W}=\left\{\mathbf{W}_{l}\right\}_{l=1}^{L}\) include \(M_{l}\times(M_{l-1}+1)\) weight matrices, each containing an additional bias term. Figure 4: The proposed state context representation model. Let \(\mathbf{X}\) be a \(T\times D\) matrix containing system input values \(x_{t}\), and \(\mathbf{Y}\) be a \(T\times K\) matrix containing system output values \(y_{t}\). The vector \(\mathbf{z}\) represents a \(T\)-dimensional random variable, including random terms \(z_{1},\cdots,z_{T}\). The likelihood function is then defined as follows:  
\[p(\mathbf{Y}\mid\mathcal{W},\mathbf{z},\mathbf{X})=\prod_{i=1}^{T}p(y_{t}\mid \mathcal{W},\mathbf{z},x_{t})=\prod_{t=1}^{T}\prod_{k=1}^{K}\mathcal{N}(y_{t,k }\mid f(x_{t},z_{t};\mathcal{W}),\Sigma) \tag{12}\]  
The output is influenced by the additive noise variable \(\epsilon_{t}\sim\mathcal{N}(\mathbf{0},\mathbf{\Sigma})\) with diagonal covariance matrix \(\mathbf{\Sigma}\). For each \(\mathbf{z}\), the prior distribution follows a Gaussian distribution: \(\mathcal{N}(0,\gamma)\). Similarly, each element of the weight matrix \(\mathcal{W}\) also follows a Gaussian prior distribution:  
\[p(\mathbf{z})=\prod_{t=1}^{T}\mathcal{N}(z_{t}\mid 0,\gamma),p(\mathcal{W})= \prod_{t=1}^{L}\prod_{i=1}^{M_{t}}\prod_{j=1}^{M_{t+1}}\mathcal{N}\Big{(}w_{ ij,l}\mid 0,\lambda\Big{)} \tag{13}\]  
where \(w_{ij,l}\) is the \(i,j\)-th element in the matrix \(\mathbf{W}_{l}\), and \(\gamma\) and \(\lambda\) are the variances of the respective prior distributions.