# Decision-Time Learning and Planning Integrated Control for the Mild Hyperbaric Chamber
## 4 Control Planning
### Decision-Time Planning with Model Predictive Control  
Model Predictive Control (MPC) is an iterative, model-based control method [25]. We apply MPC for decision-time planning, where at each time step, an optimization problem is iteratively solved to obtain an optimal action sequence. The first control input of this sequence is applied to the system, and the process is repeated at the next time step using the system's updated state. The optimization objective of decision-time planning based on model predictive control is defined as follows:  
\[\max_{a_{0},\ldots,a_{H-1}}\sum_{t=0}^{H-1}R(s_{t},a_{t}) \tag{21}\]  
where \((s_{t},a_{t})\) represents the state and control input at time step \(t,H\) denotes the prediction horizon, and \(R(s_{t},a_{t})\) is the reward function. The control input \(a_{t}\) is constrained within the range \(a_{\text{min}}\leq a_{t}\leq a_{\text{max}}\), where \(a_{\text{min}}\) and \(a_{\text{max}}\) represent the lower and upper bounds of the control input, respectively. These constraints ensure that the mild hyperbaric chamber con trol valve operates within a permissible range. MPC adopts a receding horizon approach, allowing the controller to dynamically adapt to changing conditions and disturbances at decision time. The control task is optimized by a sequence reward \(R(s_{t},\hat{a})\). Starting from state \(s_{t}\), we apply an action sequence \(\hat{a}\) to predict the future trajectory over a horizon \(H\). The expected cumulative reward is defined as the sum of the expected rewards of all particles \(s\) within the planning horizon \(H\) along the trajectory:  
\[R(s_{t},\hat{a})=\sum_{\Delta t=1}^{H}\frac{1}{K}\sum_{k=1}^{K}R\left(s_{t+ \Delta t}^{k},a_{t+\Delta t}\right) \tag{22}\]  
For action selection, we employ the Cross-Entropy Method (CEM) [26] to optimize the action sequence \(\hat{a}\) for reward maximization. To account for uncertainty in decision-time planning, the planner must avoid unpredictable regions in the state space, reducing their likelihood of selection. In the previous chapter, we designed methods to separately model aleatoric uncertainty and epistemic uncertainty for output optimization. Here, we incorporate different uncertainty and define negative feedback penalty of aleatoric uncertainty as follows:  
\[R_{A}(s_{t},\hat{a})=-w_{A}\cdot\sum_{\Delta t=1}^{H}\sqrt{\frac{1}{K}\sum_{k =1}^{K}\Sigma_{\theta}^{k}(s_{t},a_{t})} \tag{23}\]  
where \(\Sigma_{\theta}^{k}(\tilde{s}_{t},a_{t})\) represents the predicted covariance matrix at the next time step \(t+1\). The term \(w_{A}>0\) is a weighting coefficient, where higher uncertainty will result in lower reward. To guide the exploration, we define the positive feedback reward for epistemic uncertainty as follows:  
\[R_{E}(s_{t},\hat{a})=w_{E}\cdot\sum_{\Delta t=1}^{H}\sqrt{\text{Var}\left[\mu_ {\theta}^{k}(\tilde{s}_{t},a_{t})\right]+\text{Var}\left[\sum_{\theta}^{k}( \tilde{s}_{t},a_{t})\right]} \tag{24}\]  
where Var represents the variance across \(k=1,\ldots,K\) ensemble models, and \(w_{E}>0\) is a weighting coefficient.