# Decision-Time Learning and Planning Integrated Control for the Mild Hyperbaric Chamber
## 2 Problem Formulation
This process is very similar to the concept of RL, which is often regarded as a computational method that learns through interaction with the environment. By employing trial-and-error, RL establishes an effective operational mechanism within a given environment. As previously discussed, RL algorithms are difficult to apply directly to real-world tasks due to data efficiency issues. The cost of trial-and-error learning in real-world scenarios can be prohibitively high. However, model-based reinforcement learning (MBRL) can mitigate this issue to some extent, as it is similar to human intelligence that construct an imaginary world in their minds and predict the potential consequences of different actions before executing them in reality [14]. In MBRL, there are generally two approaches  
Figure 3: Pipeline diagram of the MHBC. to utilizing the model. The first approach is to use model-simulated results for data augmentation. The model acts as a substitute for the real environment, generating simulated trajectories that are used to augment the training dataset and improve policy learning. Typical algorithms include Dyna-Q [15] and Model-Based Policy Optimization (MBPO) [16]. The second approach is to assist the learning algorithm in gradient training. White-box models are leveraged to directly optimize the learning process through gradient-based training. Typical algorithms include Guided Policy Search (GPS) [17] and Probabilistic Inference for Learning Control (PILCO) [18]. The dual-system theory in cognitive psychology proposes two distinct modes of thinking in human decision-making, known as System 1 and System 2 [19]. System 1 relies on intuition for fast thinking, characterized by speed, emotion, and unconscious processing. In contrast, System 2 relies on slow thinking for reasoning, characterized by deliberation and logical inference. In sequential decision-making, model-free and model-based reinforcement learning often exhibit characteristics similar to System 1, which we refer to as background planning. For background planning, the objective is to answer the question: \"How should I learn to act optimally in any possible situation to achieve my goal?\" Conceptually, background planning can be understood as learning reusable behavioral patterns, resembling a fast-thinking process. On the other hand, model-based planning resembles the characteristics of System 2, which we refer to as decision-time planning. Decision-time planning seeks to answer: \"Given my current situation, what is the optimal sequence of actions to achieve my goal?\". This approach can be interpreted as real-time conscious decision-making, resembling a slow-thinking process. For MHBC control, we compare advantages and disadvantages with background planning and decision-time planning from perspectives:  
1. Decision-time planning generates actions based on the latest state, considering the most up-to-date environmental conditions. In contrast, background planning relies on a pre-trained policy, which may become outdated and require possible retraining upon changing system configuration. In MHBC control, decision-time planning enables adaptive planning based on current conditions, making it more responsive to sudden environmental changes. 2. Decision-time planning does not require prior policy learning to take action. As long as an environment model is available, planning can be conducted immediately. In contrast, background planning relies on a policy trained in advance. In MHBC control, decision-time planning directly leverages the environment model trained from previously collected operational data, eliminating the need for further policy learning, which could possibly introduce extra compounding error. 3. Decision-time planning performs better in environments that are not visited before.