# Thrust and Pressure Control in a Solid Propulsion System via Reinforcement Learning
## 2 Model and Algorithm  
### Solid Attitude and Divert Propulsion System (SADPS)  
As shown in Figure 1, the four divert thrust nozzles are arranged at the spacecraft's center of mass, generating thrust in the \(y\)-axis and \(z\)-axis directions. Additionally, six attitude thrust nozzles are positioned at a certain distance from the center of mass, providing pitch, yaw, and roll control along the three axes.  
A and B are cross-sectional diagram symbols, Ai (i = 4-9) and Dj (j = 0-3) are engine serial numbers respectively. The SADPS operates in three modes: power-saving mode, performance mode, and overload mode.  
* **Performance mode** (Figure 2) requires a medium level of thrust to adjust the spacecraft's orbit and pitch angle.
* **Power-saving mode** minimizes propellant loss by reducing pressure as much as possible while maintaining flight attitude and compensating for gravity.
* **Overload mode** necessitates rapid changes in the spacecraft's trajectory and flight attitude.  
Typically, chamber pressure levels are set to constant values to ensure stability and predictability in performance.  
From time \(t_{0}\) to \(t_{1}\), the SADPS operates in performance mode, during which the spacecraft adjusts its attitude and changes its orbit to aim at the target. From time \(t_{1}\) to \(t_{2}\)  
Figure 1: Nozzle layout of the solid attitude and divert propulsion system.  
Figure 2: Three operating states of the solid attitude and divert propulsion system. Ai (i = 4–9) and Dj (j = 0–3) are engine serial numbers respectively.  
the SADPS switches to power-saving mode, maintaining orbital altitude and overcoming Earth's gravity. From time \(t_{2}\) to \(t_{3}\), the engine enters overload mode, performing the final rapid maneuvers to correct any deviation from the target.  
The control system model is depicted in Figure 3. The gas generator primarily produces the working medium required for thrust. The hot gas can be expelled simultaneously through four divert thrust nozzles and six attitude thrust nozzles. The pressure controller regulates the gas generator's pressure and provides the total opening command for the nozzles. The thrust controller allocates the opening of different nozzles and, within the constraints of the total nozzle opening, generates the required five-axis thrust. The opening controller drives the motor to rotate according to each valve's opening instructions, completing the closed loop of the nozzle operation. Through the coordinated hierarchical work of the three controllers, the switching of the engine's operating modes and the output of thrust are achieved.  
Each nozzle's aperture size is adjusted by a throat plug-type gas valve. This valve comprises a plug-shaped element located in the center of the valve throat. When this element interacts with the inner wall of the nozzle, it forms an annular gas channel known as the equivalent throat area. By altering the position of the throat plug, the equivalent throat area is adjusted, thereby controlling the nozzle's output thrust [25]. The throat plug regulating valve consists of several components, including a brushless DC motor, a decelerator, a valve body, a nozzle, thermal protection components, and transmission components. The throat plug is driven by the rotary motor through a rotary-to-linear transmission mechanism, causing it to reciprocate horizontally.