# Thrust and Pressure Control in a Solid Propulsion System via Reinforcement Learning
## 1 Introduction  
The attitude and divert propulsion system (ADPS) is a propulsion system in aerospace engineering designed to adjust spacecraft attitude, ensuring three-axis stability and enabling orbital maneuvers [1]. ADPS functions by ejecting gas through multiple nozzles; attitude thrust nozzles at the spacecraft's rear control pitch, yaw, and roll, while divert thrust nozzles at the center of mass provide lateral thrust [2]. ADPS types include the liquid attitude and divert propulsion system (LADPS) and the solid attitude and divert propulsion system (SADS), each with specific advantages and disadvantages tailored to particular spacecraft design objectives and performance requirements [3]. The LADPS uses liquid oxidizer and fuel as propellants, allowing for repeated thrust start and stop and providing a higher specific impulse. However, it requires complex fuel storage and delivery systems that need regular maintenance [4]. In contrast, the SADPS uses solid oxidizers and fuels, which eliminates the need for complex pipeline delivery systems, offering ease of maintenance, safety, and storage. Variable thrust in the SADPS is achieved by altering the throat area of the nozzle in real time [5]. The precise control of nozzle openings and the propellant combustion rate enables pressure and thrust regulation in the combustion chamber [6]. However, the shared combustion chamber among multiple nozzles and the pressure-dependent combustion rate result in a gradual increase in free volume, necessitating intricate control algorithms for safe and reliable operation. The dynamics of the solid attitude and divert propulsion system (SAPS) are characterized by nonlinear and time-varying behavior due to nozzle control, pressure regulation, and thrust management. These characteristics make traditional control methods like proportional integral derivative (PID) inadequate for meeting performance standards [7]. [PERSON]'s review examines closed-loop pressure control methods in rocket propulsion, typically using linearized models around specific operating points to derive steady-state controllers, predominantly of the PID type. [PERSON] emphasizes that future control strategies must robustly achieve the desired thrust levels. Potential solutions include enhanced nonlinear approaches and optimal closed-loop control [8]. [PERSON] designed a pressure control system for pintle solid rocket motors using parameter identification through adaptive control methods, demonstrating significant advantages in terms of stability, robustness, and precision [9]. [PERSON] and [PERSON] introduced an adaptive matching technique that adjusts pre-designed controllers based on evolving system dynamics [10], while [PERSON] and [PERSON] proposed adaptive controllers to mitigate uncertainty effects [11]. [PERSON] proposed a data-driven method for early warnings for thermoacoustic instabilities in rocket engine thrust chambers, successfully using machine learning to predict these instabilities in advance [12]. Previous research has extensively investigated spacecraft propulsion, aiming to minimize fuel consumption for effective thrust command allocation and stable attitude control [13; 14; 15]. [PERSON] and [PERSON] applied adaptive feedback linearization techniques to achieve pressure regulation and conducted extensive work on five-axis thrust distribution [16; 17; 18]. The integration of artificial intelligence into control systems has led to significant advancements, with reinforcement learning (RL) emerging as a prominent approach for addressing sequential decision problems [19]. RL offers advantages such as learning from experience, flexibility, autonomous decision-making, robustness, and resource efficiency [20].