## 3 Wavefront Sensorless AO-RL Environment
### Action Space  
In the context of AO, the actions of the RL agent involve movements of the actuators located beneath the deformable mirror, as shown in Figure 3. The number of actuators determines the degree of freedom of the mirror's shape. The actuators are responsible for controlling the continuous reflective surface of the deformable mirror. The range of movement for each actuator is \(\pm 5\)\(\mu\)m, providing a high degree of precision in the mirror's surface shape and position.  
According to [PERSON] [6], the wavefront is a 2D map of the phase at a plane that is normal to the line of sight from the origin of the beam to the target. In the field of AO, multiple methods are utilized to represent this 2D wavefront map, including the power-series representation [6] and the Zernike series [47; 48; 49]. An effective approach to analyzing phase aberrations involves decomposing the wavefront into a series of  
Figure 2: Atmospheric turbulence conditions with respect to \(D/r_{0}\).  
a complete set of orthogonal polynomials defined over a circular pupil [47; 50]. This series comprises sums of power series terms with appropriate normalizing factors [49; 51].  
Our RL environment presents two choices within the action space: the number of actuators or Zernike polynomials [47]. Given our available setup, the action space can be configured either directly with 64 actuators or with deformable mirror Zernike polynomials truncated at the first six modes (second radial orders) in a circular geometry. The decision to use six-mode Zernike polynomials stemmed from experiments conducted with Shack-Hartmann wavefront sensor-based AO. These experiments demonstrated that employing six modes of Zernike polynomials is adequate to achieve a Strehl ratio of more than 50% under average atmospheric turbulence conditions (where \(D/r_{0}<5\)).  
In the former case, a 64-actuator segmented deformable mirror is simulated in a quasi-static environment, with eight actuators along each linear dimension. As a result, the RL agent is tasked with selecting actions from a 64-dimensional action space, where each actuator can be independently and continuously adjusted within a certain motion range. This setup allows for precise and smooth control of the deformable mirror. The actuation has a limit corresponding to the maximum optical phase error that is possible under the atmospheric conditions used in training. High-dimensional action space can potentially pose difficulties for RL algorithms in non-static environments due to the curse of dimensionality. For this reason, within our RL environment, we implemented the dimensionally reduced action space based on the Zernike series [47], which can be obtained from the observation space representing the wavefront.  
It is assumed that the deformable mirror operates at a sufficiently fast speed to allow for approximating the atmosphere as quasi-static, given that most deformable mirrors are capable of correction at speeds of up to 1-2 kHz [52]. Faster deformable mirrors can be achieved using smaller mirrors. For a 50 cm telescope and this deformable mirror choice, the system can be expected to achieve reasonable correction for turbulence conditions of less than \(r_{0}=6.25\). We expect \(r_{0}\) conditions to range from 5 cm to 15 cm for satellite elevation angles above 15\({}^{\circ}\).