## 7 Conclusions  
We presented a cost-effective RL-based approach for wavefront sensorless AO in optical satellite communication downlinks. To gain a comprehensive understanding of the performance of the RL algorithms, we developed and shared the first optical satellite communications downlink RL environment. By using this RL environment, we implemented off-policy algorithms like SAC and DDPG, along with an on-policy algorithm, PPO, to optimize the coupling of 1550 nm light into a single-mode fiber under varying turbulence conditions. In a quasi-static environment, our results showed that the PPO algorithm is particularly effective in achieving a high average correction with a low number of training iterations. Under atmospheric turbulence conditions with \(D/r_{0}=2\) and 5, we observe that the PPO algorithm achieves performance close to 90% of Shack-Hartmann wavefront sensor-based AO with 12 lenslets across the aperture diameter. In more turbulent conditions, such as \(D/r_{0}=10\), PPO outperforms the Shack-Hartmann wavefront sensor-based AO by approximately 10% Strehl ratio. This was accomplished while working with a low-dimensional observation space and a high-dimensional action space. This approach eliminates the need for wavefront sensor measurements, thereby reducing the cost and latency of optical satellite communications downlinks. Towards the development of a real-world dynamic environment, we implemented a semi-dynamic environment, which can be helpful for initializing agents' weights for dynamic scenarios. The results of our experiment on a semi-dynamic environment using a quasi-static configuration with the PPO algorithm showed that the latter is insufficient for learning a policy within a semi-dynamic environment, reaching a mean value of 20% coupling efficiency. From this, we hypothesize that partial observability and the curse of dimensionality can have significant influence. To address these challenges while maintaining cost-effectiveness, we increased the observation-space dimension and reduced the action-space dimension by employing the Zernike series. This resulted in an average coupling efficiency of 36% compared to the Shack-Hartmann wavefront sensor-based AO, which reaches an average efficiency of 48%. Additionally, we observed that this configuration exhibits a faster response in the semi-dynamic environment compared to the previous configuration with lower dimensional observation space and higher dimensional action space. Future work aims at developing robust RL models for wavefront sensorless AO in fully dynamic environments. Cost savings of 30% to 40% are expected through the use of smaller telescope systems and without the need for Shack-Hartmann wavefront sensors. This can be achieved by developing the framework of a deep RL model and training it to adapt to dynamically changing wavefront distortions across different satellite trajectories and deformable mirror actuator geometries. Our immediate focus involves investigating the performance of deep RL algorithms under various dynamic turbulence conditions within a simulated environment. Once this challenge is addressed, our next step involves training our model using the available setup, omitting wavefront sensors and cameras, but using low-pixel count photodetectors instead. **Author Contributions:** Conceptualization, C.B., R.C. and D.S. ; Methodology, P.P., R.Z., C.B. and R.C. ; Software, P.P. ; Validation, P.P., C.B. and R.C. ; Formal analysis, P.P., C.B. and R.C. ; Investigation, R.Z., P.P., C.B., R.C. and D.S. ; Resources, C.B., R.C. and D.S. ; Writing-original draft, P.P. and R.Z. ; Writing-review & editing, P.P., C.B., R.C. and D.S. ; Supervision, C.B., R.C. and D.S. ; Project administration, C.B., R.C. and D.S. ; Funding acquisition, C.B., R.C. and D.S. All authors have read and agreed to the published version of the manuscript.