## 1 Introduction
Likewise, in [15] a MIMO FSO system is studied and the PA problem is solved via reinforcement learning (RL) in a deep deterministic policy gradient (DDPG) approach. More power allocation problems regarding FSO, RF, and terrestrial and non-terrestrial networks exploiting ML and deep RL techniques can be found in [16; 17; 18; 19; 20; 21; 22]. In this paper, several channel-model-free methodologies and heuristics are explored for optimal PA, and then compared to the exact, model-based solution for an all-optical, multi-aperture satellite downlink between a geostationary (GEO) satellite and an optical ground station (OGS). Deep learning (DL) constitutes a powerful tool to handle data from complex and fading communication channels [23; 24; 25]. Therefore, the PA problem is formulated as a constrained learning optimization problem with peak and total expected power inequality constraints. Specifically, a deep reinforcement learning (DRL) module is proposed that assists the agents in producing actions via a stochastic policy gradient technique, given channel state information (CSI) observations from the environment. The policy is structured as a deep neural network (DNN) and trained according to the REINFORCE (REward Increment = Nonnegative Factor times Offset Reinforcement times Characteristic Eligibility) algorithm [26], but modified to include the power constraints, and the multi-agent optical environment where the agents act, independently accessing only their local observations but collaboratively trying to maximize the global reward. The proposed PA strategy is deemed appropriate for difficult optical conditions since it learns explicitly from observation. Whilst the optimization problem has been approached in the literature, it has never been presented in a MIMO optical satellite scenario along with available experimental data. Summarizing the main contributions of this paper:  
* We propose a DRL-aided algorithm to optimally allocate power in an optical GEO-to-ground multi-aperture system. The proposed method accurately adjusts the expected power for each optical channel, without any knowledge of path losses and scintillation conditions. Only CSI samples are utilized. The total expected power and peak power are constrained. Although in an LEO/MEO scenario deep learning would be even more beneficial, the GEO optical CSI is still unstable due to variations in the refractive index structure parameter along the slant path, thin clouds that may attenuate or block the laser beam, and pointing and tracking errors. Thus, it is challenging to have a more accurate system model and fading distribution knowledge in the short and long term. * Instead of simulated data, experimental irradiance time series from the European Space Agency's (ESA's) ARTEMIS optical satellite sessions were employed to evaluate the performance of the proposed methodology [27; 28]. * The achieved ergodic system capacity from the application of the proposed algorithm greatly exceeds the performance of the model-free Equal Power, Random Power, and Deep Q-Learning Network (DQN) schemes [20; 21; 22] and approaches the model-based, unparameterized solution with very good agreement. * An investigation of the impact of the number of hidden layers and neurons, policy distribution, and hyperparameter selection and overfitting effects was carried out. * The proposed solution differs from other standard learning formulas because it applies to a multi-agent optical satellite PA problem based on the parameter sharing approach, allowing centralized learning under a single policy for faster convergence. The learning model is scalable as it has been tested in scenarios with a large number of optical satellite downlinks and a great amount of data and retained its performance. It is especially more scalable than the DQN algorithm because the Q-table is not scalable when there are large, high-dimensional, and continuous state-action pairs [20; 21; 22].