# Deep Learning Methods for Space Situational Awareness
## 2 Related Works
### Reinforcement Learning-Based Methods  
Over the years, solutions have been devised that do not involve the use of a convolutional neural network. Most of these works exploited deep neural networks in Reinforcement Learning (RL) frameworks applied to the SSA domain. Reinforcement Learning is an area of AI, wherein agents are trained to take decisions or actions by interacting with the environment in order to maximize a return [108]. For example, in [109], authors simulated a controllable ground telescope observing satellites in LEO in an RL environment (Double Deep Q-Learning). Once the number of satellites observed over a period is maximized and the number of successful measurements is increased, they used an extended Kalman filter to significantly reduce the uncertainties of position and velocity measurements for the observed satellites. This work forms the basis of a Reinforcement Learning based framework to be explored in future research in the field of space situational awareness. Devices such as Floating Space Manipulators (FFSM) are increasingly used in various space activities, and Active Object Tracking (AOT) is the basis of many space missions. The AOT of FFSM systems presents two major challenges: the modeling and control of FFSM systems and the planning of the tracking motion of spatial manipulators. To address these issues, the paper [110] presented a strategy for active object tracking of FFSM systems employing Deep Reinforcement Learning (DRL), Proximal Policy Optimization (PPO), and a fuzzy neural network in order to address these challenges (FNN). The DRL paradigm does offer a fresh perspective on how to handle challenging space missions, and it works well in conjunction with more conventional approaches. Data acquisition costs, training efficiency, sensitivity to parameters, and environment are just a few of these problems. The algorithm training process was then completed in a simulation environment that is as realistic as possible, where the algorithm was then directly applied to a real physical problem. Another example of using the DRL can be found in [79]. An internal SSA environment is developed to efficiently train and evaluate the performance of DRL agents. The SSA environment can support arbitrary sensor position, various RSOs, observation windows, and sensor properties (action, change time, stabilization time, dwell time, measurement model, measurement, and process noises). Four DRL agents are trained using population-based training and proximal policy optimization for the SSA sensor tasking problem. Both are fully connected, and convolutional neural network DRL agents are explored in this work. DRL agents are also studied for their resilience to changes in orbital regimes, observation window lengths, observer positions, and sensor change rates. DRL agents have shown solidity to most of these variations and continue to outperform short-sighted policies. The paper [111] provides the first results to solve the sensor-tasking and sensor-management (SM) problem for Space Situational Awareness (SSA) using the asynchronous advantage Actor Critical Method (A3C), i.e., a Reinforcement Learning (RL) approach. It can learn how to make optimal decisions by interacting with the environment. The A3C method combines the benefits of both of the two main kinds of RL approaches--policy-based and value-based approaches. The A3C approach can also pick up information asynchronously from a collection of agents that engage with the environment. The A3C approach has the advantage of delivering more accurate political gradient estimations with less noise, which can enhance convergence. In this study, the preliminary findings for two