## 4 Categorization of Optimization Methods
On the other hand, recent and advanced methods are suitable for all types of mathematical optimization problems seen in Table 3. The complex search algorithms, such as Simulated Annealing (SA) [48], and evolutionary algorithms, such as GA [49] and Particle Swarm Optimization (PSO) [50], are methods that originally dealt with a single-objective optimization problem again, but they now have improved versions that extend the method to multi-modal and multi-objective problems (i.e., Multiple-Objective Simulated Annealing (MOSA) [51], Multiple-Objective Genetic Algorithm (MOGA) [52], Non-Dominated Sorting Genetic Algorithm II (NSGA-II) [53], Multiple-Objective Particle Swarm Optimization (MOPSO) [54], and Hybrid-Resampling Particle Swarm Optimization (HRPSO) [55]. Similarly, the Neural Network (NN)-based methods [56], such as Deep Neural Network (DNN) [57] and Convolutional Neural Network (CNN) [58], are typically designed for a particular problem, which can be any type of optimization problem from Table 3.  
Other methods, such as Kalman Filtering (KF)-based methods (i.e., basic KF [59], Extended Kalman Filter (EKF) [60], Unscented Kalman Filter (UKF) [61], Adaptive Kalman Filter (AKF) [62]), and Support-Vector Machine (SVM) [63], are methods that solve single  
Figure 3: Categorization of Optimization Methods. objective multi-modal optimization problems, but, similarly to the above-mentioned cases, they are flexible in the sense that they can be extended to other types of optimization problems via modifications. KF methods, in particular, are very suitable for estimation problems under accurate underlying problem models. An important method to discuss in detail is the MC optimization. Despite also being a brute-force optimization method that relies on the problem model, MC is unique in the sense that most simulations regarding LEO networks are constructed as a Monte-Carlo loop due to its ability to directly propagate parameters through the system model. This makes it so that the methods that require environment observations or interactions, such as evolutionary algorithms, are methods that are jointly utilized with MC loops. However, it should also be noted that the MC loop does not refer to an MC optimization in such cases; the parameters are optimized according to the actual optimization method utilized. For detailed explanations regarding MC, we advise dedicated sources such as [64, 65]. Since LEO networks have a high number of optimization-related problems from all the types shown in Table 3, it should not be surprising that many of the discussed methods are suitable for multiple types of optimization problems. However, it should be noted that a method being suitable for a problem does not mean that it performs optimally or that it is feasible to apply for that particular problem. We will discuss the differences and trade-offs between different methods in detail in the next sections, when we examine the optimization problems in different segments of a LEO system. Last but not least, an important aspect, which has a significant impact to the overall system in practical applications concerning autonomous vehicles, is the overhead of the optimization engines. In the case of an online optimization (i.e., optimization that takes place during the system operation), the overhead mainly refers to the time delay added to the system by the optimization procedure as well as to the required system resource allocation to perform the optimization. Examples of online optimization tasks are network routing and handover management (see Table 2). When we perform the optimization before the system is operational, this is referred to as an offline optimization part. Examples of offline optimization are the GS planning and constellation optimization (see Table 2). In an offline optimization task, the main overhead is the required time-to-converge and complexity of the optimization method. These two parameters typically determine the feasibility of a method.