## 7 Ground-Segment Optimization Aspects
### Applicable Optimization Methods for Ground Segment
#### 7.2.2 Long Short-Term Memory (LSTM)  
LSTM, originally introduced in [154], is a special kind of RNN that is particularly good at learning long-term dependencies within a system. All RNN have the form of a chain of repeating modules of NN. In a standard RNN, this repeating module has a very simple structure, such as a single _tanh_ layer. The upper plot in Figure 7 illustrates this structure, where \(x_{t}\) is the input to the network at time \(t\) and \(h_{t}\) is the output of the network at time \(t\)```
Step 1: Population Initialization Initialize the population with \(N\) individuals based on problem range and constraints. Step 2: Value Evaluation Evaluate the value of the objective function for each individual in the population. Step 3: Non-Dominated Sort with Crowding Distance Select \(\frac{N}{2}\) individuals to be the 'parents' in the next generation, according to the following criteria: a. Prioritize individuals with better non-domination criteria. b. Between individuals of similar non-domination criteria, prioritize individuals with lower crowding distance. Step 4: Cross-Over Apply the cross-over genetic operator to the parents (i.e., produce a total of \(\frac{N}{2}\) off-springs from the parents, with parameters carrying over from either parent with some probability). Step 5: Mutation Apply the mutation genetic operator to the parents and off-springs (i.e., change parameters of an individual according to some joint probability of the mutation happening, what parameter to change, and how it changes) Step 6: Termination Repeat steps 2-5 until a termination criterion is met (i.e., total iterations, change in optimality metric, etc.). ```

**Algorithm 4**NSGA-II

Figure 7: Upper Plot: The Block Diagram of a Repeating Module in a Standard RNN, which is the generalization of LSTM. Note that the modules are identical in structure, and the time propagation comes from the inherent self loop of an RNN. Lower Plot: Schematic Diagram of a Standard LSTM Module. LSTM also has this chain-like structure as it is a RNN itself, but the repeating module has a different, more complex structure seen in the lower plot of Figure 7. In the lower plot of Figure 7, \(x\) and \(h\) are also the inputs and outputs of the network, respectively, \(C_{t}\) is the cell state of the network at time \(t\), \(\Theta\) is a chosen activation function (typically the sigmoid function), the circular red elements represent component wise operations, and the rectangular yellow elements represent the network layers who perform operations according to Equations (24)-(29). In these equations, \(f\) represents the output of the 'forget' layer, \(i\) represents the output of the 'input' layer, \(o\) presents the output of the 'output' layer, and \(\hat{C}\) represents the output of the cell state estimation layer. The variables \(W\). and \(b\). refer to the weight matrix and bias of the layer, noted by their subscripts \((o,i,f,c)\), respectively. The subscript \(t\) refers to time, and the other subscripts \(f\), \(i\), \(c\), and \(o\) denote the different layers, as explained above. \[f_{t}=\Theta(W_{f}[h_{t-1},x_{t}]+b_{f}) \tag{24}\]

\[i_{t}=\Theta(W_{i}[h_{t-1},x_{t}]+b_{i}) \tag{25}\]

\[\hat{C}_{t}=tanh(W_{c}[h_{t-1},x_{t}]+b_{c}) \tag{26}\]

\[o_{t}=\Theta(W_{o}[h_{t-1},x_{t}]+b_{o}) \tag{27}\]

\[C_{t}=f_{t}c_{t-1}+i_{t}\hat{C}_{t} \tag{28}\]

\[h_{t}=o_{t}tanh(C_{t}) \tag{29}\]

We remark that the Equations (26) and (29) use the \(tanh(\cdot)\) operator only as an example of an output activation function to emphasize that such operator is typically a different one than the other layer's activation functions. Any other activation function can also be used in its place for the output depending on the application, but \(sigmoid\) and \(tanh\) are typical for LSTM. As for the overhead, studies such as [155; 156] provide optimal LSTM designs that boost the performance and reduce the overhead for their cases of interest, but for the general purpose, it remains a necessity to perform per case evaluations as any other ML-based method.