## 5 Proposed Solution
### Long Short-Term Memory to Approximate the UAV Trajectory Parameters  
We propose an LSTM to approximate the complex nonlinear functions in the above convex optimization sub-problems. As the LSTM approximates the parameters of the ob  
Figure 3: An illustration of the proposed federated reinforcement learning-based solution approach for UAV trajectory control and power optimization for low-latency C-V2X communications. \begin{table}
\begin{tabular}{l l} \hline \hline
**Symbol** & **Definition** \\ \hline \(v_{i}(\mathcal{H}_{i})\) & Covariance matrix for each sampling value on the trajectory point \\ \(a_{i,t}(\mathbf{q}_{i})\) & UAV selects action to move to the next coordinates; maximizes expected delay minimization \\ \(\mathbb{E}\) & Expected maximum value function when UAV visited the past coordinates \\ \(r_{i}(\mathbf{s}_{i},\mathbf{a}_{i})\) & UAV overall reward function \\ \(\gamma\) & Discount factor \\ \(r_{i,\varphi}(t)\) & Reward that achieves optimal model transmission from vehicles to UAV \\ \(a_{i,t}\) & UAV action in each TTI \\ \(e(s_{i},a_{i})\) & Entropy \\ \(Q_{i}^{\psi}\) & Actor–critic Q-function \\ \(\beta\in(0,1]\) & Learning rate \\ \hline \hline \end{tabular}
\end{table} 
Table 3: _Cont._jective function and the constraints, the resulting model is integrated with the optimization framework. In \(P1\), parameters such as the UAV trajectory (\(\mathbf{q}_{(x,y)}\)) in the next TTI or the vehicle data rate \(s_{i,t}\) cannot be precisely known, and vary significantly with time. LSTM is used to learn these parameters from training data, and the learned parameters are incorporated in the solutions of the optimization sub-problems (\(SP_{1}\)-\(SP_{4}\)). As the UAV trajectory changes, these parameters change, and as there are temporal dependencies, LSTM models these dynamics. The optimization problem is solved in an iterative manner, where the LSTM adapts the optimization strategy based on the updated parameters. In addition, the LSTM models the time-varying constraints or complex relationships between variables in \(P1\) and predicts constraint violations, as well as adapting to changing constraints. The problem \(P1\) is addressed using the actor-critic framework using two sets of LSTM networks to approximate the policy function (\(Q^{\pi}(\mathbf{s}_{t},\pi(\mathbf{s}_{t}|\psi)\)) and value functions (\(V(\psi)\)). The parameterized actor network generates a deterministic action to maximize the value function based on the steady-state distribution of the actors' policy. The critic network approximates the value function by taking the derivative of \(V(\psi)\), given by \(\
abla V(\psi)\) with respect to the policy parameter and updates the actor network by gradient ascent to improve the value function \(Q^{\pi}(\mathbf{s}_{t},\mathbf{a}_{t}|\psi)\
abla_{\psi}\pi(\mathbf{s}_{t}|\psi)\). We use the _LTE I/Q_ dataset to train the LSTM to learn a mapping from the sequences of states to the optimal trajectories and from the input features to the objective function and the constraints. The _LTE I/Q_ dataset includes sequences of state-action pairs for the UAV [69]. As illustrated in Figure 4, the state includes information about the position, velocity, and environmental conditions of the UAV, while the action represents the trajectory. We use the mean squared error (MSE) loss function to predict future trajectories based on the current state and environmental conditions of the UAV to approximate the behavior of the constraints. We periodically retrain the actor-critic LSTM using updated data to evaluate its accuracy in predicting the UAV trajectory. The actor LSTM predicts trajectory, and the critic updates the parameters based on the predictions. We monitor the convergence of the optimization process and update the LSTM training strategy. The process is repeated throughout the trajectory of the UAV to model the sequential dependencies over time. Figure 4: UAV trajectory varies in a random manner, and the vehicles capture varying sensor data at different TTIs.