## 4 Solution Approach
### Personalized Federated Learning with Attention Mechanism  
At the UAV, we fit the global model, denoted as \(\vec{\mathfrak{T}}_{T}\), which is a set of sequential data collected and stored during repeated transmissions. A UAV's global model hyper-parameter vector is  
\[\vec{\mathfrak{T}}_{T}=\{\mathcal{D}_{t}\}_{t=1}^{T}, \tag{18}\]  
where \(\mathcal{D}_{t}\) is the data gathered during the \(t^{th}\) TTI, and \(T\) implies the total number of coordinate changes traversed by the UAV. At each time step \(t\), the UAV's location is characterized over the hovering time \(T\). The progression of states correspond to different trajectory coordinates \(\boldsymbol{\Delta}_{(x,y)}\). We model the joint distribution of the UAV's states and past observations as  
\[p_{\theta}(\vec{\mathfrak{T}}_{T},\vec{\mathfrak{V}}_{T})=\prod_{t=1}^{T} \underbrace{p_{\theta}(x_{t}\,|\,\psi_{t})}_{\textbf{Past}-\textbf{trajectory}} \underbrace{p_{\theta}(\psi_{t}\,|\,\vec{\mathfrak{T}}_{t-1},\vec{\mathfrak{ V}}_{t-1})}_{\textbf{UAV's future trajectories}} \tag{19}\]  
where \(\vec{\mathfrak{V}}_{t}\) is the set of local model hyper-parameters. The transition probability in Equation (19) assumes that the UAV's state at time \(t\) depends on the past observations \((\vec{\mathfrak{T}}_{t-1},\vec{\mathfrak{V}}_{t-1})\). This is different from the Markovian assumption, where \(p_{\theta}(\psi_{t}\,|\,\vec{\mathfrak{T}}_{t-1},\vec{\mathfrak{V}}_{t-1})=p _{\theta}(\psi_{t}\,|\,\psi_{t-1})\), i.e., future states depend only on the current state. To capture non-Markovian dynamics using PFL and an attention mechanism, the state transitions are modeled as  
\[p_{\theta}(\psi_{t}\,|\,\vec{\mathfrak{T}}_{t-1},\vec{\mathfrak{V}}_{t-1})=p_{ \theta}(\psi_{t}\,|\,\vec{\mathfrak{V}}_{t},\vec{\mathfrak{V}}_{t-1}) \tag{20}\]  
where \(\vec{\mathfrak{V}}_{t}=\{\psi_{1}^{t},\ldots,\psi_{t-1}^{t}\}\) is a set of attention weights to infer the statistical characteristics of the future states. The attention weights are used to determine the influence of a UAV's past states on the future state transitions through a non-linear mapping based on the attention mechanism [87; 88; 89; 90; 91]. The attention mechanism is commonly used in Transformer architectures and utilizes query, key, and value vectors to calculate the attention weights [87]. In PFL, a vehicle is designated to communicate with the UAV in a TTI. In that TTI, the other vehicle's model parameter transmissions will be redundant until the next TTI. An adaptive weighting approach compares the statistical similarity in model weight parameters of different vehicles as well as vehicles and the UAV. If two vehicles have a high similarity, they are considered to belong to a single subset, so the UAV can communicate with any one vehicle out of the two. If there are three such vehicles in a subset, the UAV can communicate with any one out of these three vehicles because all three vehicles are communicating similar data. Moreover, in a TTI, more than one such subset can exist. Hence, in order to ensure that the UAV does not miss different types of information transmitted by different vehicles, the UAV should communicate with at least one vehicle from each subset. The attention mechanism trains the vehicles and the UAV to learn the state transitions from a sequence of past data [88]. Note, the vehicles generate local models from a limited number of data samples from each TTI. The previously learned local models are augmented with the arrival of new data, allowing the agents to adapt the current PFL model to new model updates. The attention function \(A_{t}\) is introduced to utilize the features from the previous time steps. As the UAV's trajectory coordinates are determined, the corresponding attention weights cluster the vehicles with similar local models within a selected TTI. This approach utilizes the past knowledge of optimal trajectory coordinates, where the UAV can serve a maximum number of vehicles while adapting to fresh local model hyper-parameters received from the vehicles.