### Related Work and Motivations  
In this section, related survey articles are reviewed along with the motivation for writing this survey article to highlight our key contributions. In [35], existing distributed machine learning-based approaches for UAVSNs are reviewed and classified into federated learning and multi-agent reinforcement learning for trajectory optimization, sensing, edge computing, and resource allocation. In [13], the authors reviewed the existing trajectory and topology control algorithms by classifying them based on their underlying mathematical model. In [36], the existing edge computing algorithms for the metaverse environment are reviewed, focusing on optimization techniques. In [37], the authors reviewed the existing edge computing algorithms for UAVSNs to support various artificial intelligence-based applications in computer vision, navigation, power control, security and privacy, and communication. In [38], the authors investigated the application of transformer-based encoder-decoder networks in UAV-assisted wireless networks to perform compression, schematic communication, and channel estimation. In [39], the existing task offloading algorithms in UAVSNs are reviewed and classified based on value-based and policy-based methods. Additionally, they have discussed the tasks in UAVSNs by classifying them as dependent tasks and independent tasks. The independent task can be processed in parallel, whereas dependent tasks must be processed sequentially by assuming a directed acyclic graph of task arrival. Task size, CPU cycle requirements, and task execution latency requirements depend on the application scenario. In [40], the existing trajectory-aware task offloading algorithms are reviewed and classified into general iterative optimization and reinforcement learning-based optimization methods. In [41], the existing data aggregation techniques for UAVSN-assisted IoT devices are reviewed. However, all of this existing survey did not focus on joint optimization problems based on modifications of actor-critic MA-DRL framework neural network structure in UAVSNs, which has drawn significant research attention.  
Owing to the dynamic topology of UAVSNs, actor-critic-based MA-DRL algorithms are highly suitable for sequential decision-making. Nevertheless, an actor-critic network solely depending on a fully connected network (FCN) or multi-layer perceptron (MLP) cannot deal with time series sequential data such as UAV trajectories and channel states. Additionally, FCN requires a higher number of learning parameters, and this FCN-based learning is less scalable and not adaptive to environmental changes [42]. Fortunately, the recurrent neural network (RNN) such as long short-term memory (LSTM) and gated recurrent unit (GRU) can remember a finite amount of the most recent historical sequential data and utilize it to predict more accurate state in the next timeslot by mining the temporal relationship with the state information in the current timeslot [43]. Nevertheless, RNN-based LSTM or GRU-based encoder-decoder networks may encounter gradient vanishing problems during backpropagation and cannot efficiently deal with lengthy time series data. Moreover, RNN cannot adopt parallelization in feature extraction.  
The transformer-based encoder-decoder networks can overcome these limitations via multi-head attention mechanisms [44]. However, how to employ these networks in actor-critic frameworks to address joint optimization problems in dynamically resource-constrained UAVSNs remains a research question. Although plenty of research has been conducted to address joint optimization problems in UAVSNs by leveraging various actor-critic-based MA-DRL frameworks, no in-depth survey has focused on the variations in actor-critic frameworks, their key limitations, research challenges, and potential solutions as future research directions. It motivates us to write this survey. According to the above discussions, a comparison of key contributions in this article relative to existing survey articles is summarized in Table 1.