## 2 Recent Evolution in Actor-Critic Frameworks
### Quantum Neural Network (QNN)-Based Frameworks  
This type of framework employs QNNs in both the actor and critic networks to adopt powerful quantum computing for making quicker decisions while extracting key features via quantum state encoder from complex high-dimensional state-action spaces in UAVSNs [54]. By utilizing the principles of superposition, entanglement, and quantum interference, it has powerful computational benefits that are highly suitable for dealing with learning-based complex optimization problems.  
The work in [55] leverages the unique capabilities of quantum computing in distributed actor and centralized critic framework to obtain the optimal trajectories of UAVs by utilizing millimeter wave (mmWave) communication. This framework overcomes the scalability challenges and mitigates quantum errors produced by the increasing number of qubits during the noisy intermediate-scale quantum process. Additionally, it introduces noise both in state and action spaces to make the UAVSN environment more realistic and the learning process more effective. In [56], the authors utilize QNNs in an actor-critic framework to execute task scheduling global access services given by geometry-aware coordinated large-scale cube satellites toward terrestrial ground stations. Owing to the advantages of modified QNN-based MA-DRL, it can reduce not only the scheduling action spaces but also the number of qubits. Moreover, due to application-oriented reward function design, the model ensures better quality of service (QoS) performance in global access in terms of maximizing transmission capacity and the residual energy of cube satellites.  
QNNs also can ignore unworthy results and maintain focus on the best-suited outputs, which results in better optimization along with being resourceful in computing because of their flexibility to changes in the environment. It includes an enhanced level of data security due to quantum encryption. This is one of the major concerns for modern UAVs, particularly in military and sensitive commercial applications. Despite the several benefits of QNNs, there are a couple of challenges impeding the practical application of QNNs. One of the most critical limitations is the availability of hardware resources to support QNNs. The current quantum hardware is still in its infancy, with problems like noise, qubit decoherence, and a limited number of qubits affecting the reliability and scalability of QNNs. Quantum systems are also susceptible to high error rates that may compromise the accuracy of decisions made by QNNs. These issues demonstrate the need for further developments in quantum hardware and techniques for error correction. Another challenge is the very limited availability of software tools and libraries designed for QNN development. Whereas classical neural networks can rely on a mature ecosystem of libraries and frameworks, QNNs are still lacking similar support. This lack of public development tools causes more complexity in designing, training, and deploying QNN-based systems.  
Despite these challenges, quantum-enhanced actor-critic frameworks are especially promising for UAVs's that require high-dimensional control and adaptive resource allocation under stringent QoS and resource-constrained dynamic environments. It aims to tackle traditional MA-DRL challenges like slow convergence and local optima by exploiting quantum acceleration in policy optimization in actor networks and value evaluation processes in critic networks.