# A Survey on Intelligent-Reflecting-Surface-Assisted UAV Communications
## 4 Optimization Algorithms Used for IRS-Assisted UAV Communications
### ML Algorithms  
ML is a promising technology that learns from huge data sets and plays an important role in automating systems and devices in response to external conditions. Both deep learning (DL) and reinforcement learning (RL) belong to ML, in that they learn autonomously and enable a computer to develop rules on its own to solve problems. In particular, DL is a hierarchical learning process that learns from vast amounts of raw data and applies the progressed learning to new data [95]. It learns an algorithm to find a specific pattern in raw data through an artificial neural network that mimics the neuron structure of the human brain. The pattern is narrowed by learning through successive cycles, and the prediction is improved in each cycle. On the other hand, RL continues to dynamically learn and coordinate the behavior for the best reward [96]. It receives feedback on the results of various state changes and learns to obtain the best reward. It always revises, transforms, and reinforces the algorithm through much learning until it gives the best result. DL based on complicated raw data tends to result in the wrong classification [97], whereas RL is not adequate when the data dimension is large and the environment is anomalous [98]. To make up for the weak points of DL and RL, deep reinforcement learning (DRL) was proposed by incorporating a deep neural network (DNN) into RL. Recently, various studies have been conducted to apply ML techniques to improve the performance of wireless communications systems with UAVs or IRSs. However, only a few studies considered IRS-assisted UAV communications systems. This subsection describes research works for solving complex optimization problems in IRS-assisted UAV communications systems from the viewpoint of the adopted ML techniques, which can be divided into DL with neural networks and RL. The authors of [69] considered an integration of NOMA and IRS-assisted UAV communications systems. They aimed to minimize the power consumption of the system by optimizing the UAV movement, IRS phase shift, and power allocation for data transmission between UAVs and users. To achieve the goal, the authors developed and proposed ML algorithms based on a D-DQN. Unlike the conventional DQNs, which do not jointlysolve the non-convex problem, the proposed D-DQN technique was found to optimize minor constraints, resulting in reduced power consumption. Reference [58] considered an uplink NOMA with IRSs for air-to-ground communications. The formulated problem was to maximize the network sum rate, while ensuring the combat safety of UAVs and satisfying minimum data rate requirements for both UAVs and ground users. The authors proposed sample-efficient DRL algorithms to simultaneously optimize the UAV trajectory IRS configuration and power control. In addition, they proposed a DRL algorithm robust to uncertainties to guarantee the worst-case performance. The two proposed DRL algorithms were shown to outperform the existing algorithms in terms of learning efficiency and robustness. In [65], the authors considered a UAV-mounted IRS system to support the strict constraints of URLLC. In this study, UAVs equipped with IRS panels serve as relays that reflect signals from MBSs to all users of the network. Due to the highly nonconvex and complex nature of the formulated optimization problem, they adopted a DNN to solve the optimal UAV deployment. A UAV-based IoT network supporting an IRS was proposed in [66], wherein the throughput was aimed to be maximized by jointly optimizing the UAV trajectory and transmit power under battery capacity, same-side distance constraint, and other practical constraints. To solve the established optimization problem in an accurate and efficient manner, a modified multi-agent DRL (MADRL)-based algorithm called the pre-activation penalty multi-agent deep deterministic policy gradient was proposed. Reference [84] aimed to deliver robust and secure IRS-assisted UAV communications with incomplete CSI.