# UAV Formation Shape Control via Decentralized Markov Decision Processes
## 5 Simulation Results  
We assume that each UAV has its own on-board computer to compute the local optimal control decisions. We implement the above-discussed NBO approach to solve the swarm control problem in MATLAB. We test our methods in two scenarios--a spherical shape with and without an obstacle. The UAVs are aware of the shape dimensions and the exact location of shape. Each UAV randomly picks a location on the formation shape, and uses  
Figure 1: UAV formation shape control architecture. the NBO approach to arrive at this location. We use MATLAB's _fmincon_ to solve the NBO optimization problem. Here, we set the horizon length to \(H=3\) time steps. We define the following metrics to measure the performance of our formation control approach: (1) \(T_{c}\)-_average computation time to evaluate the optimal control commands_ and (2) \(T_{f}\): _time taken for the swarm to arrive on the formation shape_. As a benchmark method, we use a centralized approach to solve the above-discussed swarm formation control problem. In other words, we use a single NBO algorithm, which optimizes the motion control commands for all the UAVs together based on the global state of the system. We implement this centralized algorithm in MATLAB. We implement the Dec-MDP approach with a spherical formation shape with and without an obstacle. The resulting swarm motion is shown in Figure 2 for the spherical formation shape in the absence of any obstacle using the cost function described in Equation (1). The scenario with an obstacle considers the following cost function. \[c(b_{k}^{i},a_{k}^{i},a_{k}^{nm}) =w_{1}\Big{[}\text{dist}(s_{k}^{i,\text{pos}},d^{i})+\text{dist}( s_{k}^{\text{nn},\text{pos}},d^{\text{nn}})\Big{]}\] \[+w_{2}\Big{[}\text{dist}(s_{k}^{i},s_{k}^{nm})^{-1}\mathbb{I}( \text{dist}\Big{(}s_{k}^{i},s_{k}^{nm})<d_{\text{coll},\text{thresh}})\Big{]}\] \[+w_{3}\Big{[}\text{dist}(s_{k}^{i},s_{k}^{obstacle})^{-1}\mathbb{ I}(\text{dist}\Big{(}s_{k}^{i},s_{k}^{obstacle})<d_{\text{coll},\text{obstacle}} \Big{)}\Big{]}\]  
where \(s_{k}^{obstacle}\) is the location of an obstacle, \(d_{\text{coll},\text{obstacle}}\) is a collision threshold with the obstacle, and \(w_{3}\) is a weighting parameter. The indicator function \(\mathbb{I}(b)=1\), if the argument \(b\) is true and \(0\) otherwise. The resulting motion of the scenario with the obstacle is shown in Figure 3. For this scenario, we also plot the distance between every pair of UAVs in the swarm, as shown in Figure 4. Here, we assume that there is a collision risk between a pair of UAVs when the distance between them is less than \(5\) m. Clearly, the Figures 3 and 4 demonstrate that our decentralized algorithm drives the swarm to the destination while successfully avoiding collisions between the UAVs. Figure 2: UAV swarm converging to the spherical formation shapes in 3D. We now compute average computation time and average pairwise distance with respect to neighborhood threshold where each UAV communicates with other UAVs within the radius of neighborhood threshold. If neighborhood threshold is infinity, a UAV can communicate with all other UAVs in the swarm. UAVs optimize their decisions together with neighbors, which depend on neighborhood threshold and implement its own control. We expect that, with the increase in neighborhood threshold, average computation time rises and, after certain neighborhood threshold, average computation time saturates. Figure 6 shows average computation time rise until neighborhood threshold reach 240 m and then waves between 20 to 25 s.  
Figure 5: Computation time (\(T_{c}\)): centralized vs. decentralized method. Figure 6: Average computation time with respect to neighborhood threshold. We also expect that with the increase of neighborhood threshold, average pairwise distance drops. The reason we are interested in analyzing average pairwise distance is, we expect the swarm to be as close as possible while avoiding collision between UAVs.