# Hierarchical Task Assignment and Path Finding with Limited Communication for Robot Swarms
## 4 Experimental Evaluation
### Results  
The evaluation of performance of the proposed HTAPF approach is compared against the greedy and CNP approaches in different experimental settings. For each setting, we performed 50 independent simulations with different random seeds and random initial robot positions. In each simulation, we measure the number of completed tasks looking for the approach that maximizes the demand satisfaction. For each tested environment and experimental condition, a one-way non-parametric ANOVA (Kruskal-Wallis) test was conducted to examine the differences in the completed tasks according to the different strategies considered (greedy, CNP, and HTAPF). Moreover, the post-hoc Dunn's test was employed to examine the significance of all possible pairwise comparisons (\(p<0.05\)). Figure 5 provides the results obtained in all environments in ideal conditions, that is, when communication among robots is perfect and there is no failure. The former aspect implies that every robot can receive without error the information about (i) tasks available and (ii) motion plans from other robots. This condition is clearly unrealistic for practical scenarios but is useful as a baseline on which to evaluate the impact of limitations, like local communication and possible failures. Generally speaking, we observe good results from HTAPF, which performs on par or better than the greedy and CNP approaches (see Figure 5 and Table 1 for the outcome of statistical tests). Specifically, in split16 HTAPF performs significantly better than both greedy and CNP. Here, the bottleneck constituted by the small opening between the two rooms may produce congestion when the task assignment to different areas is not properly managed, which is something that neither CNP nor the greedy approach properly do. In empty32, instead, CNP is properly assigning tasks to the robots and performs slightly better than HTAPF, while the greedy approach presents a significantly lower performance, due to the inability to quickly adapt to changes in the areas where tasks are spawned. It is not surprising that CNP performs well in this environment, considering that there is no limitation in motion from walls or obstacles, and, therefore, the bidding estimates closely correspond to the actual costs incurred, as motion planning has ample freedom to select a collision-free route. This is not the case for the facility environment, where the motion constraints in relocating from one area to another have a similar impact on all approaches due to the narrow corridors that in some cases force the passage of a single robot at the time. Finally, the most challenging environment in terms of size and structure is lab, where HTAPF again outperforms all other approaches. Note that, thanks to the perfect communication, CNP is close to a centralized approach for task assignment, as all robots can participate to the bidding process. In addition, the greedy approach can take advantage of the perfect information available, making task switching very efficient. Hence, the competitor strategies for HTAPF are working in the best possible conditions for their characteristics. HTAPF also exploits all the available information well, but the stochastic task assignment maintains some randomness that best suites for scenarios with limited knowledge, as we show later. Additionally, the communication requirements for HTAPF are minimal, while CNP imposes a considerable communication load. Hence, the advantage presented by HTAPF in these baseline experiments is commendable. Figure 5: Number of accomplished tasks by the different approaches (greedy, Contract Net Protocol (CNP), and HTAPF) across the different environments, obtained with unlimited communication and no robot failures. For each condition, 50 independent simulations are performed and the data displayed by a box-and-whiskers plot. Each box represents the inter-quartile range of the data, and the median is displayed as a line cutting the box in two parts. Whiskers extend to 1.5 times the top and bottom quartiles. Dots represent outliers.