## 1 Introduction  
The usage of UAVs finds its roots where human intervention might be difficult, risky or costly. Initially thought for military purposes, UAVs have demonstrated their tremendous potential in civilian applications such as parcel delivery, rescue mission or environment monitoring. Current applications nonetheless rely on the usage of a single UAV (remetely piloted or autonomous), which faces multiple limitations, such as its range of action, payload capacity and system resilience.  
Using several autonomous UAVs simultaneously as a swarm is one promising solution to address these limitations. Inspired by natural phenomena (e.g., birds flocks or ant colonies), swarm intelligence allows one to achieve complex tasks while solely relying on local decisions and interactions. Such a distributed and self-organised approach allows multi-UAV systems to be more scalable, resilient and flexible.  
However, defining a priori the behaviour of each individual swarm member to obtain a desired collective behaviour is difficult and time-consuming due to the high uncertainty of a swarm operation. As a consequence, classical multi-robot design techniques are not applicable, as they require the global specifications of the systems to define the behaviour of individual robots [1]. Other methods, mainly belonging to the field of evolutionary swarm robotics, ref. [2] have been proposed to automate the design of more complex systems like robot swarms. Here, the design problem is modelled as an optimisation problem which consists in finding an optimal parameterisation and architecture of the neural network used to control each swarm member. However, as outlined in [3], these have limitations, for instance, they are typically applied to a single use case (i.e., no generalisation study).  
Automating the design of flexible/reusable swarming behaviours would, thus, overcome these challenges. However, as outlined by [PERSON] et al. [1], automatic robot swarm design still remains an open research problem.  
The work proposed in this article, thus, aims to automate the design phase of UAV swarms thanks to a novel hyper-heuristic based on Reinforcement Learning (RL) and, more particularly, Q-Learning (QL). More precisely, it seeks to improve the state-of-the-art in automated algorithm design to generate efficient swarming behaviours in the context of area coverage. The problem of covering an area consists of moving UAVs so that any location of the area is visited by at least one UAV at some point in time.  
This article introduces several novel contributions: (1) a three-objective variant of the \"Coverage of a Connected-UAV Swarm\" problem referred to as CCUS3O, (2) a generative hyper-heuristic based on Q-learning (QLHH-II), which aims at obtaining efficient and scalable heuristics, (3) a factorial experiment to analyse the sensitivity of QLHH-II to its parameterisation, and (4) an empirical comparison of QLHH-II performance to QLHH and two state-of-the-art heuristics on CCUS3O.  
The remainder of this article first presents a state-of-the-art analysis of the manual and automatic design of robot/UAV swarms in Section 2. The CCUS3O optimisation model is then formally described in Section 3, preceding the presentation of the proposed hyper-heuristic, QLHH-II, in Section 4. The following sections are related to the experiments. In Section 5, the experimental setup and process are described, while the results are shown in Section 6. The last section contains conclusions and perspectives on the work.