Attitude Control of a Ducted VTOL UAV with a Certifiable (DO-178C/DO-326/DO-356) Adaptive and Learning Avionic Agent Trained with Reinforcement Learning
Creators: name:  J. PYRGIES 
name:   M. YACOUBI 
Publisher: name: Proceedings of  the 9th European Conference for Aerospace Sciences. Lille, France, 27 June - 1 July, 2022
Year: 2022
Subjects: UAV
FUTURE APPLICATIONS
UAV SERVICES
UAV SPECIFIC TECHNOLOGIES

description: Ducted-Fan Vertical Taking Off and Landing (VTOL) Unmanned Aerial Vehicles (UAV) combine the advantages of high precision landing and taking off without the need of runways, like helicopter and multi-copter UAVs, with high speed horizontal flight, like fixed-wing UAVs. The duct offers protection to their rotor, improves their hovering, decreases their power consumption and increase their stealth capabilities. However, the control of their attitude i.e. the orientation of the aircraft on the yaw, pitch and roll axis taking into account engine thrust represents challenges mainly in the transitions between the vertical and the horizontal flight modes. Aircraft attitude is typically controlled with a Proportional-Integral-Derivative (PID) control law which delivers acceptable performance for simple flight conditions in stable environments. However, it is expected PID shows limitations in more complex use cases like the one of a ducted VTOL UAV used for ingesting a hostile UAV or dropping a bomb to blast a nascent forest fire. The proposed research will pursue a State-of-the-Art whose first exploration shows interest in finding alternatives to the PID control law for ducted VTOL UAVs [1] and the relevancy of leveraging Reinforcement Learning for multi-copter UAVs attitude control [2]. System Requirements allocated to Software will be elicited from a research project team (Aero-Thermo-Mecanique Dpt, Universite Libre de Bruxelles) busy developing a ducted VTOL UAVs. Software Requirements will be specified with SysML diagrams to avoid pitfalls of Natural Language Specification and to facilitate Validation by Aeronautic Engineers. Software Architecture will be designed with SysML block diagrams elaborated with ANSYS SCADE Architect. A first model of the flight controller software will be produced with ANSYS SCADE Suite, exploiting both the data flows and state-machines diagrams offered by this tool. Safety and liveness properties, also specified with ANSYS SCADE Suite, will be verified with ANSYS SCADE Suite Verifier. Flight Controller model will be trained with Reinforcement Learning algorithms in order to compute the parameters that optimize an objective function (minimize error between observed attitude and target attitude). A simulation will benchmark the Flight Controller model trained with Reinforcement Learning algorithms against a baseline i.e. a flight controller implementing a PID control law. Throughout the whole Software Development Life Cycle, Safety and Cybersecurity requirements will be taken into account as the UAV attitude control software delivered by the research is aimed at being certified against the DO-178C (Safety) and the DO-256/DO-356 (Cybersecurity) standards. The rationale is that this ducted VTOL UAV shall operate in non-segregated airspaces (EASA Certified category) and in hostile environments (battle fields) where cyberattacks from enemy forces could occur. DO-178C/ED-12C certification of Adaptive and Learning Avionics Agents represent several challenges: Demonstration of deterministic behaviour, absence of emergent behaviours, Specification and Verification of Adaptability and Learning requirements as mentioned in [3] and [4]. The article will conclude with a presentation of the first design model of the ducted VTOL UAV flight controller software whose parameters have been optimized with Reinforcement Learning algorithms and with future works in the scope of the PhD thesis of the authors, in progress.