日本機械学会 2012 年度年次大会 [2012.9.9－12]
CopyrightⒸ2012 一般社団法人 日本機械学会

W011005
Development of an Autonomous UAV Managed by Virtual Reality
Hafid NINISS*1*1

Forum8 Tokyo, System Development Group, Robot Development TeamNakameguro GT Tower 15F, Kamimeguro 2-1-1, Tokyo, 153-0051 Japan

The purpose of this work is to develop an autonomous Unmanned Aerial Vehicle (UAV) as a platform for several
types of applications, including inspection, monitoring. The first project presented in this paper concerns anautonomous UAV used in conjunction of Virtual Reality to monitor crop fields to ensure a healthy growing process. We
used the AR.Drone, a widely commercialized UAV, initially designed for video gaming and customized it to suit theproject’s requirements. The resulting prototype is the AGUL, equipped with a GPS and a compass for localization as
well as other sensors for real-time monitoring of crops (temperature sensor, humidity sensor, high vision camera). The
first tests performed indoor showed promising results regarding the autonomous capability of this drone. The next step
will be to perform outdoor tests to assess the robustness of the drone to perturbations like wind or GPS localization
errors.
Key Words : Autonomous UAV, Virtual Reality, Monitoring, Teleoperation

1. Introduction
For years Unmanned Aerial Vehicles (UAV) have been used in a wide range of civil and military applications, including
aerial photography/video, communication relays, and surveillance(1) (pipelines, nuclear facilities, law enforcement…).
According to the United States congressional Research Service, about one of three military aircrafts in United States aredrones(2) and are expected to expend in the future. One of the obvious benefits to use an UAV is to scout areas which arepotentially harmful or difficult of access in order to retrieve data samples or evaluate the possible risks or threats before any
human presence is allowed on-site. More recently UAV’s nature has shifted to execute more active missions. One of the most
famous one is the Predator drone(3) used by the US army for reconnaissance missions but also combat. Although the civil
UAV market in 2000 was representing only 3% of the total market revenue(4), this market keeps growing at an increasing
speed(5), with the costs beginning to be affordable and the apparition of new needs such as aerial photography for real estateadvertising.
Despite their sophisticated design and extensive sensing capabilities, UAVs are usually manually controlled remotely by anoperator by using a radio controller. This type of control usually requires training and practice in order to operate the UAV ina safe and accurate manner.
Our approach is to develop or customize existing miniature UAV platforms (maximum takeoff weight less than 30kg,
maximum altitude of 150~300m) and integrate Virtual Reality to simplify the drone management. The expected output is asystem assisting the path planning and task scheduling processes, managing the autonomous control and real time monitoringof the drone location, status, real time video feed…). The user will only be required to specify high level “missions” (path ofthe drone, location where to collect data/pictures/videos…). The final goal is to allow any user unfamiliar with UAV to safely
operate a drone for a predefined purpose, in a natural and user friendly manner. In this paper we present our firstdevelopments of an autonomous UAV used in agriculture for monitoring crops and its link to UC-win/Road, a multi purpose
Virtual Reality software. At the current stage of development we use Virtual Reality tools mainly for visualization and to plan
*1

Forum8 Tokyo, System Development Group, Robot Development Team
Nakameguro GT Tower 15F, Kamimeguro 2-1-1, Tokyo, 153-0051
E-mail: niniss@forum8.co.jp[No.12-1] 日本機械学会 2012 年度年次大会講演論文集 〔2012.9.9－12，金沢〕

scheduled tasks.
After introducing the UAV platform used for this project, we will present the prototype developed for agriculture purposes
and the link to the Virtual Reality software. In the last part we will present the autonomous control principles and theexperimental testing, before concluding with the future works.

2. Autonomous UAV
2・1 The AR.Drone
In 2010 Parrot released the AR.Drone (Fig.1.a, 1.b), a new concept of video game which originality comes from using an
actual micro quadrotor UAV (4 propellers), instead of a flight simulator alike as most traditional flight videogames do.

Fig. 1.a: Original AR.Drone with indoor hull

Fig.1.b: AR.Drone with an outdoor hullThat video game was initially developed to run on the iPhone platform which allows the user to connect wirelessly to thedrone by an ad-hoc connection and control the drone in a simple and natural way, by tilting the iPhone body, as it is equippedof a 3 axis accelerometer and two gyrometers. Instead of having a view of a flight simulator alike, the player has a real-timevideo feed of one of the two cameras (front camera and vertical camera). The colors of the hull can be personalized to identify
each player so the drone can detect the drone of other players, and virtually chase and “shoot” them to gain points.
The particularity of the AR.Drone is that semi autonomous behaviors are integrated in the default version. For instance, theauto balancing, hovering or take off and landing are performed internally by using a 3-axis accelerometer, a 2-axis gyrometerand a single-axis yaw precision gyrometer. The altitude is estimated from an ultrasonic range sensor mounted vertically.
When the drone doesn’t receive any command from the user or in case of Wi-Fi connection loss, the drone enters
automatically in hovering mode during a certain period of time, before landing. In that mode, the drone tries to keep it
position steady while balancing itself even in case of perturbations.
2・2 AGUL prototype
One of the applications of UAVs is the monitoring and surveillance of a specific area. To answer a customer request, wedeveloped the AGUL prototype (Fig.2.a, 2.b), an UAV to monitor and check the growth of crops in an autonomous way.

Fig.2.a: AGUL drone

Fig. 2.b: AGUL drone, bottom view

The principle is to have a drone that takes aerial views at low altitude, as well as data measures (temperature, humidity) andsend them back to the user. To do this we kept the original physical body of the AR.Drone as well as the internal sensors andcontrollers, and equipped it with additional devices to extend its capabilities so it can match the project’s requirements:. Sensors: Attitude and Heading Reference System (AHRS) based on a 9DOF IMU including a compass (Razor 9DOF), GPS,
Proximity sensors for collision avoidance (Sharp Infrared Range sensors), High Vision Camera with a remote shutter,
Temperature and Humidity sensors. Communication: 2.4GHz wireless transceiver to extend the range to ~100 m. Control: GUI on PC (Fig. 3) with a 3D mouse (manual mode, for development) or autopilot (autonomous control mode)

Fig.3 : ARDrone interface, with real time video feed from the drone (front camera)

3. Autonomous navigation and task scheduling
From UC-win/Road, a 3D-interactive Virtual Reality software for interactive virtual reality modeling for construction
planning, urban planning, civil engineering and traffic modeling. In this project we use its Virtual Reality modelingcapabilities to plan the drone tasks, and visualize its data in real time. For an effective and practical use of the system in theagriculture context (monitoring of crops), the presence on-site of the end user is not necessary: the VR-Cloud version ofUC-win/Road allows him to check on the drone status or data in real time (Fig.4.a, 4.b), from virtually anywhere and anytime.
For instance the user can be in a remote location and still be able to modify the drone “mission” (e.g. change of flight plan orrescheduling of tasks in case of change of weather).

Fig.4.a, 4.b: Path planning in UC-win/Road

4. Autonomous control
The 3D path is planned using UC-win/Road, and defined as a series of waypoints (position: xi, yi, zi and heading θi), which
are sent to the drone as a reference to follow. In that control mode, the default autonomous behaviors like hovering aredisabled. From the waypoints, the drone computes a more refined path by linear or Bezier interpolation. The computation ofthe commands based on the definition of the waypoints is made in 3 phases:. at each time step, the location of the nearest point on the interpolated path is estimated. a pursuit point is defined at a fixed distance ahead of the nearest point (look ahead distance) by using the simple pure pursuit
tracking algorithm(6).
. finally, the pursuit point coordinates are then estimated in the drone local reference frame to establish the (roll, pitch, yaw
angular rates and vertical speed) to feed the AR.Drone attitude controller that computes the angular speed for each motor. The
control architecture in manual mode(7) is given in Fig.5.

Fig.5: AR.Drone control architecture(7)At anytime the drone will switch automatically to manual mode and land for safety when a problem is detected (loss ofcommunication or flying over the transmission range), to avoid loosing the drone when carried away by mild to strong wind.
We chose to study the behavior of the drone in a controlled environment, to get rid of possible perturbation elements, like thewind or the GPS localization errors. Thus the development has been performed indoor, and to overcome the inability to use
the GPS indoor, we use our absolute localization system AuReLo (AUgmented REality LOcalization system). It is amulti-purpose tracking system based on AR (Augmented Reality), designed to track position andd orientation of stationary ormoving objects, as well as laser spots, with a relatively good accuracy (1~2cm error at 2.5m range). The principle is quitesimple; the AR technique allows to estimate the relative position/orientation of AR markers in the camera’s reference frame.
To track a moving target, we use two markers, one defining the origin of an absolute reference frame and the other one fixed
on the object to track.
The Figure 6 shows the development tests of the autonomous control algorithm, for instance by an implementation of theautonomous hovering function over a specified target.

Fig.6: Autonomous behavior: hovering over a specified area, tracking with AuReLo

The next step is to put this control algorithm to test in outdoor environment with the GPS, ideally with no wind. The system
will then be ready for evaluation over an actual crops field.

5. Conclusion and Future WorkIn this paper we presented the current state of development of an agriculture autonomous UAV controlled at a high level by a
Virtual Reality software. The UAV platform used is the AR.Drone, designed initially for video gaming and homeentertainment purposes. The main motivation of the choice of this drone was its low cost, easiness to handle and semiautonomous behavior like takeoff or landing, available in the original version. The first tests indoor showed the effectivenessof the drone to perform predefined tasks in an autonomous way and give good hopes regarding outdoor applications such as
monitoring, inspection and surveillance.
The current drone configuration can lift only few grams payloads, which is not sufficient for loading additional sensors. For
this reason we will use in the next project an octorotor UAV which maximum payload is about 1kg with 30 min endurance.
Those specifications will make it possible to use an UAV in a wider variety of applications implying specific types of sensors
like in telemetry or real time mapping of 3D environments.

References(1) Bento, M.F., “Unmanned Aerial Vehicles: an Overview”, Inside GNSS, January/February 2008.
(2) Gertler , J., "US unmanned Aerial Systems", Report of the Congressional Research Service, January 3, 2012.
(3) Banks, R.L., “The Integration of Unmanned Aerial Vehicles Into the Function of Counterair”, Publisher: Air Command and Staff
College, Air University, Maxwell Air Force Base, Alabama (2000), ASIN B0006RU28W.
(4) Sarris, Z., “Survey of UAV Applications in Civil Markets (June 2001)”, STN ATLAS-3 Sigma AE and Technical University ofCrete, Greece, 2001.
(5) Degarmo, M.T., “Issues Concerning Integration of Unmanned Aerial Vehicles in Civil Airspace”, MITRE, Center for Advanced
Aviation System Development, November, 2004.
(6) Coulter, R. C., “Implementation of the Pure Pursuit Path Tracking Algorithm”, Tech. report CMU-RI-TR-92-01, Robotics Institute,Carnegie Mellon University, January, 1992
(7) Bristeau, P.J., Callou F. &Al., “The Navigation and Control Technology Inside the AR.Drone Micro UAV”, 18th IFAC WorldCongress, Milano (Italy) August 28th-September 2nd, 2011.