Patent Description:
Large-scale industrial companies, especially in utilities, oil, and gas, may own hundreds of miles of asset (e.g., powerline, pipeline) infrastructure that needs to be inspected periodically to ensure high productivity. Recently, some entities have begun utilizing small unmanned aerial vehicles (UAVs) to perform these periodic inspections due to the UAVs' ability to quickly collect high-quality data. Federal Aviation Administration (FAA) regulations require that a UAV operator or pilot maintain a line-of-sight (LOS) with the UAV at all times.

<CIT> and <CIT> are concerned with UAVs and methods of controlling the same.

The invention is defined in the attached independent claims to which reference should now be made. Optional features are defined in the sub-claims appended to the independent claims.

In general, this disclosure relates to systems and techniques for determining and indicating a position of at least one unmanned aerial vehicle (UAV) with respect to the UAV's pilot, in order to assist the pilot to maintain line-of-sight (LOS) with the UAV.

The use of Unmanned Aerial Vehicles (UAVs) is becoming increasingly common in non-military operations, including, but not limited to, surveillance, search and rescue, shipping and delivery, and inspections. For example, <FIG> is a conceptual diagram depicting the inspection of a utility infrastructure asset with Unmanned Aircraft System (UAS) <NUM>, including UAV <NUM>. Unmanned Aircraft System <NUM> includes UAV operator or pilot <NUM>, UAV <NUM>, augmented-reality (AR) head-mounted device (HMD) <NUM>, ground station <NUM>, and UAV controller <NUM>.

UAV pilot <NUM> uses UAV controller <NUM> to control the flightpath <NUM> of UAV <NUM>, for example, in order to conduct an inspection of structures 22A-22E (collectively, "structures <NUM>"). In the example depicted in <FIG>, structures <NUM> are depicted as electrical transmission towers, but structures <NUM> may include any structure, including pipelines, flare stacks, solar panels, bridges, etc..

For example, pilot <NUM> may launch UAV <NUM> from launch location <NUM> and guide UAV <NUM> around structures <NUM> such that a camera or other sensor mounted on UAV <NUM> may capture data. UAV <NUM> may, for example, capture photographic or video data through the use of one or more camera-based sensors and may additionally or alternatively capture other types of data, such as temperature data, environmental data, or electromagnetic field data, using other types of sensors mounted on UAV <NUM>.

Pilot <NUM> may guide UAV <NUM> along an extended and/or tortuous flightpath <NUM> in order to capture extensive data during a single launch. For example, at any time during the inspection, UAV <NUM> could be <NUM> meters away from pilot <NUM> while conducting the inspection mission. The distance UAV <NUM> may travel from pilot <NUM> may be limited only by the battery life of UAV <NUM> and/or the radio communication range of controller <NUM>.

Federal Aviation Administration (FAA) Small-UAS Rule §<NUM><NUM> requires that a UAV pilot must maintain a line-of-sight (LOS) with the UAV at all times during operation. A number of factors, including, but not limited to, tortuous terrain, background (e.g., sky) color contrast, weather, or other obstacles may hinder the ability of pilot <NUM> to maintain a line-of-sight with UAV <NUM>.

In some examples of this disclosure, pilot <NUM> may wear HMD <NUM>, having therein processing circuitry configured to determine a location of UAV <NUM> with respect to the field-of-view (FOV) <NUM> of HMD <NUM>, and output for display on a transparent display screen of HMD <NUM> an indication of the relative location of UAV <NUM>, in order to assist pilot <NUM> to maintain and/or re-obtain line-of-sight <NUM> with UAV <NUM>. In some examples, such as when UAV <NUM> is localized within the FOV <NUM> of HMD <NUM>, HMD <NUM> may display a bounding box around the approximate location of UAV <NUM> on the screen. In other examples, such as when UAV <NUM> is not localized within the FOV <NUM> of HMD <NUM>, HMD <NUM> may display information indicating the relative location of UAV <NUM>, such as a set of arrows directing pilot <NUM> to turn his or her head in order to bring UAV <NUM> back within FOV <NUM> of HMD <NUM>. As detailed further below, HMD <NUM> may be configured to display other relevant UAV flight information to pilot <NUM>, including but not limited to a UAV airspeed and/or direction, a distance from the UAV, a remaining UAV battery life, an intended UAV flightpath, or an indication of a controlled airspace <NUM> that UAV <NUM> must avoid.

<FIG> is a block diagram illustrating system <NUM> of <FIG> for determining a position of UAV <NUM> with respect to the FOV of HMD <NUM>. Example system <NUM> includes pilot <NUM>, UAV <NUM>, HMD <NUM>, ground station <NUM>, and UAV controller <NUM>.

Pilot <NUM> is a person who guides UAV <NUM> along a flightpath, such as to take aerial photographs, collect sensor data, or deliver a package, for example. UAV <NUM> is depicted (in <FIG>) as a four-rotor "quadcopter," however, UAV <NUM> may be any type of UAV including, but not limited to, a rotorcraft, a fixed-wing aircraft, compound aircraft such as tilt-rotor, X2 and X3, an aerostat, or any other such type of UAV including all vertical take-off and landing (VTOL), tail-sitter, etc. UAV <NUM> may be configured to fly with various degrees of autonomy. Although the techniques of this disclosure are not limited to any particular type of UAV, UAV <NUM> may, for example, be a relatively small, low-altitude, and low-airspeed UAV, where in this context, "small" corresponds to under <NUM> lbs. , "low-altitude" corresponds to operating altitudes less than <NUM> feet above ground, and "low-airspeed" corresponds to air speeds less than <NUM> knots. Furthermore, it is contemplated that UAV <NUM> may have hovering capabilities, meaning UAV <NUM> may have the capability of remaining at an approximately constant location in the air.

UAV <NUM> includes positioning system (PS) <NUM>, inertial measurement unit (IMU) <NUM>, orientation unit (OU) <NUM>, altimeter <NUM>, processing circuitry (PC) <NUM>, memory <NUM>, transceiver <NUM>, camera <NUM>, and sensor <NUM>. Although shown separately in <FIG> for purposes of illustrations, many of the described components of UAV <NUM> may in fact be highly integrated. For example, many of the described components of UAV <NUM> may be implemented into a single circuit or into a system on a chip. Various units or more modules may be combined into a single hardware unit or provided by a collection of interoperative hardware units, including one or more processors, in conjunction with suitable software and/or firmware.

Positioning System (PS) <NUM> includes any hardware and/or software configured to determine a relative location, such as a geolocation, of UAV <NUM>. In some examples, PS <NUM> may include a GPS system configured to determine a latitude and longitude of UAV <NUM>. In other examples, PS <NUM> may be configured to determine a location of UAV <NUM> based on nearby wireless internet signals, cell tower signals, or transponder signals. In yet other examples, positioning system <NUM> may include camera-based positioning systems. Generally speaking, positioning system <NUM> may include any one of or variety of types of positioning systems and is not limited to any one particular type of positioning system.

Inertial Measurement Unit (IMU) <NUM> is an electronic component or device configured to detect an acceleration, motion, and/or orientation of UAV <NUM>. IMU <NUM> may include one or more accelerometers, gyroscopes, and/or magnetometers. IMU <NUM>, alone or in combination with OU <NUM>, may be configured to output data indicative of a direction of flight of UAV <NUM>.

Orientation Unit (OU) <NUM> includes one or more devices configured to determine a relative orientation of UAV <NUM> with respect to the cardinal directions of the Earth. For example, OU <NUM> may include a magnetometer configured to measure a strength and direction of the Earth's magnetic field to identify the northern direction. In other examples, OU <NUM> includes a simple compass configured to identify the direction of magnetic north. In some examples, processing circuitry <NUM> may be configured to combine magnetic-based orientation data from OU <NUM> with location data from PS <NUM> in order to determine the orientation of UAV <NUM> relative to True North (as defined by Earth's axis of rotation). In other examples, OU <NUM> may be camera-based and usual visual landmarks to determine an orientation of UAV <NUM>.

Altimeter <NUM> is a device or component for determining a height of UAV <NUM> above the ground. In some examples, altimeter <NUM> may include a device configured to determine the altitude of UAV <NUM> based on atmospheric pressure. Altimeter <NUM> may include data from positioning system <NUM> to determine altitude based on a difference between a measured atmospheric pressure and the expected atmospheric pressure at the known elevation of ground level at the local latitude and longitude, as indicated by positioning system <NUM>. In other examples, altimeter <NUM> may include a signal transceiver configured to reflect a signal, such as an electromagnetic, sonar, or other signal, off of the ground and measure the time until the reflected signal is detected. In other examples, altimeter <NUM> may use stereoscopic images to determine an altitude.

UAV <NUM> includes processing circuitry (PC) <NUM> and memory <NUM> configured to process and store data, such as data received from transceiver <NUM> or from any other internal component of UAV <NUM>. PC <NUM> may include one or more processors configured to execute instructions, such as one or more digital signal processors (DSPs), general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuitry. Processing circuitry <NUM> may include analog and/or digital circuitry. The term "processor" or "processing circuitry," as used herein, may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein.

Memory <NUM> may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Memory <NUM> may include one or more of a hard drive, flash memory, volatile or nonvolatile memory, or any other suitable digital storage media for storing encoded video data.

Transceiver <NUM> includes one or more electronic components configured to wirelessly send and receive data, such as from ground station <NUM>, controller <NUM>, or HMD <NUM>. Transceiver <NUM> may represent any one or more of wireless transmitters/receivers, modems, networking components, wireless communication components that operate according to any of a variety of IEEE <NUM> standards, or other physical components. Transceiver <NUM> may be configured to transfer data according to a cellular communication standard, such as <NUM>, <NUM>-LTE (Long-Term Evolution), LTE Advanced, <NUM>, or the like, or according to other wireless standards, such as an IEEE <NUM> specification, an IEEE <NUM> specification (e.g., ZigBee™), a Bluetooth™ standard, or the like.

UAV <NUM> includes camera <NUM> and/or additional sensor <NUM> configured to capture photographic or other data of a designated target, such as a structure. Camera <NUM> may represent one or both of a monoscopic camera or stereoscopic camera configured to acquire images and/or video data. UAV <NUM> may store the images and video in memory <NUM> or may additionally or alternatively stream the images and video to another device, such as ground station <NUM>.

HMD <NUM> includes positioning system (PS) <NUM>, inertial measurement unit (IMU) <NUM>, orientation unit (OU) <NUM>, display <NUM>, camera <NUM>, processing circuity (PC) <NUM>, memory <NUM>, transceiver <NUM>, and altimeter <NUM>. HMD <NUM> is a head-mounted device that can include a variety of electronic components found in a computing system, including one or more processor(s) <NUM> (e.g., microprocessors or other types of processing units) and a memory <NUM> that may be mounted on or within a frame. Furthermore, HMD <NUM> may include a transparent display screen <NUM> that is positioned at eye level when HMD <NUM> is worn by a user, such as UAV pilot <NUM>. In some examples, display screen <NUM> can include one or more liquid crystal displays (LCDs) or other types of display screens on which images are perceptible to UAV pilot <NUM> who is wearing or otherwise using HMD <NUM> via display <NUM>. Other display examples include organic light emitting diode (OLED) displays. In some examples, HMD <NUM> can operate to project 3D images onto the user's retinas using techniques known in the art.

In some examples, display <NUM> may include see-through holographic lenses. sometimes referred to as waveguides, that permit a user to see real-world objects through (e.g., beyond) the lenses and also see holographic imagery projected into the lenses and onto the user's retinas by displays, such as liquid crystal on silicon (LCoS) display devices, which are sometimes referred to as light engines or projectors, operating as an example of a holographic projection system within HMD <NUM>. In other words, HMD <NUM> may include one or more see-through holographic lenses to present virtual images to a user. Hence, in some examples, HMD <NUM> can operate to project 3D images onto the user's retinas via display <NUM>, e.g., formed by holographic lenses. In this manner, HMD <NUM> may be configured to present a 3D virtual image to a user within a real-world view observed through display <NUM>, e.g., such that the virtual image appears to form part of the real-world environment. In some examples, HMD <NUM> may be a Microsoft HOLOLENS™ headset, available from Microsoft Corporation, of Redmond, Washington, USA, or a similar device, such as, for example, a similar MR visualization device that includes waveguides. The HOLOLENS ™ device can be used to present 3D virtual objects via holographic lenses, or waveguides, while permitting a user to view actual objects in a real-world scene, i.e., in a real-world environment, through the holographic lenses.

In other examples, display screen <NUM> may include an opaque (e.g., non-transparent) digital screen configured to display rendered imagery. Some non-limiting examples of devices having opaque display screens include the RealWear HMT-<NUM>, the Vuzix M300, or any standard smartphone or tablet. In the case of an AR HMD having an opaque display screen <NUM>, the user's optical axis (e.g., line of sight, or the center of the field of view) may differ slightly from the optical axis of camera <NUM>. In some examples, an AR HMD may have a display screen <NUM> configured to cover only one eye of a user ("monocular") or both eyes ("binocular").

As shown in <FIG>, HMD <NUM> can also generate a graphical display or user interface (UI) that is visible to the user, e.g., as holographic imagery projected into see-through holographic lenses as described above. Imagery presented by HMD <NUM> may include, for example, one or more 3D virtual objects. Details of example UIs are described elsewhere in this disclosure. HMD <NUM> also can include a speaker or other sensory devices that may be positioned adjacent the user's ears. Sensory devices can convey audible information or other perceptible information (e.g., vibrations) to assist the user of HMD <NUM>.

HMD <NUM> can also include a transceiver <NUM> to connect HMD <NUM> to a second processing device, such as intermediate ground station <NUM>, or directly to UAV <NUM>, and/or to a network and/or to a computing cloud, such as via a wired communication protocol or a wireless protocol, e.g., Wi-Fi, Bluetooth, etc. Transceiver <NUM> may represent any one or more of wireless transmitters/receivers, modems, networking components, wireless communication components that operate according to any of a variety of IEEE <NUM> standards, or other physical components. Transceiver <NUM> may be configured to transfer data according to a cellular communication standard, such as <NUM>, <NUM>-LTE (Long-Term Evolution), LTE Advanced, <NUM>, or the like, or according to other wireless standards, such as an IEEE <NUM> specification, an IEEE <NUM> specification (e.g., ZigBee™), a Bluetooth™ standard, or the like.

HMD <NUM> also includes a variety of sensors to collect sensor data, such as one or more optical camera(s) <NUM> (or other optical sensors) and one or more depth camera(s) (or other depth sensors), mounted to, on or within the frame. In some examples, the optical sensor(s) <NUM> are operable to scan the geometry of the physical environment in which user of HMD <NUM> is located and collect two-dimensional (2D) optical image data (either monochrome or color). Depth sensor(s) are operable to provide 3D image data, such as by employing time of flight, stereo or other known or future-developed techniques for determining depth and thereby generating image data in three dimensions. Camera <NUM> may represent one or both of a monoscopic camera or stereoscopic camera configured to acquire images and/or video data. HMD <NUM> may store the images and video in memory <NUM> or may additionally or alternatively stream the images and video to another device, such as ground station <NUM>. Other sensors can include motion sensors (e.g., Inertial Mass Unit (IMU) sensors <NUM>, accelerometers, etc.) to assist with tracking movement.

System <NUM> processes the sensor data so that geometric, environmental, textural, etc. landmarks (e.g., corners, edges or other lines, walls, floors, objects) in the user's environment or "scene" can be defined and movements within the scene can be detected. As an example, the various types of sensor data can be combined or fused so that the user of HMD <NUM> can perceive 3D images that can be positioned, or fixed and/or moved within the scene. When fixed in the scene, the user can walk around the 3D image, view the 3D image from different perspectives, and manipulate the 3D image within the scene using hand gestures, voice commands, gaze line (or direction) and/or other control inputs. As another example, the sensor data can be processed so that the user can position a 3D virtual object (e.g., a bounding box) on an observed physical object in the scene (e.g., UAV <NUM>) and/or orient the 3D virtual object with other virtual images displayed in the scene.

HMD <NUM> may include one or more processors, or processing circuitry (PC) <NUM> and memory <NUM>, e.g., within the frame of the HMD. PC <NUM> may include one or more processors configured to execute instructions, such as one or more digital signal processors (DSPs), general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuitry. Processing circuitry <NUM> may include analog and/or digital circuitry. The term "processor" or "processing circuitry," as used herein, may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein.

Memory <NUM> may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Memory <NUM> may include one or more of a hard drive, flash memory, volatile or nonvolatile memory, or any other suitable digital storage media for storing encoded video data.

In some examples, one or more external computing resources process and store information, such as sensor data, instead of or in addition to in-frame PC <NUM> and memory <NUM>. In this way, data processing and storage may be performed by one or more PC <NUM> and memory <NUM> within HMD <NUM> and/or some of the processing and storage requirements may be offloaded from HMD <NUM>. Hence, in some examples, one or more PC that control the operation of HMD <NUM> may be within the HMD, e.g., as PC <NUM>. Alternatively, in some examples, at least one of the processors that controls the operation of HMD <NUM> may be external to the HMD. Likewise, operation of HMD <NUM> may, in some examples, be controlled in part by a combination one or more PC <NUM> within the visualization device and one or more processors external to the visualization device.

For instance, in some examples, when HMD <NUM> is in the context of <FIG>, processing of the sensor data can be performed by PC <NUM> in conjunction with memory <NUM> or storage device(s). In some examples, PC <NUM> and memory <NUM> mounted to the frame may provide sufficient computing resources to process the sensor data collected by cameras <NUM> and motion sensors <NUM>. In some examples, the sensor data can be processed using a Simultaneous Localization and Mapping (SLAM) algorithm, or other known or future-developed algorithm for processing and mapping 2D and 3D image data and tracking the position of HMD <NUM> in the 3D scene. In some examples, image tracking may be performed using sensor processing and tracking functionality provided by the Microsoft HOLOLENS™ system, e.g., by one or more sensors and processors <NUM> within HMD <NUM> substantially conforming to the Microsoft HOLOLENS™ device or a similar mixed reality (MR) visualization device.

HMD <NUM> includes Positioning System (PS) <NUM>. PS <NUM> includes any hardware and/or software configured to determine a relative location, such as a geolocation, of HMD <NUM>. In some examples, PS <NUM> may include a GPS system configured to determine a latitude and longitude of HMD <NUM>. In other examples, PS <NUM> may be configured to determine a location of HMD <NUM> based on nearby wireless internet signals, cell tower signals, or transponder signals. In yet other examples, positioning system <NUM> may include camera-based positioning systems. Generally speaking, positioning system <NUM> may include any one of or variety of types of positioning systems and is not limited to any one particular type of positioning system.

HMD <NUM> includes Inertial Measurement Unit (IMU) <NUM>. IMU <NUM> is an electronic component or device configured to detect an acceleration, motion, and/or orientation of HMD <NUM>, such as with respect to gravity. IMU <NUM> may include one or more accelerometers, gyroscopes, and/or magnetometers. IMU <NUM> may be configured to output data indicative of an angle of HMD <NUM> with respect to gravity, correspondingly indicative of an angle of the field-of-view of HMD <NUM> with respect to the ground, which may be assumed to be locally perpendicular to the direction of gravity.

HMD <NUM> includes orientation unit (OU) <NUM>. OU <NUM> includes one or more devices configured to determine a relative orientation of HMD <NUM> with respect to the cardinal directions of the Earth. For example, OU <NUM> may include a magnetometer configured to measure a strength and direction of the Earth's magnetic field to identify the northern direction. In other examples, OU <NUM> includes a simple compass configured to identify the direction of magnetic north. In some examples, processing circuitry <NUM> may be configured to combine magnetic-based orientation data from OU <NUM> with location data from PS <NUM> in order to determine the orientation of HMD <NUM> relative to True North (as defined by Earth's axis of rotation). In other examples, OU <NUM> may be camera-based and usual visual landmarks to determine an orientation of HMD <NUM>.

In some examples, HMD <NUM> includes altimeter <NUM>. Altimeter <NUM> includes hardware and/or software for determining a height of HMD <NUM> above the ground. In some examples, the height of HMD <NUM>, when worn on the head of pilot <NUM>, is essentially negligible relative to the height of UAV <NUM> above the ground while in flight. In these examples, PC <NUM> determines the height of HMD <NUM> to be zero for the purposes of determining the relative location of UAV <NUM>. In other examples, altimeter <NUM> may include a device configured to determine the altitude of HMD <NUM> based on atmospheric pressure. Altimeter <NUM> may include data from positioning system <NUM> to determine altitude based on a difference between a measured atmospheric pressure and the expected atmospheric pressure at the known elevation of ground level at the local latitude and longitude, as indicated by positioning system <NUM>. In other examples, altimeter may include a signal transceiver configured to reflect a signal, such as an electromagnetic, sonar, or other signal, off of the ground and measure the time until the reflected signal is detected. Since HMD <NUM> is worn on the head of pilot <NUM>, in other examples, altimeter <NUM> may include a data input device for pilot <NUM> to input his or her height, approximately corresponding to the height of HMD <NUM> above the ground when pilot <NUM> is standing erect. In other examples, altimeter <NUM> may use stereoscopic images to determine an altitude.

Ground Station (GS) <NUM> is a ground-based computing device that includes transceiver <NUM>, controls <NUM>, and display <NUM>. In some examples, GS <NUM> is a fixed communication station, such as a radio tower. In other examples, GS <NUM> is a mobile computing device, such as a laptop, tablet, smartphone, or other computing device. As shown in <FIG>, GS <NUM> is configured to send and receive data, via transceiver <NUM>, from both UAV <NUM> and HMD <NUM>. In some examples, UAV <NUM> is configured to communicate directly with HMD <NUM>.

GS <NUM> includes one or more controls <NUM>, such as a keyboard, touchscreen, buttons, throttle, or other similar user-input devices. GS <NUM> further includes display <NUM>, such as a screen configured to display camera, sensor, and/or flight data from UAV <NUM>. In some examples, display screen <NUM> is configured to display a live video feed, as captured by camera <NUM> on UAV <NUM>. In some examples, ground station <NUM> includes transceiver <NUM> configured to receive telemetry data from sensors <NUM> of UAV <NUM> and wirelessly communicate the telemetry data to transceiver <NUM> of HMD <NUM>.

In some examples, system <NUM> can also include one or more user-operated control device(s) (controller(s)) <NUM> that allow the user to operate UAV <NUM>, interact with UIs and/or otherwise provide commands or requests to processing device(s) or other systems connected to the network. As examples, controller <NUM> can include a microphone, a touch pad, a control panel, a motion sensor or other types of control input devices with which the user can interact. In some examples, controller <NUM> may be incorporated within GS <NUM>. In other examples, controller <NUM> may be incorporated within HMD <NUM>. In other examples, controller <NUM> is a distinct, handheld device having one or more buttons, toggles, touchscreens, or other similar control inputs configured to control the motion of UAV <NUM>.

In some examples in accordance with this disclosure, and as detailed further with respect to <FIG>, below, processing circuitry <NUM> of HMD <NUM> is configured to receive location data from UAV <NUM> and at least location data from HMD <NUM>, and use the received data to determine a location of UAV <NUM> with respect to the FOV of display screen <NUM> of HMD <NUM>. The FOV of HMD <NUM>, between HMD <NUM> and UAV <NUM>, may assume the shape of a rectangular frustum, having an expanding horizontal width dimension and an expanding height dimension, both dimensions taken perpendicular to a line extending from HMD <NUM> to the center "C" of the field-of-view at the distance of UAV <NUM>. For example, processing circuitry <NUM> within HMD <NUM> may be configured to determine, based on the received location and orientation data, both a horizontal position (<FIG>) and a vertical position (<FIG>) of UAV <NUM> with respect to the center "C" of the FOV of HMD <NUM>. The example methods and processes (e.g., the specific disclosed calculations) of <FIG> are recited only as examples, and are not intended to be limiting.

<FIG> is a conceptual diagram illustrating an example process for determining a horizontal position of UAV <NUM> with respect to FOV <NUM> of HMD <NUM>, according to one or more techniques of this disclosure. In order to determine whether UAV <NUM> is located within the horizontal FOV of HMD <NUM>, HMD <NUM> (e.g., processing circuitry <NUM> within HMD <NUM>) determines the magnitudes of two horizontal distances: (<NUM>) a width FOVwidth between the center "C" of the field-of-view <NUM> of HMD <NUM> and the left edge FOVleft (or equivalently, the right edge FOVright) of the FOV at the distance of the UAV <NUM>; and (<NUM>) the horizontal distance dhor between UAV <NUM> and FOVcenter. Once HMD <NUM> determines the values (e.g., magnitudes) of FOVwidth and dhor, HMD <NUM> compares the two values to determine whether UAV <NUM> is located within the horizontal width of FOV <NUM>. For example, if dhor is less than FOVwidth, then UAV <NUM> is located within the horizontal width of FOV <NUM>. If dhor is greater than FOVwidth, then UAV <NUM> is not located within the horizontal width of FOV <NUM>.

Equivalently (e.g., additionally or alternatively), HMD <NUM> may determine whether the horizontal angle αΔ between FOV center C, HMD <NUM>, and UAV <NUM> is greater than or less than ½ fθ, wherein fθ represents the fixed angle of the horizontal field-of-view of HMD <NUM>. If αΔ is less than ½ fθ, then UAV <NUM> is located within the horizontal width of FOV <NUM>.

<FIG> is an overhead view of the diagram of <FIG>. In one example process, HMD <NUM> determines the values of αΔ, dhor, and/or FOVwidth based on one known value and three measured values. The horizontal angle fθ, defining the horizontal component of field-of-view <NUM>, is a fixed value defined by the size and relative position of display screen <NUM> and/or camera <NUM> (<FIG>) of HMD <NUM> as manufactured. For example, display screen <NUM> and camera <NUM> may each define a field of view which, in some cases, may be substantially aligned with one another. In some examples, the fields of view of display screen <NUM> and camera <NUM> may differ slightly, e.g., may not be perfectly aligned, however may be similar enough to be interchanged for the example techniques described herein. Accordingly, HMD <NUM> (e.g., PC <NUM> and memory <NUM>) is aware of the fixed value of ½ fθ.

HMD <NUM> (e.g., transceiver <NUM> and/or PC <NUM>) receives three other values measured by various sensors and/or detectors. Positioning system <NUM> (<FIG>) of HMD <NUM> determines a relative location of HMD <NUM>. For example, positioning system <NUM> may include a GPS device configured to determine a latitude and longitude (Hlat, Hlon) of HMD <NUM>. Similarly, positioning system <NUM> (<FIG>) of UAV <NUM> determines a relative location of UAV <NUM>. For example, positioning system <NUM> may include a GPS device configured to determine a latitude and longitude (Ulat, Ulon) of UAV <NUM>. Orientation unit <NUM> (<FIG>) within HMD <NUM> determines a relative orientation of HMD <NUM>. For example, orientation unit <NUM> may include a compass or magnetometer configured to determine a value for angle αHMD, the horizontal angle between FOV center C, HMD <NUM>, and north <NUM> (e.g., magnetic north and/or true north). Transceiver <NUM> and/or PC <NUM> (<FIG>) within HMD <NUM> receive data indicative of (Hlat, Hlon), (Ulat, Ulon), and αHMD.

Based on (Hlat, Hlon) and (Ulat, Ulon), HMD <NUM> determines distance dground, the horizontal "ground distance" along a straight line between HMD <NUM> and UAV <NUM>. For example, HMD <NUM> may implement the Pythagorean Theorem a<NUM> + b<NUM> = c<NUM>, wherein "a" is the difference "ΔLat" between Ulat and Hlat, "b" is the difference "ΔLon" between Ulon and Hlon, and "c" is equal to dground.

Using the determined values of dground, ΔLat, and ΔLon, HMD <NUM> may determine the value of αUAV, the horizontal angle between UAV <NUM>, HMD <NUM>, and north <NUM>. For example, using the known mathematical relationship that sine(θ) = (opposite / hypotenuse), HMD <NUM> may determine that the angle αUAV is equal to the inverse sine of (ΔLon / dground).

Using the measured value of αHMD and the determined value of αUAV, HMD <NUM> may determine the value of αΔ, as the difference between αHMD and αUAV. HMD <NUM> may then compare the determined value of αΔ to the fixed value of ½ fθ to determine whether αΔ is less than ½ fθ, and therefore, whether UAV <NUM> is within the horizontal field-of-view <NUM> of HMD <NUM>. Using further trigonometric ratios, HMD <NUM> may similarly determine the values of FOVwidth and dhor to determine the approximate horizontal location of UAV <NUM> within FOV <NUM> (e.g., the approximate horizontal location of UAV <NUM> on display screen <NUM>). For example, dhor is equal to dground * sine(αΔ). Horizontal distance dCH from HMD <NUM> to FOV center C is equal to dground * cosine(αΔ). FOVwidth is equal to dCH * tangent(½ fθ).

If HMD <NUM> determines that UAV <NUM> is within the horizontal field-of-view, PC <NUM> may take the ratio of (dhor / FOVwidth) to determine the approximate location (as a percent of the screen width) on display screen <NUM> to place a bounding box indicating the location of UAV <NUM>.

<FIG> is a conceptual diagram illustrating an HMD determining a vertical location of a UAV with respect to an FOV of the HMD, according to one or more techniques of this disclosure. HMD <NUM> (e.g., processing circuitry within HMD <NUM>) determines the magnitudes of two vertical distances: a height FOVheight between the center "C" of the field-of-view <NUM> of HMD <NUM> and the top edge FOVtop (or equivalently, the bottom edge FOVbottom) of the FOV at the distance of UAV <NUM>; as well as the vertical distance dvert between UAV <NUM> and FOV center C. Once HMD <NUM> determines the values of FOVheight and dvert, HMD <NUM> compares the two values to determine whether UAV <NUM> is located within the vertical height of FOV <NUM>. For example, if dvert is less than FOVheight, then UAV <NUM> is located within the vertical height of FOV <NUM>. If dvert is greater than FOVheight, then UAV <NUM> is not located within the vertical height of FOV <NUM>.

Equivalently, HMD <NUM> may determine whether the vertical angle βΔ between FOV center C, HMD <NUM>, and UAV <NUM> is greater than or less than ½ fϕ, wherein fϕ represents the fixed angle of the vertical field-of-view of HMD <NUM>. If βΔ is less than ½ fϕ, then UAV <NUM> is located within the vertical height of FOV <NUM>.

<FIG> is a side view of the diagram of <FIG>. HMD <NUM> determines the values of βΔ, dvert, and/or FOVheight based on or more known and measured values. The vertical angle fϕ of the vertical field-of-view <NUM> is a fixed value defined by the size and relative position of display screen <NUM> (<FIG>) of HMD <NUM> as manufactured. Accordingly, HMD <NUM> is aware of the value of ½ fϕ.

HMD <NUM> receives three other values input or measured by various sensors and/or detectors. HMD <NUM> determines a height or altitude AHMD of HMD <NUM> above the ground. For example, since HMD <NUM> is intended to be worn on the head of pilot <NUM>, AHMD may be approximately equal to the height of pilot <NUM> if pilot <NUM> is standing erect while controlling HMD <NUM>. System <NUM> may include means for receiving input indicating the height of pilot <NUM>. In some examples, HMD <NUM> may include its own altimeter, similar to altimeter <NUM> of UAV <NUM>. In some examples, system <NUM> may assume for simplicity of computation that AHMD is significantly smaller (e.g., negligible) compared to the altitude of UAV <NUM> while in flight, and accordingly, set AHMD equal to zero.

Altimeter <NUM> (<FIG>) of UAV <NUM> determines a height or altitude AUAV of UAV <NUM> above the ground. Transceiver <NUM> (<FIG>) within HMD <NUM> receives data indicative of AUAV. Inertial measurement unit (IMU) <NUM> (<FIG>) within HMD <NUM> determines a relative orientation of HMD <NUM> with respect to gravity. For example, IMU <NUM> may include an accelerometer or other sensor configured to determine a value for angle βHMD, the vertical angle between a horizontal axis (e.g., parallel to the ground) extending from HMD <NUM> and field-of-view center C. HMD <NUM> determines the horizontal ground distance dground between HMD <NUM> and UAV <NUM> as described with respect to <FIG>, above.

HMD <NUM> determines distance dcv, the vertical distance between the horizontal axis and FOV center C, as equal to dground * tan(βHMD). Using dCV, HMD <NUM> determines distance dvert, the vertical distance between UAV <NUM> and field-of-view center C, as equal to (AUAV - AHMD - dCV).

Using dvert, HMD <NUM> determines angle βUAV, the angle between the horizontal axis, HMD <NUM>, and UAV <NUM>, as equal to the inverse tangent of [(dvert + dCV) / dground]. Using βUAV, HMD <NUM> determines βΔ as the difference between βUAV and βHMD. HMD <NUM> determines fheight as equal to [dground * tangent(βHMD + ½ fϕ)] - dCV.

HMD <NUM> may then compare the determined value of βΔ to the fixed value of ½ fϕ to determine whether βΔ is less than ½ fϕ, and therefore respectively whether UAV <NUM> is within the vertical field-of-view <NUM> of HMD <NUM>. HMD <NUM> may similarly use the values of FOVheight and dvert to determine the approximate vertical location of UAV <NUM> within FOV <NUM> (e.g., the approximate vertical location of UAV <NUM> on display screen <NUM>). Additionally, HMD <NUM> may use the determined values to further determine the values of AC (the altitude of field-of-view center "C" at the distance of UAV <NUM>), dC (the distance between HMD <NUM> and field-of-view center C), and dH-U (the distance between HMD <NUM> and UAV <NUM> along a direct line-of-sight <NUM>).

If HMD <NUM> determines that UAV <NUM> is within the vertical field-of-view, PC <NUM> may determine the ratio of (dvert / FOVheight) to determine the approximate location (as a percent of the screen height) on display screen <NUM> to place a graphical object, such as a bounding box, indicating the location of UAV <NUM>.

<FIG> is an example user interface (UI) or graphical user interface (GUI) <NUM> that may be generated and displayed on display screen <NUM> of HMD <NUM> (<FIG>), according to one or more techniques of this disclosure. In the example GUI <NUM> depicted in <FIG>, processing circuitry <NUM> of HMD <NUM> (<FIG>) has determined that a location of UAV <NUM> is within the field-of-view of display screen <NUM>, according to the example techniques described with respect to <FIG>. In this case, processing circuitry <NUM> generates and outputs for display on display screen <NUM> a graphical object, such as bounding box <NUM>, indicating the location of UAV <NUM> with respect to the screen. Although <FIG> depicts the graphical object as a rectangular bounding box <NUM>, the graphical object may take the form of any graphical indication of the UAV's location, such as any other geometric shape such as a circle or triangle, an approximate outline around UAV <NUM>, or an indication of highlighting, shading, blinking or other visual identification around the image of UAV <NUM> on display screen <NUM>.

In some examples, HMD <NUM> (e.g., transceiver <NUM> within HMD <NUM>) may receive orientation data and inertial data from OU <NUM> and IMU <NUM>, respectively, such that PC <NUM> may "predict" a subsequent relative location of UAV <NUM> and update bounding box <NUM> accordingly. By determining a position of UAV <NUM> before UAV <NUM> arrives at that location, HMD <NUM> may reduce the "lag time" between the arrival of UAV <NUM> at a particular location and the display of a bounding box <NUM> over that location.

GUI <NUM> further includes a number of virtual elements indicative of data obtained by sensors within UAV <NUM>, HMD <NUM>, or both. For example, GUI <NUM> includes elements indicating a distance <NUM> between UAV <NUM> and HMD <NUM>, an airspeed and orientation <NUM> of UAV <NUM>, a number of satellites <NUM> connected to positioning system components <NUM>, <NUM> (<FIG>), and an estimated remaining battery life <NUM> of UAV <NUM>.

<FIG> is an example UI or GUI <NUM> that may be generated and displayed on display screen <NUM> of HMD <NUM> (<FIG>), according to one or more techniques of this disclosure. In the example GUI <NUM> depicted in <FIG>, processing circuitry <NUM> of HMD <NUM> (<FIG>) has determined that the location of UAV <NUM> is not within the field-of-view of display screen <NUM>, according to the example techniques described with respect to <FIG>. In this case, processing circuitry <NUM> generates and outputs for display on display screen <NUM> a textual alert <NUM> indicating that UAV pilot <NUM> has lost the line-of-sight with UAV <NUM>. In some examples, HMD <NUM> may additionally or alternatively output an audio alert indicating the same. GUI <NUM> further includes a graphical object, such as a set of arrows <NUM>, indicating to the pilot <NUM> which direction to turn their head in order to bring UAV <NUM> back within the field-of-view of display screen <NUM>. Although <FIG> depicts the graphical object as set of arrows <NUM>, the graphical object may take any other direction-indicating form, such as a single arrow or a flashing light along a respective edge of display screen <NUM>. In some examples, additional or alternative to a graphical object, HMD <NUM> may output an audible indication, such as a tone or sound in the respective ear of the wearer, indicating a direction for the wearer to turn his or her head.

<FIG> is an example UI or GUI <NUM> that may be generated and displayed on display screen <NUM> of HMD <NUM> (<FIG>), according to one or more techniques of this disclosure. In the example GUI <NUM> depicted in <FIG>, processing circuitry <NUM> of HMD <NUM> (<FIG>) has determined that the location of UAV <NUM> is within the field-of-view of display screen <NUM> and has generated bonding box <NUM> around the UAV's location. However, transceiver <NUM> of HMD <NUM> has received data from altimeter <NUM> of UAV <NUM> indicating that UAV <NUM> is flying too low and may be at risk of crashing into the ground. In this case, processing circuitry <NUM> generates and outputs for display on display screen <NUM> a textual alert <NUM> recommending that UAV pilot <NUM> should operate controller <NUM> so as to increase the altitude of UAV <NUM>. In some examples, HMD <NUM> may additionally or alternatively output an audio alert indicating the same.

In some other examples, processing circuitry <NUM> may generate and output for display on screen <NUM> several other virtual or graphical elements not shown in <FIG>. For example, screen <NUM> may display a planned flightpath for UAV <NUM>, including an indication, such as a bounding box, of any potential obstacles within the flightpath. In another example, screen <NUM> may display an indication of a controlled airspace or a "no-fly zone". For example, screen <NUM> may receive data indicative of an oil refinery and generate a virtual boundary indicating a threshold distance, such as <NUM> meters, that UAV <NUM> must remain from the refinery. In some examples, screen <NUM> may display the live video feed from camera <NUM> of UAV <NUM>, displayed as a "picture-in-picture" within the larger field-of-view of screen <NUM>. In some examples, screen <NUM> may display a graphical element indicating the field-of-view of camera <NUM> of UAV <NUM>, such as a triangular graphical element with UAV <NUM> at its vertex. In some examples, screen <NUM> may display a number of other flight indications or warnings, such as indicating a low UAV battery, a lost GPS signal, an obstacle in close proximity, or the completion of one or more mission milestones. For example, processing circuitry <NUM> may perform image processing on image data captured by camera <NUM> of HMD <NUM>, camera <NUM> of UAV <NUM>, or both, in order to recognize one or more obstacles in close proximity to UAV <NUM>, such as birds or nearby aircraft. In other examples, processing circuitry may determine the presence of nearby aircraft when transceiver <NUM> receives an automatic dependent surveillance broadcast (ADS-B) signal from the aircraft.

<FIG> is a flowchart illustrating an example operation for determining and displaying a location of a UAV with respect to an FOV of an HMD, in accordance with a technique of this disclosure. The example techniques of <FIG> are described with respect to system <NUM> of <FIG> and <FIG>, but the techniques may be performed by any adequate computing system. Processing circuitry <NUM> receives, via transceivers <NUM> and <NUM>, data indicative of a geolocation of UAV <NUM>, such as from GPS device <NUM> and altimeter <NUM> installed within the UAV. Processing circuitry <NUM> further receives data indicative of an orientation of UAV <NUM>, such as from compass <NUM> and IMU <NUM> installed within the UAV (<NUM>).

Processing circuitry <NUM> further receives data indicative of a geolocation of HMD <NUM>, such as from GPS device <NUM> installed within the HMD. Processing circuitry <NUM> further receives data indicative of an orientation of HMD <NUM>, such as from compass <NUM> and IMU <NUM> installed within the HMD (<NUM>).

Processing circuitry <NUM> determines, from the position and orientation data of both UAV <NUM> and HMD <NUM>, a relative location of UAV <NUM> with respect to a field-of-view of display screen <NUM> of HMD <NUM> (<NUM>). For example, processing circuitry <NUM> determines whether UAV <NUM> is within the field-of-view or outside the field-of-view of the screen.

Processing circuitry <NUM> generates and outputs for display on screen <NUM> an indication of the location of UAV <NUM> (<NUM>). For example, if UAV <NUM> is within the field-of-view of display screen <NUM>, processing circuitry <NUM> generates and outputs a rectangular-shaped bounding box around the approximate location of UAV <NUM> with respect to the screen. In examples in which UAV <NUM> is not within the field of view of display screen <NUM>, processing circuitry generates and outputs an indication, such as an arrow or set of arrows, indicating the location of UAV <NUM> with respect to screen <NUM>, so that the user of HMD <NUM> may turn his or her head to bring UAV <NUM> back within the field-of-view.

<FIG> is a flowchart illustrating an example operation for determining and displaying a location of a UAV with respect to an FOV of an HMD, in accordance with a technique of this disclosure. The example techniques of <FIG> are described with respect to system <NUM> of <FIG> and <FIG>, but the techniques may be performed by any adequate computing system. In some examples, HMD <NUM> establishes a data-communication connection with ground station <NUM>, such as a laptop or other data-transfer device (<NUM>). In other examples, HMD <NUM> communicates directly with UAV <NUM>. HMD <NUM> receives UAV position data, such as from GPS sensor <NUM> and an altimeter <NUM> within UAV <NUM> (<NUM>). HMD <NUM> receives HMD position data, such as from GPS sensor <NUM> within HMD <NUM> (<NUM>).

HMD <NUM> receives UAV orientation data, such as from compass <NUM> and IMU <NUM> within UAV <NUM> (<NUM>). HMD <NUM> receives HMD position data, such as from compass <NUM> and IMU <NUM> within HMD <NUM> (<NUM>).

Using the position and orientation data of both HMD <NUM> and UAV <NUM>, HMD <NUM> determines a distance between HMD <NUM> and UAV <NUM> (<NUM>). Processing circuitry <NUM> within HMD <NUM> generates a spherical coordinate plane, centered within HMD <NUM> such that the boundary of the field-of-view of display screen <NUM> extends along a set of radii of the sphere (<NUM>). Based on the spherical coordinate plane, processing circuitry <NUM> determines a frustum bounded by the field-of-view of camera <NUM> (<NUM>). Processing circuitry <NUM> determines the distance between the approximate position of UAV <NUM> and the center of the determined frustum (e.g., the center of the spherical coordinate plane) (<NUM>). Processing circuitry <NUM> determines, based on the distance, whether the location of UAV <NUM> is within the determined frustum (<NUM>).

If UAV <NUM> is outside the frustum dimensions, processing circuitry <NUM> determines and indicates a direction to move the frustum (e.g., by the wearer of HMD <NUM> turning his or her head) in order to reduce the distance between the frustum and UAV <NUM> (<NUM>), and then repeats the previous steps as necessary until UAV <NUM> is within the frustum dimensions.

If UAV <NUM> is within the frustum dimensions, processing circuitry <NUM> generates and outputs for display a graphical indication, such as a rectangular bounding box, around the UAV location (<NUM>).

<FIG> is a flowchart illustrating an example operation for determining and displaying a location of a UAV with respect to an FOV of an HMD, in accordance with a technique of this disclosure. The example techniques of <FIG> are described with respect to system <NUM> of <FIG> and <FIG>, but the techniques may be performed by any adequate computing system. An Unmanned Aerial Vehicle System <NUM> includes an augmented-reality (AR) head-mounted display (HMD) device <NUM> having processing circuitry (PC) <NUM> configured to at least determine a location of an unmanned aerial vehicle (UAV) <NUM> (<NUM>). For example, UAV <NUM> may include a positioning system (PS) <NUM>, such as a global positioning system (GPS), an altimeter <NUM>, and a transceiver <NUM> configured to transmit a UAV latitude Ulat, a UAV longitude Ulon, and an altitude AUAV of UAV <NUM> above the ground, to HMD <NUM>.

PC <NUM> of HMD <NUM> may also be configured to determine a position and orientation of HMD <NUM> (<NUM>). For example, HMD <NUM> may include its own positioning system (PS) <NUM> and an orientation unit (OU) <NUM> configured to determine an HMD latitude Hlat, an HMD longitude Hlon, a cardinal direction heading or bearing αHMD, and an angle with respect to gravity βHMD.

present, on the AR HMD, an indication of the location of the UAV based on the orientation of the AR HMD.

Using at least this data, PC <NUM> of HMD <NUM> is configured to determine whether the location of the UAV (including the geolocation and the altitude) of the UAV is within a field-of-view (FOV) of HMD <NUM> (<NUM>). In some examples, PC <NUM> may construct a grid-based coordinate system to determine whether UAV <NUM> is positioned within both a horizontal field-of-view and a vertical field-of-view of HMD <NUM>. In other examples, PC <NUM> may construct a spherical coordinate system centered at HMD <NUM> and determine whether the angle between the center C of the FOV and UAV <NUM> is greater than or less than the fixed angle of the FOV of HMD <NUM>. In some examples, PC <NUM> may determine the relative position of UAV <NUM> using a combination of both grid-based distances and spherical angles.

Responsive to (e.g., based on) determining whether UAV <NUM> is within the FOV of HMD <NUM>, PC <NUM> is configured to generate and present on display screen <NUM> of AR HMD <NUM> an indication of the relative location of the UAV. For example, responsive to determining that UAV <NUM> is within the FOV of HMD <NUM>, PC <NUM> is configured to present a first graphical object on HMD <NUM> (<NUM>). The first graphical object may include, for example, a bounding box indicating possible locations of the UAV, or any other visual-based UAV-location indication.

PC <NUM> is also configured, responsive to determining that UAV <NUM> is not within the FOV of HMD <NUM>, to present a second graphical object on HMD <NUM> (<NUM>). The second graphical object may include, for example, a set of arrows or other visual-based indication of the relative location of UAV <NUM> with respect to HMD <NUM>.

In some examples, PC <NUM> is also configured to determine a location of an obstacle near the flightpath of UAV <NUM>, and present, on the AR HMD, an indication of the location of the obstacle based on the orientation of the AR HMD. For example, PC <NUM> may determine a location of the obstacle by processing image data from an HMD camera <NUM> or a UAV camera <NUM>, and then display a bounding box around the obstacle on display screen <NUM>. Examples of flightpath obstacles include structures (e.g., buildings, radio towers), birds, manned aircraft (such as airplanes, helicopters, etc.), a second UAV, or terrain (such as rocks, trees, hills, etc.).

In some examples, PC <NUM> is further configured to determine a location of a controlled airspace and present, on the AR HMD, an indication of the location of the controlled airspace based on the orientation of the AR HMD. For example, a nearby controlled airspace may include airports, oil refineries, sporting venues, or other regulated airspaces having surrounding no-fly threshold distances.

In some examples, PC <NUM> is configured to determine a field of view of camera <NUM> of UAV <NUM> and output for display on HMD <NUM> an indication of the field of view of camera <NUM>, such that the wearer of HMD <NUM> may visually determine the general direction of camera <NUM> and its target capture window.

In some examples, PC <NUM> may be further configured to determine and output for display one or more of a remaining battery life of UAV <NUM>, an altitude AUAV of UAV <NUM>, an airspeed of UAV <NUM>, a number of satellites in communication with UAV <NUM>, or a compass heading of UAV <NUM>.

In some examples, PC <NUM> may be further configured to output for display on HMD <NUM> an alert, such as responsive to determining, and indicative of one or more of a low UAV battery, a GPS-signal lost, an obstacle-proximity warning, a UAV mission milestone reached.

In some examples, transceiver <NUM> and PC <NUM> may be configured to receive, from UAV camera <NUM>, video data, and output for display on HMD <NUM> the video data. For example, display screen <NUM> of HMD <NUM> may include a "picture in picture" type window featuring a live feed of recorded video data from camera <NUM> of UAV <NUM>.

In some examples, transceiver <NUM> and PC <NUM> may be configured to receive a planned flightpath of UAV <NUM> and output for display on HMD <NUM> an indication of the flightpath, such as with respect to the current position or location of UAV <NUM>.

In some examples, PC <NUM> may be further configured to determine a loss of line-of-sight between HMD <NUM> and UAV <NUM> and output for display on HMD <NUM> an indication of the loss of line-of-sight.

If implemented in software, the functions may be stored on or transmitted over, as one or more instructions or code, a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media <NUM>, <NUM>, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol.

By way of example, and not limitation, such computer-readable storage media <NUM>, <NUM> can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.

Instructions may be executed by one or more processors <NUM>, <NUM>, such as one or more DSPs, general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuitry.

Cloud technology used to automatically save the images on web server is not limited to local or global internet cloud. It can be a private and/or public cloud which is protected by the user ID and passwords. The passwords may not limit to one or two.

Claim 1:
A device comprising:
an augmented-reality (AR) head-mounted display (HMD) (<NUM>); and
processing circuitry configured to:
receive a planned flightpath of an unmanned aerial vehicle (UAV) (<NUM>);
determine a location of the UAV (<NUM>);
determine a position and orientation of the AR HMD (<NUM>);
output for display on the AR HMD (<NUM>) a graphical element indicating the planned flightpath with respect to the location of the UAV (<NUM>);
based on the location of the UAV (<NUM>) and the position and orientation of the AR HMD (<NUM>), determine that the UAV (<NUM>) is outside of a field of view (<NUM>) of the AR HMD (<NUM>);
in response to determining that the UAV is outside of the field of view (<NUM>) of the AR HMD (<NUM>), present, on the AR HMD (<NUM>), an indication of a movement to bring the UAV (<NUM>) within the field of view (<NUM>) of the AR HMD (<NUM>);
determine a location of an obstacle near the planned flightpath; and
present, on the AR HMD (<NUM>), a graphical element indicating the location of the obstacle based on the orientation of the AR HMD (<NUM>).