Patent ID: 12216481

DETAILED DESCRIPTION

In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context indicates otherwise. The illustrative implementations described in the detailed description, drawings, and claims are not meant to be limiting. Other implementations can be utilized, and other changes can be made, without departing from the spirit or scope of the subject matter presented. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.

The systems, apparatuses, and methods described herein navigate aerial vehicles responsive to a deviation from a route. For example, a pilot of the aerial vehicle can manually steer an aerial vehicle to deviate from the route, or an obstacle along the route can cause a deviation therefrom. This technical solution determines a return path to the route. This technical solution can generate a vector field for an environment including the route. The vector field can include vectors merging to the route. A return path can be generated traversing the vector field to return to the route. The return path can be generated or filtered based on kino dynamic constraints of an aerial vehicle via a motion planner such as closed-loop RRT. The return path can be associated with a cost. For example, the cost can be indicative of a portion of the return path traveled against the vector field. The execution of the systems, apparatuses, and methods described herein can cause the aerial vehicle, or data processing system thereof, to determine a return path to a route without violating kino dynamic constraints of the aerial vehicle.

Referring now toFIG.1, illustrated is a block diagram of an example data processing system100, in accordance with one or more implementations. The data processing system100can include at least one vehicle interface102. The data processing system100can include at least one route generator104. The data processing system100can include at least one vector field analyzer106. The data processing system100can include at least one vehicle sensor108. The data processing system100can include at least one trajectory follower110. The data processing system100can include at least one data repository120.

The vehicle interface102, route generator104, vector field analyzer106, vehicle sensor108, or trajectory follower110can each include one processing unit or other logic device such as programmable logic array engine, or module configured to communicate with the data repository120or database. The vehicle interface102, route generator104, vector field analyzer106, vehicle sensor108, or trajectory follower110can be separate components, a single component, or part of the data processing system100. The data processing system100can include hardware elements, such as one or more processors, logic devices, or circuits. For example, the data processing system100can include one or more components, structures or functionality of a computing device depicted inFIG.8.

The data repository120can include one or more local or distributed databases, and can include a database management system. The data repository120can include computer data storage or memory and can store one or more data elements for a world model122, local sensor data124, or kino dynamic constraint126. The world model122can include major terrain features such as mountains or valleys, or restricted airspace. Restricted airspace can include lateral or vertical components. For example, restricted airspace can include some or all elevations within a lateral boundary, or all airspace in excess of an elevation above ground level or a terrain feature, such as for nape-of-the-earth (NoE) flight. The predefined route can be based on the kino dynamic constraints126of an aerial vehicle, and include motion along a three dimensional environment (e.g., by a vehicle having six degrees of freedom). The local sensor data124can include information relating to a local environment such as the position of additional vehicles, vegetation, or snow drifts. The local sensor data124can include environmental data such as vehicle information (e.g., speed, position, heading, or rotation) or weather conditions (e.g., wind speed, air density, or the like).

The kino dynamic constraints126can include information relating to a locomotion of an aerial vehicle. For example, a maximum or minimum airspeed, a maximum allowed climb rate, a maximum allowed rotational speed, minimum turn radius, or a maximum allowed thrust associated with the aircraft which are a function of vehicle mass, density, or altitude. The kino dynamic constraints126can be constraints of the aerial vehicle, the crew or other passengers, or a cargo thereof. The data processing system100can select, from the data repository120, various kino dynamic constraints126such as a crewed kino dynamic constraint126or un-crewed kino dynamic constraint126. The various kino dynamic constraints126can be selected by the data processing system100by manual entry, aircraft condition, route type, environmental condition, or the like.

Still referring toFIG.1, and in further detail, the data processing system100can include at least one vehicle interface102designed, constructed or operational to control the path of travel of an aerial vehicle. The vehicle interface102can include autonomous navigation controls or manual navigation controls. The vehicle interface102can include a state control to alternate between various states or modes of operation. For example, the state control of the vehicle interface102can transition the aerial vehicle between autonomous and manual modes of operation (e.g., semi-autonomous modes of operation) based on a presence or absence of user input, or responsive to a user command. Manual modes of operation can include a manual actuation of various aircraft portions by a pilot of the aerial vehicle, or a manual entries of airspeed, headings, or other flight controls in real time, such as by a pilot or remote operator of the aircraft.

The vehicle interface102can include manual flight controls (e.g., throttle, anti-torque controls or a cyclic control). The manual flight controls can include manual control of various aircraft portions such as main rotor tilt (e.g., via a swashplate), throttle, or tail rotor control. The manual controls can include combinatorial control of various aircraft portions such as via a collective control. The manual flight controls can receive user input during manual operation to navigate the aircraft. A mixing unit, detection unit for the state control, or semi-autonomous system can intermediate the various manual controls and respective aircraft portions. For example, a semi-autonomous flight control input (e.g., an autonomy inceptor) can receive an input from a pilot, determine one or more instructions to realize the input, and convey the instructions to various portions of the aircraft.

Although various examples provided herein may refer to rotary aircraft or components thereof, such references are not intended to be limiting. The systems and methods disclosed herein can be applied to various vehicles including fixed wing aircraft. For example, the vehicle control interface102can include a control yoke or autonomous systems to manipulate ailerons, elevators, rudders, or the like, and may omit anti-torque controls or a cyclic control according to a particular vehicle control interface102.

The detection unit can detect an engagement of a manual control. A manual flight control (e.g., a semi-autonomous input) can detect a user input during operation. The vehicle interface102can transition, responsive to the detect input, from an autonomous flight mode to a manual flight mode. The detection unit can detect a lack of user input. For example, the detection unit can detect a lack of user input for a time exceeding a threshold (e.g., 3 seconds, 5 seconds, or 10 seconds). Responsive to the detected lack of input, the vehicle interface102can transition to an autonomous mode of operation.

The vehicle interface102can present information to a pilot via a graphical user interface (GUI) or various visual or audio indicators (e.g., audible alarms, LEDs, or haptic feedback). The GUI can present information such as a position of a vehicle, a route associated with the aerial vehicle, or a return path to the route. The vehicle interface102can include one or more inputs such as buttons (e.g., a keypad) or touchscreens, levers, pedals, and the like. The vehicle interface102can receive indications of transitions between modes of operation (e.g., manual or autonomous modes of operation) from the pushbuttons. For example, prior to an expiration of a time exceeding a threshold, the vehicle interface102can receive an indication to transition to an autonomous mode, or to remain in a manual mode of operation. Various routes, paths to return to routes, or indications of a mode of operation can be presented by the GUI.

The data processing system100can include at least one route generator104designed, constructed or operational to establish a path of travel between locations. As discussed herein, a route can be a path of travel which is defined continuously or as a series of waypoints. Each waypoint can include heading information, a 3D position, or a velocity. The route generator104can determine a route or a return path thereto based on the world model122data, local sensor data124, or kino dynamic constraints126. For example, the route generator104can determine a route or a return path thereto based on weather conditions, terrain, or other obstacles (e.g., structures, air space restrictions, or desired travel corridors). The route generator104can generate the route prior to the aerial vehicle traversing all or a portion of the route, such as prior to a liftoff of the aerial vehicle. The route generator104can base the route on a various parameters. For example, the route generator104can implement a cost function to minimize or otherwise adjust a route distance, fuel use, time to transit the route, or an amount of acceleration at one or more points along the route.

During navigation, the aerial vehicle can depart from the route. For example, a pilot in or otherwise associated with the aerial vehicle can manually adjust the flight path of the aerial vehicle to deviate from the route, or an autonomous flight system can receive an indication of an obstacle or other condition (e.g., airspace or weather restriction) indicative of a departure from the route, such as a condition detected by a vehicle sensor108. The route generator104can, responsive to the detection of the obstacle or deviation from the route, generate a path to return to the route.

The route generator104can determine a route or return path based on a search-based, sample-based, or optimization-based algorithm. For example, the route generator104can employ at least one A*, rapidly exploring random trees (RRT), or particle swarm optimization (PSO) technique. The route generator104can employ various predictive based motion planning techniques such as model predictive control (MPC) or closed loop RRT (CL-RRT). The route generator104can avoid potential obstacles on the return path, on the route, or otherwise present in an environment. The route generator104can determine the return path based on a kino dynamic constraint126of the aerial vehicle, such that the return path is viable for the aerial vehicle or any associated crew or cargo. The route generator104can provide one or more routes to the vector field analyzer106to determine a cost therewith. The route generator104can generate paths or portions thereof based on feedback from the vector field analyzer106. For example, the route generator104can increase a number of portions of a potential path for paths having a relatively low cost associated therewith, or decrease a number of portions of a path for paths having a relatively high cost associated therewith. The route generator104can increase a path length or select a direction of a portion of the cost based on feedback from the vector field analyzer.

The data processing system100can include at least one vector field analyzer106designed, constructed or operational to determine a cost associated with a return path or a portion thereof. The vector field analyzer106can generate a vector field based on a position of an aerial vehicle, a route associated with the aerial vehicle, a destination associated with the route, or an obstacle. For example, the vector field analyzer106can define a substrate for the vector field, such as a rectilinear grid. The rectilinear grid can be oriented to cardinal directions. For example, gridlines can be aligned with latitudinal lines, or longitudinal lines. The grid can include a resolution based on a size, speed, or turn radius of an aerial vehicle, or a compute power of the data processing system100. The grid can include a same or different resolution as other portions of the data processing system100. For example, an obstacle avoidance system can operate at 10 meter or 1 meter intervals, and the gridlines can be defined at 50 meter, 100 meter interval, or 200 meter intervals. The grid can be a two dimensional grid, or a three dimensional grid. For example, the grid can be aligned to Cartesian, polar, or other coordinate systems in real space, or along a projection thereof.

For each gridline intersection of the grid, or intersections within a distance (e.g., a radius) of the aerial vehicle or the route, an origin can be defined, and a vector can be calculated with respect to the route. For example, the vector can be determined based on a direction to the route or a waypoint thereof, or by a direction of travel along the route. Transit along the vector can thus merge into the route. In some implementations, travel directly along the vectors can violate a kino dynamic constraint126of an aerial vehicle (e.g., can exceed an available thrust, turn radius, g-force, attitude, or the like). The vector field analyzer106can compare a return path to the vector field. For example, the vector field analyzer106can determine a cost for each segment of a return path corresponding to one or more vectors of the vector field that are adjacent to the discretized points along the return path (e.g., based on a dot product therebetween). The vector field analyzer106can sum the costs to generate a return path cost. The vector field analyzer106can compare the return path costs associated with return paths to each other or to a threshold. For example, the vector field analyzer106can determine a return path cost is below a threshold, and thereafter provide the return path to the trajectory follower110to steer the aircraft along the return path having the cost below the threshold. The vector field analyzer106can select a return path based on the cost. The vector field analyzer106can select a return path having a lowest cost for provision to the trajectory follower110to navigate the selected return path.

The data processing system100can include at least one vehicle sensor108designed, constructed or operational to detect environmental conditions associated with an aerial vehicle. For example, the vehicle sensor108can detect a wind speed (e.g., averaged, maximum, or current). The vehicle sensor108can detect additional vehicles, terrain features, structures, and the like. For example, the vehicle sensor108can include a terrain following radar or other terrain awareness or monitoring systems. The vehicle sensors108can provide additional terrain detail relative to the world model122. The vehicle sensors108can detect a position, direction of travel, or other information of additional vehicles. The vehicle sensor108can provide the local sensor data124to the route generator104, the vehicle interface102, or the data repository120for distribution throughout the data processing system100.

The vehicle sensor108can be mounted on the interior or the exterior of the aerial vehicle. Non-limiting examples of the vehicle sensors108include LiDAR sensors, visible light sensors (e.g., cameras, video capture devices, etc.), infrared light sensors, accelerometers, gyroscopes, magnetometers, elevation sensors, pressure sensors, temperature sensors, force sensors, proximity sensors, radar sensors, angle-of-attack sensors, global positioning system (GPS) sensors, thermal infrared cameras, and thermal imaging cameras, among others. In some implementations, one or more of the sensors can provide local sensor data124periodically (e.g., in a batch transmission, etc.). In some implementations, one or more of the sensors can provide local sensor data124upon receiving a corresponding request for sensor data from the route generator104. The sensors can provide raw measurements, which can be stored in the memory as part of the local sensor data124. A vehicle sensor108can determine an air speed or ground speed of the aerial vehicle, an elevation of the aerial vehicle, or an orientation of the aerial vehicle. The orientation of the aerial vehicle can include roll, pitch, and yaw, as well as a relative position of a control surface (e.g., a rotor, or an aerodynamic surface) relative to the wind or another object.

The data processing system100can include at least one trajectory follower110designed, constructed or operational to navigate the aerial vehicle along a route or a return path thereto. For example, the trajectory follower110can directly control the flight of the aerial vehicle (e.g., in an autonomous aerial vehicle) to navigate a route or return path determined or received by the data processing system100. The trajectory follower110can maneuver the aircraft based on sensed or determined information. For example, the trajectory follower110can receive a set of waypoints determined from the route generator104based on a detection of an obstacle, and control the operation of the aircraft responsive to the waypoints. For example, the trajectory follower110can maneuver the aerial vehicle to avoid an obstacle or return to a route based on a route or heading provided by the route generator104. The trajectory follower110can maneuver the aerial vehicle to traverse the return path. For example, an autonomous or semiautonomous flight control system can provide one or more waypoints to the trajectory follower110. The trajectory follower110can perform one or more navigational actions to maneuver the vehicle from a first waypoint to a second waypoint.

Referring toFIG.2, illustrated is an example representation of a gridline intersection205disposed along an gridded environment200including a portion of a route210including various waypoints. The vector field analyzer106can define vectors at each of the gridline intersections205of the environment. The environment can be centered on the aerial vehicle, or defined with respect to the route210, destinations or waypoints thereof. The route generator104or the vector field analyzer106can define waypoints along the route210. The waypoints can be defined according to a distance along the route210such as a route distance, or a transit time. The waypoints can be spaced regularly (e.g., equally) along the route210. The vector field analyzer106can associate the waypoints with a direction of travel along the route210. For example, the vector field analyzer106can number, name, or otherwise identify waypoints according to a sequence of travel along the route210. The vector field analyzer106can identify a nearest waypoint220to the gridline intersection205. The vector field analyzer106can identify other waypoints along the route210with reference to the nearest waypoint220. For example, a previous waypoint225, and subsequent waypoint230can be identified according to a direction of travel. The vector field analyzer106can determine a distance, d215between the gridline intersection205and a nearest waypoint220thereto. The vector field analyzer106can determine d215by subtracting the three-dimensional location of waypoint220from the gridline intersection205, and normalizing the resulting vector.

The vector field analyzer106can determine a direction of travel at the nearest waypoint220. The vector field analyzer106can determine the direction of travel with respect to a subsequent waypoint230or a previous waypoint225. For example, the vector field analyzer106can define a direction vector, R by subtracting the nearest waypoint220from the subsequent waypoint230. The vector field analyzer106can decompose the direction vector, R, to determine constituent portions thereof (e.g., rxand ryaccording to a two dimensional Cartesian environment, or rx, ry, and rzaccording to a three dimensional Cartesian environment). The vector field analyzer106can define a direction, Ψ, by the inverse tangent of (rx/ry). The vector field analyzer106can determine a relative vector of the gridline intersection205with respect to the nearest waypoint220, f. The vector field analyzer106can decompose the relative vector f into, for example, an x component, fx, and a y component, fy. The vector field analyzer106can compare a product of the x component of the relative vector and the y component of the direction vector to determine a greater of the two, such as by determining a positivity or negativity of a difference thereof (e.g., according to s=(sign(−rxfy+ryfx)).

The vector field analyzer106can thereafter determine the vector direction for the gridline intersection205according to the following formula. As provided below, k depicts a constant which can be selected according to a desired rate of return to a route. Larger values of k can cause the vehicle to return to the route relatively sharply; smaller values may cause the vehicle to return to the route relatively gradually. For example, k can be selected from values greater than 0 and less than 1 (e.g., a value which is less than 0.1, such as about 0.01). The vector magnitude can be normalized (e.g., can be defined as 1):

[cos⁡(ψ)-sin⁡(ψ)sin⁡(ψ)cos⁡(ψ)][1k*d*s]

The various vectors can be generated responsive to an environment surrounding the vehicle (e.g., the vector field can be constantly refreshed during aircraft transit) or the vectors can be generated responsive to a deviation of an aerial vehicle from the route210, which can reduce compute demands or energy usage. The vehicle sensors108can determine a position of the aerial vehicle. The route generator104can determine the aerial vehicle has deviated from the route210. For example, the route generator104can receive an indication from the vehicle interface102to resume route pathing or determine that the aerial vehicle has exceeded a threshold distance from the route210(e.g., based on local sensor data124). The vector field analyzer106can generate a vector to determine a cost function at various positions within an environment. The environment can be a three dimensional environment associated with an aerial vehicle having six degrees of freedom. Merely for brevity and clarity of description,FIG.2depicts a two dimensional representation of the environment. One skilled in the art will realize that the techniques of the present disclosure can be applied to an arbitrary number of dimensions and corresponding coordinate systems (e.g., three-dimensional real space). Some vectors can be omitted which may accelerate processing. For example, a vector corresponding to an obstacle, such as terrain, or a restricted elevation for nape-of-the-earth flight can be omitted. The process can be repeated to generate each vector for a desired vector field. For example, the vector field ofFIG.3can be defined thusly.

Referring now toFIG.3, illustrated is an example representation of a vector field305. The vector field305can be generated along a route210terminating at one or more destinations315or waypoints. The vector field305can be generated along an entirety of the route210or a portion thereof, such as a portion centered around a position of an aerial vehicle. For example, the vector field305can be generated for a distance around an aerial vehicle according to a predefined distance or computational performance (e.g., based on a variable speed, gridline distance, available compute resources, or the like). Although depicted in two dimensions for brevity and clarity of the figures, the vector field305can be generated in three dimensional space around a vehicle, such that the paths and courses through the vector field can be traversed by a path or course in three dimensions. The vector field305can include paths of travel not realizable in conjunction with one or more sets of kino dynamic constraints126. For example, the vector field305can include discontinuities310(e.g., areas of abrupt directional change within the vector field305). The discontinuities310can include adjacent vectors having changes of directionality exceeding a rotational acceleration of one or more aerial vehicles. Indeed, the vector field305can be determined absent an input of kino dynamic constraints126, such that a return path to the route210having a minimum cost associated therewith may not be realizable by one or more aircraft, crews, cargoes, or the like.

Referring now toFIG.4, the vector field analyzer106determines cost for one or more paths traversing the vector field305. For example, the directionality of the vector field305can be associated with the cost function such that a portion of a path aligned with the vector field305can generate a minimum cost and a portion of the path unaligned with the vector field305(e.g., opposing the vector field305) can generate a maximum cost. The vector field analyzer106can overlay a potential return path405over the vector field305through three-dimensional space, as depicted in two dimensions for clarity and brevity of the figures. The vector field analyzer106can determine a series of waypoints along the vector field analyzer106. The vector field analyzer106can associate a first waypoint410with a first ground course415. The first ground course415can include a direction of travel or a velocity for the aerial vehicle. The vector field analyzer106can associate at least one vector of the vector field305with the first waypoint410. For example, the vector field analyzer106can select a closest vector or vector origin or can select multiple vectors (e.g., for linear interpolation therebetween or a selection of a closest coordinate in the x direction for an x direction component of a vector and a closest coordinate in a y direction for a y direction component of the vector).

The vector field analyzer106can determine a conformity of the first ground course415along the path of travel with any associated vectors of the vector field305. For example, the vector field analyzer106can determine a dot product between the first ground course415and a closest vector of the vector field305. Thusly, travel perpendicular to the vector field305can generate zero cost, and any deviation therefrom can assume cost (e.g., according to a sine or tangent thereof). A path of travel along the vector field305can accrue negative cost and a path of travel opposed to the vector field305can accrue positive cost. Such a cost can be determined for each waypoint along the potential return path405until a final waypoint420and final ground course425is reached. The costs can be summed to determine a total cost of the potential return path405.

Because the route generator104can generate various paths in view of kino dynamic constraints126of the aerial vehicle, the realizable paths of travel may not be aligned with the vector field305(e.g., because the vector field305can include discontinuities310or other paths which are not compatible with one or more kino dynamic constraint126associated with an aerial vehicle.) The function is generated for one or more paths traversing the vector field305. By summing values for various segments of the vector field305, a path having a minimum cost can be determined or selected. Put differently, satisfying the kinodynamic constraints can be performed by the closed loop RRT or model predictive control algorithm. The vector fields can guide the return path along a lowest cost path of travel.

Referring toFIG.5, a graphical user interface (GUI)500is illustrated in accordance with one or more implementations. The vehicle interface102can include various output devices, such as the GUI500. The vehicle interface102can include any type of device capable of presenting information to an operator of the aerial vehicle505(e.g., a human operator or an autonomous flight control system). The GUI500can be positioned within the aerial vehicle505such that it can be accessed by the operator during operation of the aerial vehicle505, or can be remote from the vehicle, such that it can be accessed by a remote operator. The graphical user interface500can include devices that present specific sensor information, such as speed, direction, velocity, or location. The GUI500can present information related to the environment. For example, the GUI500can display a two-dimensional (2D) or three-dimensional (3D) representation of the environment, such as according to features present in a world model122or features detected by vehicle sensors108.

The GUI500depicts an aerial vehicle505. The aerial vehicle505can be any type of aerial vehicle505, such as an airplane, a helicopter, a drone, a vertical take-off and landing (VTOL) aircraft, or any other type of aerial vehicle505. The aerial vehicle505can also be referred to as an aircraft. The aerial vehicle505can be operated by one or more pilots, for example, using the vehicle interface102, or can be piloted autonomously. The aerial vehicle505can be equipped with any type of device or system that can be used to complete a task or mission that has been assigned to the aerial vehicle505. The aerial vehicle505can include one or more communications interfaces, via which the aerial vehicle505or the vehicle interface102can transmit or receive information, including any of the information used in the techniques described herein.

The GUI500depicts the aerial vehicle505as deviated from a route210, and an obstacle510along the route210. The GUI500depicts a first return path515from the position of the aerial vehicle505to the route210, and a second return path520which departs from the route210, avoids the obstacle510, and returns to the route210. Although depicted via the GUI500for ease of presentation, the data processing system100can generate or employ return paths to merge with a route210, or avoid obstacles510according to the present disclosure regardless of a presence of a graphical user interface500. Obstacles510can include terrain features, environmental objects, other vehicles, or restricted airspace. The graphical user interface500can include one or more directional orientations525. For example, the directional orientation525can include an ordinal direction, a direction to a waypoint, or the like. The local sensor data124or world model122can include obstacle510information such as a height, size, or priority of an obstacle510.

The route generator104can generate the route210, the first return path515, or the second return path520based on a search-based, sample-based, or optimization-based algorithm, as discussed above. For example, a predictive based motion planning techniques such as model predictive control (MPC) or closed loop RRT (CL-RRT) can generate various return paths or routes210satisfying kino dynamic constraints126associated with the aerial vehicle505. The vector field analyzer106can select at least one path according to a cost function such as a dot product of a vector field305and a direction of travel with respect to the vector field305. The route generator104can generate the first return path515incident to a generation of the vector field305, a deviation from the route210in excess of a threshold, or a release of an autonomy inceptor. The route generator104can generate the second return path520incident to a detection of the obstacle510. The obstacle510can be detected based on local sensor data124, world model122data, or a manual entry thereof.

Referring now toFIG.6, a flow diagram for a method600of navigation of an aerial vehicle505is illustrated, in accordance with one or more implementations. The method600can be executed, performed, or otherwise carried out by the data processing system100, the computer system800described herein in conjunction withFIG.8, or any other computing devices described herein. In brief overview, the method600can include engaging in autonomous flight (ACT605), detecting a deviation from autonomous flight (ACT610), engaging in manual flight operations (ACT615), detecting a condition indicative of an engagement of an autonomous operation (ACT620), generating a vector field305(ACT625), determining kinodynamic return path (ACT630), and providing the trajectory to return to the route210(ACT635).

In further detail, the method600can include engaging in autonomous flight (ACT605). For example, the aerial vehicle505can proceed autonomously along a predefined route210by conveying a route210to a trajectory follower110to navigate the route210. During autonomous operation, the aerial vehicle505can navigate the route210, detect a position of the aerial vehicle505relative to the route210(e.g., to detect a deviation therefrom), or detect an obstacle510along the route210.

The method600can include detecting a deviation from autonomous flight (ACT610). The data processing system100can detect the deviation based on a position of the aerial vehicle505(e.g., as detected by a vehicle sensor108). The data processing system100can detect the deviation based on an input received at the vehicle interface102. A manual entry to exit automated operation can be received by the vehicle interface102. An inceptor such as an inceptor configured to manually control a portion of the aerial vehicle505in a manual mode of operation, or an autonomy inceptor configured to adjust a course or position of the aerial vehicle505in a semi-autonomous mode of operation can receive an input. The data processing system100can enter a manual or semi-autonomous mode responsive to the detection of the input.

The method600can include engaging in manual flight operations (ACT615). For example, the data processing system100(e.g., the trajectory follower110) can cause a displacement from the route210responsive to the input of an autonomy inceptor. The data processing system100can monitor the status of the autonomy inceptor during manual flight operations to maintain a manual or semi-autonomous state. The method600can include detecting a condition indicative of an engagement of an autonomous operation (ACT620). For example, a manual entry to resume automated operation can be received by the vehicle interface102. The vehicle interface102can detect a lack of input to a control such as the autonomy inceptor. For example, the vehicle interface102can detect the autonomy inceptor is at a neutral position, or a lack of a change of input to the autonomy inceptor for a predefined amount of time, travel distance, proximity to an obstacle510, elevation, a status of an attention monitoring system for a pilot, or other conditions.

The method600can include generating a vector field305(ACT625). For example, the vector field analyzer106can generate the vector field305responsive to the determination that the aerial vehicle505has deviated from the route210, or responsive to the determination of the condition indicative of the engagement of autonomous operation. The vector field analyzer106can generate the vector field305around a radius or square around the aerial vehicle505, the route210, or an obstacle510. The vector field analyzer106can generate the vector field305or a portion thereof (e.g., the gridline spacing) based on the speed, distance deviated from the route210, or a condition of the environment.

The method600can include determining a kinodynamic return path (ACT630). The route generator104can determine one or more return paths to the route210. For example, the route generator104can employ a CL-RRT to generate various return paths, and convey the return paths to the vector field analyzer106. The vector field analyzer106can determine a cost of the return paths based on a comparison of the directional vectors of the return path with the vector field305. The vector field analyzer106can select a return path based on the cost of the return path in comparison to other return paths, or to a threshold. For example, a first return path having a cost below a cost threshold can be selected, or a lowest cost return path can be selected. A return path can be selected iteratively. For example, a first return path can be conveyed to the trajectory follower110. The data processing system100can continue to execute the method600to determine a lower cost return path, and adjust the return path upon a generation thereof by providing the updated return path to the trajectory follower110.

The method600can include providing the trajectory to return to the route210to the trajectory follower (ACT635). The trajectory follower110can follow the trajectory to traverse the return path to return to the route210. For example, the trajectory follower110can actuate portions of the aerial vehicle505to cause adjustments to the thrust, attitude, or control surfaces of the aerial vehicle505to cause the aerial vehicle505to navigate the return path to the route210. The return path and the route210can be maintained or received by the trajectory follower110, or the trajectory follower110can receive an updated route210, including the return path portion.

Referring now toFIG.7, a flow diagram for a method700of navigation of an aerial vehicle505is illustrated, in accordance with one or more implementations. The method700can be executed, performed, or otherwise carried out by the data processing system100, the computer system800described herein in conjunction withFIG.8, or any other computing devices described herein. In brief overview, the method800can include receiving an indication to return to a route210(ACT705), generating a vector field305to merge into the route210(ACT710), or providing instructions to traverse the vector field305(ACT715).

In further detail, the method700can include receiving an indication to return to a route210(ACT705). The route210can include a destination315. The indication to return to the route210can be generated responsive to a detection of an obstacle510along the route210, such that the aerial vehicle505can receive the indication prior to departing from a path of travel along the route210, or based on a deviation of the aerial vehicle505from the route210. For example, the indication can be received responsive to a detection, by the vehicle interface102, that a manual mode of operation is no longer indicated, based on various control inputs or a lack thereof.

The method700can include generating a vector field305to merge into the route210(ACT710). For example, a vector field analyzer106can generate the vector field305responsive to the indication to return to the route210. The vector field analyzer106can receive various return paths to the route210from the route generator104, and select a route210therefrom. For example, the vector field analyzer106can determine a cost associated with the return paths, and select a return path having a lowest cost.

The method700can include providing instructions to traverse the vector field305(ACT715). For example, the trajectory follower110can generate instructions to traverse a provided route210, or to navigate along a series of waypoints, headings, or other instructions provided thereto. The trajectory follower110can steer the aircraft towards the destination315. For example, the trajectory follower110can steer the aircraft along the route210, such as merging to the route210on a return path. The return path can follow along one or more vectors of the vector field305. For example, the return path can be parallel to, overlaid with, or at an acute angle with a vector of the vector field305(e.g., the return path can be a low-cost path).

Referring now toFIG.8, depicted is a block diagram of an example computer system800. The computer system or computing device800can include or be used to implement the data processing system100, or its components. The computing system800includes at least one bus805or other communication component for communicating information and at least one processor810or processing circuit coupled to the bus805for processing information. The computing system800can also include one or more processors810or processing circuits coupled to the bus805for processing information. The computing system800also includes at least one main memory815, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus805for storing information, and instructions to be executed by the processor810. The computing system800can further include at least one read only memory (ROM)820or other static storage device coupled to the bus805for storing static information and instructions for the processor810. A storage device825, such as a solid state device, magnetic disk, or optical disk, can be coupled to the bus805to persistently store information and instructions.

The computing system800can be coupled via the bus805to a display835, such as a liquid crystal display, or active matrix display, for displaying information to a user such as an administrator of the data processing system100. An input device830, such as a keyboard or voice interface can be coupled to the bus805for communicating information and commands to the processor810. The input device830can include a touch screen display835. The input device830can also include a cursor control, such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor810and for controlling cursor movement on the display835. The display835can be part of the display devices835, or other components ofFIG.8.

The processes, systems, and methods described herein can be implemented by the computing system800in response to the processor810executing an arrangement of instructions contained in main memory815. Such instructions can be read into main memory815from another computer-readable medium, such as the storage device825. Execution of the arrangement of instructions contained in main memory815causes the computing system800to perform the illustrative processes described herein. One or more processors in a multi-processing arrangement can also be employed to execute the instructions contained in main memory815. Hard-wired circuitry can be used in place of or in combination with software instructions together with the systems and methods described herein. Systems and methods described herein are not limited to any specific combination of hardware circuitry and software.

Although an example computing system has been described inFIG.8, the subject matter including the operations described in this specification can be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.

Some of the description herein emphasizes the structural independence of the aspects of the system components and illustrates one grouping of operations and responsibilities of these system components. Other groupings that execute similar overall operations are understood to be within the scope of the present application. Modules can be implemented in hardware or as computer instructions on a non-transient computer readable storage medium, and modules can be distributed across various hardware- or computer-based components.

The systems described above can provide multiple ones of any or each of those components, and these components can be provided on either a standalone system or on multiple instantiation in a distributed system. In addition, the systems and methods described above can be provided as one or more computer-readable programs or executable instructions embodied on or in one or more articles of manufacture. The article of manufacture can be cloud storage, a hard disk, a CD-ROM, a flash memory card, a PROM, a RAM, a ROM, or a magnetic tape. In general, the computer-readable programs can be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA. The software programs or executable instructions can be stored on or in one or more articles of manufacture as object code.

Example and non-limiting module implementation elements include sensors providing any value determined herein, sensors providing any value that is a precursor to a value determined herein, datalink or network hardware including communication chips, oscillating crystals, communication links, cables, twisted pair wiring, coaxial wiring, shielded wiring, transmitters, receivers, or transceivers, logic circuits, hard-wired logic circuits, reconfigurable logic circuits in a particular non-transient state configured according to the module specification, any actuator including at least an electrical, hydraulic, or pneumatic actuator, a solenoid, an op-amp, analog control elements (springs, filters, integrators, adders, dividers, gain elements), or digital control elements.

The subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. The subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more circuits of computer program instructions, encoded on one or more computer storage media for execution by, or to control the operation of, data processing apparatuses. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to a suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. While a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate components or media (e.g., multiple CDs, disks, or other storage devices include cloud storage). The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

The terms “computing device,” “component,” or “data processing apparatus” or the like encompass various apparatuses, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array), a GPU, or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services and/or distributed computing infrastructures.

A computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program can correspond to a file in a file system. A computer program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatuses can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array), a GPU, or an ASIC (application specific integrated circuit). Devices suitable for storing computer program instructions and data can include non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

The subject matter described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification, or a combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

While operations are depicted in the drawings in a particular order, such operations are not required to be performed in the particular order shown or in sequential order, and all illustrated operations are not required to be performed. Actions described herein can be performed in a different order.

Having now described some illustrative implementations, it is apparent that the foregoing is illustrative and not limiting, having been presented by way of example. In particular, although many of the examples presented herein involve specific combinations of method acts or system elements, those acts and those elements can be combined in other ways to accomplish the same objectives. Acts, elements, and features discussed in connection with one implementation are not intended to be excluded from a similar role in other implementations or implementations.

The phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” “having,” “containing,” “involving,” “characterized by,” “characterized in that,” and variations thereof herein is meant to encompass the items listed thereafter, equivalents thereof, and additional items, as well as alternate implementations consisting of the items listed thereafter exclusively. In one implementation, the systems and methods described herein consist of one, each combination of more than one, or all of the described elements, acts, or components.

Any references to implementations or elements or acts of the systems and methods herein referred to in the singular can also embrace implementations including a plurality of these elements, and any references in plural to any implementation or element or act herein can also embrace implementations including only a single element. References in the singular or plural form are not intended to limit the presently disclosed systems or methods, their components, acts, or elements to single or plural configurations. References to any act or element being based on any information, act or element can include implementations where the act or element is based at least in part on any information, act, or element.

Any implementation disclosed herein can be combined with any other implementation or embodiment, and references to “an implementation,” “some implementations,” “one implementation” or the like are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the implementation can be included in at least one implementation or embodiment. Such terms as used herein are not necessarily all referring to the same implementation. Any implementation can be combined with any other implementation, inclusively or exclusively, in any manner consistent with the aspects and implementations disclosed herein.

References to “or” can be construed as inclusive so that any terms described using “or” can indicate any of a single, more than one, and all of the described terms. For example, a reference to “at least one of ‘A’ and ‘B’” can include only “A,” only “B,” as well as both “A” and “B.” Such references used in conjunction with “comprising” or other open terminology can include additional items.

Where technical features in the drawings, detailed description or any claims are followed by reference signs, the reference signs have been included to increase the intelligibility of the drawings, detailed description, and claims. Accordingly, neither the reference signs nor their absence have any limiting effect on the scope of any claim elements.

Modifications of described elements and acts such as variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, and orientations can occur without materially departing from the teachings and advantages of the subject matter disclosed herein. For example, elements shown as integrally formed can be constructed of multiple parts or elements, the position of elements can be reversed or otherwise varied, and the nature or number of discrete elements or positions can be altered or varied. Other substitutions, modifications, changes and omissions can also be made in the design, operating conditions and arrangement of the disclosed elements and operations without departing from the scope of the present disclosure.

The systems and methods described herein can be embodied in other specific forms without departing from the characteristics thereof. Scope of the systems and methods described herein is thus indicated by the appended claims, rather than the foregoing description, and changes that come within the meaning and range of equivalency of the claims are embraced therein.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what can be claimed, but rather as descriptions of features specific to particular embodiments of particular aspects. Certain features described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination can be directed to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated in a single software product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results.