Patent ID: 12208893

DETAILED DESCRIPTION

This disclosure is generally directed to systems and/or processes for updating a virtual aerial map using sensor data obtained from one or more aerial vehicles. The virtual aerial map may include information related to one or more aerial views of terrain and one or objects located on the terrain. The virtual aerial map may, in some examples, also include any information related to the geographic location corresponding to the terrain, such as, for example, traffic conditions, weather conditions, agricultural conditions (e.g., crop status), and/or any other information that may be derived from information generated by one or more sensors associated with (e.g., mounted to) one or more aerial vehicles.

Such virtual aerial maps may be useful for providing a variety of information related to particular geographic locations for a number of purposes. For example, delivery services may use traffic-related information and/or weather-related information associated with geographic regions for improving delivery time. In other examples, agricultural parties may use crop status and/or weather-related information related to particular geographic locations to improve crop yields. Emergency responders may use environment-related information, such as, for example, forest fire location, speed, and/or direction to more effectively suppress forest fires. Thus, improvements in an ability to update such virtual aerial maps, even with respect to geographic locations separated from road networks, may be beneficial to such parties and others.

For example, aerial vehicles such as manned, semi-autonomous, and autonomous aerial vehicles, each including one or more sensors configured to generate one or more sensor data signals indicative of the environment in which they travel, may generate sensor data and communicate such sensor data via one or more networks to a system configured to incorporate at least portions of the sensor data into a virtual aerial map and, in some examples, generate an updated virtual aerial map. Such vehicles may include, for example, unmanned aerial vehicles (UAVs), delivery drones, private drones of both professionals and hobbyists, air taxis, agricultural drones, manned airplanes, manned helicopters, etc. The one or more sensors may include imagers, light detection and ranging (LIDAR) sensors, and/or any other sensor types configured to generate sensor data capable of being incorporated into a virtual aerial map. In some examples, the sensor data may be communicated instantaneously (within technical limitations), or nearly-instantaneously, for example, via radio transmission to one or more networks, from at least some of the respective aerial vehicles to a system configured to update a virtual aerial map. In some examples, the aerial vehicles may be configured to communicate via one or more networks upon return to a base at which the aerial vehicle lands. For example, sensor data may be stored in memory aboard the aerial vehicle, and the stored sensor data may be downloaded once the aerial vehicle returns to the base, either directly from the memory to the system, or via one or more communication networks.

In some examples, the systems and processes may be configured to receive sensed data (e.g., the image data, the LIDAR data, etc.) included in the sensor data, and in some examples, determine whether the quality of the sensed data (e.g., one or more of the resolution, the signal noise, the signal strength, the data density, the data range, etc.) meets or exceeds a threshold data quality. If the sensed data meets or exceeds the threshold data quality, the systems and/or processes may be configured to incorporate, based at least in part on determining that the data quality meets or exceeds the threshold data quality, at least a portion of the sensed data into the virtual aerial map, and in some examples, generate an updated virtual aerial map. If the data quality is below the threshold data quality, the systems and/or processes may be configured to exclude at least a portion of the sensed data from the virtual aerial map. In this example manner, the systems and/or processes may be able to facilitate incorporation of sensed data into the virtual aerial map when received from a wide variety of types of aerial vehicles and/or sensor types, while also substantially preventing incorporation of sensed data into the map that is of inferior data quality.

Because the sensor signals are generated by aerial vehicles, the sensor data generated by sensors mounted on the aerial vehicles is not limited to information obtainable from sensors mounted on ground-borne vehicles and is not limited to information obtainable while constrained by a road network. Thus, some examples of the systems and processes may result in providing updated information about areas remote from road networks and/or information difficult to obtain from the ground. Some examples may result in information being more frequently updated, even where sensors mounted on ground-borne vehicles may be available. As a result, some examples of the systems and processes described herein may result in providing more frequent and/or more comprehensive information about environments through which aerial vehicles travel.

The techniques and systems described herein may be implemented in a number of ways. Example implementations are provided below with reference to the following figures.

FIG.1illustrates an example system100for updating a virtual aerial map102using sensor data104obtained from one or more sensors associated with (e.g., mounted to) one or more aerial vehicles to generate an updated virtual aerial map106, as explained herein. For example, a plurality of aerial vehicles may be operated in the same or different environments, and the aerial vehicles may each have one or more sensors associated with the respective aerial vehicles. The sensors may be configured to generate sensor data including map data related to an aerial view from the respective aerial vehicle of terrain and/or objects on the terrain over which the aerial vehicle maneuvers. The map data may include data indicative of any information related to the geographic area based on which the sensor data is generated, including but not limited to, for example, information related to the geography of the geographic area, information related to the topography of the geographic area, information related to objects (e.g., machines, structures, vegetation, animals, and people) in the geographic area, a road or road network in the geographic area, information related to agriculture (e.g., crop status) in the geographic area, etc. The aerial vehicles may include any aerial vehicles capable of carrying one or more sensors configured to generate sensor data indicative of terrain over which the aerial vehicle maneuvers, such as, for example, manned aerial vehicles, semi-autonomous aerial vehicles, autonomous aerial vehicles, fixed-wing aerial vehicles, multicopters, helicopters, vertical take-off and landing (VTOL) aerial vehicles, military aerial vehicles, gliders, ultralights, or any other type of aerial vehicle. The sensors may include one or more imagers including, for example, RGB-cameras, monochrome cameras, intensity (grey scale) cameras, infrared cameras, ultraviolet cameras, depth cameras, video cameras, or stereo cameras, one or more LIDAR sensors, which may be any type of LIDAR-based sensor, and any other types of sensors configured to generate sensor data that could be incorporated into the virtual aerial map. The system100, in some examples, may be operated and/or controlled by one or more third party entities, either related or unrelated to entities operating and/or controlling the one or more aerial vehicles.

FIG.1schematically depicts an example first aerial vehicle108(e.g., an example fixed-wing aerial vehicle) including one or more sensors110configured to generate sensor data104including map data related to an aerial view from the first aerial vehicle108of an example mountainous terrain112and/or objects on the mountains terrain112over which the first aerial vehicle108maneuvers. For example, the example sensor110may have a field of view114directable at the example mountainous terrain112, and the sensor110may be configured to generate sensor data representative of a portion of the mountainous terrain112within the field of view114of the sensor110. The example first aerial vehicle108shown includes a transmitter116configured to transmit the sensor data104from the first aerial vehicle108via one or more network(s)118to one or more processors at a location120remote from the first aerial vehicle108, which may use at least a portion of the sensor data104to update the virtual aerial map102and/or generate an updated virtual aerial map106, except as noted herein. The network(s)118may facilitate such communications/interactions via any type of network, such as a public wide-area-network (WAN) (e.g., the Internet), which may utilize various different technologies including wired and wireless technologies.

As shown inFIG.1, an example second aerial vehicle122(e.g., a helicopter) includes one or more sensors110configured to generate sensor data104including map data related to an aerial view from the second aerial vehicle122of example urban terrain124and/or objects on the urban terrain124, such as buildings and objects typically found in urban settings. The example sensor110may have a field of view126directable at the example urban terrain124, and the sensor110may be configured to generate sensor data representative of a portion of the urban terrain124within the field of view126of the sensor110. The example second aerial vehicle122shown includes a transmitter128configured to transmit the sensor data104from the second aerial vehicle122via the one or more networks118to one or more processors at the location120, which may be remote from the second aerial vehicle122, which may use at least a portion of the sensor data104to update the virtual aerial map102and/or generate the updated virtual aerial map106, except as noted herein.

Similarly,FIG.1also shows an example third aerial vehicle130(e.g., a manned or unmanned multicopter) including one or more sensors110configured to generate sensor data104including map data related to an aerial view from the third aerial vehicle130of example agricultural terrain132and/or objects on the agricultural terrain132, such as crops, farm animals, related buildings and machines, and/or other objects typically found in agricultural settings. The example sensor110may have a field of view134directable at the example agricultural terrain132, and the sensor110may be configured to generate sensor data representative of a portion of the agricultural terrain132within the field of view134of the sensor110. The example third aerial vehicle130shown includes a transmitter136configured to transmit the sensor data104from the third aerial vehicle130via the one or more networks118to one or more processors to the location120, which may be remote form the third aerial vehicle130, and which may use at least a portion of the sensor data104to update the virtual aerial map102and/or generate the updated virtual aerial map106, except as noted herein.

FIG.1also depicts an example fourth aerial vehicle138(e.g., a manned or unmanned multicopter) including one or more sensors110configured to generate sensor data104including map data related to an aerial view from the fourth aerial vehicle138of example neighborhood terrain140and/or objects on the neighborhood terrain140, such as a house142, trees144, and a drop-off zone146for deliveries. For example, the example fourth aerial vehicle138shown carries a package148for delivery to the house142at the example drop-off zone146. The example sensor110may have a field of view150directable at the example neighborhood terrain140, and the sensor110may be configured to generate sensor data representative of a portion of the neighborhood terrain140within the field of view150of the sensor110. The example fourth aerial vehicle138shown includes a transmitter152configured to transmit the sensor data104from the fourth aerial vehicle138via the one or more networks118to one or more processors, which may use at least a portion of the sensor data104to update the virtual aerial map102and/or generate an updated virtual aerial map106, except as noted herein.

FIG.1also shows an example fifth aerial vehicle154(e.g., a manned or unmanned multicopter) at an example base station156. In some examples, the aerial vehicles may not be capable of transmitting information directly from the aerial vehicle to the network(s)118. Rather, in some examples, the aerial vehicles may return to a base station at which they land and/or receive service (e.g., power, maintenance, etc.), and at the base station, the aerial vehicle may communicate information including the sensor data104stored in memory of the aerial vehicle via the base station to the location120of the one or more processors. In some such examples, the base station156may include a transmitter configured to transmit the sensor data104via the one or more network(s)118.

Although the example first through fourth aerial vehicles108,122,130, and138are shown as each maneuvering and generating sensor data individually and related to a single environment, more than one aerial vehicle may concurrently maneuver and generate sensor data in the one or more environments. For example, some examples of the system100may leverage sensor data generated intermittently or continuously by more than a single aerial vehicle in a given geographic area, which may, in some examples, result in near continuous updating of the virtual aerial map102associated with the given geographic area.

In some examples, the system100may be configured to receive from one or more of the aerial vehicles the sensor data104including the sensed data (e.g., image data, LIDAR data, and/or other data representative of the environment through which the aerial vehicle maneuver), and in some examples, the system100may be configured to determine whether the quality of the sensed data (e.g., one or more of the resolution, the signal noise, the signal strength, the data density, the data range, etc.) meets or exceeds a threshold data quality. If the sensed data meets or exceeds the threshold data quality, the system100may be configured to incorporate, based at least in part on determining that the data quality meets or exceeds the threshold data quality, at least a portion of the sensed data into the virtual aerial map102, and in some examples, generate an updated virtual aerial map106. If the data quality is below the threshold data quality, in some examples, the system100may be configured to exclude at least a portion of the sensed data from the virtual aerial map102(e.g., the system100may not incorporate at least a portion of the sensed data). In this example manner, the system100may be able to facilitate incorporation of sensed data into the virtual aerial map102when received from a wide variety of types of aerial vehicles and/or sensor types, while also substantially preventing incorporation of sensed data into the map that is of inferior data quality.

In some examples, the threshold data quality may be empirically determined and/or adjusted over time as the quality of the data encompassing the virtual aerial map102improves over time. In some examples, the threshold data quality may be based at least in part on geographic location in the virtual aerial map102and/or the quality of the data in the virtual aerial map102at the geographic location. For example, if at a certain geographic location in the virtual aerial map102the quality of the existing data is at a certain level, the system100may be configured to determine whether the quality of the sensed data received from an aerial vehicle is improved relative to the existing data already in the virtual aerial map102, and if so, incorporate the sensed data from the aerial vehicle into the virtual aerial map102. If the quality of the sensed data received from the aerial vehicle is below the level of quality of the existing data, then the system100may be configured to exclude at least a portion of the sensed data received from the aerial vehicle.

In some examples, the virtual aerial map102may be updated with the sensor data104using, for example, sensed data stitching techniques. For example, one or more algorithms may be used to relate data points (e.g., pixels) in the sensor data104(e.g., sensed data) to data points in an existing virtual aerial map102. In some examples, one or more features from the terrain and/or objects on the terrain represented in the sensor data104may be matched with one or more corresponding features in the virtual aerial map102. In some such examples, at least some portions of the sensor data104may be incorporated into the virtual aerial map102using one or more algorithms that blend at least a portion of the sensor data104into the virtual aerial map102. Other techniques for incorporating the sensor data104into the virtual aerial map102are contemplated.

As shown inFIG.1, the geographic location, orientation, and/or navigation of one or more of the aerial vehicles may be assisted by a global positioning system (GPS)158, which may include a plurality of GPS satellites160sending GPS signals162, which may be used by one or more of the aerial vehicles to assist with determining one or more of the geographic location and/or orientation of the aerial vehicle, and/or to assist with navigation and/or control of the aerial vehicle. It is contemplated that one or more of the aerial vehicles may use other known navigation technologies for determining one or more of the geographic location and/or orientation, and/or to assist with navigation, such as, for example, dead-reckoning, image-aided navigation, inertial measurement units, gyroscopes, and/or accelerometers.

FIG.2illustrates an example virtual aerial map updating system200, which may correspond to the example system100shown inFIG.1. The example virtual aerial map updating system200may be configured to receive sensor data104via the one or more network(s)118from one or more aerial vehicles202and, based at least in part on the sensor data104, update a virtual aerial map102and generate an updated virtual aerial map106, for example, as explained with respect toFIG.1. Although the example aerial vehicles202depicted inFIG.2are multicopters, as noted previously herein, the aerial vehicles may be any type of aerial vehicle. In the example shown inFIG.2, the virtual aerial map updating system200includes one or more content server(s)204. The network(s)118may facilitate such communications/interactions via any type of network, such as a public wide-area-network (WAN) (e.g., the Internet), which may utilize various different technologies including wired and wireless technologies. Moreover, the content server(s)204may contain any number of servers that are possibly arranged as a server farm. Other server architectures may also be used to implement the content server(s)204. As shown, the content server(s)204include the one or more processor(s)206, and computer-readable media208. In the example shown, the computer-readable media208includes a communication module210including a receiver212and a transmitter214, an analysis module216, a position and orientation module218including a position finder220and an orientation finder222, a timing module224, a data quality module226, an alignment module228, an incorporation module230, and a map builder module232.

The one or more processors206may execute the one or more of the above-noted modules and/or processes to cause the virtual aerial map updating system200and/or the content servers204to perform a variety of functions, as set forth above and explained in further detail herein. In some examples, the processor(s)206may include a central processing unit (CPU), a graphics processing unit (GPU), both CPU and GPU, or other processing units or components known in the art. Additionally, each of the processors206may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems.

The computer-readable media208may include volatile memory (e.g., RAM), non-volatile memory (e.g., ROM, flash memory, miniature hard drive, memory card, or the like), or some combination thereof. The computer-readable media208may be non-transitory computer-readable media. The computer-readable media208may include or be associated with the one or more of the above-noted modules, which perform various operations associated with the virtual aerial map updating system200and/or content server(s)204. In some examples, one or more of the above-noted modules may include or be associated with computer-executable instructions that are stored by the computer-readable media208and that are executable by the one or more processors206to perform such operations. The virtual aerial map updating system200and/or the content server(s)204may also include additional components not listed above that may perform any function associated with the virtual aerial map updating system200and/or the content server(s)204.

In some examples, the communication module210may be configured to facilitate communication of data between one or more aerial vehicles202and the virtual aerial map updating system200. For example, the communication module210may include a receiver212configured to receive one or more signals from one or more of the aerial vehicles202via, for example, the one or more network(s)118. Such signals may include the sensor data104from one or more sensors110associated with (e.g., mounted to) the aerial vehicles202. In some examples, for example, as shown inFIG.2, the communications module210may include a transmitter214configured to transmit one or more signals. For example, continuously or intermittently, as the system updates the virtual aerial map102, the system may transmit an updated virtual aerial map106, for example, via the one or more network(s)118. Transmission of other data available to the virtual aerial map updating system200is also contemplated. In some examples, the receiver212and the transmitter214may take the form of a transceiver.

In some examples, the analysis module216may be configured to identify and/or extract sensed data from the sensor data104received from the aerial vehicles202. For example, the sensor data104may include data other than the sensed data, for example, as described herein. The analysis module216may be configured to identify the sensed data and in some embodiments, preform image analysis techniques on the sensed data, for example, to extract more information from the sensed data. For example, the analysis module216may be configured to perform one or more of two-dimensional or three-dimensional object recognition, image segmentation, motion detection, video tracking, optical flow, three-dimensional position and/or orientation estimation, etc. For example, the sensed data may be segmented and/or objects in the sensed data may be classified, for example, according to known image-analysis techniques to identify features and/or objects in the sensed data. Sensed data may include any data related to images and/or point clouds generated by imagers, LIDAR sensors, and/or any other types of sensors configured to generate signals indicative of an environment that may be incorporated into the virtual aerial map102. Such objects and features identified by the analysis module216may include, for example, topography of terrain, vegetation and/or its condition, buildings, bridges, vehicles, traffic flow, road conditions, construction zones, weather conditions, crop status, locations of vehicles in a fleet of vehicles, locations of machines and people at a work-site, etc.

The example shown inFIG.2also includes a position and orientation module218configured to determine geographic location and/or orientation of the sensor from which the sensor data104is received. In some examples, the sensor data104itself (e.g., the one or more signals including the sensor data104) may include the geographic location and/or orientation corresponding to the sensor data104. In such examples, the position and orientation module218may be configured to identify and/or extract the geographic location and/or orientation from the sensor data104. For example, the geographic location may correspond to the north east down (NED) position of the aerial vehicle202and/or the sensors110, where the NED position is represented by three coordinate values corresponding respectively to the position along the northern axis, the position along the eastern axis, and the position along the vertical axis. As used herein, the position may refer to one or more of the three axis positions. Orientation may correspond to the pitch, roll, and/or yaw about the three axis positions of the aerial vehicle202and/or the sensors110. The geographic location and/or the orientation may be used to align the sensor data104with data in the virtual aerial map102, for example, in order to incorporate the sensor data104into the virtual aerial map102, as described herein. In some examples, the sensor data104may not include the geographic location and/or orientation. In some examples, the position and orientation module218may include one or more of a position finder220or an orientation finder222. In such examples, the position finder220and/or the orientation finder222may be configured to determine the geographic location and/or orientation related to the sensor data104. For example, the position finder220and/or the orientation finder222may be configured to determine the geographic location and/or orientation based at least in part on sensed data in the sensor data104, for example, according to known techniques, such as, for example, simultaneous localization and mapping (SLAM) and/or other techniques.

The example shown inFIG.2also includes a timing module224configured to determine a time associated with the sensor data104, for example, the absolute time at which the sensed data associated with the sensor data104was generated or captured by the one or more sensors110. In some examples, the sensor data104itself (e.g., the one or more signals including the sensor data104) may include the time associated with the sensor data104. In such examples, the timing module224may be configured to identify and/or extract the time from the sensor data104. In some examples, the sensor data104may not include the time. In such examples, the timing module224may determine the time at which the sensor data104was received by the virtual aerial map updating system200. The time associated with the sensor data104may be used to prioritize more recently received sensor data104over less recently received sensor data104, for example, with respect to determining whether to update the virtual aerial map102based on the sensor data104. For example, the virtual aerial map updating system200may discount or not use sensor data104that is less recent than other sensor data104that is more recent, for example, to the extent that sensor data104associated with a geographic location conflicts. The virtual aerial map updating system200may be configured to use the more recent sensor data104for updating the virtual aerial map102, unless, for example, some other attribute of the more recent sensor data104might indicate that it is relatively less reliable than the less recent sensor data104. For example, the more recent sensor data104may not have as high a resolution as the less recent sensor data104, and the differences between the more recent sensor data104and the less recent sensor data104may be attributed, at least in part, to the difference in resolution.

For example, some examples of the virtual aerial map updating system200may include a data quality module226configured to determine the relative quality (e.g., data indicative of one or more of the resolution, the signal noise, the signal strength, the data density, the data range, etc.) of the sensor data104(e.g., the sensed data). In some examples, the sensor data104itself (e.g., the one or more signals including the sensor data104) may include the data quality (e.g., the resolution) associated with the sensor data104. In such examples, the data quality module226may be configured to identify and/or extract the data quality from the sensor data104. In some examples, the sensor data104may not include the data quality, and in some examples, the data quality module226may determine the data quality of the sensor data104. The data quality, in some examples, may be used to prioritize relatively higher quality sensor data104over relatively lower quality sensor data104, for example, with respect to determining whether to update the virtual aerial map102based on the sensor data104. For example, the virtual aerial map updating system200may discount or not use sensor data104that has a lower quality, for example, to the extent that sensor data104associated with a geographic location conflicts. The virtual aerial map updating system200may be configured to use the higher quality sensor data104for updating the virtual aerial map102, unless, for example, some other attribute of the higher quality sensor data104might indicate that it is relatively less reliable than lower quality sensor data104. For example, the more recent sensor data104having a lower quality may include additional objects or changes that is not included in the higher quality sensor data104.

In some examples, the data quality module226may be configured to receive from one or more of the aerial vehicles202the sensor data104including the sensed data (e.g., image data, LIDAR data, and/or other data representative of the environment through which the aerial vehicle maneuvers), and determine whether the quality of the sensed data (e.g., one or more of the resolution, the signal noise, the signal strength, the data density, the data range, etc.) meets or exceeds a threshold data quality, for example, as explained previously herein. If the sensed data meets or exceeds the threshold data quality, the data quality module226may be configured to incorporate, based at least in part on determining that the data quality meets or exceeds the threshold data quality, at least a portion of the sensed data into the virtual aerial map102, and in some examples, generate an updated virtual aerial map106. If the data quality is below the threshold data quality, in some examples, the system100may be configured to exclude at least a portion of the sensed data from the virtual aerial map102(e.g., the system100may not incorporate at least a portion of the sensed data). In this example manner, the system100may be able to facilitate incorporation of sensed data into the virtual aerial map102when received from a wide variety of types of aerial vehicles and/or sensor types, while also substantially preventing incorporation of sensor data into the map that is of inferior data quality.

As shown inFIG.2, some examples of the virtual aerial map updating system200may also include an alignment module228configured to align the sensor data104(e.g., the sensed data (e.g., image data and/or point cloud data) in the sensor data104) with existing data in the virtual aerial map102. For example, sensor data104associated with a particular geographic location, in order to be incorporated into the virtual aerial map102, may be incorporated at the correct geographic location and/or orientation in the virtual aerial map102. The alignment module228may align the sensor data104with the data in the virtual aerial map102according to known alignment techniques. For example, one or more algorithms may be used to relate data points (e.g., pixels) in the sensor data104(e.g., sensed data) to data points in an existing virtual aerial map102. In some examples, one or more features from the terrain and/or objects on the terrain represented in the sensor data104may be matched with one or more corresponding features in the virtual aerial map102.

Once the sensor data104(e.g., the sensed data) has been aligned with the data in the virtual aerial map102, the virtual aerial map updating system200may include an incorporation module230configured to incorporate the sensor data104into the virtual aerial map102, for example, according to known techniques. Once the sensor data104has been incorporated into the virtual aerial map102, the virtual aerial map updating system200may include a map builder module232configured to generate an updated virtual aerial map106, for example, according to known techniques. For example, the virtual aerial map102may be updated with the sensor data104using, for example, sensed data stitching techniques. One or more algorithms may be used to relate data points (e.g., pixels) in the sensor data104(e.g., sensed data) to data points in an existing virtual aerial map102. In some examples, one or more features from the terrain and/or objects on the terrain represented in the sensor data104may be matched with one or more corresponding features in the virtual aerial map102. In some such examples, at least some portions of the sensor data104may be incorporated into the virtual aerial map102using one or more algorithms that blend at least a portion of the sensor data104into the virtual aerial map102. Other techniques for incorporating the sensor data104into the virtual aerial map102are contemplated.

FIG.3illustrates an example flow of information between example aerial vehicles202and an example virtual aerial map updating system200, which may correspond to the system100shown inFIG.1. In the examples shown, one or more sensors110associated with the aerial vehicles202capture, receive, and/or generate sensor data104including sensed data300associated with a time at which the sensor data104is generated (or captured or received). For example, as shown the sensed data300may be generated at times T1, T2, T3, and so on, and the sensed data300associated with one or more of the times may be communicated via the one or more network(s)118to the virtual aerial map updating system200. The virtual aerial map updating system200may access the virtual aerial map102, which may be stored locally or remotely in memory, and the virtual aerial map updating system200may incorporate at least portions of the sensor data104into the virtual aerial map102(e.g., into the most recently updated version of the virtual aerial map102) and generate an updated virtual aerial map106, for example, as described herein. For example, as shown inFIG.3, one or more of the aerial vehicles202may be maneuvering above one or more of mountainous terrain112, an urban terrain124, an agricultural terrain132, or a neighborhood terrain140, and one or more sensors110associated with the respective aerial vehicles202may be generating sensor data104, including, for example, the sensed data300, which may be communicated via the one or more network(s)118to the virtual aerial map updating system200, for example, as described herein.

FIG.4illustrates an example of sensor data104input into an example virtual aerial map updating system200to generate an example updated virtual aerial map106. As shown inFIG.4, in some examples, the sensor data104generated by the one or more sensors110associated with the one or more aerial vehicles may include map data400. The map data400may include sensed data402, which may be incorporated into the virtual aerial map102. The sensed data402may correspond to the sensed data300shown inFIG.3. For example, the sensed data402may include one or more of image data404generated by an imager, LIDAR data406generated by a LIDAR sensor, or other sensor type data408, generated by sensor types other than an imager or a LIDAR sensor. In some examples, an imager may include one or more of RGB-cameras, monochrome cameras, intensity (grey scale) cameras, infrared cameras, ultraviolet cameras, depth cameras, video cameras, or stereo cameras. The LIDAR sensor may be any type of LIDAR-based sensor, and the other sensor type may be any sensor type configured to generate sensor data that could be incorporated into the virtual aerial map102.

In some examples, the map data400may also include one or more of geographic location data410, orientation data412, time data414, or data quality data416. The location data410and/or the orientation data412may provide the geographic location and/or orientation of a sensor110by which corresponding sensed data402was generated. For example, the aerial vehicle202(seeFIG.2) on which the sensor110is mounted may have a navigation and/or control system that determines and updates the geographic location and/or orientation of the aerial vehicle202. For example, the aerial vehicle202may include a GPS receiver configured to receive GPS signals162from three or more GPS satellites160(seeFIG.1) and determine its geographic location, which may be included in geographic location data410and incorporated into the sensor data104communicated to the virtual aerial map updating system200. In some examples, the aerial vehicle202may have additional navigational and control systems, for example, configured to determine the orientation of the aerial vehicle202and/or one more of the sensors110it carries. This information may also be included in orientation data412, which may be incorporated into the sensor data104communicated from the aerial vehicle202to the virtual aerial map updating system200. In some examples, as mentioned above with respect toFIG.2, the virtual aerial map updating system200may be configured to determine the geographic location data410and/or the orientation data412based, at least in part, on the sensed data402, for example, using the position and orientation module218, including the position finder220and orientation finder222.

The time data414may be included in the sensor data104communicated to the virtual aerial map updating system200. For example, the aerial vehicle202may include an internal clock and may determine the time (e.g., a timestamp) at which the sensor data104(e.g., the sensed data402) is captured, received, and/or generated by the one or more sensors110, and the time data414may be incorporated into the sensor data104communicated to the virtual aerial map updating system200. In some examples, the time data414may be determined by the virtual aerial map updating system200(e.g., by the timing module224) based, for example, on the time at which the virtual aerial map updating system200receives the sensor data104. Similarly, the data quality data416may be included in the sensor data104communicated to the virtual aerial map updating system200. In some examples, the data quality data416may be determined by the virtual aerial map updating system200(e.g., by the data quality module226) based, for example, on the sensed data402.

As shown inFIG.4, the sensor data104may be received by the virtual aerial map updating system200, and the virtual aerial map updating system200may determine the geographic location418and/or the orientation420associated with the sensor data104, for example, as described above. The virtual aerial map updating system may also access the latest version of the virtual aerial map102, which may be stored in memory of the virtual aerial map updating system200or in memory remote from the virtual aerial map updating system200. As described herein, the alignment module228(FIG.2) may align the sensor data104(e.g., the sensed data402) received from an aerial vehicle202with the virtual aerial map102to create aligned map data422. For example, the alignment module228may use the geographic location418and/or the orientation420as inputs for one or more algorithms that relate data points (e.g., pixels) in the sensor data104(e.g., sensed data) to data points in the virtual aerial map102. In some examples, one or more features from the terrain and/or objects on the terrain represented in the sensor data104may be matched with one or more corresponding features in the virtual aerial map102.

In some examples, thereafter the incorporation module230(FIG.2) may incorporate the aligned map data422into the virtual aerial map102creating incorporated map data424. In some examples, the incorporation module230may use sensed data stitching techniques. For example, once the aligned map data422has been determined, one or more algorithms may be used to blend at least a portion of the aligned map data422into the virtual aerial map102. Other techniques for incorporating the sensor data104into the virtual aerial map102are contemplated. In some examples, the incorporated map data424may be used by the map builder232(FIG.2) of the virtual aerial map updating system200to generate an updated virtual aerial map106, for example, by replacing the virtual aerial map102in its form prior to the updating with an updated virtual aerial map106including the incorporated map data424. In some examples, the updated virtual aerial map106may be stored in memory, either in addition to the virtual aerial map102or as a replacement for the virtual aerial map102. In some examples, only differences between updated virtual aerial map106and the virtual aerial map102may be determined, and only the differences may be stored and/or replaced.

FIG.5illustrates an example process for updating a virtual aerial map using sensor data from one or more sensors associated with one or more aerial vehicles. This process is illustrated as a logical flow graph, operation of which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the process.

FIG.5illustrates a flow diagram of an example process500for updating a virtual aerial map using sensor data from one or more sensors associated with one or more aerial vehicles, for example, as described herein. The following actions described with respect toFIG.5may be performed by the virtual aerial map updating system200and/or the content server(s)204, for example, as illustrated with respect toFIGS.1-4.

The example process500, at502, may include receiving map data from an aerial vehicle, the map data related to an aerial view from the aerial vehicle of terrain and objects on the terrain. For example, the map data may include at least one of image-based data or laser-based data received from the aerial vehicle, for example, as described herein.

At504, the process500, in some examples, may include identifying at least one of a geographic location associated with the map data or an orientation associated with the map data. In some examples, this may include determining the geographic location and/or the orientation based at least in part on the map data. In some examples, the map data may include the geographic location and/or the orientation, and in some examples, the process500may include determining the geographic location and/or the orientation based at least in part on the sensed data, for example, as described herein.

Some examples of the process500may include determining whether the quality of the sensor data (e.g., one or more of the resolution, the signal noise, the signal strength, the data density, the data range, etc.) meets or exceeds a threshold data quality. If the sensed data does not meet or exceed the threshold data quality, the process500, in some examples, may include excluding at least a portion of the sensed data from the virtual aerial map (e.g., the process500may not incorporate at least a portion of the sensed data, for example, the process500may not incorporate any of the sensed data received from the aerial vehicle into the virtual aerial map). In this example manner, the process500may be able to substantially prevent incorporation of sensed data into the map that is of inferior data quality. If, however, it is determined that the sensed data meets or exceeds the threshold data quality, the process500may continue, for example, as outlined herein.

At506, the process500may include aligning, based at least in part on the geographic location and/or the orientation, the map data with a virtual aerial map providing an aerial view of terrain and objects on the terrain, for example, as described herein.

The example process500may also include, at508, incorporating at least a portion of the map data into the virtual aerial map to generate an updated virtual aerial map, for example, as described herein.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.