Patent ID: 12198347

DETAILED DESCRIPTION

As discussed above, the imaging data transmitted from satellites to ground stations is subject to bandwidth constraints. One technique for conserving bandwidth is to filter the downlinked data such that imaging data that is likely to be of high value is selectively downlinked to the ground station. Imaging data that is unlikely to be useful may, in contrast, be omitted from the downlinked data when filtering is performed. One technique that may be used to address bandwidth limitations is to filter data before sending, so that all of the data is not sent. However, a technical challenge in the satellite context is determining what data to filter since the importance or relevance of the data is not known at the satellite.

In addition to the bandwidth constraints on the data transmitted from the satellites to the ground stations, the satellites are subject to energy and compute constraints. Satellites are typically powered by photovoltaic cells that provide small quantities of energy to electronic components. In addition, computing hardware included in a satellite typically has low mass and volume in order to decrease the overall size of the satellite and the cost of launching the satellite into orbit. Thus, the energy and compute constraints of the satellites may limit what filtering algorithms are practical to perform at the satellites when filtering the imaging data.

In order to address the above challenges, a computing device10is provided, as shown in the example ofFIG.1. The computing device10may be configured to communicate with a satellite20, which may, for example, orbit the Earth in LEO. Communication between the computing device10and the satellite20may be conducted via a ground station30that is configured to transmit and receive wireless electromagnetic signals to and from the satellite20. In addition, the ground station30may be configured to communicate with the computing device10via a wired or wireless connection. In some examples, the computing device10may be located on-premises at the ground station30. In other examples, the computing device10may be configured to communicate with the ground station30over a network. For example, the computing device10may be a server computing device located at a data center.

The computing device10may include a processor12that is communicatively coupled to memory14. Other components such as one or more user input devices and/or one or more user output devices may also be included in the computing device10in some examples. Although the computing device10is depicted in the example ofFIG.1as a single physical computing device, the functionality of the processor12and/or the memory14may alternatively be distributed between a plurality of physical computing devices, thereby forming the computing device10as a virtual computing device.

FIG.2schematically shows the computing device10in additional detail when the computing device10communicates with the satellite20via the ground station30, according to one example. As depicted in the example ofFIG.2, the processor12of the computing device10may be configured to receive imaging relevance data40for a geographic area34. The imaging relevance data40may be received from one or more external sensors32configured to communicate with the computing device10. Each external sensor32of the one or more external sensors32may be configured to communicate with the computing device10directly or over a network. Types of data that may be included in the imaging relevance data40are discussed in further detail below.

Based at least in part on the imaging relevance data40, the processor12may be further configured to generate image mask instructions50specifying a region of interest52included in the geographic area34. The image mask instructions50may specify a boundary between the region of interest52and one or more deprioritized regions53of the geographic area34that are outside the region of interest52. In some examples, the processor12may be configured to generate the image mask instructions50at least in part by executing a machine learning model42configured to receive the imaging relevance data40as input. The machine learning model42may be further configured to output the image mask instructions50in such examples. Other types of algorithms may additionally or alternatively be used to segment the geographic area34into the region of interest52and one or more regions outside the region of interest52.

In addition to the imaging relevance data40received from the one or more external sensors32, map data41may additionally be utilized in some examples when generating the image mask instructions50. For example, the map data41may be used as an additional input to the machine learning model42. The map data41may include data related to the geographic area34that is not collected from the one or more external sensors32. For example, the map data41may indicate a location of a political boundary between regions of the geographic area34. As another example, the map data41may indicate one or more locations of electrical infrastructure, water supply infrastructure, transportation infrastructure, or some other type of infrastructure. Other types of data for the geographic area34that are not collected from the one or more external sensors32or computed from the external sensor data may additionally or alternatively be included in the map data41.

The processor12may be further configured to transmit the image mask instructions50to the satellite20. As discussed above, the image mask instructions50may be transmitted to the satellite20via a ground station30that is configured to convey wireless signals to the satellite20.

In some examples, the image mask instructions50may be an image mask51that is used to filter satellite image data60at the satellite20, as discussed in further detail below. In other examples, the image mask instructions50may be data with which the image mask51is configured to be generated at the satellite20. For example, at least a portion of the imaging relevance data40and/or the map data41may be included in the image mask instructions50and may be used to generate the image mask51at the satellite20.

As depicted in the example ofFIG.2, the satellite20may include a satellite processor22and satellite memory24that provide the satellite20with onboard computing capabilities. The satellite20may further include a satellite imaging sensor26configured to collect satellite image data60of the Earth. The satellite image data60may include image data of the geographic area34for which the image mask51is generated.

At the satellite20, the satellite processor22may be configured to apply the image mask instructions50to the satellite image data60to generate filtered satellite image data62. In examples in which the image mask instructions50directly specify the image mask51, the satellite processor22may be configured to apply the image mask51received from the computing device10to the satellite image data60. In examples in which the image mask instructions50instead include data from which the image mask51is configured to be generated, the satellite processor22may be configured to compute the image mask51from the image mask instructions50. The one or more deprioritized regions53of the geographic area34outside the region of interest52may be excluded from the filtered satellite image data62. Accordingly, a file size of the filtered satellite image data62may be lower than a file size of the satellite image data60.

The satellite processor22may be further configured to transmit the filtered satellite image data62to the computing device10. Thus, the processor12of the computing device10may be further configured to receive, from the satellite20, the filtered satellite image data62of the region of interest52. Since the file size of the filtered satellite image data62may be lower than that of the satellite image data60, the satellite processor22may utilize the limited bandwidth of the connection between the satellite20and the ground station30more efficiently by selectively transmitting imaging data of the region of interest52to the ground station30.

FIG.3shows example types of imaging relevance data40and example types of external sensors32from which the imaging relevance data40may be at least partially received. The imaging relevance data40may include data that has been pre-processed at the processor12in addition to raw data collected from the one or more external sensors32. The processor12may, for example, utilize multiple different types of sensor data as inputs from which at least a portion of the imaging relevance data40is computed.

The imaging relevance data40may, for example, include meteorological data40A. The meteorological data40A may be received at least in part from one or more ground-based weather stations32A. In some examples, the meteorological data40A may be collected at least in part from one or more aerial sensors, such as one or more radar sensors32C or one or more laser imaging, detection, and ranging (LIDAR) sensors32D mounted on aircraft and respectively configured to collect radar data40D and LIDAR data40E. The meteorological data40A may, for example, include temperature data, precipitation data, wind speed and direction data, cloud cover data, lighting condition data, air pressure data, humidity data, dew point data, visibility range data, and/or other types of meteorological data.

The imaging relevance data40may also, for example, include additional satellite image data40B received from the satellite20or an additional satellite21. The additional satellite image data40B may be collected at the satellite imaging sensor26of the satellite20and/or an additional satellite imaging sensor27included in the additional satellite. In some examples, the additional satellite image data40B may be received at least in part from a plurality of additional satellites21that include respective additional satellite imaging sensors27.

Other types of imaging relevance data40, such as the meteorological data40A, may be computed based at least in part on the additional satellite image data40B in some examples. Computing other imaging relevance data40from the additional satellite image data40B may, in such examples, include performing image classification on the additional satellite image data40B. This image classification may be performed at an additional machine learning model or using some other algorithm. For example, regions of the geographic area34may be classified by surface type (e.g. water, forest, concrete, cropland, grassland, desert, tundra, marsh, etc.) based at least in part on the additional satellite image data40B.

In some examples, the imaging relevance data40may additionally or alternatively include vehicle tracking data40C for one or more vehicles. The one or more vehicles may be one or more land vehicles, aircraft, watercraft, or spacecraft, and may be manned or unmanned. The vehicle tracking data40C may, for example, be received from one or more vehicle-mounted sensors32B included in the one or more vehicles. The vehicle tracking data40C may additionally or alternatively include the additional satellite image data40B, radar data40D collected from one or more radar sensors32C, LIDAR data collected from one or more LIDAR sensors32D, or data collected from one or more other types of external sensors32located separately from the one or more vehicles.

In some examples, the imaging relevance data40may include seismographic data40F collected from one or more seismographic sensors32E. The seismographic data40F may, in such examples, be used to estimate a region of the geographic area that is affected by an earthquake. When estimating the region affected by the earthquake in such examples, the processor12may further utilize other types of imaging relevance data40, such as additional satellite image data40B and/or LIDAR data40E.

In some examples, the imaging relevance data40may include an estimated location of a fire boundary40G. The estimated location of the fire boundary40G may be computed from one or more other types of imaging relevance data40, such as meteorological data40A, additional satellite image data40B, radar data40D, LIDAR data40E, and/or one or more other types of imaging relevance data40.

FIG.4Ashows a first geographic area34A that may be imaged by the satellite20according to one example. In the example ofFIG.4A, the first geographic area34A is partially obscured by clouds. Regions of the first geographic area34A that are obscured by clouds may be of lower relevance to the user than the unobscured regions of the first geographic area34A. Accordingly, in the example ofFIG.4A, the imaging relevance data40may be meteorological data that provides evidence of which regions of the first geographic area34A are covered by clouds. For example, the imaging relevance data40may include ground-based meteorological data received from a ground-based weather station. The ground-based weather station may, in such examples, include the one or more external sensors32from which the processor12is configured to receive the imaging relevance data40.

FIG.4Bshows a first image mask51A generated for the example first geographic area34A ofFIG.4A. The first image mask51A includes a first region of interest52A, as well as a first deprioritized region53A outside the region of interest52A. The first deprioritized region53A in the example ofFIG.4Bis a region of the first geographic area34A that the processor12, based at least on the imaging relevance data40, determines is likely to be covered by clouds. The first deprioritized region53A includes non-contiguous regions of the geographic area34in the example ofFIG.4B. In other examples, the region of interest52may additionally or alternatively be non-contiguous.

Returning toFIG.2, in some examples, the region of interest52may include a first subregion54A and a second subregion54B. The image mask instructions50may indicate a first priority level56A and a second priority level56B for a first subregion54A and a second subregion54B of the region of interest52, respectively. The first priority level56A and the second priority level56B may indicate different respective resolutions for the filtered satellite image data62in the first subregion54A and the second subregion54B, respectively. Thus, when the processor12receives the filtered satellite image data62, the filtered satellite image data62may include first image data62A of the first subregion54A and second image data62B of the second subregion54B that differ in resolution. Thus, the image mask instructions50may be utilized at the satellite processor22to assign different priority levels to subregions within the region of interest52. In some examples, the region of interest52may include three or more different subregions with respective priority levels.

FIG.4Cshows an example second image mask51B generated for the first geographic area34A ofFIG.4A. The second image mask51B ofFIG.4Cindicates a region of interest that includes a first subregion54A and a second subregion54B. The first subregion54A and the second subregion54B may have a first priority level56A and a second priority level56B, respectively. In addition, the second image mask51B ofFIG.4Cindicates a second deprioritized region53B, which covers the same area as the first deprioritized region53A ofFIG.4B.

In some examples, when the processor12generates the image mask instructions50, the processor12may be configured to estimate a probability distribution44over the geographic area34. The probability distribution44may indicate, for a plurality of locations within the geographic area34, respective probabilities that satellite image data60of those locations is relevant for a user-specified application of the satellite image data60. In the example ofFIGS.4A-4C, the probability distribution44may be a probability distribution of cloud cover over the geographic area34. In some examples, the probability distribution44may be generated at least in part at the machine learning model42. Subsequently to generating the probability distribution44, the processor12may be further configured to assign the first priority level56A to the first subregion54A and assign the second priority level56B to the second subregion54B based at least in part on the probability distribution44. For example, the processor12may be configured to assign the first priority level56A or the second priority level56B to a location based on whether the probability estimated for that location is above or below a predetermined probability threshold.

FIG.5Ashows a second geographic area34B according to another example. In the example ofFIG.5A, the imaging relevance data40includes vehicle tracking data40C for one or more vehicles70. The vehicles70depicted in the example ofFIG.5Aare ships. As discussed above with reference toFIG.3, the vehicle tracking data40C may be received at least in part from one or more vehicle-mounted sensors32B. In some examples, at least a portion of the vehicle tracking data40C may additionally or alternatively be additional satellite image data40B, radar data40D, LIDAR data40E, or some other type of sensor data. In addition, vehicle tracking data40C may be collected for other types of vehicles in other examples.

As depicted in the example ofFIG.5A, the processor12may be further configured to compute one or more predicted locations72of the one or more vehicles70. For example, when the vehicle tracking data indicates respective positions and velocities of the one or more vehicles70, the processor12may be configured to extrapolate a respective future location of each of the one or more vehicles70based at least in part on the position and velocity indicated for that vehicle70.

As shown in a third image mask51C depicted in ofFIG.5B, the region of interest52may include the one or more predicted locations72in examples in which the processor12is configured to predict one or more predicted locations72for one or more vehicles70. The example third image mask51C ofFIG.5Bincludes a third region of interest52C surrounding the predicted locations72of the plurality of vehicles70shown inFIG.5A. The third image mask51C also includes a third deprioritized region53C located outside the third region of interest52C. By filtering the satellite image data60with the third image mask51C ofFIG.5B, the satellite processor22may be configured to selectively downlink filtered satellite image data62that has a high probability of showing the one or more vehicles70while excluding portions of the satellite image data60that are less likely to be relevant.

FIG.6Ashows an example third geographic area34C in which a wildfire is occurring. The imaging relevance data40received for the third geographic area34C may include an estimated location of a fire boundary74, as shown in the example ofFIG.6B.FIG.6Bshows an example fourth image mask51D generated for the third geographic area34C. The fourth image mask51D includes a fourth region of interest52D within which the estimated location of the fire boundary74is located, and further includes a fourth deprioritized region53D located outside the fourth region of interest52D. As discussed above, the processor12may be configured to compute the estimated location of the fire boundary74based at least in part on one or more other types of imaging relevance data40received from the one or more external sensors32. For example, the processor12may be configured to estimate the location of the fire boundary74based on meteorological data40A received from a ground-based weather station32A combined with additional satellite image data40B received from an additional satellite imaging sensor27.

In examples in which the imaging relevance data40includes additional satellite image data40B received from the satellite20or an additional satellite21, as discussed above with reference toFIG.3, the region of interest52may be a region of the geographic area34not depicted in the additional satellite image data40B.FIG.7Ashows an example fourth geographic area34D that partially overlaps with an example fifth geographic area34E that is imaged in the additional satellite image data40B. As shown in the example ofFIG.7B, the processor12may be further configured to generate a fifth image mask51E in which the region of interest52E is the region of the fourth geographic area34D that is not also included in the fifth geographic area34E. The portion of the fourth geographic area34D that overlaps with the fifth geographic area34E is indicated as a fifth deprioritized region53E. Accordingly, applying the fifth image mask51E to the satellite image data60may reduce redundancy between the filtered satellite image data62and the additional satellite image data40B.

In the example ofFIGS.7A-7B, the satellite20may be configured to perform collaborative filtering with one or more additional satellites21to determine the respective regions of interest52for which the satellite20and the additional satellite21are configured to downlink filtered satellite image data62. This collaborative filtering may be performed at least in part at the processor12of the computing device10by utilizing additional satellite imaging data40B when generating the image mask instructions50. In some examples, collaborative filtering may additionally or alternatively be performed via inter-satellite communications in which the satellite20and the one or more additional satellites21transmit at least a portion of their respective image mask instructions50to each other without transmitting those image mask instructions50through the ground station30. In some examples, at least a portion of the image mask instructions50may be transmitted between the satellite20and the one or more additional satellites21via one or more intermediate satellites included in a satellite network. In examples in which satellite-to-satellite communication is used in collaborative filtering, the image mask instructions50may be shared between the satellites is a manner that avoids low-bandwidth connections with the ground station30.

As discussed above with reference toFIG.2, the processor12may be configured to generate the image mask instructions50at least in part by executing a machine learning model42in some examples.FIG.8schematically depicts the computing device10during training of the machine learning model42. Although the training of the machine learning model42is depicted as occurring at the computing device10in the example ofFIG.8, the machine learning model42may, in other examples, be trained at another computing device and loaded into the memory14of the computing device10to be executed at runtime.

During training of the machine learning model42, training data130including training imaging relevance data140and a plurality of training image mask instructions150may be utilized. The training imaging relevance data140may include a plurality of training imaging relevance inputs142that are respectively paired with the training image mask instructions150. Each of the training imaging relevance inputs142may, for example, include training meteorological data, training additional satellite image data, training vehicle tracking data, training radar data, training LIDAR data, training seismographic data, a training estimated location of a fire boundary, and/or one or more other types of training imaging relevance data140. The training imaging relevance inputs142may each include data of a type associated with one or more external sensors. In some examples, the plurality of training imaging relevance inputs142may be paired with respective training map data141that indicates training data not associated with a type of external sensor. For example, the training map data141may include one or more respective locations of one or more political boundaries. As another example, the training map data141may include one or more infrastructure locations.

The training image mask instructions150may be a plurality of training image masks or may alternatively be data from which training image masks may be generated. The training image mask instructions150may each indicate a respective training region of interest152and a respective training deprioritized region153. In some examples, the training region of interest may include at least a first training subregion154A and a second training subregion154B. In such examples, the first training subregion154A and the second training subregion154B may respectively have a first training priority level156A and a second training priority level156B, which may indicate a corresponding first training resolution158A and second training resolution158B. In some examples, the training region of interest152may include three or more training subregions with respective training priority levels and training resolutions.

The training of the machine learning model42may include a plurality of parameter updating iterations. During each parameter updating iteration, the machine learning model42may be configured to receive a training imaging relevance input142of the plurality of training imaging relevance inputs142. The processor12may be further configured to generate candidate image mask instructions160at the machine learning model42based at least in part on the training imaging relevance input142. The candidate image mask instructions160may indicate a candidate region of interest162and a candidate deprioritized region163. In some examples, the candidate region of interest162may include a first candidate subregion164A and a second candidate subregion164B. The first candidate subregion164A may have a first candidate priority level166A that corresponds to a first candidate resolution168A. Similarly, the second candidate subregion164A may have a second candidate priority level166B that corresponds to a second candidate resolution168B. In some examples, the candidate region of interest162may include three or more candidate subregions with respective candidate priority levels and candidate resolutions.

In each of the plurality of parameter updating iterations, subsequently to generating the candidate image mask instructions160, the processor12may be further configured to compute a loss172for the machine learning model42. The processor12may be configured to compute the loss172using a loss function170. The loss function170may be a function that is minimized when the candidate image mask instructions160are an exact match for the training image mask instructions150, and that increases as a distance between the candidate image mask instructions160and the training image mask instructions150increases. The processor12may be further configured to compute a loss gradient174of the loss172and to perform gradient descent at the machine learning model42using the loss gradient174. Parameters of the machine learning model42may accordingly be updated in each parameter updating iterations. Thus, over the plurality of parameter updating iterations, the machine learning model42may learn to generate candidate image mask instructions160that more closely match the corresponding training image mask instructions150when the machine learning model42receives the training imaging relevance inputs142.

FIG.9Ashows a flowchart of an example method200for use with a computing device that is configured to communicate with a satellite. The method200may be used with the computing device10ofFIG.1or with some other computing device. At step202, the method200may include receiving imaging relevance data for a geographic area. The imaging relevance data may be received at least in part from one or more external sensors that are configured to communicate with the computing device. Some example types of data that may be included in the imaging relevance data are meteorological data, additional satellite image data, vehicle tracking data, radar data, LIDAR data, seismographic data, and an estimated location of a fire boundary. The imaging relevance data may, in some examples, include information that is computed from raw data received from the one or more external sensors. In such examples, at least a portion of the imaging relevance data may be computed by combining data collected from a plurality of different types of external sensors. For example, when the imaging relevance data includes meteorological data, the imaging relevance data includes ground-based meteorological data received from a ground-based weather station. The imaging relevance data in this example may further include radar data that is combined with the ground-based meteorological data.

At step204, the method200may further include generating, based at least in part on the imaging relevance data, image mask instructions specifying a region of interest included in the geographic area. For example, when the imaging relevance data includes meteorological data, the region of interest may be a region of the geographic area that is not covered by clouds. The image mask instructions may further specify a deprioritized region included in the geographic area. The image mask instructions may include an image mask or may alternatively include data with which the image mask is configured to be generated at the satellite.

Step204may further include additional steps that may be performed when the imaging relevance data includes specific types of information. In examples in which the imaging relevance data includes vehicle tracking data for one or more vehicles, step204may include, at step204A, computing one or more predicted locations of the one or more vehicles. In such examples, the region of interest may include the one or more predicted locations.

In some examples, the imaging relevance data may include additional satellite image data received from the satellite or an additional satellite. In such examples, step204may further include, at step204B, determining a region of the geographic area that is not depicted in the additional satellite image data. At least a portion of the region that is not depicted in the additional satellite image data may be used as the region of interest in such examples.

At step204C, step204may further include, in some examples, computing an estimated location of a fire boundary. The estimated location of the fire boundary may be computed from one or more other types of imaging relevance data. In such examples, the image mask instructions may be generated such that the region of interest includes the estimated location of the fire boundary.

In some examples, at step204D, step204may further include executing a machine learning model configured to receive the imaging relevance data as input. The machine learning model may be further configured to output the image mask instructions. Thus, the machine learning model may be configured to perform segmentation of the geographic area based at least on the imaging relevance data.

At step206, subsequently to generating the image mask instructions, the method200may further include transmitting the image mask instructions to a satellite. The image mask instructions may be transmitted to the satellite via a ground station configured to communicate with the satellite and the computing device. At the satellite, an image mask that is included in the image mask instructions or computed from the image mask instructions may be applied to satellite image data collected by one or more satellite imaging sensors. Thus, the amount of satellite image data downlinked to the ground station may be reduced by removing portions of the satellite image data that are likely to be of low importance to the user.

At step208, the method200may further include receiving, from the satellite, filtered satellite image data of the region of interest. The one or more deprioritized regions of the geographic area outside the region of interest may be excluded from the filtered satellite image data. By excluding the one or more deprioritized regions from the filtered satellite image data, the bandwidth of the downlink from the satellite to the ground station may be utilized more efficiently.

FIG.9Bshows additional steps of the method200that may be performed in some examples when generating the image mask instructions at step204. At step204E, step204may further include estimating a probability distribution over the geographic area. For example, the probability distribution may be a probability distribution of cloud cover over that indicates, for a plurality of locations in the geographic area, respective probabilities that those locations are obscured by clouds.

At step204F, step204may further include determining a first priority level and a second priority level for a first subregion and a second subregion of the region of interest, respectively. Determining the first priority level and the second priority level may include, at step204G, assigning the first priority level to the first subregion and the second priority level to the second subregion based at least in part on the probability distribution. For example, a predetermined probability threshold may be utilized to determine the location of a boundary between the first subregion and the second subregion. In some examples, the region of interest may be divided into three or more subregions based at least in part on the probability distribution.

In examples in which the region of interest includes at least a first subregion and a second subregion, the image mask instructions may indicate a first resolution for the satellite image data within the first subregion and a second resolution for the satellite image data within the second subregion. Thus, in such examples, the filtered satellite image data may include first image data of the first subregion and second image data of the second subregion that differ in resolution. In examples in which the region of interest includes three or more subregions, the three or more subregions may have three or more corresponding resolutions.

FIG.10shows a flowchart of an example method300for use at a satellite. The satellite at which the steps of the method300are performed may be the satellite20ofFIG.1or some other satellite. The satellite at which the method300is performed may include an imaging sensor configured to collect satellite image data of a geographic area. In addition, the satellite may include a satellite processor and satellite memory configured to provide onboard computing capabilities. The satellite may be configured to communicate with a ground station via wireless electromagnetic signals.

At step302, the method300may include receiving, from a ground-based computing device, image mask instructions specifying a region of interest included in a geographic area. The image mask instructions may further specify a deprioritized region of the geographic area that is located outside the region of interest. In some examples, the region of interest may be subdivided into at least a first subregion that has a first priority level and a second subregion that has a second priority level, thereby indicating different levels of prioritization for satellite imaging data of different subregions of the region of interest.

At step304, the method300may further include collecting imaging data of the geographic area via the imaging sensor.

At step306, the method300may further include filtering the satellite image data using the image mask instructions to obtain filtered satellite image data. In examples in which the image mask instructions include the image mask, step306may be performed by applying the image mask to the satellite image data. In other examples, performing step306may include generating the image mask based at least in part on the image mask instructions and using the image mask generated in this manner to filter the satellite image data. The one or more deprioritized regions of the geographic area outside the region of interest may be excluded from the filtered satellite image data, whereas the region of interest may be included. In examples in which the region of interest includes at least a first subregion and a second subregion, the first subregion and the second subregion may have different resolutions in the filtered satellite image data, as specified by the image mask instructions.

At step308, the method300may further include transmitting the filtered satellite image data to the ground-based computing device. The satellite image data may be transmitted to the ground-based computing device via the ground station.

Using the systems and methods discussed above, satellite image data may be filtered such that regions of satellite images that are more likely to be useful to a user are selectively downlinked to a ground station, whereas other regions of the satellite images are excluded. This selective downlinking allows the limited bandwidth of the connection between the satellite and the ground station to be utilized more efficiently. In addition, in the systems and methods discussed above, the image mask instructions with which the satellite image data is filtered are computed at a ground-based computing device rather than at an onboard processor of the satellite. Generating the image mask instructions at the ground-based computing device allows an image mask to be generated using more compute-intensive processes than would be feasible to perform at the onboard processors of many satellites. Accordingly, the filtering may be performed without exceeding energy and compute constraints of the satellite.

In some embodiments, the methods and processes described herein may be tied to a computing system of one or more computing devices. In particular, such methods and processes may be implemented as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product.

FIG.11schematically shows a non-limiting embodiment of a computing system400that can enact one or more of the methods and processes described above. Computing system400is shown in simplified form. Computing system400may embody the computing device10described above and illustrated inFIG.1. Components of the computing system400may be included in one or more personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices, and wearable computing devices such as smart wristwatches and head mounted augmented reality devices.

Computing system400includes a logic processor402volatile memory404, and a non-volatile storage device406. Computing system400may optionally include a display subsystem408, input subsystem410, communication subsystem412, and/or other components not shown inFIG.11.

Logic processor402includes one or more physical devices configured to execute instructions. For example, the logic processor may be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.

The logic processor may include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logic processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. Processors of the logic processor402may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and/or distributed processing. Individual components of the logic processor optionally may be distributed among two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the logic processor may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration. In such a case, these virtualized aspects are run on different physical logic processors of various different machines, it will be understood.

Volatile memory404may include physical devices that include random access memory. Volatile memory404is typically utilized by logic processor402to temporarily store information during processing of software instructions. It will be appreciated that volatile memory404typically does not continue to store instructions when power is cut to the volatile memory404.

Non-volatile storage device406includes one or more physical devices configured to hold instructions executable by the logic processors to implement the methods and processes described herein. When such methods and processes are implemented, the state of non-volatile storage device406may be transformed—e.g., to hold different data.

Non-volatile storage device406may include physical devices that are removable and/or built-in. Non-volatile storage device406may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., ROM, EPROM, EEPROM, FLASH memory, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), or other mass storage device technology. Non-volatile storage device406may include nonvolatile, dynamic, static, read/write, read-only, sequential-access, location-addressable, file-addressable, and/or content-addressable devices. It will be appreciated that non-volatile storage device406is configured to hold instructions even when power is cut to the non-volatile storage device406.

Aspects of logic processor402, volatile memory404, and non-volatile storage device406may be integrated together into one or more hardware-logic components. Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC/ASICs), program- and application-specific standard products (PSSP/ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.

The terms “module,” “program,” and “engine” may be used to describe an aspect of computing system400typically implemented in software by a processor to perform a particular function using portions of volatile memory, which function involves transformative processing that specially configures the processor to perform the function. Thus, a module, program, or engine may be instantiated via logic processor402executing instructions held by non-volatile storage device406, using portions of volatile memory404. It will be understood that different modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms “module,” “program,” and “engine” may encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.

When included, display subsystem408may be used to present a visual representation of data held by non-volatile storage device406. The visual representation may take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the non-volatile storage device, and thus transform the state of the non-volatile storage device, the state of display subsystem408may likewise be transformed to visually represent changes in the underlying data. Display subsystem408may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with logic processor402, volatile memory404, and/or non-volatile storage device406in a shared enclosure, or such display devices may be peripheral display devices.

When included, input subsystem410may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may comprise or interface with selected natural user input (NUI) componentry. Such componentry may be integrated or peripheral, and the transduction and/or processing of input actions may be handled on- or off-board. Example NUI componentry may include a microphone for speech and/or voice recognition; an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity; and/or any other suitable sensor.

When included, communication subsystem412may be configured to communicatively couple various computing devices described herein with each other, and with other devices. Communication subsystem412may include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide-area network, such as a HDMI over Wi-Fi connection. In some embodiments, the communication subsystem may allow computing system400to send and/or receive messages to and/or from other devices via a network such as the Internet.

The following paragraphs discuss several aspects of the present disclosure. According to one aspect of the present disclosure, a computing device is provided, including a processor configured to receive imaging relevance data for a geographic area. The processor may be further configured to generate, based at least in part on the imaging relevance data, image mask instructions specifying a region of interest included in the geographic area. The processor may be further configured to transmit the image mask instructions to a satellite. The processor may be further configured to receive, from the satellite, filtered satellite image data of the region of interest. One or more deprioritized regions of the geographic area outside the region of interest may be excluded from the filtered satellite image data.

According to this aspect, the imaging relevance data may include meteorological data. The region of interest may be a region of the geographic area that is not covered by clouds, as indicated by the meteorological data.

According to this aspect, the image mask instructions may indicate a first priority level and a second priority level for a first subregion and a second subregion of the region of interest, respectively. The filtered satellite image data may include first image data of the first subregion and second image data of the second subregion that differ in resolution.

According to this aspect, the processor may be further configured to estimate a probability distribution over the geographic area. The processor may be further configured to assign the first priority level to the first subregion and the second priority level to the second subregion based at least in part on the probability distribution.

According to this aspect, the processor may be configured to generate the image mask instructions at least in part by executing a machine learning model configured to receive the imaging relevance data as input.

According to this aspect, the processor may be further configured to train the machine learning model using training data that includes a plurality of training imaging relevance inputs paired with a respective plurality of training image mask instructions.

According to this aspect, the imaging relevance data may include additional satellite image data received from the satellite or an additional satellite.

According to this aspect, the region of interest may be a region of the geographic area not depicted in the additional satellite image data.

According to this aspect, the imaging relevance data may include vehicle tracking data for one or more vehicles. The processor may be further configured to compute one or more predicted locations of the one or more vehicles. The region of interest may include the one or more predicted locations.

According to this aspect, the imaging relevance data may include seismographic data.

According to this aspect, the imaging relevance data may include an estimated location of a fire boundary.

According to another aspect of the present disclosure, a method for use with a computing device is provided. The method may include receiving imaging relevance data for a geographic area. The method may further include generating, based at least in part on the imaging relevance data, image mask instructions specifying a region of interest included in the geographic area. The method may further include transmitting the image mask instructions to a satellite. The method may further include receiving, from the satellite, filtered satellite image data of the region of interest. One or more deprioritized regions of the geographic area outside the region of interest may be excluded from the filtered satellite image data.

According to this aspect, the imaging relevance data may include meteorological data. The region of interest may be a region of the geographic area that is not covered by clouds, as indicated by the meteorological data.

According to this aspect, the image mask instructions may indicate a first priority level and a second priority level for a first subregion and a second subregion of the region of interest, respectively. The filtered satellite image data may include first image data of the first subregion and second image data of the second subregion that differ in resolution.

According to this aspect, the method may further include estimating a probability distribution over the geographic area. The method may further include assigning the first priority level to the first subregion and the second priority level to the second subregion based at least in part on the probability distribution.

According to this aspect, generating the image mask instructions may include executing a machine learning model configured to receive the imaging relevance data as input.

According to this aspect, the imaging relevance data may include additional satellite image data received from the satellite or an additional satellite.

According to this aspect, the region of interest may be a region of the geographic area not depicted in the additional satellite image data.

According to this aspect, the imaging relevance data may include vehicle tracking data for one or more vehicles. The method may further include computing one or more predicted locations of the one or more vehicles. The region of interest may include the one or more predicted locations.

According to another aspect of the present disclosure, a satellite is provided, including an imaging sensor configured to collect satellite image data of a geographic area. The satellite may further include a processor configured to receive, from a ground-based computing device, image mask instructions specifying a region of interest included in the geographic area. The processor may be further configured to receive the satellite image data from the imaging sensor. The processor may be further configured to filter the satellite image data using the image mask instructions to obtain filtered satellite image data. One or more deprioritized regions of the geographic area outside the region of interest may be excluded from the filtered satellite image data. The processor may be further configured to transmit the filtered satellite image data to the ground-based computing device.

“And/or” as used herein is defined as the inclusive or v, as specified by the following truth table:

ABA ∨ BTrueTrueTrueTrueFalseTrueFalseTrueTrueFalseFalseFalse

It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.

The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.