Patent Description:
Manual interpretation of remote sensing imagery using commercial software is time consuming and costly. Manual interpretational also adds a degree of human induced error. This error can arise from a combination of poor analysis, or error in the sensor data (from cloud cover, atmospheric distortion, or geographic projection for example). Furthermore, these single image observations are often taken at a single point or snapshots in time, which means that they are both out of date the minute they are taken as well as the next increment in time (daily, weekly, monthly, etc.) producing new values for the same locations. This can further skew results in imagery analysis, especially when the analysis includes incremental replication in time within years and between years. A system that can aggregate temporal replication of raw satellite imagery could reduce error while simultaneously inflating the processing time in imagery analysis.

<NPL>, discloses a method of using signal processing to clean up within-year signal to then generate statistics that are used with machine learning.

<NPL> discloses a method for classifying multi-temporal satellite images using RFs, annual statistics and smoothed sessional data as features.

<NPL>, focuses on classification analysis in their tests of parallel architectures.

<NPL>, focuses on classification analysis in their tests of parallel architectures, not regression.

<NPL> describes a linear unmixing method that is used to convert rasters in the Modis imagery to forest inventory stand data which is polygon/vector spatial information. Linear mixing is used to downscale the spatial resolution between Modis pixels and forest inventory stand polygons. The purpose is to convert the raster information to a <NUM>:<NUM> with the stand polygons that exist as multiple areas, some smaller than a pixel, some larger and many pixels fell along the boundary of polygons.

In accordance with a first aspect of the invention there is provided a computer implemented process for Earth observation and analysis according to claim <NUM>.

The present invention will now be described with reference to the accompanying drawings in which:.

The present invention provides Earth observations and analytics in a three stage process. <FIG> is a diagram showing process steps in an embodiment of the present invention. <FIG> is a flow chart <NUM> with the following features. A preprocessing function <NUM>, a machine learning and Random Forrest Training function <NUM> and a Random Forrest Scoring function <NUM> all with a range of associated inputs. The preprocessing function has a Modis per Day data input <NUM> a Sat Tif <NUM> input and a calcstats function <NUM>. Year filter <NUM> Per Day - Per Year Geotiff files <NUM> with daily filtering <NUM> and Yearly filtering <NUM>. Non Modis inputs <NUM> such as Soil and Digital Elevation Models are shown. reference numerals <NUM>, <NUM>, <NUM>, <NUM> show collation of processed data and Tiles <NUM> include collated geotiffs <NUM>, per x tile <NUM>, <NUM>, geotiff annual <NUM> Geotiff yearly/per day.

RF train <NUM> requires input from and provides output to a number of data sources and functions as shown in <FIG>. These are normalised Y <NUM>, X and Y parameters <NUM>, identify files for training <NUM>, trees <NUM>, quantile info <NUM> and QRF <NUM>.

RF score <NUM> requires input from and provides output to a number of data sources and functions as shown in <FIG>. These are Tiles and year identification <NUM>, Year filter <NUM>, Scores <NUM>, Geotiff yearly tiles (scored) <NUM>, Geotiff statistic tiles <NUM>.

Remap <NUM> requires input from and provides output to a number of data sources and functions as shown in <FIG>. These are Filtered Scores <NUM>, Region of interest command line <NUM>, Shape file <NUM>, Geotiff images <NUM>, PDF <NUM> and producing a final output of an annotated Image <NUM>.

The functionality of the flowchart is described below.

Pre-processing - extracts required image bands from downloaded files, and remaps them to either global tiles, or specific Area of Interest (PreProc). Generates per- year statistics from within year satellite imagery at any temporal resolution (ie min/mean/max/median across all n images acquired each year, n being any integer). Time-series filtering may be applied for both within-year (for example CalcStats) and across years (YearFilter for example), where filtering across years is for the descriptive statistics extracted from within year imagery over more than <NUM> years. This step in the process is facilitated by the application of temporal signal processing to filter a time- series of remote sensing images.

The signal processing step is run on individual pixel locations within the image.

The signal processing smooths out the signal noise for each pixel within a year or with respect to a growth cycles within a year.

Descriptive statistics are extracted from statics from the pixel's filtered frequency data. The signal processing can then be run between years when comparing values between years.

The Machine Learning System as shown in the schematic flow diagram <NUM> of <FIG> which has the following features. The user interface <NUM> comprises calibration/filer options <NUM>, model training parameters <NUM>, AOI, model, time, filter options <NUM>, specific AOI Y parameter, display format and download options <NUM>. Training data <NUM> is input to machine learning training <NUM>, a Random Forest model <NUM>, Machine learning predictive analysis <NUM>, predicted parameter valves <NUM>, interactive predictive analysis <NUM>.

Sentinel <NUM> earth observation input data <NUM> is pre-processed <NUM> and calibrated <NUM>. Raw modus data <NUM> is pre-processed <NUM> and filtered modus data <NUM> used in the machine learning predictive analytics along with other parameters <NUM>. Display images <NUM> and files for download <NUM> are provided.

Machine Learning is utilised to produce the deliverable data, for regression. A Random Forest based approach is an machine learning software that is optimized and used in the process. The machine learning uses as input a variety of images, each representing a specific parameter; either as a target parameter (to be used for training, and to be subsequently provided as the results of the predictive analysis), or as a reference parameter (upon which the predictions will be made; these include satellite imagery, such as from MODIS, Proba-V, Landsat or Sentinel sensors).

The embodiment supports use of any remote sensing imagery, including MODIS, Proba-V, Landsat and Sentinel, including data obtained from both passive and active sensors.

Pre-Processing involves the remapping and co-registration of data from other sources, for use in the training process. These include the independent variables (X-parameters), such as satellite-derived products, including imagery from MODIS, Proba- V, Landsat and/or Sentinel, Discrete element method (DEM) derived parameters (such as slope, aspect, elevation), and soil-derived parameters. They also include any target variables (Y-parameters), which is user-supplied information for a number of measured samples, or reference (historic) data obtained from other sources, such as land-use maps, or other more targeted information provided by regional or national organisations, such as gridded data containing forestry-specific attributes.

Pre-processing further comprises creating per-pixel time-series filtering, to reduce the effect of anomalous values (e.g. cloud contamination and/or other sources of signal noise). The pre-processed images and data are stored in a defined directory structure, ready for ingestion into the main Machine Learning part of the processing <FIG> shows an example of the signal filtering utilized in the Pre-Processing. It is a graph <NUM> which plots NDVI on its Y axis <NUM> and Year <NUM> on its x axis. Curves <NUM>, <NUM> show NDVI Raw data and NDVI filtered data respectively. The graph shows <NUM> pixel sampled every <NUM> days from MODIS data for NDVI between <NUM> and <NUM>. The blue line is the raw data that which clearly shows the noise from frequency spikes.

The line <NUM> (B) is what is standard in academic research. The line <NUM> (R) provides users with additional statistical information, including error in the blue line on a per-pixel basis, the system and method of the present invention can implement the same time-series filtering on any imagery at any resolution, including Proba-V and Landsat. This allows for the creation of new highly accurate geospatial data used as inputs to the machine learning.

The present invention provides a way to make more accurate and quantifiable predictions whilst also reducing costs. Accordingly, the present invention is capable of not just analyzing one or two images, but hundreds of thousands, to help develop a big data time-series analysis. With machine learning you provide more accurate, quantifiable observations whilst removing the need for costly manual interpretation.

<FIG> is a diagram that shows an example of machine learning training in accordance with the present invention. The machine learning/training function <NUM> has inputs from Training AOI, source, X and Y parameters <NUM>, sentinel <NUM> data <NUM>, other parameters such as DEM, slope and soil <NUM>, reference training data <NUM> and user specific training data <NUM>. The output is a reference model <NUM> and a user specific model <NUM>.

The training phase of the method of the present invention, <FIG>, incorporates a novel implementation of the "Random Forest" machine learning model to generate decision trees, based on the specified training datasets (Y- parameters), and the selected independent variables (X-parameters), which can
include satellite-derived parameters, such as NDVI, as well as other datasets, such as elevation, soil type, etc..

The training data can be provided by the user (if they have appropriate historical in-situ or "ground truth" information) in the form of CSV files, with latitude/longitude and associated measured value(s) at each sample location, with the output from the Training being a user-specific Model. If the user has no available training data, then an off-the-shelf database repository of "reference" training data covering a range of land attribute data, such as forestry attributes at different spatial resolutions can be used as an alternative (though less- tailored to local environments) training solution, with the output from the Training being a Reference Model, as illustrated below.

The present invention comprises a model which can run regression functions for:.

The outputs from the Training step are Models which each consist of a "forest" of decision trees, along with associated statistical quality information, derived from quantile regression that quantifies the confidence associated with the subsequent predicted value for each pixel. The quantile regression allows us to produce maps for the uncertainties of all target data processed through our system. This allows us to also give clients spread of possible values for the land assets in which the client is interested.

Having developed a Model by training the Machine Learning system with a suitable set of training data, this Model can then be applied to any other region for which the same set of X-parameters are available, using these data, the corresponding Y-parameters can be predicted, on a per-pixel basis as is shown in <FIG>. The machine learning predictive analytics module <NUM> has inputs related to the general AOI model, time and filter options <NUM>, Sentinel <NUM> data <NUM>, other parameters such as DEM, slope and soil <NUM>, a reference model <NUM> and a user specific model <NUM>. It outputs predicted parameter valves <NUM>. Interactive predictive analysis <NUM> uses specific AOI, Y parameters, display format and download information along with the predicted parameter valves <NUM> to create files for download <NUM> and images <NUM>. There are two stages to the Predictive Analytics; the first stage is to define the processing to be performed and the second stage involves interactive selection of the specific outputs for display and/or download.

For the first stage, the inputs for the Predictive Analytics are defined by the user, and the processing is started. The Predictive Analytics software will then generate output images covering the specified general Aol corresponding to the specified time, for each of the Y-parameters originally used in training for the selected Model. This processing could take a significant amount of time, if a large general Aol has been specified and/or multiple time periods have been selected.

For the second stage, the user analyses the results, and generates consolidated output for the precise Area of Interest (including mosaicking across remote sensing tiles tiles, if necessary, and applying shapefile-defined masks for the specific area of interest, as well as generating "user-friendly" colour look-up tables for presenting the resulting imagery in a clear and meaningful format (as. png files or pdf files, ready for inclusion in web pages or in reports.

The most processing-intensive part of the original machine learning code (Random Forest in source C) regression was in the training part, where all the training data for all parameters needs to be accessible more-or-less simultaneously, as well as the derived decision tree structures; this involves complex data and memory management, as well as significant amounts of arithmetic operations. Thus the focus for optimisation was this core part of the decision tree growing algorithms.

The present invention uses Random Forest processing and includes a process for optimisations performance by;.

<FIG> is a block diagram which shows an example of Random Forrest Processing <NUM>. It shows X samples <NUM>, Y samples <NUM> which are combined in a Grow Tree process <NUM> and in a validate solution process <NUM>. The output of the Grow Tree process <NUM> is Tree Data <NUM> which is, along with the X sample data, an input to the valifate solution function which gives apredicted Y value <NUM>. X modis <NUM> is combined with the Tree Data at the Score Data function <NUM> to give a Y Modis output <NUM>.

In this example of known Random Forrest processing, nearly all the memory is being consumed by a number of massive 2D "double" arrays (called "Tree Data" in the diagram), indexed by Nsamples (up to <NUM>*Nsamples =<NUM>*<NUM> million) by Ntrees (up to <NUM>). This contains all the information needed by the random forest algorithm for each "tree" in the "forest", which includes information about the structure of each tree, in terms of its branches, nodes and decision criteria.

Each tree is quite independent of every other tree (a fundamental property of "random Forest").

<FIG> shows a technical solution of the present invention wherein the 2D arrays are replaced by 1D arrays <NUM> (indexed only by N samples). The information for each tree is stored <NUM> in a disk file (one per tree); these files can then be collated <NUM> once all the trees have been generated, which is the time-consuming part of the processing.

This novel technical solution considerably reduces the memory requirement by a theoretical factor of <NUM>, if there are <NUM> trees (in reality, the memory reduction is not quite so great, as there are numerous other data structures needed). This means that instead of needing many hundreds of Gigabytes of RAM with the original RF software in C, the updated a version of RandomForests was implemented needing only a few Gbytes of RAM (well within the scope of "normal" desktop machines). This core change in structure of processing is shown in the following diagram, which reflects the new processing:
With the revised approach, only data associated with a single tree needs to be held in memory at any one time, instead of data for all trees simultaneously.

A further memory optimisation was to store and process all data internally as <NUM>-bit integers, which as well as reducing the overall memory requirement by a factor of four (compared to storing as doubles), also has the benefit of faster processing (see below).

The above approach for reducing memory also lends itself very well to parallel processing as shown in <FIG>. Each tree can be processed (though multi-threading) on a separate core of one CPU <NUM>, <NUM> or on separate CPUs (if the machine has more than once CPU), or indeed on separate machines (if all machines have a shared disk/file structure). Thus, the revised scheme allows for N-fold parallelisation of the core "tree growing" part of the processing, giving an effective (N-<NUM>)-fold reduction in processing time, as the final step of validation and scoring takes no longer than growing a single tree.

Significant performance improvements were also obtained by extensive re-design of much of the processing code, affecting all parts, with systematic optimisations covering: Examples of performance optimization.

Given the same set of input X-variables, and required output Y-values, with the same number of input samples and the same number of trees, the comparable elapsed time for processing the Random Forest (Training part, which was the most CPU-intensive, and memory-intensive part) can be summarised as follows:<MAT>.

Where:
Original Time to grow one tree is unoptimised, and all processing is sequential; ie each tree is grown sequentially, one after the other, and each Y-variable is solved-for sequentially, one after the other, each requiring a new set of trees to be computed in turn.

New Elapsed Time -= New Time to grow one Tree Where:
New Time to grow one tree is -<NUM>* faster than before (following code optimization), and Assuming:
all trees processed in parallel (eg <NUM> trees => <NUM> times faster [overall <NUM>,<NUM> times faster]) Multivariate Random Processing, so all output Y-variables computed in parallel (eg if <NUM> Y-variables, then <NUM> times faster again [overall <NUM>,<NUM> times faster]).

If processing multiple MODIS tiles in parallel, using HPC batch processing, then even faster processing times are achievable (though in practice, such batch processing parallelisation may have been possible with the original software, but this was never tested, as it was incapable of processing the volumes of data needed for a single tile).

It is a complex task to compare the current performance of the present invention with the original Open Source implementation of Random Forest training.

It is therefore necessary to scale-up from the original system, using models of the projected performance improvement, based on the (limited) range of performance figures obtained from the original system (up to the data volumes at which it crashed - due primarily to memory limitations). The performance has been optimised through a combination of measures, including:
Complete re-design of internal data structures and type-casting for optimum performance, and minimisation of memory usage.

Optimisation of performance-critical sections of code (eg changing ordering of data indexing, optimising use of re-initialisations of arrays, optimising extent This section has been provided to show the calculations behind the statement that the present invention is <NUM>,<NUM> times faster when compared to standard open source software-of array copying, minimising processing performed within innermost loops, etc.).

Use of multithreading, within the process, and through running multiple processes in parallel across multiple CPUs and machines.

Introduction of multivariate regression, thereby solving for multiple target values in parallel.

Porting to higher-performance machines, with faster processors and larger RAM.

A summary of the results is shown in the table of <FIG> which shows a calculations table <NUM>, columns for Sample size <NUM>, No of calculations open source <NUM>, No of calculations present nvention185 and No of calculations scaled up version of present invention <NUM>.

Throughout the optimisation process, frequent checks were made to ensure that the results generated by the optimised code remained consistent with those generated by the original code. Broadly speaking, the main performance gains can be broken down according to the following: <MAT> <MAT> <MAT> <MAT>
<MAT>
<MAT>.

Thus, an increase in performance of <NUM>,<NUM>-fold is achievable on the same original machine, and if multiple more powerful machines are used, then the performance can be improved by at least <NUM>,<NUM> times (or even more, if more cores and machines are used in parallel - the revised architecture allows for a considerable degree of parallelisation).

These performance improvements (compared to the original) mean that analyses which would have taken several years to run (assuming a machine with sufficient memory could be used), can now be completed in a few hours, making it feasible to apply the sophisticated Random Forest regression techniques to large-scale Earth Observation applications using long time-series of data, such as those already addressed by GSi, covering forestry, agriculture and the environment. Embodiments of the present invention use a super computer cluster with extremely fast processors, massive RAM and low latency disk interconnect The server can be accessed from anywhere in the world via the users favoured.

<FIG> shows an embodiment of a system <NUM> in accordance with the present invention with the following features system architecture. The figure shows a user <NUM> with an internet connection <NUM>. A control portal <NUM>, front end <NUM>, a computing array <NUM> and a local file system <NUM>.

<FIG> is a diagram <NUM> that shows a mechanism for using CSV files containing training samples. GeoTiff tiles <NUM>, user data for training <NUM>, user CSV files <NUM>, X and Y parameters <NUM> and normalized Y parameters <NUM> are input to the RF train function <NUM>. The output from RF train is input to trees <NUM> along with QRF (Quantile Regression Forest) <NUM>. The output from Trees <NUM> is an input to RF score <NUM> along with tiles <NUM> and Training command line <NUM> and Identify tiles and years for scoring <NUM>. The output of which is scores <NUM> which are GeoTiff yearly scored files and GeoTiff yearly statistics files and QRF sample results <NUM>.

It illustrates the differences regarding input and output files when using (user-provided) CSV files (as target Y values) for training, with MODIS and other imagery as X-values, relying on the latitude and longitude specified in the CSV files for locating pixels in the imagery. The results of this training (the decision trees) can then be applied to whole MODIS tiles imagery.

The following examples illustrate the operation of the method and system of the present invention. As shown below, the present invention provides a quantifiably improved technical result because the data processing used in the present invention provides the user an improved ability to monitor biomass without the need for extensive, ongoing ground measurements. In addition, the system and method of the present invention achieves this result with a significant improvement in computing performance.

Within Year Filtering means tracking the change in vegetation as it goes through the seasons. For herbaceous vegetation such as grassland and cropland, this is the full amount of vegetative matter produced per unit area through the duration of a year. For forested areas, this means the growth of all live matter (leaves, stems, roots and all). The present invention does this by first building a database directory for a specified senor and imagery product.

<FIG> is a graph <NUM> which plots MODIS <NUM>-day NaVI on a Y axis <NUM> and days in <NUM> on the X axis <NUM>. Curves are shown for raw data <NUM>, 1st Order data <NUM>, 2nd Order data <NUM>, 3rd Order data <NUM> and 4th Order data <NUM>.

The raw data here represents <NUM>-day NDVI processed with <NUM>-day PR layer to remove low quality pixels. The <NUM>-day replication is the 1st day (<NUM> Jan <NUM>) through the 365th day (<NUM> Dec <NUM>) of the year. The raw data contains noise typical of sensed data over time. This noise is exemplified in the ups and down between <NUM>-day sensed measures. None of the standard MODIS data products are processed with filtering methods typically found in the field of signal processing.

The present invention runs signal processing filters for within year remote sensing data. It processes remote sensing with a range of signal processing methods, including Fourier and Kalman filtering and smoothing. <FIG> shows the application of the Fourier filter to NDVI where the <NUM>-day NDVI is processed and smoothed at the 1st, 2nd 3rd and 4th Fourier order of magnitude. The <NUM>-day frequency is smoothed depending on the order of magnitude processed through the Fourier equation.

<FIG> replicates the filtering and smoothing analyses, but with Proba-V <NUM> NDVI Values. <FIG> is a graph <NUM> which plots Proba-V <NUM> NDVI on a Y axis <NUM> and days in <NUM> on the X axis <NUM>. Curves are shown for raw data <NUM>, 1st Order data <NUM>, 2nd Order data <NUM>, 3rd Order data <NUM> and 4th Order data <NUM>.

Whilst there are less replicates in the time series, the same basic frequency pattern exists as with MODIS, which again is a measure for annual phenology. The signal processing software is then run on the raw Proba-V NDVI data to smooth it with the same orders of magnitude from the Fourier filter.

In <FIG>, signal processing is shown with applied to Landsat <NUM> AWS NDVI data at <NUM> resolution. <FIG> is a graph <NUM> which plots Landsat <NUM> AWS NDVI on a Y axis <NUM> and days in <NUM> on the X axis <NUM>. Curves are shown for raw data <NUM>, 1st Order data <NUM>, 2nd Order data <NUM>, 3rd Order data <NUM> and 4th Order data <NUM>.

As with both MODIS and Proba-V, the raw data contains a degree of noise over time. Smoothing of the raw NDVI frequency, and the 1st to 4th order of magnitude from the Fourier equation is demonstrated here. It is noted that growth phenology is generally consistent over time for all sensors and resolutions.

<FIG> shows a black and white image for Landsat <NUM> AWS NDVI at day <NUM> (June <NUM>) of <NUM>. It is important to note the darker shape noise <NUM> through the centre of image. This dark noise is cloud cover, which distorts the raw data in the image.

<FIG> shows output for the same day using data processing in accordance with the present invention. It shows a smoothed image at the 1st order magnitude from the Fourier filter. The noise derived from the cloud cover has been removed <NUM> from the Landsat image. Without the noise removed, the Landsat image cannot be used to assess the underlying AOI because of the cloud cover contamination.

The novelty of signal processing of the present invention as demonstrated here is that it applies signal processing methods with image processing software and can be run on any remote sensing data. It can be run on any group of images representing a time series from one sensor and remove or reduce sensor noise over time. This means it can increase the value of existing remote sensing imagery by reducing and removing signal noise. In application, the newly cleaned imagery output creates a new high quality imagery set which is unavailable through other processing methods.

Once the images have been processed as described above, on the within year data, the next element is to automatically generate annual descriptive statistics per pixel from the filtering and smoothing. By annual descriptive statistics, we mean the annual minimum, annual median, annual mean and annual maximum value for all pixels in an image.

These descriptive statistics are useful for calculating vegetation attribute values that do not reflect within year phenology, such as growth. Examples of such values include tree volume, tree height, tree dbh (diameter at breast height), percentage land cover per pixel, and carbon stocks. It is impossible to track these values using within year data because of the effects of growing season phenology. The point here is that the annual descriptive statistics change very little in undisturbed places. For example, when considering NDVI, annual descriptive statistics values should go up in value year on year, but the variance between year should be little. When there is a disturbance, the value is initially measured by the within value, but can only be tracked year on year. For NDVI, a disturbance in July <NUM> should be measured as a decrease in <NUM> from <NUM>, and then a decrease again between <NUM><NUM>, because half of <NUM> had no change prior to the disturbance.

<FIG> is a graph which shows the CalcStats annual descriptive statistics outputs for the same coordinate annual from MODIS. The graph <NUM> has MODIS annual NDVl <NUM> on its Y axis <NUM> and data type on the X axis <NUM>. Four types of data are plotted namely raw, <NUM>, <NUM>st order <NUM>, <NUM>nd order <NUM>, <NUM>rd order <NUM> and <NUM>th order <NUM>. In each case min <NUM>, median <NUM>, mean <NUM> and max <NUM> values are provide.

<FIG> is a graph which shows the CalcStats annual descriptive statistics outputs for the same coordinate annual from Proba-V. The graph <NUM> has Proba-V annual NDVI <NUM> on its Y axis <NUM> and data type on the X axis <NUM>. Four types of data are plotted namely raw, <NUM>, <NUM>st order <NUM>, <NUM>nd order <NUM>, <NUM>rd order <NUM> and <NUM>th order <NUM>. In each case min <NUM>, median <NUM>, mean <NUM> and max <NUM> values are provide.

<FIG> is a graph which shows the CalcStats annual descriptive statistics outputs for the same coordinate annual from Landsat <NUM> AWS. The graph <NUM> has Landsat <NUM> AWS annual NDVI <NUM> on its Y axis <NUM> and data type on the X axis <NUM>. Four types of data are plotted namely raw, <NUM>, <NUM>st order <NUM>, <NUM>nd order <NUM>, <NUM>rd order <NUM> and <NUM>th order <NUM>. In each case min <NUM>, median <NUM>, mean <NUM> and max <NUM> values are provide.

The annual descriptive statistics are shown for each sensor for the 1st to 4th orders of magnitude from the Fourier equation. Note that the values change between orders of magnitude, so that each descriptive statistic from each order of magnitude is slightly different. This means that when filtering NDVI for annual descriptive statistics at the 1st to 4th order of magnitude, <NUM> different cleaned values are newly created describing the within year growth.

<FIG> show the difference in image between Landsat <NUM> AWS NDVI for raw annual descriptive statistics (i.e. those Figures ending with a) versus filtered annual descriptive statistics for the 1st order of magnitude from the Fourier filter using data processing in accordance with the present invention. <FIG> and <FIG> show the difference between minimum raw data (<FIG>) and data filtered in accordance with the present invention (<FIG>). <FIG> and <FIG> show the difference between median raw data (<FIG>) and data filtered in accordance with the present invention (<FIG>). <FIG> and <FIG> show the difference between mean raw data (<FIG>) and data filtered in accordance with the present invention (<FIG>). <FIG> and show the difference between maximum raw data (<FIG>) and data filtered in accordance with the present invention (<FIG>). Note the difference is especially pronounced in the annual minimum, median and maximum.

In addition to the descriptive statistics, a pixel count used to create the annual descriptive statistics (<FIG>) and a per pixel Root Mean Squared Error (i.e. RMSE; see <FIG>) quantifying the difference between the noisy raw annual frequency and the filtered annual frequency are generated. These added measures allow for the use of pixel-level quality control measures to identify which pixels have the lowest count over time and highest RMSE. It is these individual pixels that we have the least certainty in their annual descriptive statistics. One novel feature of the present invention is that new clean annual imagery can be created to add quantitative quality control measures at the pixel level.

The underlying processing for the machine learning has been described in prior section.

Here a case study is provided for total volume in forests. Two different data sets were used to train the model. The first uses Canadian National Forest Inventory (NFI) estimates that are mapped at <NUM> resolution as an initial off the shelf proxy when there is no ground data available, which is sometimes the case prior to purchasing a land asset that will be harvested for timber extraction. Thus, the off the shelf data can be used when there is no ground data. In addition, we also develop a model for forest with some ~<NUM> ground plots we were collected for an AOI in Ontario.

<FIG> shows an x/y scatter plot for the off the model developed between NF! total volume and the Landsat <NUM> AWS annual descriptive statistics for 1st to 4th order of magnitude from the Fourier filtering. The model had an overall fit of R2 = <NUM> with the NF! data with a count of n = <NUM>. The key assumption here is that the NFI data is good and reliable data. <FIG> shows an uncertainty distribution based on the xly plot, where the yellow lines are set to the <NUM>% Confidence. Meaning the range in the uncertainty spread is <NUM>. 5th quantile at the minimum and <NUM>. 5th at the maximum.

<FIG> and <FIG> reproduce the modelling, but with local ground data where the count is only n = ~<NUM> points. The model based on locally measured ground data had an R2 of <NUM> with total volume. Again, the uncertainty threshold was calibrated to the <NUM>% Confidence Level in 22b. It is also relevant to note here that machine learning software in accordance with the present invention can be developed quickly with hundreds of millions of data point. In order to visualize the correlations and uncertainties, visualization tools were created which were meant very big datasets. Small datasets like the locally scaled data shown in <FIG> and <FIG> do not visually depict the modelling as accurately as it does with big data. <FIG> and <FIG> are four modelling diagnostics that measure that influence of each of the <NUM> NDVI descriptive statistics for making the modelled inference for total volume in both NF! total volume and locally measured total volume.

In <FIG>, the graph <NUM> with a Y axis <NUM> that plots NVDI min, median, mean and maximum values for first, second third and fourth order values each in respect of four measures which are measure <NUM><NUM>, <NUM>, <NUM> and <NUM>. The X axis plots percentage values <NUM>.

Note that the descriptive statistic for the annual mean NDVI for 1st order shows thegreatest influence out of all <NUM> variables for both NFI and locally derived total volume. This means there is similar spatial consistence for the total volume pattern with annual mean NDVI for 1st order.

<FIG> and <FIG> show the conditional mean prediction for total volume at the whole Aol, where 29a is the NFl-derived inference and 29b is the locally measured inference with field data. The quantile-based uncertainty analysis allows us to get around the need to cut the data into proportions for training and cross-validation, which are accuracy approaches tied specifically to a conditional mean inference alone. This approach is especially useful with when locally modelling with small sample sizes.

<FIG> show the range of quantile prediction at the <NUM>% Confidence Interval. Meaning, <FIG> and <FIG> show the conditional minimum (i.e., <NUM>. 5th quantile). Figure <NUM> shows the conditional lower quartile (i.e., 25th quantile); Figure <NUM> shows the conditional median (i.e., 50th quantile), Figure <NUM> shows the conditional upper quartile (i.e., 75th quantile) and Figure <NUM> shows the conditional maximum (<NUM>. 5th quantile).

The point to the quantile-based uncertainty analysis is to provide a range of predictions when the modelled inference in made for the whole AOI or managed sub-regions within the AOI. This translates to the client in valuing risk in the numbers versus investment into the site. For example, the client may want to start by harvesting timber at the site for sub-regions where the quantiles have the least variability in the modelled inference or where all inferences show consistently higher values.

Thus, when the client has no field data, machine learning in accordance with the present invention can provide initial test proxy data based on an assumption that national data is more accurate than no data.

However, field sampling methods differ and can produce different results for the same location. In application to a client this means:.

In this way, the machine learning and mapping system of the present invenion can be quickly updated with new data, so one map should never be assumed accurate for all users.

Beyond modelling for a single value (i.e. total volume), the regression-based machine learning function has been applied to the following modelling scenarios:.

The present invention is applied to active satellite sensors information to tracking
vegetation attribute assets in forestry and agriculture. Examples here are using active sensors from a variety of data sources to clean the daily imagery at the raster level.

The cleaned daily data is then used to better infer vegetation attributes such as agriculture and forest growth rates throughout a year. The cleaned annual data is used to infer vegetation attributes related to stocks, such as standing timber or carbon in the soil. The novelty is that existing data can be cleaned and new data products created. One element of the software that should be clearly understood is that the processing can be applied to any set of imagery over time. <FIG> shows an example of a generic processing system for the system and method of the present invention. The
figure <NUM> shows a generic database image as an input to calcstats <NUM> which outputs statistics <NUM> and provides a cleaned daily image <NUM> and a cleaned annual image <NUM>. Machine learning <NUM> provides new daily vegetation attribute data <NUM> and new annual vegetation attribute data <NUM>.

Conceptually, the system treats any image as a raster layer (i.e. a GeoTiff), so the values in the raster layers could be a red, blue, green, any index, near-infrared, height from a Lidar sensor or back scatter / texture from a SAR sensor. In all cases, the remotely sensed data is still just numbers stored as pixel values over space in an digital image. The digital image is replicated over the same place in time. If the SAR and Lidar imagery are collected at the same place (i.e. regular repeat cycles in the satellite's orbit) throughout a year, these data can be setup in the same way as the NDVI described herein, to run the software to clean the sensor noise in the pixels overtime.

If the SAR and/or Lidar imagery is the collected on an annual basis, a second version of the software is used which filters and smoothes pixel-level noise in annual imagery similar to the inventions treatment of within year imagery. The second version can then be run on any annual SAR and/or Lidar imagery where one snapshot is collected each year around the same proximate same day of year within each year(i.e. not winter one year and summer the next). It uses either he Fourier or Kalman filtering to produce clean annual data and can also be used to adjust the amount of year to year change that is measured per pixel by either changing <NUM>) the order of magnitude in the Fourier series or <NUM>) the gain factor in the Kalman filter. It also produces a count of samples and RMSE per pixel for quality control measures in the annual filtering.

Another application is if the SAR and/or Lidar imagery are only collected once a year, these data can be treated as annual data used with the annual descriptive statistics generated from CalcStats, NDVI for example. <FIG> shows a generic Multi-Sensor Data Fusion system <NUM> with a multisensory data fusion module <NUM>. SAR and Lidar imagery <NUM> is added to the annual descriptive statistics for use in the machine learning modelling with either an off the shelf data product or with ground data in a user defined CSV <NUM>. Training CSV module <NUM> and Training image module <NUM> are also input into the learning machine <NUM> and an output <NUM> of new annual vegetation attributedata is obtained.

The present invention has been designed as a forward-thinking solution which will change the way we can analyse our planet by embracing big data predictive analytics. It is the solution that can turn raw satellite data into commercially value information, which is a key differentiator in what will become an increasingly busy market of upstream providers hoping to sell the low margin raw data or imagery. The present invention may, for example, be used to monitor land, water, sea or air, by extracting any satellite data product and importantly, it then filters and removes the inherent errors in the raw data. It has the processing power to do this on global scales. The result is a capability to access and analyse more accurate data, giving anyone a clearer and better understanding of our changing planet.

Claim 1:
A computer implemented process for Earth observation and analysis (<NUM>), the process comprising:
i) pre-processing (<NUM>) remote sensing images by a) applying, on individual pixel locations a temporal signal processing filter (<NUM>, <NUM>) to a time series of remote sensing images to smooth out the signal noise for each pixel within a year or with respect to growth cycles within a year and b) extracting within-year and/or across year descriptive statistics from image pixels of the remote sensing images to create input independent variables (<NUM>, <NUM>) for use in a machine learning training process (<NUM>);
ii) applying the machine learning training process (<NUM>) to create a model (<NUM>) which determines how the input independent variable values (<NUM>, <NUM>) map to a range of possible target variable values (<NUM>, <NUM>), through a regression process;
iii) applying the model created in step ii) to a potentially new Area of Interest to predict target variable values (<NUM>, <NUM>) for known independent variables (<NUM>, <NUM>); and
iv) generating output images (<NUM>) corresponding to the specific regions defining the Area of Interest, for each of the target variables (<NUM>, <NUM>) used in the machine learning training process;
wherein the step of applying a machine learning training process (<NUM>) comprises using a Random Forest machine learning software application (<NUM>, <NUM>) which creates a <NUM> dimensional data array (<NUM>) indexed by n samples in the data set, stores the information for each tree and collates (<NUM>) the information for processing to improve RAM allocation and parallelizing in the software and processors used during the machine learning training process, wherein parallelizing is achieved by processing each tree through multi-threading to allow for N-fold parallelisation (<NUM>) of the core tree growing part of the processing, giving an effective (N-<NUM>)-fold reduction in processing time.