Process parameter prediction using multivariant structural regression

Multivariant feature extraction is used for training volumes or 2D images, (real or synthetic) coupled to process (effective) values probably obtained from direct simulation. These features are coupled with machine learning/regression algorithms to make a predictive model for the effective property. This model can then be used on a real geometry of a sample for effective parameter prediction.

BACKGROUND OF THE INVENTION

Various imaging modalities have been used to identify and visualize the internal microscopic structure of natural and synthetic samples both in two dimensions (2D) and in three dimensions (3D). For example, these imaging modalities can analyze various types of samples such as rock samples from oil and gas extraction operations, additively manufactured items, or other natural or man-made things.

Non-destructive imaging systems include x-ray computed tomography (CT) microscopy and Scanning Electron Microscopy (SEM) systems. These systems provide the ability to visualize features such as pores, organics and minerals in the samples. Other examples include light microscopy including light based tomography techniques such as optical coherence tomography.

The X-ray CT microscopy systems irradiate the sample with x-rays, typically in a range between 1 and several hundred keV. 2D projection images are collected at multiple angles and a 3D volume of the sample is reconstructed from the projections.

SEM systems instead irradiate the sample surface with a beam of high energy electrons, typically between 500 eV and 30 keV. The signals derived from electron-sample interaction are used in constructing high resolution 2D images of the sample surface. This enables the simultaneous operation of SEM in multiple modes such as Back Scattered Electron (BSE), Secondary Electron (SE), Energy Dispersive X-ray (EDX), and Cathodoluminescence (CL) modes. EDX is typically the primary system on a SEM that offers quantitative mineralogy information which enables 2D mineral mapping of the sample surface.

Destructive imaging systems include Focused Ion Beam Scanning Electron Microscope (FIB-SEM) systems. A FIB-SEM is a multiple beam system that integrates ion beam and electron beam systems. The FIB system irradiates the sample with a focused high-current beam of ions of a source material such as gallium to mill the sample surface with high precision. The milled surface is then imaged at high resolution using the integrated SEM system. The FIB milling and SEM imaging process is repeated until a desired volume is sampled. The SEM images from each slice are stacked to construct a 3D volume of the milled region of the sample.

Often these 2D and 3D imaging modalities are used determine predict parameters of the samples. In typical operation, these imaging modalities create image datasets such as 3D volumes or 2D images. Image analysis techniques are then employed to infer composition and structures from the volumes and/or images. Physics simulations can then be used to determine or predict various parameters of interest. Porosity and mineralogy, flow parameters and other mechanical parameters are derived from these simulations.

SUMMARY OF THE INVENTION

The present invention concerns an analysis method that employs a regression technique that offers a way by which macroscopic (effective) parameters of samples, imaged in X-ray microscopes, SEM's and/or other imaging systems, can be predicted. The parameters of interest can be computed directly from multivariant structural statistics without having to perform expensive full physics simulations/computations. It also allows for predictive models to be created which are effective when imaging a sample using merely 2D imaging techniques, eliminating the need to image them with 3D techniques which can be expensive, slow, and limited in the spatial lengthscale they have accessible to them.

The approach contrasts with the state of the art. Typically, the approach to predict parameters employs expensive computationally complex full physics simulations, requiring extensive resources, or utilizes highly simplified/analytical approaches to prediction, leading to significantly inaccurate results.

The present approach instead takes a multivariant (machine learning based) approach, which can greatly speed up predictions (estimating orders of magnitude speedups) while maintaining prediction robustness. This machine learning based model is trained using a large number of representative structures, which can either be taken from a library or database, or can be created using a geometry creation technique.

As a result, any specific geometry creation routine may be able to be substituted—the critical factor is that the simulated geometries should well represent the real (imaged) geometries, and so act as an extrapolative guide. That said, it is possible, given enough real geometries, that the whole step of creating simulated geometries for structural regression could be bypassed, only using real geometries to define both feature vectors and target functions.

In general, according to one aspect, the invention features a method for determining parameters of a sample. The method comprises performing parameter estimation process by creating a prediction model from images and making final predictions using real geometry of a sample and the prediction model.

In examples, the prediction model is created from synthetic images and/or actual images.

Preferably the prediction model is generated using multivariant regression, such as by computing process parameters from the images, segmenting the images into objects, extracting features from the images and computing statistics across the objects of the images.

In some examples, the prediction model is created by feeding images into a convolutional neural network which maps from image area to a variable or other machine learning approach.

Then, the final predictions are performed by extracting feature vectors from the sample and employing the prediction module to determine the parameters of the sample.

In examples, the sample is from mining and/or oil/gas extraction. Other examples involve an additive manufacturing process.

In current embodiments, the parameter estimation process is performed on 3D image datasets, such as from an x-ray CT microscope, and the final predictions are made from 2D image datasets, such as from 2D electron or optical microscope image datasets.

In general, according to one aspect, the invention features a system for determining parameters of a sample. The system comprises a machine learning application, executing on a computer system, for example, performing parameter estimation process by creating a prediction model from images. Then, the machine learning application makes final predictions using real geometry of a sample and the prediction model.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Important macroscopic or effective parameters of a sample include but are not limited to the sample's permeability, diffusivity, elastic moduli, electrical characteristics, and single or multiphase transport processes. These parameters are frequently calculated from 3D tomographic images using computational simulation of partial differential equations.

FIG.1illustrates the conventional approach of using partial differential equations to computationally simulate a sample's parameters.

A 3D volume108of a sample, such as one created by tomographic reconstruction from x-ray projections through the sample, is used to create a full physics simulation110. This simulation110is then used to predict the sample's process parameters of interest such as the sample's permeability112.

Performing a full physics simulation is extremely computationally expensive, however. The physical processes must be computed explicitly step by step in time.

FIG.2illustrates another conventional approach is to estimate parameters of a sample. A set of univariant measurements are made to the sample. In the illustrated example, the measurements114are made from a 2D slice116taken from a 3D tomographic image or 2D SEM images from cross-sections of the sample, for example. These measurements are then used as the input into quasi-analytical equations 118, which predict process parameters112.

Such quasi-analytical approaches are extremely simplified, however. As a result, they tend to have large prediction errors. The predicted parameters, however, do inherently depend on the structural features of the network, and as such it should be possible to make a direct prediction from these features.

FIG.3Ais a flow diagram showing a parameter estimation process according to the present invention.

In step210, series of synthetic or real images are generated or collected bearing similar statistical and process parameter properties.

Such synthetic images can be generated using a suite of statistical or object based techniques. An example of such an object based technique can be one of the following:1. Statistical reconstruction of three-dimensional porous media from two-dimensional images, by Anthony P. Roberts, Phys. Rev. E 56, 3203. 1997.2. Prediction of permeability for porous media reconstructed using multiple-point statistics, Hiroshi Okabe and Martin J. Blunt, Phys. Rev. E 70, 066135, 2004.3. 3D Stochastic Modelling of Heterogeneous Porous Media—Applications to Reservoir Rocks, Transport in Porous Media volume 65, pages 443-467(2006).4. Comparing organic-hosted and intergranular pore networks: topography and topology in grains, gaps and bubbles, Matthew Andrew, Geological Society, London, Special Publications, 484, 3 Sep. 2018.5. Reconstruction of three-dimensional porous media using generative adversarial neural networks, Lukas Mosser, Olivier Dubrule, and Martin J. Blunt, Phys. Rev. E 96, 043309, 2017.

In other examples, the images are not necessarily synthetic. For example, the real images could be images obtained from actual samples. In general, however, the synthetic images have the advantage that many can be created with relative ease.

The synthetic or hybrid or actual images are then segmented into individual phases, such as phases, pores, grains, or other structures in the images in step212.

The process then bifurcates.

The process parameters and other physical properties are computed from these images using traditional modelling or physics simulation techniques in step214. Examples of appropriate modeling tools include such open source projects as OpenFOAM, a large number of academic codes (e.g. https://www.imperial.ac.uk/earth-science/research/research-groups/perm/research/pore-scale-modelling/software/ or http://openpnm.org/), or internal industrial software projects (such as Schlumberger Direct HydroDynamic (DHD) simulation) as well as a suite of commercial software packages, including ThermoFisher Avizo Xlab, ThermoFisher eCore, ThermoFisher Pergeos, VolumeGraphics VGStudio, Comsol, Abaqus and Math2Market GeoDict simulation packages.

In parallel, the images are analyzed to separate physically touching objects in step216. Objects are defined as contiguous regions belonging to a single phase. Approaches include morphological techniques or deep learning techniques.

Then in step218, measurements of the objects are made.

Statistical feature vectors are constructed from the measurements in step220. If separation/measurement/analysis is exclusively done in 2D, model is “2D.” If any or all is done in 3D, model is 3D.

In general, the extracted features could relate to a wide array of properties measured in the image, and may vary from predicted process (effective) parameter to predicted process (effective) parameter, and may be extracted in 2D or in 3D. Note that as it is possible to create a model from features extracted from a 2D slice from a 3D network, it is possible to create a predictive model which operates only from 2D data. This allows for effective multivariate predictions to be made even when the only data available is 2D.

A prediction model is then creates using multivariant regression of feature vectors to predicted physical property based on the constructed feature vectors and the computed physical properties in step222.

Generally, statistics are computed across all the objects in the image. These statistics are then used in a multivariant description of the network as a whole. This multivariant description is then regressed against the parameter values using multivariant linear or non-linear regression to create the prediction model. In fact, any number of regression techniques may be used.

Then, in step230, the prediction model is used to determine the physical properties of samples from 2D or 3D images. These samples are taken from mineral and/or oil/gas exploration and production. The samples could also come from manufacturing operations such as the manufacture of batteries and powder bed 3D additive manufacturing. The features vectors are extracted and given to the prediction model of step222to predict the parameters of the samples.

FIG.3Bis a flow diagram showing a parameter estimation process according to another embodiment.

Here, the images from steps210and212are fed into a convolutional neural network which maps from image area to single continuous variable.

Then, as before, a prediction model is created in step222using multivariant regression of feature vectors to predicted physical property based on the constructed feature vectors and the computed physical properties from step214.

FIG.4illustrates this process of creating a prediction model by multivariant regression of X features or parameters on a target function created by simulating process parameters on N synthetic volumes.

In more detail, the N images310are simulated as described in connection with step210ofFIGS.3A and3B. Or real images or a combination of real and synthetic images are used.

Another approach would be to perform a direct regression from the images using 2D or 3D networks (bypassing the need for structural statistical extraction).

Structural parameters can then be calculated312on real and/or synthetic (imaged) geometries as in step218and applied to predict the process parameter.

Incremental imaged and simulated volumes can be added to the training set for multivariant regression314, or used as to create new regression sequences, or as the basis for transfer learning from the original set.

FIG.5shows the final step of making final predictions as set forth in step230ofFIGS.3A and3B, using (imaged) real geometry of a sample S. The feature vectors are extracted350from the volume of the same and the statistics of analytical features calculated. These vectors are then input into previously trained multivariant model to make process parameter predictions352as described in step230.

The present approach could be extended to other machine learning tools such as neural networks. The model could be a trained neural network which takes as an input directly the 2D or 3D image structures from the training set, or the statistical feature set extracted from these images. Other alternatives are random forest regression or other multivariant regression models.

An example application of this technology is the prediction of permeability parameters from 2D images generated from light and/or electron microscopy.

Typically, such prediction of permeability is performed (at the pore scale) by performing 3D computational fluid dynamics on x-ray microscopy image datasets. The challenge with this approach is that the addressable spatial lengthscale with these (or, more precisely, its ratio with the voxel resolution of the volume) is extremely limited.

In contrast, the present approach can use multivariant regression based prediction, applied to 2D light and/or electron microscopy image data, which can be acquired over a much larger spatial lengthscale.

In general, sample permeability is critical to understand a range of applications from subsurface oil and gas flow, to carbon capture and storage, to filter performance, battery performance and more.

This takes the following process/workflow, which is described in connection withFIG.6:

First, a suite of segmented training images is required. These can be produced by a range of approaches as set forth in step210:a. Image library. If samples have been imaged then these image datasets can be used for training, but a broad range of samples is required. In one example, the image datasets are 3D dataset from an x-ray microscope. This is shown inFIG.6, where a manufacturing or extraction operation410produces actual, physical samples412that are imaged in an x-ray CT microscope414.b. Synthetic image creation. This has the advantage of not requiring many images of many samples, but may only be representative of certain pore systems.c. AI based statistical image realization. Generative networks (e.g., GANs) can be trained on a relatively limited subset of original images, extending them over a much broader range of input parameters.

Second, flow is simulated in each of these geometries. This is performed using standard computational fluid dynamics techniques using a physics simulation application416.

Such a physics simulation application416is executed on a computer system450such as computer workstation or computer cluster or a cloud-based computer system. The computer system450has a hardware system454including one or more microprocessors and attendant memory, along with other storage resources. An operating system452will typically execute on the hardware system. The operating system452provides access to the compute resources for the various applications executing on the operation system such as the physics simulation applications416.

In parallel, each geometry is analyzed to extract a statistical feature vector of the geometry in step220. This uses image or 3D volume analysis app418that executes on the computer system450or another computer system. The 3D volume analysis app418performs following sub-workflow in one example:a. 2D slices are randomly extracted from the volume. Each slice is analyzed independently.b. Objects within each slice are separated as in step216. This uses a multi-scale object separation routine. Pores and grains are separated independently. The multiscale object separation can be performed as follows: 1. a Euclidian (or chamfer) distance transform is calculated on the objects, 2. small object seeds are created by identifying local maxima in the distance transform, 3. large object seeds are created by segmenting the distance transform, 4. the union between the two seed objects is evaluated, 5. this image is then analyzed to identify connected components, with each forming the seed of a separated object, and 6. These seeds are then grown on the landscape of the distance transform image using a watershed algorithm.c. This generates a separated pore and grain image. A final “pore throat” image is created by 3D volume analysis app418evaluating the regions when two separated pore objects touch.d. Measurements are then made on each of these objects, creating a set of features as in step218by the 3D volume analysis app418. Many different measurements are possible. Example measurements include:

TABLE 1Feature/unitInscribed circle radius/μmArea/μm2Euler numberCoordination numberPerimeter/μmConvexitye. From this list of features, statistical measures are made on each feature as in step220. A range of different statistical measures are possible, but an example evaluated for permeability prediction from thin section are:

TABLE 2Feature/unitThroat inscribed radius standard deviation/μmPore inscribed radius mean/μmAverage throat inscribed radius/μmLargest throat inscribed radius/μmPore inscribed radius standard deviation/μmLargest pore inscribed radius/μmPore inscribed radius area weighted mean/μmPorosityFraction of pores with coordination number 2Pixel size/μmGrain inscribed radius standard deviation/μmCoordination number 0Grain inscribed radius mean/μmArea weighted mean pore area/μm2Largest grain inscribed radius/μm2Lognorm Fitted Pore Area/μm2Pore area standard deviation/μm2Largest pore area/μm2Grain inscribed radius weighted mean/μmFraction of pores with coordination number 3Mean pore area/μm2Maximum pore area/μm2Fraction of pores with coordination number 4Fraction of pores with coordination number 1Euler number (porosity)Largest grain area/μm2Fraction of pores with coordination number 5Lognorm fitted grain area/μm2Euler number (volume weighted, grain)Euler number (grain)Lognorm fitted pore number standard deviationEuler number (volume weighted, pore)Fraction of pores with coordination number 6Fraction of pores with coordination number 7f. This then provides a statistical feature vector description of the pore network. Measurements (from step d) can be aggregated across multiple slices prior to statistical measurement to increase the robustness of the measure.

Third, a prediction model426is created using multivariant regression, e.g. using a Random Forest, multi-layer perceptron, support vector machine or other machine learning approach as in step222. In the illustrated example, the machine learning is performed by a machine learning app420executing on the computer system450which received that statistical feature vector descriptions from the geometry app418.

Finally, once this model426is created, it can be applied on any 2D image dataset (e.g. image dataset from a light microscope or electron microscope), once that data has been segmented into pore and grain (features were only extracted from 2D slices) as in step230. This allows for much larger spatial lengthscales to be applied in a reasonable period of time.

In the illustrated example, new samples412produced by the manufacturing or extraction operation410are imaged in a SEM422and the images are provided directly to the machine learning app420that applies the prediction model426or first provided to the geometry app that then provides their statistical feature vector descriptions to the machine learning app420.