Package: geobo
GeoBO: A Python package for Multi-Objective Bayesian Optimisation and Joint Inversion in GeosciencesGeoBOis build upon a probabilistic framework using Gaussian Process (GP) priors to jointly solve multi-linear forward models. This software generates multi-output 3D cubes of geophysical properties (e.g. density, magnetic susceptibility, mineral concentrations) and their uncertainties from 2D survey data (e.g. magnetics and gravity) and any pre-existing drillcore measurements. The reconstructed 3D model is then used to query the next most promising measurement location given an expensive cost function (e.g. for drillcores). A ranked list of new measurements is proposed based on user-defined objectives as defined in the acquisition function which typically aims to optimize exploration (reducing global model uncertainty) and exploitation (focusing on highly promising regions) while minimizing costs.Table of ContentsDefinitionsFunctionalityInstallation And RequirementsRequirementsInstallationDocumentationUsage and SettingsExamples and TestsSynthetic ModelsDrillcore Test ExampleResults and Output FilesOptions and CustomizationCustom Linear Forward ModelsGaussian Process Kernel FunctionsLiteratureRelated SoftwareAttribution and AcknowledgmentsProject ContributorsLicenseDefinitionsBayesian Optimisation (BO) is a powerful framework for finding the extrema of objective functions that are noisy, expensive
to evaluate, do not have a closed-form (e.g. black-boxfunctions), or have no accessible derivatives. The model used for approximating the objective function is called surrogate model, which is typically based on aGaussian Process modelsfor tractability. Gaussian Processes define a prior over functions (typically given by a kernel function) and is used to propose points in the search space where sampling is likely to yield an improvement. The specific set of objectives for the improvement are defined in an acquisition function, which guides the search for a user-defined optimum. An example use case scenario is described in a nutshell inOPTIMIZATION_FOR_ACTIVE_SENSORFUSION_IN_A_NUTSHELL.pdf.Acquisition functionThe key of BO is the acquisition function, which typically has to balance between:a) exploration, i.e., querying points that maximise the information gain and minimize the uncertainty of a model
b) exploitation, i.e. querying points that maximise the reward (e.g. concentrating search in the
vicinity locations with high value such as minerals)
c) minimize the number of samples given an expensive cost function for any new measurement.Forward Models and Joint InversionIn geology and geophysics, inversion problems occur whenever the goal is to reconstruct the geological conditions, i.e. the 3D distribution of physical rock properties, that give rise to a set of (2D) geophysical observations. Since the number of possible geological configurations is typically greater than the number of observational constraints, the problem is nearly always under-determined.
Forward models transform the localized measurement of a remote sensor grid into a 3D representation of geophysical properties of a region. The most common geophysical linear forward model are gravity and magnetic forward models, which are computed using Li’s tractable approximation. Joint inversion is  simultaneously interpreting  multiple (distinct) sensor measurements using a single model to provide a better constrained joint solution rather than taking individual solutions that only satisfy their aspect of data on their own.FunctionalityGeoBO's probabilistic framework includes all steps from  prior selection, data fusion and inversion, to sensor optimisation and real world model output. The main functionalities of GeoBO are summarised in the following:Joint probabilistic inversion tool by solving simultaneously multi-linear forward models (e.g. gravity, magnetics) using cross-variances between geophysical properties (cross-variance terms can be specified by user).Output 1: Generation of cubes and computation of complete posterior distribution for all geophysical properties (described by their mean and variance value at each location (cubecell aka voxel).Output 2: Generation of ranked proposal list for new most promising drillcores based on global optimisation of acquisition functionTemplates for acquisition function to use in Bayesian OptimisationFlexible parameter settings for exploration-exploitation trade-off and inclusion of local 3D cost function in acquisition functionOther features are:Generation of simulated geophysical data with a choice of three different modelsPackage includes geological survey/drillcore sample as well as synthetic data and functions for synthetic data generationGeneration of 2D/3D visualisation plots of reconstructed cubes and survey data3D Cube export in VTK format (for subsequent analysis, e.g., in Python or with ParaView)Options to include any pre-existing drillcore dataIncluded linear forward models: density-to-gravity and magnetic susceptibility-to-magnetic field; custom linear forward models can be added (seeCustom Linear Forward Models)Library of Gaussian Process (GP) kernels including sparse GP kernelsFlexible settings for any cube geometry and resolution(Optional) Optimization of GP hyperparameters and cross-correlation coefficients via computation of marginal GP likelihoodExample outputs can be found in the directoryexamples/results/.Installation And RequirementsInstallationTo install GeoBO locally using setuptools:pythonsetup.pybuild
pythonsetup.pyinstallor using pip:pip3installgeoboThe installation can be tested by running the example with included synthetic data and default settings:cdgeobo/
pythonmain.pytests/settings_example1.yamlRequirementspython >=3.6numpymatplotlibscikit_imagescipyrasteriopandaspyvistaskimagePyYAMLDocumentationDocumentation conversion is generated using pandoc. The README markdown file can be converted to PDF:pandoc-Vgeometry:margin=1.0inREADME.md-oREADME.pdfA complete API documentation for all modules can be found here:run_geobo.pyinversion.pykernels.pycubeshow.pysensormodel.pysimcube.pyutils.pyUsage and SettingsChange the main settings such as filenames and parameters insettings.yaml. These settings specify:directory, filenames, and geophysical drillcore propertiesthe generated cube's geometry, size, and resolutionGaussian Process settings (lengthscale, input data uncertainty, correlation coefficients, kernel function)local Earth's magnetic field vectorBayesian Optimisation Settings (vertical/non-vertical drillcores, the exploration/exploitation and cost weighting)plotting settingsoptional generation of simulated dataThen run geobocdgeobo/
pythonmain.pysettings.yamlThe main functions for the acquisition function can be found inrun_geobo.py; visualisation functions and VTK export are defined incubeshow.py; inversion functions are defined ininversion.py.Examples and TestsSynthetic ModelsSynthetic geophysical models can be created by setting switching ongen_simulationin the settings yaml file.
Three different models are so far implemented:two-cylindric dipping bodies (modelname: 'cylinders')two layer model (modelname: 'layers2')three layer model (modelname: 'layers3')
For each model a 3D voxel cube with geological structures is generated with density and magnetic susceptibility properties, plus
the corresponding 2D gravity and magnetic remote sensor measurements. Other custom models can be included by adding a new model in functioncreate_syncube()insimcube.pyResult examples of the synthetic models are stored in the subfolderexamples/testdata/synthetic/.
An example settings file is given insettings_example1.yamland can be run bycdgeobo/
pythonmain.pytests/settings_example1.yamlDrillcore Test ExampleAnother examples includes drillcore and gravity/magnetic survey data (examples/testdata/sample/). This example can be run withcdgeobo/
pythonmain.pytests/settings_example2.yamland creates the reconstructed density and magnetic susceptibility cubes, uncertainty cubesResults and Output FilesThe output results include the generated reconstructed density and magnetic susceptibility cubes and their corresponding uncertainty cubes, visualisations of original survey data and reconstructed properties, and list of new measurement proposals.List of Joint Inversion Result Outputs:3D Cube files in vtk format (to use, e.g., with PyVista or ParaView): Output of cross-correlated reconstructed properties (density:cube_density.vtk, magnetic susceptibility:cube_magsus.vtk, property from drill:cube_drill.vtk) and their uncertainity cubes in terms of standard deviation (cube_density_variance.vtk, cube_drill_variance.vtk, cube_drill_variance.vtk). In case the cross-covariance terms (gp_coeffin the settings file) are all set to zero, the solutions for each forward model are independent from each other.Optional (Default optiion: plot=True in functionread_surveydata()): Images of the input gravitational field (gravfield.png) and magnetic field (magfield.png) and their corresponding downsampled images (gravfield_downsampled.png, magfield_downsampled.png)Optional (ifplot3d:True in settings): 3D Preview of reconstructed properties: density (density-mesh3D.png), magnetic suscpetibility (magsus-mesh3D.png), and drill property (drill-mesh3D.png). However, it is recommended to use PyVista or Paraview instead. Future updates for 3D visualisations will be based on PyVista.Optional (ifplot_vertical:True in settings): 2D maps of vertically (along z-axis) mean value of cube properties (dens_rec2D_loc2.png, magsus_rec2D_loc2.png, drill_rec2D_loc2.png)List of Bayesian Optimisation Output:List of all new measurement proposals (here for drillcores) ranked from maximum (hightest gain) to minimum of optimisation function. The results are saved as csv file (newdrill_proposals_non-vertical.csvornewdrill_proposals_vertical.csv) and include for vertical drillcores the position (Easting, Northing) and for non-vertical drillcores position and drill angles (Azimuth, Dip).Points of proposed measurement positions on top of reconstructed drill property image (mean projection along z-axis): The output figure (non-vertical drillcores:newdrill_proposals.pngor vertical drillcores:newdrill_vertical_proposals.png) shows the location of the already existing drills as given by input measurements (black points), the new proposed drill positions (white), and the best (maximum of optimsation function) new drill location (red)..Options and CustomizationCustom Linear Forward ModelsThe relationship between a physical system (or its model parameters)Pand the observed sensor datayis described by a linear forward modely=GP,whereGis the transformation operator or matrix. The gravitational and magnetic forward model can be determined analytically by using Li's tractable approximation (see Li and Oldenburg 1998) for a 3D field of prisms of constant susceptibility and density, and GeoBO applies this prism shape model to compute the corresponding sensor sensitivity for gravity and anomalous magnetic field related to each prism cell.The current implementation includes magnetic and gravity forward models, which are defined in the modulesensormodel.pyby the functionsA_sens(),grav_func(), andmagn_func(). The easiest way to add custom models is to create a new forward model function similar to the included functionsgrav_func()ormagn_funcand to compute the forward model matrix withA_sens(), if possible. The custom function need to describe the sensitivity or relationship for a particular point relative to the sensor origin (see, e.g.,grav_func()).In general any linear forward model can be added by changing accordingly the forward model matrix as computed byA_sens()as long as this function returns the matrixGthat satisfies the linear relationy=GP.Gaussian Process Kernel FunctionsGaussian Processes (GPs) are a flexible, probabilistic approach using kernel machines and can propagate consistently uncertainties from input to output space under the Bayesian formalism. Another advantage of GPs is that their marginal likelihood function is well defined by the values of their hyper-parameters, and can thus be optimized.
The choice for an appropriate covariance (kernel) function is important and there are many stationary (invariant to translation in input space) and non-stationary covariance functions available (for an overview, see, e.g., Rasmussen and Williams 2006). To handle the computational problem of inverting a large covariance matrix, GeoBO uses by default an intrinsically sparse covariance function (Melkumyan et, al. 2009). However, other standard kernel functions are available (see modulekernels.py), which includes the squared exponential
and Matern32 function and their their corresponding multi-kernel covariance functions (see Melkumyan et. al. 2011).The settings yaml file allows you to choose the kernel function by configuring the parameterkernelfunc,  which can be set either to 'sparse' (Default), 'exp' (squared exponential) or 'matern32'. New custom kernels can be a added in the modulekernels.py, which requires to write their covariance function (see as examplegpkernel()) and cross-covariance function (see as examplegpkernel_sparse()), and then to add their function name to settings.yaml and tocreate_cov()inkernels.py.The hyperparameters of the GP kernel can be configured in the settings yaml file (see Gaussian Process Settings) and are given by the lengthscale (gp_lengthscale), the noise regularization terms (gp_err) per forward model, and the cross-covariance amplitude terms which (w1,w2,w3) that coorrepond to the correlation coefficients between the model properties (e.g., rock density ,magnetic susceptibility, and drillcore measurements). The mathematical details for construction of the Multi-Kernel Covariance Functions are described in Haan et al 2020.Bayesian Optimisation OptionsTo find the optimal new sampling point, GeoBO maximises the objective function (acquisition function) which is defined by the Upper Confidence Bound (UCB)UCB(x) =m(x) +ksigma(x) -bc(x)with the mean value for the predictionm(x), the variancesigma2(x), and a cost functionc(x), which is defined by the cost of obtaining a measurement at the sample point x. The parameterkandbdefine the trade-off in exploration-to-exploitation and gain-to-cost, respectively. For example, maximizing the mean value can be beneficial if the goal is to sample new data at locations with high density or mineral content, and not only where the uncertainty is high.
The parameterskandbcan be accordingly specified by the user in the settings yaml file.  Moreover, the settings allow the user to choose between vertical and non-vertical drillcore; in the latter case GeoBO is optimising also dip and azimuthal angle of the  drillcore in addition to drillcore position.LiteratureSebastian Haan, Fabio Ramos, Dietmar Muller, "Multi-Objective Bayesian Optimisation and Joint Inversion for Active Sensor Fusion", Geophysics, 86(1), pp.1-78.arXiv PreprintCarl Edward Rasmussen and Christopher KI Williams, Gaussian process for machine learning, MIT press, 2006.Li Yaoguo and Douglas W Oldenburg, “3d-inversion of gravity data,” Geophysics, vol. 63, no. 1, pp. 109–119, 1998.Arman Melkumyan and Fabio Ramos, “A sparse covariance function for exact gaussian process inference in large datasets.,” in IJCAI, 2009, vol. 9, pp. 1936–1942Armon Melkuyman and Fabio Ramos, “Multi-kernel gaussian processes,” in IJCAI, 2011, vol. 22, p. 1408Reid, A., O. Simon Timothy, E. V. Bonilla, L. McCalman, T. Rawling, and F. Ramos, 2013, Bayesian joint inversions for the exploration of earth resources.: IJCAI, 2877Eric Brochu, Vlad M Cora, and Nando De Freitas, “A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning,” arXiv preprint arXiv:1012.2599, 2010.Related SoftwareFor the inversion part, GeoBO uses a direct inversion method via transformation of Gaussian Process priors, which enables joint inversion but is limited to linear forward models (e.g. gravity, magnetics, drillcores). For solving more complex non-linear forward models (e.g., seismic, or prior geological knowledge), the following bayesian inversion methods can potentially be applied to generate 3D geophysical surrogate models or to further refine GeoBo's 3D posterior model:hIPPYlib: an Extensible Software Framework for Large-scale Deterministic and Bayesian Inverse Problems.Publication Link; the software code is available athippylib.github.ioObsidian: a flexible software platform for MCMC sampling of 3-D multi-modal geophysical models on distributed computing clusters.Publication Link; the code for version 0.1.2 of Obsidian is available athttps://doi.org/10.5281/zenodo.2580422GemPy: open-source stochastic geological modeling and inversion; geoscientific model development. Seegempy.orgAttribution and AcknowledgmentsAcknowledgments are an important way for us to demonstrate the value we bring to your research. Your research outcomes are vital for ongoing funding of the Sydney Informatics Hub.If you make use of this code for your research project, please include the following acknowledgment:“This research was supported by the Sydney Informatics Hub, a Core Research Facility of the University of Sydney.”Project ContributorsKey project contributors to the GeoBO project are:Dr. Sebastian Haan (USYD, Sydney Informatics Hub): Expert in machine learning and physics, main contributor and software development of GeoBO.Prof. Fabian Ramos (USYD): Computational scientist and research expert in machine learning and bayesian computational techniques.Prof. Dietmar Muller (USYD, School of Geoscience): Research expert in geophysics and geoscience applications.Dr. Ben Mather (USYD, Sydney Informatics Hub/School of Geoscience ): Computational Geophysicist, GeoBO testing.LicenseCopyright 2020 Sebastian HaanGeoBO is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License (AGPL version 3) as published by the Free Software Foundation.This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more details.You should have received a copy of the GNU Affero General Public License along with this program (see LICENSE.md). If not, seehttps://www.gnu.org/licenses/.