Repository: alan-turing-institute/mogp-emulator (language: python)

Issue:
Feature/gpu meanfunc
Implement mean functions in the C++ code used for the GPU implementation.  (Note that the mean function calculations are not currently done on the GPU - they are in the C++ code executed on the CPU).
Currently implemented:
* ZeroMeanFunc (default - zero everywhere)
* FixedMeanFunc (no parameters, fixed constant value everywhere)
* ConstMeanFunc (one parameter, constant value everywhere)
* PolyMeanfunc (polynomial of any degree in any of the input dimensions, but no cross-terms (e.g. x[0]*x[1]) permitted.

* All C++ classes have pybind11 bindings to create corresponding python classes.
* The Python interface in GaussianProcessGPU uses the python MeanFunction as an argument to the GP constructor.   A function `parse_meanfunc_formula` uses the string representation of the formula from MeanFunction to create the appropriate C++ instance (i.e. it relies on the Python version having done some formula parsing to put things in a reasonably standard format).
Fix threadblock size calculation in some of the CUDA kernel calls

Rename variables in mogp_gpu to be consistent with the python implementation



Discussion / hints:

Fix patch:
diff --git a/.gitignore b/.gitignore
index 8746f4ac..34f610bf 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,9 @@
 *.pyc
 *.DS_Store
+*~
+docs/_build/*
+mogp_gpu/obj
+mogp_gpu/bin
 docs/_build/*
 mogp_emulator/benchmarks/*.png
 mogp_emulator/benchmarks/*.pdf
diff --git a/docs/benchmarks/benchmarks.rst b/docs/benchmarks/benchmarks.rst
index 27a9c1dd..9a28190b 100644
--- a/docs/benchmarks/benchmarks.rst
+++ b/docs/benchmarks/benchmarks.rst
@@ -8,69 +8,86 @@ Benchmarks
    rosenbrock
    branin
    tsunami
-   mcmc_benchmark
    mice_benchmark
+   pivot_benchmark
    gkdr_benchmark
    histmatch_benchmark
 
-The code includes a series of benchmarks that illustrate various pieces of the implementation. Benchmarks
-can be run from the ``mogp_emulator/tests`` directory by entering ``make all`` or ``make benchmarks`` to
-run all benchmarks, or ``make rosenbrock``, ``make branin``, ``make tsunami``, ``make mcmc``, ``make mice``, ``make gKDR``, or ``make histmatch`` to run the individual benchmarks.
+The code includes a series of benchmarks that illustrate various
+pieces of the implementation. Benchmarks can be run from the
+``mogp_emulator/benchmarks`` directory by entering ``make all`` or ``make
+benchmarks`` to run all benchmarks, or ``make rosenbrock``, ``make
+branin``, ``make tsunami``, ``make mice``, ``make pivot``, ``make
+gKDR``, or ``make histmatch`` to run the individual benchmarks.
 
 Single Emulator Convergence Tests
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The first benchmark examines the convergence of a single emulator applied to the Rosenbrock function in
-several different dimensions (more details can be found `here <https://www.sfu.ca/~ssurjano/rosen.html>`__).
-This illustrates how the emulator predictions improve as the number of training points is increased
-for different numbers of input parameters. The benchmark evaluates the Rosenbrock function in 4, 6, and
-8 dimensions and shows that the mean squared prediction error and the mean variance improve with the
-number of training points used. Matplotlib can optionally be used to visualize the results.
+The first benchmark examines the convergence of a single emulator
+applied to the Rosenbrock function in several different dimensions
+(more details can be found `here
+<https://www.sfu.ca/~ssurjano/rosen.html>`__).  This illustrates how
+the emulator predictions improve as the number of training points is
+increased for different numbers of input parameters. The benchmark
+evaluates the Rosenbrock function in 4, 6, and 8 dimensions and shows
+that the mean squared prediction error and the mean variance improve
+with the number of training points used. Matplotlib can optionally be
+used to visualize the results.
 
 Multi-Output Convergence Tests
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The second benchmark examines the convergence of multiple emulators derived from the same input values.
-This benchmark is based on the 2D Branin function (more details on this function can be found `here <https://www.sfu.ca/~ssurjano/branin.html>`__)
-. The code uses 8 different realizations of the Branin
-function using different parameter values, and then examines the convergence of the 8 different
-emulators fit using different number of parameter values based on the prediction errors and
-variance values. The results can optionally be visualized using Matplotlib.
+The second benchmark examines the convergence of multiple emulators
+derived from the same input values.  This benchmark is based on the 2D
+Branin function (more details on this function can be found `here
+<https://www.sfu.ca/~ssurjano/branin.html>`__) . The code uses 8
+different realizations of the Branin function using different
+parameter values, and then examines the convergence of the 8 different
+emulators fit using different number of parameter values based on the
+prediction errors and variance values. The results can optionally be
+visualized using Matplotlib.
 
 Multi-Output Performance Benchmark
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-A performance benchmark is included that uses a set of Tsunami simulation results to examine the
-speed at which the code fits multiple emulators in parallel. The code fits 8, 16, 32, and 64 emulators
-using 1, 2, 4, and 8 processess and notes the time required to perform the fitting. Note that the results
-will depend on the number of cores on the computer -- once you exceed the number of cores, the performance
-will degrade. As with the other benchmarks, Matplotlib can optionally be used to plot the results.
+A performance benchmark is included that uses a set of Tsunami
+simulation results to examine the speed at which the code fits
+multiple emulators in parallel. The code fits 8, 16, 32, and 64
+emulators using 1, 2, 4, and 8 processess and notes the time required
+to perform the fitting. Note that the results will depend on the
+number of cores on the computer -- once you exceed the number of
+cores, the performance will degrade. As with the other benchmarks,
+Matplotlib can optionally be used to plot the results.
 
-MCMC Benchmark
+MICE Benchmark
 ~~~~~~~~~~~~~~
 
-A benchmark applying the software to fitting an emulator with MCMC sampling is included. The code
-draws hyperparameter samples and compares the resulting posterior distributions with the values
-found via maximum likelihood estimation. If Matplotlib is installed, a histogram of the parameter
-samples is shown.
+A benchmark comparing the MICE Sequential design method to Latin
+Hypercube sampling is also available.  This creates designs of a
+variety of sizes and computes the error on unseen data for the 2D
+Branin function. It compares the accuracy of the sequential design to
+the Latin Hypercube for both the predictions and uncertainties.
 
-MICE Benchmark
-~~~~~~~~~~~~~~
+Pivoted Cholesky Benchmark
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-A benchmark comparing the MICE Sequential design method to Latin Hypercube sampling is also available.
-This creates designs of a variety of sizes and computes the error on unseen data for the 2D Branin
-function. It compares the accuracy of the sequential design to the Latin Hypercube for both the
-predictions and uncertainties.
+This benchmark shows how a duplicated input can degrade emulator performance
+and how using pivoting can fix this more effectively than adding a nugget.
+It compares the accuracy of the emulator and its uncertainty for pivoting
+and adaptive nugget handling to the 2D Branin function.
 
 Dimension Reduction Benchmark
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-A benchmark illustrating gradient-based kernel dimension reduction is available in ``tests/benchmark_kdr_GP.py``
-This problem contains a 100 dimension function with a single active dimension and shows how the loss depends
-on the number of dimensions included in the reduced dimension space.
+A benchmark illustrating gradient-based kernel dimension reduction is
+available in ``tests/benchmark_kdr_GP.py`` This problem contains a 100
+dimension function with a single active dimension and shows how the
+loss depends on the number of dimensions included in the reduced
+dimension space.
 
 History Matching Benchmark
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-A benchmark illustrating use of the ``HistoryMatching`` class to rule out parts of the input space
-using a GP emulator to make predictions. The benchmark contains a 1D and a 2D example.
+A benchmark illustrating use of the ``HistoryMatching`` class to rule
+out parts of the input space using a GP emulator to make
+predictions. The benchmark contains a 1D and a 2D example.
diff --git a/docs/benchmarks/mcmc_benchmark.rst b/docs/benchmarks/mcmc_benchmark.rst
deleted file mode 100644
index 5e77b938..00000000
--- a/docs/benchmarks/mcmc_benchmark.rst
+++ /dev/null
@@ -1,7 +0,0 @@
-.. _mcmc_benchmark:
-
-**********************************
-MCMC Benchmark
-**********************************
-
-.. automodule:: mogp_emulator.benchmarks.benchmark_MCMC
\ No newline at end of file
diff --git a/docs/benchmarks/pivot_benchmark.rst b/docs/benchmarks/pivot_benchmark.rst
new file mode 100644
index 00000000..c6bcaac5
--- /dev/null
+++ b/docs/benchmarks/pivot_benchmark.rst
@@ -0,0 +1,7 @@
+.. _pivot_benchmark:
+
+**********************************
+Pivoted Cholesky Benchmark
+**********************************
+
+.. automodule:: mogp_emulator.benchmarks.benchmark_pivot
diff --git a/docs/demos/gp_demoGPU.rst b/docs/demos/gp_demoGPU.rst
new file mode 100644
index 00000000..a7111dd7
--- /dev/null
+++ b/docs/demos/gp_demoGPU.rst
@@ -0,0 +1,16 @@
+.. _gpdemoGPU:
+
+Gaussian Process Demo (GPU)
+=========================
+This demo illustrates a simple example of fitting a GP emulator to results of the projectile
+problem discussed in the :ref:`tutorial`, using the GPU implementation of the emulator.
+
+Note that in order for this to work, it must be run on a machine with an Nvidia GPU, and with
+CUDA libraries available.  It also depends on Eigen and pybind.
+
+The example uses Maximum Likelihood Estimation with a Squared Exponential kernel, which is currently
+the only kernel supported by the GPU implementation.
+
+
+.. literalinclude::
+   ../../mogp_emulator/demos/gp_demo_gpu.py
diff --git a/docs/demos/historymatch_demos.rst b/docs/demos/historymatch_demos.rst
new file mode 100644
index 00000000..f9d17269
--- /dev/null
+++ b/docs/demos/historymatch_demos.rst
@@ -0,0 +1,12 @@
+.. _historymatch_demos:
+
+History Matching Demos
+========================================================
+
+This demo shows how to carry out History Matching using a GP emulator.
+The two examples show how a fit GP can be passed directly to the
+`HistoryMatching` class, or how the predictions object can be passed
+instead. The demo also shows how other options can be set.
+
+.. literalinclude::
+   ../../mogp_emulator/demos/historymatch_demos.py
diff --git a/docs/demos/kdr_demos.rst b/docs/demos/kdr_demos.rst
new file mode 100644
index 00000000..6a91fb03
--- /dev/null
+++ b/docs/demos/kdr_demos.rst
@@ -0,0 +1,13 @@
+.. _kdr_demos:
+
+Kernel Dimension Reduction (KDR) Demos
+========================================================
+
+This demo shows how to use the ``gKDR`` class to perform dimension reduction
+on the inputs to an emulator. The examples show how dimension reduction
+with a known number of dimensions can be fit, as well as how the class
+can use cross validation to infer a best number of dimensions from the
+data itself.
+
+.. literalinclude::
+   ../../mogp_emulator/demos/kdr_demos.py
diff --git a/docs/implementation/GaussianProcessGPU.rst b/docs/implementation/GaussianProcessGPU.rst
new file mode 100644
index 00000000..09304361
--- /dev/null
+++ b/docs/implementation/GaussianProcessGPU.rst
@@ -0,0 +1,15 @@
+.. _GaussianProcessGPU:
+
+**********************************
+The ``GaussianProcessGPU`` Class
+**********************************
+
+.. automodule:: mogp_emulator.GaussianProcessGPU
+    :noindex:
+
+.. autoclass:: mogp_emulator.GaussianProcessGPU.GaussianProcessGPU
+    :members:
+    
+    .. automethod:: __init__
+
+.. automodule:: libgpgpu
diff --git a/docs/implementation/implementation.rst b/docs/implementation/implementation.rst
index f1bbc70a..df4186fe 100644
--- a/docs/implementation/implementation.rst
+++ b/docs/implementation/implementation.rst
@@ -8,6 +8,7 @@ mogp_emulator Implementation Details
    :caption: Class and Function Implementation Details:
 
    GaussianProcess
+   GaussianProcessGPU
    MultiOutputGP
    fitting
    MeanFunction
@@ -18,4 +19,4 @@ mogp_emulator Implementation Details
    ExperimentalDesign
    SequentialDesign
    HistoryMatching
-   MCMC
\ No newline at end of file
+   linalg
diff --git a/docs/implementation/MCMC.rst b/docs/implementation/linalg.rst
similarity index 51%
rename from docs/implementation/MCMC.rst
rename to docs/implementation/linalg.rst
index d7e9df84..939868c8 100644
--- a/docs/implementation/MCMC.rst
+++ b/docs/implementation/linalg.rst
@@ -1,8 +1,8 @@
-.. _MCMC:
+.. _linalg:
 
 **********************************
-The ``MCMC`` Module
+The ``linalg`` Module
 **********************************
 
-.. automodule:: mogp_emulator.MCMC
+.. automodule:: mogp_emulator.linalg.cholesky
     :members:
diff --git a/docs/index.rst b/docs/index.rst
index 74321c5b..31330c02 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -33,6 +33,9 @@ details, and some included benchmarks.
 
    demos/gp_demos
    demos/mice_demos
+   demos/historymatch_demos
+   demos/kdr_demos
+   demos/gp_demoGPU
    demos/gp_demoR
    demos/excalibur_workshop_demo
 
diff --git a/mogp_emulator/GaussianProcess.py b/mogp_emulator/GaussianProcess.py
index 336d927a..b720028f 100644
--- a/mogp_emulator/GaussianProcess.py
+++ b/mogp_emulator/GaussianProcess.py
@@ -4,41 +4,55 @@
 from mogp_emulator.Priors import Prior
 from scipy import linalg
 from scipy.optimize import OptimizeResult
-from mogp_emulator.linalg.cholesky import jit_cholesky
-
-class GaussianProcess(object):
-    """
-    Implementation of a Gaussian Process Emulator.
-
-    This class provides a representation of a Gaussian Process Emulator. It contains
-    methods for fitting the GP to a given set of hyperparameters, computing the
-    negative log marginal likelihood plus prior (so negative log posterior) and its
-    derivatives, and making predictions on unseen data. Note that routines to
-    estimate hyperparameters are not included in the class definition, and are instead
-    provided externally to facilitate implementation of high performance versions.
-
-    The required arguments to initialize a GP is a set of training data consisting
-    of the inputs and targets for each of those inputs. These must be numpy arrays
-    whose first axis is of the same length. Targets must be a 1D array, while inputs
-    can be 2D or 1D. If inputs is 1D, then it is assumed that the length of the second
-    axis is unity.
-
-    Optional arguments are the particular mean function to use (default is zero mean),
-    the covariance kernel to use (default is the squared exponential covariance),
-    a list of prior distributions for each hyperparameter (default is no prior information
-    on any hyperparameters) and the method for handling the nugget parameter.
-    The nugget is additional "noise" that is added to the diagonal of the covariance
-    kernel as a variance. This nugget can represent uncertainty in the target values
-    themselves, or simply be used to stabilize the numerical inversion of the covariance
-    matrix. The nugget can be fixed (a non-negative float), can be found adaptively
-    (by passing the string ``"adaptive"`` to make the noise only as large as necessary
-    to successfully invert the covariance matrix), or can be fit as a hyperparameter
-    (by passing the string ``"fit"``).
-
-    The internal emulator structure involves arrays for the inputs, targets, and hyperparameters.
-    Other useful information are the number of training examples ``n``, the number of input
-    parameters ``D``, and the number of hyperparameters ``n_params``. These parameters can
-    be obtained externally by accessing these attributes
+from mogp_emulator.linalg import jit_cholesky, pivot_cholesky, pivot_cho_solve
+
+
+class GaussianProcessBase(object):
+    pass
+
+
+class GaussianProcess(GaussianProcessBase):
+    """Implementation of a Gaussian Process Emulator.
+
+    This class provides a representation of a Gaussian Process
+    Emulator. It contains methods for fitting the GP to a given set of
+    hyperparameters, computing the negative log marginal likelihood
+    plus prior (so negative log posterior) and its derivatives, and
+    making predictions on unseen data. Note that routines to estimate
+    hyperparameters are not included in the class definition, and are
+    instead provided externally to facilitate implementation of high
+    performance versions.
+
+    The required arguments to initialize a GP is a set of training
+    data consisting of the inputs and targets for each of those
+    inputs. These must be numpy arrays whose first axis is of the same
+    length. Targets must be a 1D array, while inputs can be 2D or
+    1D. If inputs is 1D, then it is assumed that the length of the
+    second axis is unity.
+
+    Optional arguments are the particular mean function to use
+    (default is zero mean), the covariance kernel to use (default is
+    the squared exponential covariance), a list of prior distributions
+    for each hyperparameter (default is no prior information on any
+    hyperparameters) and the method for handling the nugget parameter.
+    The nugget is additional "noise" that is added to the diagonal of
+    the covariance kernel as a variance. This nugget can represent
+    uncertainty in the target values themselves, or simply be used to
+    stabilize the numerical inversion of the covariance matrix. The
+    nugget can be fixed (a non-negative float), can be found
+    adaptively (by passing the string ``"adaptive"`` to make the noise
+    only as large as necessary to successfully invert the covariance
+    matrix), can be fit as a hyperparameter (by passing the string
+    ``"fit"``), or pivoting can be used to ignore any collinear matrix
+    rows and ensure a zero nugget is used (by passing the string
+    ``"pivot"``).
+
+    The internal emulator structure involves arrays for the inputs,
+    targets, and hyperparameters.  Other useful information are the
+    number of training examples ``n``, the number of input parameters
+    ``D``, and the number of hyperparameters ``n_params``. These
+    parameters can be obtained externally by accessing these
+    attributes.
 
     Example: ::
 
@@ -65,68 +79,92 @@ class GaussianProcess(object):
     """
     def __init__(self, inputs, targets, mean=None, kernel=SquaredExponential(), priors=None,
                  nugget="adaptive", inputdict = {}, use_patsy=True):
-        """
-        Create a new GaussianProcess Emulator
-
-        Creates a new GaussianProcess Emulator from either the input data and targets to be fit and
-        optionally a mean function, covariance kernel, and nugget parameter/method.
-
-        Required arguments are numpy arrays ``inputs`` and ``targets``, described below.
-        Additional arguments are ``mean`` to specify the mean function (default is
-        ``None`` for zero mean), ``kernel`` to specify the covariance kernel (default
-        is the squared exponential kernel), ``priors`` to indicate prior distributions
-        on the hyperparameters, and ``nugget`` to specify how the nugget
-        parameter is handled (see below; default is to fit the nugget adaptively).
-
-        ``inputs`` is a 2D array-like object holding the input data, whose shape is
-        ``n`` by ``D``, where ``n`` is the number of training examples to be fit and ``D``
-        is the number of input variables to each simulation. If ``inputs`` is a 1D array,
-        it is assumed that ``D = 1``.
-
-        ``targets`` is the target data to be fit by the emulator, also held in an array-like
-        object. This must be a 1D array of length ``n``.
-
-        ``prior`` must be a list of length ``n_params`` whose elements are either ``Prior``-derived
-        objects or ``None``.  Each element is used as the prior for the corresponding parameter (with
-        ``None`` indicating an uninformative prior).  Passing the empty list or ``None`` as this
-        argument (in its entirety) may be used as an abbreviation for a list of ``n_params`` where
-        all list elements are ``None``.
-
-        ``nugget`` controls how additional noise is added to the emulator targets when fitting.
-        This can be specified in several ways. If a string is provided, it can take the
-        values of ``"adaptive"`` or ``"fit"``, which indicate that the nugget will be
-        chosen in the fitting process. ``"adaptive"`` means that the nugget will be made only
-        as large as necessary to invert the covariance matrix, while ``"fit"`` means that
-        the nugget will be treated as a hyperparameter to be optimized. Alternatively,
-        a non-negative float can be used to specify a fixed noise level. If no value is
-        specified for the ``nugget`` parameter, ``"adaptive"`` is the default.
-
-
-        :param inputs: Numpy array holding emulator input parameters. Must be 1D with length
-                       ``n`` or 2D with shape ``n`` by ``D``, where ``n`` is the number of
-                       training examples and ``D`` is the number of input parameters for
-                       each output.
-        :type inputs: ndarray
-        :param targets: Numpy array holding emulator targets. Must be 1D with length ``n``
+        """Create a new GaussianProcess Emulator
+
+        Creates a new GaussianProcess Emulator from either the input
+        data and targets to be fit and optionally a mean function,
+        covariance kernel, and nugget parameter/method.
+
+        Required arguments are numpy arrays ``inputs`` and
+        ``targets``, described below.  Additional arguments are
+        ``mean`` to specify the mean function (default is ``None`` for
+        zero mean), ``kernel`` to specify the covariance kernel
+        (default is the squared exponential kernel), ``priors`` to
+        indicate prior distributions on the hyperparameters, and
+        ``nugget`` to specify how the nugget parameter is handled (see
+        below; default is to fit the nugget adaptively).
+
+        ``inputs`` is a 2D array-like object holding the input data,
+        whose shape is ``n`` by ``D``, where ``n`` is the number of
+        training examples to be fit and ``D`` is the number of input
+        variables to each simulation. If ``inputs`` is a 1D array, it
+        is assumed that ``D = 1``.
+
+        ``targets`` is the target data to be fit by the emulator, also
+        held in an array-like object. This must be a 1D array of
+        length ``n``.
+
+        ``prior`` must be a list of length ``n_params`` whose elements
+        are either ``Prior``-derived objects or ``None``.  Each
+        element is used as the prior for the corresponding parameter
+        (with ``None`` indicating an uninformative prior).  Passing
+        the empty list or ``None`` as this argument (in its entirety)
+        may be used as an abbreviation for a list of ``n_params``
+        where all list elements are ``None``.
+
+        ``nugget`` controls how additional noise is added to the
+        emulator targets when fitting.  This can be specified in
+        several ways. If a string is provided, it can take the values
+        of ``"adaptive"`` or ``"fit"``, which indicate that the nugget
+        will be chosen in the fitting process. The nugget can also be
+        chosen as part of the fitting process, with three options for
+        nugget handling: ``"adaptive"`` means that the nugget will be
+        made only as large as necessary to invert the covariance
+        matrix, ``"fit"`` means that the nugget will be treated as a
+        hyperparameter to be optimized, and ``"pivot"`` indicates that
+        pivoting will be used to ignore any collinear rows and ensure
+        use of a nugget of zero. Alternatively, a non-negative float
+        can be used to specify a fixed noise level. If no value is
+        specified for the ``nugget`` parameter, ``"adaptive"`` is the
+        default.
+
+
+        :param inputs: Numpy array holding emulator input
+                       parameters. Must be 1D with length ``n`` or 2D
+                       with shape ``n`` by ``D``, where ``n`` is the
+                       number of training examples and ``D`` is the
+                       number of input parameters for each output.
+                       :type inputs: ndarray
+        :param targets: Numpy array holding emulator targets. Must be
+                        1D with length ``n``
         :type targets: ndarray
-        :param mean: Mean function to be used (optional, default is ``None`` for a zero mean)
+        :param mean: Mean function to be used (optional, default is
+                     ``None`` for a zero mean)
         :type mean: None or MeanFunction
-        :param kernel: Covariance kernel to be used (optional, default is Squared Exponential)
-                       Can provide either a ``Kernel`` object or a string matching the
-                       kernel type to be used.
+        :param kernel: Covariance kernel to be used (optional, default
+                     is Squared Exponential) Can provide either a
+                     ``Kernel`` object or a string matching the kernel
+                     type to be used.
         :type kernel: Kernel or str
-        :param priors: List of priors to be used. Must be None (default) or an empty list
-                       (indicates uninformative priors) or list of length ``n_params``.
-                       Any parameter for which you wish to specify an uninformative prior,
-                       pass ``None``. Number of parameters is the number of parameters in
-                       the mean function plus ``D + 2`` (one correlation length per input
-                       plus a covariance scale and a nugget). If the nugget will not be fit,
-                       the list can have length ``n_params - 1``.
+        :param priors: List of priors to be used. Must be None
+                     (default) or an empty list (indicates
+                     uninformative priors) or list of length
+                     ``n_params``.  Any parameter for which you wish
+                     to specify an uninformative prior, pass
+                     ``None``. Number of parameters is the number of
+                     parameters in the mean function plus ``D + 2``
+                     (one correlation length per input plus a
+                     covariance scale and a nugget). If the nugget
+                     will not be fit, the list can have length
+                     ``n_params - 1``.
         :type priors: list or None
-        :param nugget: Noise to be added to the diagonal, specified as a string or a float.
-                       A non-negative float specifies the noise level explicitly, while a string
-                       indicates that the nugget will be found via fitting, either as ``"adaptive"``
-                       or ``"fit"`` (see above for a description). Default is ``"adaptive"``.
+        :param nugget: Noise to be added to the diagonal, specified as
+                     a string or a float.  A non-negative float
+                     specifies the noise level explicitly, while a
+                     string indicates that the nugget will be found
+                     via fitting, either as ``"adaptive"``, ``"fit"``,
+                     or ``"pivot"`` (see above for a
+                     description). Default is ``"adaptive"``.
         :type nugget: float or str
         :returns: New ``GaussianProcess`` instance
         :rtype: GaussianProcess
@@ -216,61 +254,73 @@ def D(self):
 
     @property
     def n_params(self):
-        """
-        Returns number of hyperparameters
+        """Returns number of hyperparameters
 
-        Returns the number of hyperparameters for the emulator. The number depends on the
-        choice of mean function, covariance function, and nugget strategy, and possibly the
-        number of inputs for certain choices of the mean function.
+        Returns the number of hyperparameters for the emulator. The
+        number depends on the choice of mean function, covariance
+        function, and nugget strategy, and possibly the number of
+        inputs for certain choices of the mean function.
 
         :returns: Number of hyperparameters
         :rtype: int
+
         """
 
         return self.mean.get_n_params(self.inputs) + self.D + 2
 
     @property
     def nugget_type(self):
-        """
-        Returns method used to select nugget parameter
+        """Returns method used to select nugget parameter
 
-        Returns a string indicating how the nugget parameter is treated, either ``"adaptive"``,
-        ``"fit"``, or ``"fixed"``. This is automatically set when changing the ``nugget``
-        property.
+        Returns a string indicating how the nugget parameter is
+        treated, either ``"adaptive"``, ``"pivot"``, ``"fit"``, or
+        ``"fixed"``. This is automatically set when changing the
+        ``nugget`` property.
 
         :returns: Current nugget fitting method
         :rtype: str
+
         """
         return self._nugget_type
 
     @property
     def nugget(self):
-        """
-        Returns emulator nugget parameter
+        """Returns emulator nugget parameter
 
-        Returns current value of the nugget parameter. If the nugget is to be selected
-        adaptively or by fitting the emulator and the nugget has not been fit,
-        returns ``None``.
+        Returns current value of the nugget parameter. If the nugget
+        is to be selected adaptively, by pivoting, or by fitting the
+        emulator and the nugget has not been fit, returns ``None``.
 
         :returns: Current nugget value, either a float or ``None``
         :rtype: float or None
 
-        The ``nugget`` parameter controls how noise is added to the covariance matrix in order to
-        stabilize the inversion or smooth the emulator predictions. If ``nugget`` is a non-negative
-        float, then that particular value is used for the nugget. Note that setting this parameter
-        to be zero enforces that the emulator strictly interpolates between points. Alternatively,
-        a string can be provided. A value of ``"fit"`` means that the nugget is treated as a
-        hyperparameter, and is the last entry in the ``theta`` array. Alternatively, if ``nugget``
-        is set to be ``"adaptive"``, the fitting routine will adaptively make the noise parameter
-        as large as is needed to ensure that the emulator can be fit.
-
-        Internally, this modifies both the way the nugget is chosen (which can be determined
-        via the ``nugget_type`` property) and the value itself (the ``nugget`` property)
-
-        :param nugget: Noise to be added to the diagonal, specified as a string or a float.
-                       A non-negative float specifies the noise level explicitly, while a string
-                       indicates that the nugget will be found via fitting, either as ``"adaptive"``
-                       or ``"fit"`` (see above for a description).
+        The ``nugget`` parameter controls how noise is added to the
+        covariance matrix in order to stabilize the inversion or
+        smooth the emulator predictions. If ``nugget`` is a
+        non-negative float, then that particular value is used for the
+        nugget. Note that setting this parameter to be zero enforces
+        that the emulator strictly interpolates between
+        points. Alternatively, a string can be provided. A value of
+        ``"fit"`` means that the nugget is treated as a
+        hyperparameter, and is the last entry in the ``theta``
+        array. Alternatively, if ``nugget`` is set to be
+        ``"adaptive"``, the fitting routine will adaptively make the
+        noise parameter as large as is needed to ensure that the
+        emulator can be fit. Finally, pivoting can be selected by
+        setting to ``"pivot"``, which will ignore any collinear rows
+        in the covariance matrix.
+
+        Internally, this modifies both the way the nugget is chosen
+        (which can be determined via the ``nugget_type`` property) and
+        the value itself (the ``nugget`` property)
+
+        :param nugget: Noise to be added to the diagonal, specified as
+                       a string or a float.  A non-negative float
+                       specifies the noise level explicitly, while a
+                       string indicates that the nugget will be found
+                       via fitting, either as ``"adaptive"``,
+                       ``"pivot"``, or ``"fit"`` (see above for a
+                       description).
         :type nugget: float or str
         :returns: None
         :rtype: None
@@ -280,31 +330,43 @@ def nugget(self):
 
     @nugget.setter
     def nugget(self, nugget):
-        """
-        Set the nugget parameter for the emulator
-
-        Method for changing the ``nugget`` parameter for the emulator. When a new emulator is
-        initilized, this is set to None.
-
-        The ``nugget`` parameter controls how noise is added to the covariance matrix in order to
-        stabilize the inversion or smooth the emulator predictions. If ``nugget`` is a non-negative
-        float, then that particular value is used for the nugget. Note that setting this parameter
-        to be zero enforces that the emulator strictly interpolates between points. Alternatively,
-        a string can be provided. A value of ``"fit"`` means that the nugget is treated as a
-        hyperparameter, and is the last entry in the ``theta`` array. Alternatively, if ``nugget``
-        is set to be ``"adaptive"``, the fitting routine will adaptively make the noise parameter
-        as large as is needed to ensure that the emulator can be fit.
-
-        Internally, this modifies both the way the nugget is chosen (which can be determined
-        via the ``nugget_type`` property) and the value itself (the ``nugget`` property)
-
-        :param nugget: Noise to be added to the diagonal, specified as a string or a float.
-                       A non-negative float specifies the noise level explicitly, while a string
-                       indicates that the nugget will be found via fitting, either as ``"adaptive"``
-                       or ``"fit"`` (see above for a description).
+        """Set the nugget parameter for the emulator
+
+        Method for changing the ``nugget`` parameter for the
+        emulator. When a new emulator is initilized, this is set to
+        None.
+
+        The ``nugget`` parameter controls how noise is added to the
+        covariance matrix in order to stabilize the inversion or
+        smooth the emulator predictions. If ``nugget`` is a
+        non-negative float, then that particular value is used for the
+        nugget. Note that setting this parameter to be zero enforces
+        that the emulator strictly interpolates between
+        points. Alternatively, a string can be provided. A value of
+        ``"fit"`` means that the nugget is treated as a
+        hyperparameter, and is the last entry in the ``theta``
+        array. Alternatively, if ``nugget`` is set to be
+        ``"adaptive"``, the fitting routine will adaptively make the
+        noise parameter as large as is needed to ensure that the
+        emulator can be fit. Finally, pivoting can be selected by
+        setting to ``"pivot"``, which will ignore any collinear rows
+        in the covariance matrix.
+
+        Internally, this modifies both the way the nugget is chosen
+        (which can be determined via the ``nugget_type`` property) and
+        the value itself (the ``nugget`` property)
+
+        :param nugget: Noise to be added to the diagonal, specified as
+                       a string or a float.  A non-negative float
+                       specifies the noise level explicitly, while a
+                       string indicates that the nugget will be found
+                       via fitting, either as ``"adaptive"``,
+                       ``"pivot"``, or ``"fit"`` (see above for a
+                       description).
         :type nugget: float or str
         :returns: None
         :rtype: None
+
         """
 
         if not isinstance(nugget, (str, float)):
@@ -318,8 +380,10 @@ def nugget(self, nugget):
                 self._nugget_type = "adaptive"
             elif nugget == "fit":
                 self._nugget_type = "fit"
+            elif nugget == "pivot":
+                self._nugget_type = "pivot"
             else:
-                raise ValueError("bad value of nugget, must be a float or 'adaptive' or 'fit'")
+                raise ValueError("bad value of nugget, must be a float or 'adaptive', 'pivot', or 'fit'")
             self._nugget = None
         else:
             if nugget < 0.:
@@ -329,31 +393,37 @@ def nugget(self, nugget):
 
     @property
     def theta(self):
-        """
-        Returns emulator hyperparameters
+        """Returns emulator hyperparameters
 
-        Returns current hyperparameters for the emulator as a numpy array if they have been fit.
-        If no parameters have been fit, returns ``None``. Note that the number of parameters
-        depends on the mean function, so the length of this array will vary across instances.
+        Returns current hyperparameters for the emulator as a numpy
+        array if they have been fit.  If no parameters have been fit,
+        returns ``None``. Note that the number of parameters depends
+        on the mean function, so the length of this array will vary
+        across instances.
 
-        :returns: Current parameter values (numpy array of length ``n_params``), or ``None`` if the
-                  parameters have not been fit.
+        :returns: Current parameter values (numpy array of length
+                  ``n_params``), or ``None`` if the parameters have
+                  not been fit.
         :rtype: ndarray or None
 
-        When set, pre-calculates the matrices needed to compute the log-likelihood and its derivatives
-        and make subsequent predictions. This is called any time the hyperparameter values are
-        changed in order to ensure that all the information is needed to evaluate the
-        log-likelihood and its derivatives, which are needed when fitting the optimal
-        hyperparameters.
-
-        The method computes the mean function and covariance matrix and inverts the covariance
-        matrix using the method specified by the value of ``nugget``. The factorized matrix
-        and the product of the inverse with the difference between the targets and the mean
-        are cached for later use, and the negative marginal log-likelihood is also cached.
-        This method has no return value, but it does modify the state of the object.
-
-        :param theta: Values of the hyperparameters to use in fitting. Must be a numpy
-                      array with length ``n_params``
+        When set, pre-calculates the matrices needed to compute the
+        log-likelihood and its derivatives and make subsequent
+        predictions. This is called any time the hyperparameter values
+        are changed in order to ensure that all the information is
+        needed to evaluate the log-likelihood and its derivatives,
+        which are needed when fitting the optimal hyperparameters.
+
+        The method computes the mean function and covariance matrix
+        and inverts the covariance matrix using the method specified
+        by the value of ``nugget``. The factorized matrix and the
+        product of the inverse with the difference between the targets
+        and the mean are cached for later use, and the negative
+        marginal log-likelihood is also cached.  This method has no
+        return value, but it does modify the state of the object.
+
+        :param theta: Values of the hyperparameters to use in
+                      fitting. Must be a numpy array with length
+                      ``n_params``
         :type theta: ndarray
         """
 
@@ -361,56 +431,69 @@ def theta(self):
 
     @theta.setter
     def theta(self, theta):
-        """
-        Fits the emulator and sets the parameters (property-based setter alias for ``fit``)
-
-        Pre-calculates the matrices needed to compute the log-likelihood and its derivatives
-        and make subsequent predictions. This is called any time the hyperparameter values are
-        changed in order to ensure that all the information is needed to evaluate the
-        log-likelihood and its derivatives, which are needed when fitting the optimal
-        hyperparameters.
-
-        The method computes the mean function and covariance matrix and inverts the covariance
-        matrix using the method specified by the value of ``nugget``. The factorized matrix
-        and the product of the inverse with the difference between the targets and the mean
-        are cached for later use, and the negative marginal log-likelihood is also cached.
-        This method has no return value, but it does modify the state of the object.
-
-        :param theta: Values of the hyperparameters to use in fitting. Must be a numpy
-                      array with length ``n_params``
+        """Fits the emulator and sets the parameters (property-based setter alias for ``fit``)
+
+        Pre-calculates the matrices needed to compute the
+        log-likelihood and its derivatives and make subsequent
+        predictions. This is called any time the hyperparameter values
+        are changed in order to ensure that all the information is
+        needed to evaluate the log-likelihood and its derivatives,
+        which are needed when fitting the optimal hyperparameters.
+
+        The method computes the mean function and covariance matrix
+        and inverts the covariance matrix using the method specified
+        by the value of ``nugget``. The factorized matrix and the
+        product of the inverse with the difference between the targets
+        and the mean are cached for later use, and the negative
+        marginal log-likelihood is also cached.  This method has no
+        return value, but it does modify the state of the object.
+
+        :param theta: Values of the hyperparameters to use in
+                      fitting. Must be a numpy array with length
+                      ``n_params``
         :type theta: ndarray
         :returns: None
         """
-        self.fit(theta)
+        if theta is None:
+            self._theta = None
+            self.current_logpost = None
+        else:
+            self.fit(theta)
 
     @property
     def priors(self):
-        """
-        The current list priors used in computing the log posterior
-
-        To set the priors, must be a list or ``None``. Entries can be ``None`` or a subclass of ``Prior``.
-        ``None`` indicates weak prior information. An empty list or ``None`` means all uninformative
-        priors. Otherwise list should have the same length as the number of hyperparameters,
-        or alternatively can be one shorter than the number of hyperparameters
-        if ``nugget_type`` is ``"adaptive"`` or ``"fixed"`` meaning that the nugget hyperparameter
-        is not fit but is instead fixed or found adaptively. If the nugget hyperparameter is not fit,
-        the prior for the nugget will automatically be set to ``None`` even if a distribution is
+        """The current list priors used in computing the log posterior
+
+        To set the priors, must be a list or ``None``. Entries can be
+        ``None`` or a subclass of ``Prior``.  ``None`` indicates weak
+        prior information. An empty list or ``None`` means all
+        uninformative priors. Otherwise list should have the same
+        length as the number of hyperparameters, or alternatively can
+        be one shorter than the number of hyperparameters if
+        ``nugget_type`` is ``"adaptive"``, ``"pivot"`` or ``"fixed"``
+        meaning that the nugget hyperparameter is not fit but is
+        instead fixed or found adaptively. If the nugget
+        hyperparameter is not fit, the prior for the nugget will
+        automatically be set to ``None`` even if a distribution is
         provided.
         """
         return self._priors
 
     @priors.setter
     def priors(self, priors):
-        """
-        Sets the priors to a list of prior objects/None
-
-        Sets the priors, must be a list or ``None``. Entries can be ``None`` or a subclass of ``Prior``.
-        ``None`` indicates weak prior information. An empty list or ``None`` means all uninformative
-        priors. Otherwise list should have the same length as the number of hyperparameters,
-        or alternatively can be one shorter than the number of hyperparameters
-        if ``nugget_type`` is ``"adaptive"`` or ``"fixed"`` meaning that the nugget hyperparameter
-        is not fit but is instead fixed or found adaptively. If the nugget hyperparameter is not fit,
-        the prior for the nugget will automatically be set to ``None`` even if a distribution is
+        """Sets the priors to a list of prior objects/None
+
+        Sets the priors, must be a list or ``None``. Entries can be
+        ``None`` or a subclass of ``Prior``.  ``None`` indicates weak
+        prior information. An empty list or ``None`` means all
+        uninformative priors. Otherwise list should have the same
+        length as the number of hyperparameters, or alternatively can
+        be one shorter than the number of hyperparameters if
+        ``nugget_type`` is ``"adaptive"``,``"pivot"``, or ``"fixed"``
+        meaning that the nugget hyperparameter is not fit but is
+        instead fixed or found adaptively. If the nugget
+        hyperparameter is not fit, the prior for the nugget will
+        automatically be set to ``None`` even if a distribution is
         provided.
         """
 
@@ -423,14 +506,14 @@ def priors(self, priors):
         if len(priors) == 0:
             priors = self.n_params*[None]
 
-        if self.nugget_type in ["adaptive", "fixed"]:
+        if self.nugget_type in ["adaptive", "fixed", "pivot"]:
             if len(priors) == self.n_params - 1:
                 priors.append(None)
 
         if not len(priors) == self.n_params:
             raise ValueError("bad length for priors; must have length n_params")
 
-        if self.nugget_type in ["adaptive", "fixed"]:
+        if self.nugget_type in ["adaptive", "fixed", "pivot"]:
             if not priors[-1] is None:
                 priors[-1] = None
 
@@ -442,32 +525,36 @@ def priors(self, priors):
 
 
     def get_K_matrix(self):
-        """
-        Returns current value of the covariance matrix as a numpy array. Does not include the nugget
-        parameter, as this is dependent on how the nugget is fit.
+        """Returns current value of the covariance matrix as a numpy
+        array. Does not include the nugget parameter, as this is
+        dependent on how the nugget is fit.
+
         """
         switch = self.mean.get_n_params(self.inputs)
 
         return self.kernel.kernel_f(self.inputs, self.inputs, self.theta[switch:-1])
 
     def fit(self, theta):
-        """
-        Fits the emulator and sets the parameters
-
-        Pre-calculates the matrices needed to compute the log-likelihood and its derivatives
-        and make subsequent predictions. This is called any time the hyperparameter values are
-        changed in order to ensure that all the information is needed to evaluate the
-        log-likelihood and its derivatives, which are needed when fitting the optimal
-        hyperparameters.
-
-        The method computes the mean function and covariance matrix and inverts the covariance
-        matrix using the method specified by the value of ``nugget_type``. The factorized matrix
-        and the product of the inverse with the difference between the targets and the mean
-        are cached for later use, and the negative marginal log-likelihood is also cached.
-        This method has no return value, but it does modify the state of the object.
-
-        :param theta: Values of the hyperparameters to use in fitting. Must be a numpy
-                      array with length ``n_params``
+        """Fits the emulator and sets the parameters
+
+        Pre-calculates the matrices needed to compute the
+        log-likelihood and its derivatives and make subsequent
+        predictions. This is called any time the hyperparameter values
+        are changed in order to ensure that all the information is
+        needed to evaluate the log-likelihood and its derivatives,
+        which are needed when fitting the optimal hyperparameters.
+
+        The method computes the mean function and covariance matrix
+        and inverts the covariance matrix using the method specified
+        by the value of ``nugget_type``. The factorized matrix and the
+        product of the inverse with the difference between the targets
+        and the mean are cached for later use, and the negative
+        marginal log-likelihood is also cached.  This method has no
+        return value, but it does modify the state of the object.
+
+        :param theta: Values of the hyperparameters to use in
+                      fitting. Must be a numpy array with length
+                      ``n_params``
         :type theta: ndarray
         :returns: None
         """
@@ -485,13 +572,18 @@ def fit(self, theta):
 
         if self.nugget_type == "adaptive":
             self.L, self._nugget = jit_cholesky(Q)
+            self.P = np.arange(0, self.n)
+        elif self.nugget_type == "pivot":
+            self.L, self.P = pivot_cholesky(Q)
+            self._nugget = 0.
         else:
             if self.nugget_type == "fit":
                 self._nugget = np.exp(self.theta[-1])
             Q += self._nugget*np.eye(self.n)
             self.L = linalg.cholesky(Q, lower=True)
+            self.P = np.arange(0, self.n)
 
-        self.invQt = linalg.cho_solve((self.L, True), self.targets - m)
+        self.invQt = pivot_cho_solve(self.L, self.P, self.targets - m)
 
         self.current_logpost = 0.5*(2.0*np.sum(np.log(np.diag(self.L))) +
                                     np.dot(self.targets - m, self.invQt) +
@@ -503,19 +595,21 @@ def fit(self, theta):
 
 
     def logposterior(self, theta):
-        """
-        Calculate the negative log-posterior at a particular value of the hyperparameters
+        """Calculate the negative log-posterior at a particular value of the hyperparameters
 
-        Calculate the negative log-posterior for the given set of parameters. Calling this
-        method sets the parameter values and computes the needed inverse matrices in order
-        to evaluate the log-posterior and its derivatives. In addition to returning the
-        log-posterior value, it stores the current value of the hyperparameters and
-        log-posterior in attributes of the object.
+        Calculate the negative log-posterior for the given set of
+        parameters. Calling this method sets the parameter values and
+        computes the needed inverse matrices in order to evaluate the
+        log-posterior and its derivatives. In addition to returning
+        the log-posterior value, it stores the current value of the
+        hyperparameters and log-posterior in attributes of the object.
 
-        :param theta: Value of the hyperparameters. Must be array-like with shape ``(n_params,)``
+        :param theta: Value of the hyperparameters. Must be array-like
+                      with shape ``(n_params,)``
         :type theta: ndarray
         :returns: negative log-posterior
         :rtype: float
+
         """
 
         if self.theta is None or not np.allclose(theta, self.theta, rtol=1.e-10, atol=1.e-15):
@@ -524,26 +618,29 @@ def logposterior(self, theta):
         return self.current_logpost
 
     def logpost_deriv(self, theta):
-        """
-        Calculate the partial derivatives of the negative log-posterior
-
-        Calculate the partial derivatives of the negative log-posterior with respect to
-        the hyperparameters. Note that this function is normally used only when fitting
-        the hyperparameters, and it is not needed to make predictions.
-
-        During normal use, the ``loglike_deriv`` method is called after evaluating the
-        ``logposterior`` method. The implementation takes advantage of this by reusing
-        cached results, as the factorized covariance matrix is expensive to compute and is
-        used by the ``logposterior``, ``logpost_deriv``, and ``logpost_hessian`` methods.
-        If the function is evaluated with a different set of parameters than was previously
-        used to set the log-posterior, the method calls ``fit`` (and subsequently resets
-        the cached information).
-
-        :param theta: Value of the hyperparameters. Must be array-like with shape
-                      ``(n_params,)``
+        """Calculate the partial derivatives of the negative log-posterior
+
+        Calculate the partial derivatives of the negative
+        log-posterior with respect to the hyperparameters. Note that
+        this function is normally used only when fitting the
+        hyperparameters, and it is not needed to make predictions.
+
+        During normal use, the ``logpost_deriv`` method is called
+        after evaluating the ``logposterior`` method. The
+        implementation takes advantage of this by reusing cached
+        results, as the factorized covariance matrix is expensive to
+        compute and is used by the ``logposterior``,
+        ``logpost_deriv``, and ``logpost_hessian`` methods.  If the
+        function is evaluated with a different set of parameters than
+        was previously used to set the log-posterior, the method calls
+        ``fit`` (and subsequently resets the cached information).
+
+        :param theta: Value of the hyperparameters. Must be array-like
+                      with shape ``(n_params,)``
         :type theta: ndarray
-        :returns: partial derivatives of the negative log-posterior with respect to the
-                  hyperparameters (array with shape ``(n_params,)``)
+        :returns: partial derivatives of the negative log-posterior
+                  with respect to the hyperparameters (array with
+                  shape ``(n_params,)``)
         :rtype: ndarray
         """
 
@@ -564,13 +661,13 @@ def logpost_deriv(self, theta):
         partials[:switch] = -np.dot(dmdtheta, self.invQt)
 
         for d in range(self.D + 1):
-            invQ_dot_dKdtheta_trace = np.trace(linalg.cho_solve((self.L, True), dKdtheta[d]))
+            invQ_dot_dKdtheta_trace = np.trace(pivot_cho_solve(self.L, self.P, dKdtheta[d]))
             partials[switch + d] = -0.5*(np.dot(self.invQt, np.dot(dKdtheta[d], self.invQt)) -
                                          invQ_dot_dKdtheta_trace)
 
         if self.nugget_type == "fit":
             nugget = np.exp(self.theta[-1])
-            partials[-1] = 0.5*nugget*(np.trace(linalg.cho_solve((self.L, True), np.eye(self.n))) -
+            partials[-1] = 0.5*nugget*(np.trace(pivot_cho_solve(self.L, self.P, np.eye(self.n))) -
                                        np.dot(self.invQt, self.invQt))
 
         for i in range(self.n_params):
@@ -580,29 +677,33 @@ def logpost_deriv(self, theta):
         return partials
 
     def logpost_hessian(self, theta):
-        """
-        Calculate the Hessian of the negative log-posterior
-
-        Calculate the Hessian of the negative log-posterior with respect to
-        the hyperparameters. Note that this function is normally used only when fitting
-        the hyperparameters, and it is not needed to make predictions. It is also used
-        to estimate an appropriate step size when fitting hyperparameters using
-        the lognormal approximation or MCMC sampling.
-
-        When used in an optimization routine, the ``logpost_hessian`` method is called after
-        evaluating the ``logposterior`` method. The implementation takes advantage of
-        this by storing the inverse of the covariance matrix, which is expensive to
-        compute and is used by the ``logposterior`` and ``logpost_deriv`` methods as well.
-        If the function is evaluated with a different set of parameters than was previously
-        used to set the log-posterior, the method calls ``fit`` to compute the needed
-        information and changes the cached values.
-
-        :param theta: Value of the hyperparameters. Must be array-like with shape
-                      ``(n_params,)``
+        """Calculate the Hessian of the negative log-posterior
+
+        Calculate the Hessian of the negative log-posterior with
+        respect to the hyperparameters. Note that this function is
+        normally used only when fitting the hyperparameters, and it is
+        not needed to make predictions. It is also used to estimate an
+        appropriate step size when fitting hyperparameters using the
+        lognormal approximation or MCMC sampling.
+
+        When used in an optimization routine, the ``logpost_hessian``
+        method is called after evaluating the ``logposterior``
+        method. The implementation takes advantage of this by storing
+        the inverse of the covariance matrix, which is expensive to
+        compute and is used by the ``logposterior`` and
+        ``logpost_deriv`` methods as well.  If the function is
+        evaluated with a different set of parameters than was
+        previously used to set the log-posterior, the method calls
+        ``fit`` to compute the needed information and changes the
+        cached values.
+
+        :param theta: Value of the hyperparameters. Must be array-like
+                      with shape ``(n_params,)``
         :type theta: ndarray
-        :returns: Hessian of the negative log-posterior (array with shape
-                  ``(n_params, n_params)``)
+        :returns: Hessian of the negative log-posterior (array with
+                  shape ``(n_params, n_params)``)
         :rtype: ndarray
+
         """
 
         assert theta.shape == (self.n_params,), "Parameter vector must have length number of inputs + 1"
@@ -620,20 +721,20 @@ def logpost_hessian(self, theta):
         d2Kdtheta2 = self.kernel.kernel_hessian(self.inputs, self.inputs, self.theta[switch:-1])
 
         hessian[:switch, :switch] = -(np.dot(d2mdtheta2, self.invQt) -
-                                      np.dot(dmdtheta, linalg.cho_solve((self.L, True),
+                                      np.dot(dmdtheta, pivot_cho_solve(self.L, self.P,
                                                                         np.transpose(dmdtheta))))
 
         hessian[:switch, switch:-1] = np.dot(dmdtheta,
-                                             linalg.cho_solve((self.L, True),
+                                             pivot_cho_solve(self.L, self.P,
                                                               np.transpose(np.dot(dKdtheta, self.invQt))))
 
         hessian[switch:-1, :switch] = np.transpose(hessian[:switch, switch:-1])
 
         for d1 in range(self.D + 1):
-            invQ_dot_d1 = linalg.cho_solve((self.L, True), dKdtheta[d1])
+            invQ_dot_d1 = pivot_cho_solve(self.L, self.P, dKdtheta[d1])
             for d2 in range(self.D + 1):
-                invQ_dot_d2 = linalg.cho_solve((self.L, True), dKdtheta[d2])
-                invQ_dot_d1d2 = linalg.cho_solve((self.L, True), d2Kdtheta2[d1, d2])
+                invQ_dot_d2 = pivot_cho_solve(self.L, self.P, dKdtheta[d2])
+                invQ_dot_d1d2 = pivot_cho_solve(self.L, self.P, d2Kdtheta2[d1, d2])
                 term_1 = np.linalg.multi_dot([self.invQt,
                                               2.*np.dot(dKdtheta[d1], invQ_dot_d2) - d2Kdtheta2[d1, d2],
                                               self.invQt])
@@ -642,20 +743,20 @@ def logpost_hessian(self, theta):
 
         if self.nugget_type == "fit":
             nugget = np.exp(self.theta[-1])
-            invQinvQt = linalg.cho_solve((self.L, True), self.invQt)
+            invQinvQt = pivot_cho_solve(self.L, self.P, self.invQt)
             hessian[:switch, -1] = nugget*np.dot(dmdtheta, invQinvQt)
             for d in range(self.D + 1):
                 hessian[switch + d, -1] = nugget*(np.linalg.multi_dot([self.invQt, dKdtheta[d], invQinvQt]) -
-                                                  0.5*np.trace(linalg.cho_solve((self.L, True),
+                                                  0.5*np.trace(pivot_cho_solve(self.L, self.P,
                                                                                 np.dot(dKdtheta[d],
-                                                                                       linalg.cho_solve((self.L, True),
+                                                                                       pivot_cho_solve(self.L, self.P,
                                                                                                         np.eye(self.n))))))
 
-            hessian[-1, -1] = 0.5*nugget*(np.trace(linalg.cho_solve((self.L, True), np.eye(self.n))) -
+            hessian[-1, -1] = 0.5*nugget*(np.trace(pivot_cho_solve(self.L, self.P, np.eye(self.n))) -
                                                    np.dot(self.invQt, self.invQt))
             hessian[-1, -1] += nugget**2*(np.dot(self.invQt, invQinvQt) -
-                                          0.5*np.trace(linalg.cho_solve((self.L, True),
-                                                                        linalg.cho_solve((self.L, True),
+                                          0.5*np.trace(pivot_cho_solve(self.L, self.P,
+                                                                       pivot_cho_solve(self.L, self.P,
                                                                                          np.eye(self.n)))))
 
             hessian[-1, :-1] = np.transpose(hessian[:-1, -1])
@@ -667,52 +768,69 @@ def logpost_hessian(self, theta):
         return hessian
 
     def predict(self, testing, unc=True, deriv=True, include_nugget=True):
-        """
-        Make a prediction for a set of input vectors for a single set of hyperparameters
-
-        Makes predictions for the emulator on a given set of input vectors. The input vectors
-        must be passed as a ``(n_predict, D)``, ``(n_predict,)`` or ``(D,)`` shaped array-like
-        object, where ``n_predict`` is the number of different prediction points under
-        consideration and ``D`` is the number of inputs to the emulator. If the prediction
-        inputs array is 1D and ``D == 1`` for the GP instance, then the 1D array must have
-        shape ``(n_predict,)``. Otherwise, if the array is 1D it must have shape
-        ``(D,)``, and the method assumes ``n_predict == 1``. The prediction is returned as an
-        ``(n_predict, )`` shaped numpy array as the first return value from the method.
-
-        Optionally, the emulator can also calculate the variances in the predictions
-        and the derivatives with respect to each input parameter. If the uncertainties are
-        computed, they are returned as the second output from the method as an ``(n_predict,)``
-        shaped numpy array. If the derivatives are computed, they are returned as the third
-        output from the method as an ``(n_predict, D)`` shaped numpy array.
-
-        The final input to the method determines if the predictive variance should include
-        the nugget or not. For situations where the nugget represents observational error
-        and predictions are estimating the true underlying function, this should be set to
-        ``False``. However, most other cases should probably include the nugget, as the
-        emulator is using it to represent some of the uncertainty in the underlying simulator,
+        """Make a prediction for a set of input vectors for a single set of hyperparameters
+
+        Makes predictions for the emulator on a given set of input
+        vectors. The input vectors must be passed as a ``(n_predict,
+        D)``, ``(n_predict,)`` or ``(D,)`` shaped array-like object,
+        where ``n_predict`` is the number of different prediction
+        points under consideration and ``D`` is the number of inputs
+        to the emulator. If the prediction inputs array is 1D and ``D
+        == 1`` for the GP instance, then the 1D array must have shape
+        ``(n_predict,)``. Otherwise, if the array is 1D it must have
+        shape ``(D,)``, and the method assumes ``n_predict == 1``. The
+        prediction is returned as an ``(n_predict, )`` shaped numpy
+        array as the first return value from the method.
+
+        Optionally, the emulator can also calculate the variances in
+        the predictions and the derivatives with respect to each input
+        parameter. If the uncertainties are computed, they are
+        returned as the second output from the method as an
+        ``(n_predict,)`` shaped numpy array. If the derivatives are
+        computed, they are returned as the third output from the
+        method as an ``(n_predict, D)`` shaped numpy array.
+
+        The final input to the method determines if the predictive
+        variance should include the nugget or not. For situations
+        where the nugget represents observational error and
+        predictions are estimating the true underlying function, this
+        should be set to ``False``. However, most other cases should
+        probably include the nugget, as the emulator is using it to
+        represent some of the uncertainty in the underlying simulator,
         so the default value is ``True``.
 
-        :param testing: Array-like object holding the points where predictions will be made.
-                        Must have shape ``(n_predict, D)`` or ``(D,)`` (for a single prediction)
+        :param testing: Array-like object holding the points where
+                        predictions will be made.  Must have shape
+                        ``(n_predict, D)`` or ``(D,)`` (for a single
+                        prediction)
         :type testing: ndarray
-        :param unc: (optional) Flag indicating if the uncertainties are to be computed.
-                    If ``False`` the method returns ``None`` in place of the uncertainty
+
+        :param unc: (optional) Flag indicating if the uncertainties
+                    are to be computed.  If ``False`` the method
+                    returns ``None`` in place of the uncertainty
                     array. Default value is ``True``.
         :type unc: bool
-        :param deriv: (optional) Flag indicating if the derivatives are to be computed.
-                      If ``False`` the method returns ``None`` in place of the derivative
+        :param deriv: (optional) Flag indicating if the derivatives
+                      are to be computed.  If ``False`` the method
+                      returns ``None`` in place of the derivative
                       array. Default value is ``True``.
         :type deriv: bool
-        :param include_nugget: (optional) Flag indicating if the nugget should be included
-                               in the predictive variance. Only relevant if ``unc = True``.
-                               Default is ``True``.
+
+        :param include_nugget: (optional) Flag indicating if the
+                               nugget should be included in the
+                               predictive variance. Only relevant if
+                               ``unc = True``.  Default is ``True``.
         :type include_nugget: bool
-        :returns: Tuple of numpy arrays holding the predictions, uncertainties, and derivatives,
-                  respectively. Predictions and uncertainties have shape ``(n_predict,)``
-                  while the derivatives have shape ``(n_predict, D)``. If the ``unc`` or
-                  ``deriv`` flags are set to ``False``, then those arrays are replaced by
-                  ``None``.
+
+        :returns: Tuple of numpy arrays holding the predictions,
+                  uncertainties, and derivatives,
+                  respectively. Predictions and uncertainties have
+                  shape ``(n_predict,)`` while the derivatives have
+                  shape ``(n_predict, D)``. If the ``unc`` or
+                  ``deriv`` flags are set to ``False``, then those
+                  arrays are replaced by ``None``.
         :rtype: tuple
+
         """
 
         if self.theta is None:
@@ -723,10 +841,10 @@ def predict(self, testing, unc=True, deriv=True, include_nugget=True):
             testing = np.reshape(testing, (-1, 1))
         elif testing.ndim == 1:
             testing = np.reshape(testing, (1, len(testing)))
-        assert testing.ndim == 2
+        assert testing.ndim == 2, "testing must be a 2D array"
 
         n_testing, D = np.shape(testing)
-        assert D == self.D
+        assert D == self.D, "second dimension of testing must be the same as the number of input parameters"
 
         switch = self.mean.get_n_params(testing)
         mtest = self.mean.mean_f(testing, self.theta[:switch])
@@ -741,7 +859,7 @@ def predict(self, testing, unc=True, deriv=True, include_nugget=True):
             if include_nugget:
                 sigma_2 += self._nugget
 
-            var = np.maximum(sigma_2 - np.sum(Ktest*linalg.cho_solve((self.L, True), Ktest), axis=0),
+            var = np.maximum(sigma_2 - np.sum(Ktest*pivot_cho_solve(self.L, self.P, Ktest), axis=0),
                              0.)
 
         inputderiv = None
@@ -758,42 +876,51 @@ def __call__(self, testing):
         calling `predict` without uncertainty and derivative
         predictions, and extracting the zeroth component for the
         'mean' prediction.
+
         """
         return (self.predict(testing, unc=False, deriv=False)[0])
 
     def __str__(self):
-        """
-        Returns a string representation of the model
+        """Returns a string representation of the model
 
-        :returns: A string representation of the model (indicates number of training examples
-                  and inputs)
+        :returns: A string representation of the model (indicates
+                  number of training examples and inputs)
         :rtype: str
+
         """
 
         return ("Gaussian Process with " + str(self.n) + " training examples and " +
                 str(self.D) + " input variables")
 
 class PredictResult(dict):
-    """
-    Prediction results object
-
-    Dictionary-like object containing mean, uncertainty (variance), and derivatives with
-    respect to the inputs of an emulator prediction. Values can be accessed like a dictionary
-    with keys ``'mean'``, ``'unc'``, and ``'deriv'`` (or indices 0, 1, and 2 for the mean,
-    uncertainty, and derivative for backwards compatability), or using attributes
-    (``p.mean`` if ``p`` is an instance of ``PredictResult``). Also supports iteration and
-    unpacking with the ordering ``(mean, unc, deriv)`` to be consistent with indexing behavior.
-
-    Code is mostly based on scipy's ``OptimizeResult`` class, with some additional code to
-    support iteration and integer indexing.
-
-    :ivar mean: Predicted mean for each input point. Numpy array with shape ``(n_predict,)``
+    """Prediction results object
+
+    Dictionary-like object containing mean, uncertainty (variance),
+    and derivatives with respect to the inputs of an emulator
+    prediction. Values can be accessed like a dictionary with keys
+    ``'mean'``, ``'unc'``, and ``'deriv'`` (or indices 0, 1, and 2 for
+    the mean, uncertainty, and derivative for backwards
+    compatability), or using attributes (``p.mean`` if ``p`` is an
+    instance of ``PredictResult``). Also supports iteration and
+    unpacking with the ordering ``(mean, unc, deriv)`` to be
+    consistent with indexing behavior.
+
+    Code is mostly based on scipy's ``OptimizeResult`` class, with
+    some additional code to support iteration and integer indexing.
+
+    :ivar mean: Predicted mean for each input point. Numpy array with
+                shape ``(n_predict,)``
     :type mean: ndarray
-    :ivar unc: Predicted variance for each input point. Numpy array with shape ``(n_predict,)``
+
+    :ivar unc: Predicted variance for each input point. Numpy array
+               with shape ``(n_predict,)``
     :type mean: ndarray
-    :ivar deriv: Predicted derivative with respect to the inputs for each input point.
-                 Numpy array with shape ``(n_predict, D)``
+
+    :ivar deriv: Predicted derivative with respect to the inputs for
+                 each input point.  Numpy array with shape
+                 ``(n_predict, D)``
     :type deriv: ndarray
+
     """
 
     def __getattr__(self, name):
@@ -843,4 +970,4 @@ def __repr__(self):
             return '\n'.join([k.rjust(m) + ': ' + repr(v)
                               for k, v in zip(["mean", "unc", "deriv"], self)])
         else:
-            return self.__class__.__name__ + "()"
\ No newline at end of file
+            return self.__class__.__name__ + "()"
diff --git a/mogp_emulator/GaussianProcessGPU.py b/mogp_emulator/GaussianProcessGPU.py
new file mode 100644
index 00000000..51ba5223
--- /dev/null
+++ b/mogp_emulator/GaussianProcessGPU.py
@@ -0,0 +1,509 @@
+"""
+extends GaussianProcess with an (optional) GPU implementation
+"""
+
+import os
+import re
+import numpy as np
+
+from mogp_emulator.Kernel import SquaredExponential, Matern52
+from mogp_emulator.MeanFunction import MeanFunction, MeanBase
+
+import mogp_emulator.LibGPGPU as LibGPGPU
+
+from mogp_emulator.GaussianProcess import GaussianProcessBase, PredictResult
+
+
+class GPUUnavailableError(RuntimeError):
+    """Exception type to use when a GPU, or the GPU library, is unavailable"""
+    pass
+
+
+def ndarray_coerce_type_and_flags(arr):
+    """
+    Helper function for the GaussianProcessGPU methods that call
+    CUDA/C++ functions (those wrapped by _dense_gpgpu) and that take
+    numpy arrays as arguments.  Ensures that an array is of the
+    correct type for this purpose.
+
+    Takes an array or array-like.
+
+    Returns an ndarray with the same data, that has:
+    - dtype of float64
+    - flags.writable
+    - flags.c_contiguous
+
+    The array returned may reference the original array, be a copy of
+    it, or be newly constructed from the argument.
+    """
+
+    arr = np.array(arr, copy=False)
+
+    # cast into float64, just in case we were given integers and ensure contiguous (C type)
+    arr_contiguous_float64 = np.ascontiguousarray(arr.astype(np.float64, copy=False))
+    if not arr_contiguous_float64.flags['WRITEABLE']:
+        return np.copy(arr_contiguous_float64)
+    else:
+        return arr_contiguous_float64
+
+
+def parse_meanfunc_formula(formula):
+    """
+    Assuming the formula has already been parsed by the Python MeanFunction interface,
+    we expect it to be in a standard form, with parameters denoted by 'c' and variables
+    denoted by 'x[dim_index]' where dim_index < D.
+
+    :param formula: string representing the desired mean function formula
+    :type formula: str
+
+    :returns: Instance of LibGPGPU.ConstMeanFunc or LibGPGPU.PolyMeanFunc implementing formula,
+       or None if the formula could not be parsed, or is not currently implemented in C++ code.
+    :rtype: LibGPGPU.ConstMeanFunc or LibGPGPU.PolyMeanFun or None
+    """
+    if formula == "c":
+        return LibGPGPU.ConstMeanFunc()
+    else:
+        # see if the formula is a string representation of a number
+        try:
+            m = float(formula)
+            return LibGPGPU.FixedMeanFunc(m)
+        except:
+            pass
+    # if we got here, we hopefully have a parse-able formula
+    terms = formula.split("+")
+    def find_index_and_power(term):
+        variables = re.findall("x\[[\d+]\]",term)
+        if len(variables) == 0:
+            # didn't find a non-const term
+            return None
+        indices = [int(re.search("\[([\d+])\]", v).groups()[0]) for v in variables]
+        if indices.count(indices[0]) != len(indices):
+            raise NotImplementedError("Cross terms, e.g. x[0]*x[1] not implemented in GPU version.")
+        # first guess at the power to which the index is raised is how many times it appears
+        # e.g. if written as x[0]*x[0]
+        power = len(indices)
+        # however, it's also possible to write 'x[0]^2' or even, 'x[0]*x[0]^2' or even 'x[0]^2*x[0]^2'
+        # so look at all the numbers appearing after a '^'.
+        more_powers = re.findall("\^[\d]+",term)
+        more_powers = [int(re.search("\^([\d]+)",p).groups()[0]) for p in more_powers]
+        # now add these on to the original power number
+        # (subtracting one each time, as we already have x^1 implicitly)
+        for p in more_powers:
+            power += p - 1
+        return [indices[0], power]
+    indices_powers = []
+    for term in terms:
+        ip = find_index_and_power(term)
+        if ip:
+            indices_powers.append(ip)
+    if len(indices_powers) > 0:
+        return LibGPGPU.PolyMeanFunc(indices_powers)
+    else:
+        return None
+
+
+class GaussianProcessGPU(GaussianProcessBase):
+
+    """
+    This class implements the same interface as
+    :class:`mogp_emulator.GaussianProcess.GaussianProcess`, but using a GPU if available.
+    Will raise a RuntimeError if a CUDA-compatible GPU, GPU-interface library libgpgpu
+    could not be found.
+    Note that while the class uses a C++/CUDA implementation of the SquaredExponential or
+    Matern52 kernels for the "fit" and "predict" methods, the 'kernel' data member (and
+    hence the results of e.g. 'gp.kernel.kernel_f(theta)' will be the pure Python versions,
+    for compatibility with the interface of the GaussianProcess class.
+    """
+
+    def __init__(self, inputs, targets, mean=None, kernel=SquaredExponential(), priors=None,
+                 nugget="adaptive", inputdict = {}, use_patsy=True, max_batch_size=2000):
+
+        if not LibGPGPU.HAVE_LIBGPGPU:
+            raise RuntimeError("Cannot construct GaussianProcessGPU: "
+                               "The GPU library (libgpgpu) could not be loaded")
+
+        elif not LibGPGPU.gpu_usable():
+            raise RuntimeError("Cannot construct GaussianProcessGPU: "
+                               "A compatible GPU could not be found")
+
+        inputs = ndarray_coerce_type_and_flags(inputs)
+        if inputs.ndim == 1:
+            inputs = np.reshape(inputs, (-1, 1))
+        assert inputs.ndim == 2
+
+        targets = ndarray_coerce_type_and_flags(targets)
+        assert targets.ndim == 1
+        assert targets.shape[0] == inputs.shape[0]
+
+        self._inputs = inputs
+        self._targets = targets
+        self._max_batch_size = max_batch_size
+
+        if mean is None:
+            self.mean = LibGPGPU.ZeroMeanFunc()
+        else:
+            if not issubclass(type(mean), MeanBase):
+                if isinstance(mean, str):
+                    mean = MeanFunction(mean, inputdict, use_patsy)
+                else:
+                    raise ValueError("provided mean function must be a subclass of MeanBase,"+
+                                     " a string formula, or None")
+
+            # at this point, mean will definitely be a MeanBase.  We can call its __str__ and
+            # parse this to create an instance of a C++ MeanFunction
+            self.mean = parse_meanfunc_formula(mean.__str__())
+            # if we got None back from that function, something went wrong
+            if not mean:
+                raise ValueError("""
+                GPU implementation was unable to parse mean function formula {}.
+                """.format(mean.__str__())
+                )
+
+        self.nugget = nugget
+
+        # set the kernel.
+        # Note that for the "kernel" data member, we use the Python instance
+        # rather than the C++/CUDA one (for consistency in interface with
+        # GaussianProcess class).  However the C++/CUDA version of the kernel is
+        # used when calling fit() or predict()
+        if (isinstance(kernel, str) and kernel == "SquaredExponential") \
+           or isinstance(kernel, SquaredExponential):
+            self.kernel_type = LibGPGPU.kernel_type.SquaredExponential
+            self.kernel = SquaredExponential()
+        elif (isinstance(kernel, str) and kernel == "Matern52") \
+           or isinstance(kernel, Matern52):
+            self.kernel_type = LibGPGPU.kernel_type.Matern52
+            self.kernel = Matern52()
+        else:
+            raise ValueError("GPU implementation requires kernel to be SquaredExponential or Matern52")
+
+        # instantiate the DenseGP_GPU class
+        self._densegp_gpu = None
+        self._init_gpu()
+
+
+    def _init_gpu(self):
+        """
+        Instantiate the DenseGP_GPU C++/CUDA class, if it doesn't already exist.
+        """
+        if not self._densegp_gpu:
+            self._densegp_gpu = LibGPGPU.DenseGP_GPU(self._inputs,
+                                                     self._targets,
+                                                     self._max_batch_size,
+                                                     self.mean,
+                                                     self.kernel_type) #,
+#                                                     self.mean_type)
+
+
+    @property
+    def inputs(self):
+        """
+        Returns inputs for the emulator as a numpy array
+
+        :returns: Emulator inputs, 2D array with shape ``(n, D)``
+        :rtype: ndarray
+        """
+        return self._densegp_gpu.inputs()
+
+    @property
+    def targets(self):
+        """
+        Returns targets for the emulator as a numpy array
+
+        :returns: Emulator targets, 1D array with shape ``(n,)``
+        :rtype: ndarray
+        """
+        return self._densegp_gpu.targets()
+
+    @property
+    def n(self):
+        """
+        Returns number of training examples for the emulator
+
+        :returns: Number of training examples for the emulator object
+        :rtype: int
+        """
+        return self._densegp_gpu.n()
+
+    @property
+    def D(self):
+        """
+        Returns number of inputs (dimensions) for the emulator
+
+        :returns: Number of inputs for the emulator object
+        :rtype: int
+        """
+        return self._densegp_gpu.D()
+
+    @property
+    def n_params(self):
+        """
+        Returns number of hyperparameters
+
+        Returns the number of hyperparameters for the emulator. The number depends on the
+        choice of mean function, covariance function, and nugget strategy, and possibly the
+        number of inputs for certain choices of the mean function.
+
+        :returns: Number of hyperparameters
+        :rtype: int
+        """
+        return self._densegp_gpu.n_params()+ self.mean.get_n_params(self.inputs)
+
+    @property
+    def nugget_type(self):
+        """
+        Returns method used to select nugget parameter
+
+        Returns a string indicating how the nugget parameter is treated, either ``"adaptive"``,
+        ``"fit"``, or ``"fixed"``. This is automatically set when changing the ``nugget``
+        property.
+
+        :returns: Current nugget fitting method
+        :rtype: str
+        """
+        return self._nugget_type.__str__().split(".")[1]
+
+    @property
+    def nugget(self):
+        """
+        See :func:`mogp_emulator.GaussianProcess.GaussianProcess.nugget`
+        """
+        return self._nugget
+
+    @nugget.setter
+    def nugget(self, nugget):
+        if not isinstance(nugget, (str, float)):
+            try:
+                nugget = float(nugget)
+            except TypeError:
+                raise TypeError("nugget parameter must be a string or a non-negative float")
+
+        if isinstance(nugget, str):
+            if nugget == "adaptive":
+                self._nugget_type = LibGPGPU.nugget_type(0)
+            elif nugget == "fit":
+                self._nugget_type = LibGPGPU.nugget_type(1)
+            else:
+                raise ValueError("nugget must be a float set to 'adaptive', 'fit', or 'fixed'")
+            self._nugget = None
+        else:
+            if nugget < 0.:
+                raise ValueError("nugget parameter must be non-negative")
+            self._nugget_type = LibGPGPU.nugget_type(2) #fixed
+            self._nugget = float(nugget)
+
+    @property
+    def theta(self):
+        """
+        Returns emulator hyperparameters
+        see
+        :func:`mogp_emulator.GaussianProcess.GaussianProcess.theta`
+
+        :type theta: ndarray
+        """
+        if not self._densegp_gpu.theta_fit_status():
+            return None
+        theta = np.zeros(self.n_params)
+        self._densegp_gpu.get_theta(theta)
+        return theta
+
+    @theta.setter
+    def theta(self, theta):
+        """
+        Fits the emulator and sets the parameters (property-based setter
+        alias for ``fit``)
+
+        See :func:`mogp_emulator.GaussianProcess.GaussianProcess.theta`
+
+        :type theta: ndarray
+        :returns: None
+        """
+        if theta is None:
+            self._densegp_gpu.reset_theta_fit_status()
+            self.current_logpost = None
+        else:
+            self.fit(theta)
+
+    @property
+    def L(self):
+        """
+        Return the lower triangular Cholesky factor.
+
+        :returns: np.array
+        """
+        result = np.zeros((self.n, self.n))
+        self._densegp_gpu.get_cholesky_lower(result)
+        return np.tril(result.transpose())
+
+    def get_K_matrix(self):
+        """
+        Returns current value of the inverse covariance matrix as a numpy array.
+
+        Does not include the nugget parameter, as this is dependent on how the
+        nugget is fit.
+        """
+        result = np.zeros((self.n, self.n))
+        self._densegp_gpu.get_K(result)
+        return result
+
+    def fit(self, theta):
+        """
+        Fits the emulator and sets the parameters.
+
+        Implements the same interface as
+        :func:`mogp_emulator.GaussianProcess.GaussianProcess.fit`
+        """
+        theta = ndarray_coerce_type_and_flags(theta)
+
+        nugget_size = self.nugget if self.nugget_type == "fixed" else 0.
+        self._densegp_gpu.fit(theta, self._nugget_type, nugget_size)
+        if self.nugget_type == "adaptive" or self.nugget_type == "fit":
+            self._nugget = self._densegp_gpu.get_jitter()
+        invQt_result = np.zeros(self.n)
+        self._densegp_gpu.get_invQt(invQt_result)
+        self.invQt = invQt_result
+        self.current_logpost = self.logposterior(theta)
+
+    def logposterior(self, theta):
+        """
+        Calculate the negative log-posterior at a particular value of the hyperparameters
+
+        See :func:`mogp_emulator.GaussianProcess.GaussianProcess.logposterior`
+
+        :param theta: Value of the hyperparameters. Must be array-like with shape ``(n_params,)``
+        :type theta: ndarray
+        :returns: negative log-posterior
+        :rtype: float
+        """
+        if self.theta is None or not np.allclose(theta, self.theta, rtol=1.e-10, atol=1.e-15):
+            self.fit(theta)
+
+        return self._densegp_gpu.get_logpost()
+
+    def logpost_deriv(self, theta):
+        """
+        Calculate the partial derivatives of the negative log-posterior
+
+        See :func:`mogp_emulator.GaussianProcess.GaussianProcess.logpost_deriv`
+
+        :param theta: Value of the hyperparameters. Must be array-like with shape ``(n_params,)``
+        :type theta: ndarray
+        :returns: partial derivatives of the negative log-posterior with respect to the
+                  hyperparameters (array with shape ``(n_params,)``)
+        :rtype: ndarray
+        """
+        theta = np.array(theta, copy=False)
+
+        assert theta.shape == (self.n_params,), "bad shape for new parameters"
+
+        if self.theta is None or not np.allclose(theta, self.theta, rtol=1.e-10, atol=1.e-15):
+            self.fit(theta)
+
+        result = np.zeros(self.n_params)
+        self._densegp_gpu.logpost_deriv(result)
+        return result
+
+    def logpost_hessian(self, theta):
+        """
+        Calculate the Hessian of the negative log-posterior
+
+        See :func:`mogp_emulator.GaussianProcess.GaussianProcess.logpost_hessian`
+
+        :param theta: Value of the hyperparameters. Must be array-like with shape
+                      ``(n_params,)``
+        :type theta: ndarray
+        :returns: Hessian of the negative log-posterior (array with shape
+                  ``(n_params, n_params)``)
+        :rtype: ndarray
+        """
+        raise GPUUnavailableError(
+            "The Hessian calculation is not currently implemented in the GPU version of MOGP."
+        )
+
+
+    def predict(self, testing, unc=True, deriv=True, include_nugget=True):
+        """
+        Make a prediction for a set of input vectors for a single set of hyperparameters.
+        This method implements the same interface as
+        :func:`mogp_emulator.GaussianProcess.GaussianProcess.predict`
+        """
+
+        if self.theta is None:
+            raise ValueError("hyperparameters have not been fit for this Gaussian Process")
+
+        testing = ndarray_coerce_type_and_flags(testing)
+
+        if self.D == 1 and testing.ndim == 1:
+            testing = np.reshape(testing, (-1, 1))
+        elif testing.ndim == 1:
+            testing = np.reshape(testing, (1, len(testing)))
+        assert testing.ndim == 2
+
+        n_testing, D = np.shape(testing)
+
+        assert D == self.D
+
+        means = np.zeros(n_testing)
+
+        if unc:
+            variances = np.zeros(n_testing)
+            for i in range(0, n_testing, self._max_batch_size):
+                self._densegp_gpu.predict_variance_batch(
+                    testing[i:i+self._max_batch_size],
+                    means[i:i+self._max_batch_size],
+                    variances[i:i+self._max_batch_size])
+            if include_nugget:
+                variances += self._nugget
+        else:
+            for i in range(0, n_testing, self._max_batch_size):
+                self._densegp_gpu.predict_batch(
+                    testing[i:i+self._max_batch_size],
+                    means[i:i+self._max_batch_size])
+            variances = None
+        if deriv:
+            deriv_result = np.zeros((n_testing,self.D))
+            for i in range(0, n_testing, self._max_batch_size):
+                self._densegp_gpu.predict_deriv(
+                    testing[i:i+self._max_batch_size],
+                    deriv_result[i:i+self._max_batch_size])
+        else:
+            deriv_result = None
+        return PredictResult(mean=means, unc=variances, deriv=deriv_result)
+
+
+    def __call__(self, testing):
+        """A Gaussian process object is callable: calling it is the same as
+        calling `predict` without uncertainty and derivative
+        predictions, and extracting the zeroth component for the
+        'mean' prediction.
+        """
+        return (self.predict(testing, unc=False, deriv=False)[0])
+
+
+    def __str__(self):
+        """
+        Returns a string representation of the model
+
+        :returns: A string representation of the model
+        (indicates number of training examples and inputs)
+        :rtype: str
+        """
+        return ("Gaussian Process with " + str(self.n) + " training examples and " +
+                str(self.D) + " input variables")
+
+
+    ## __setstate__ and __getstate__ for pickling: don't pickle "_dense_gp_gpu",
+    ## and instead reinitialize this when unpickling
+    ## (Pickling is required to use multiprocessing.)
+    def __setstate__(self, state):
+        self.__dict__ = state
+        self.init_gpu()
+        if self._theta is not None:
+            self.fit(self._theta)
+
+
+    def __getstate__(self):
+        copy_dict = self.__dict__.copy()
+        del copy_dict["_densegp_gpu"]
+        copy_dict["_densegp_gpu"] = None
+        return copy_dict
diff --git a/mogp_emulator/HistoryMatching.py b/mogp_emulator/HistoryMatching.py
index e128f132..66fc1e07 100644
--- a/mogp_emulator/HistoryMatching.py
+++ b/mogp_emulator/HistoryMatching.py
@@ -1,649 +1,646 @@
-import numpy as np
-from mogp_emulator.GaussianProcess import GaussianProcess, PredictResult
-
-class HistoryMatching(object):
-    r"""
-    Class containing tools to implement history matching between the outputs of a
-    gaussian process and an observation.
-
-    Primary usage is the ``get_implausibility`` method, which calculates the
-    implausibility metric (a number of standard deviations between the emulator mean
-    and the observation) for a number of query points:
-
-    .. math::
-        {I_i(\bar{x_0}) = \frac{|z_i - E(f_i(\bar{x_0}))|}
-        {\sqrt{Var\left[z_i - E(f_i(\bar{x_0}))\right]}}}
-
-    The code allows a number of ways to specify variances, and all are summed to
-    compute the final variance that determines the implausibility measure.
-    Variances on the observation itself can be provided with the observations,
-    while variances arising from the GP prediction are included in the emulator
-    expectations. Additional variances can be included to account for model
-    discrepancy when computing the implausibility metric, and these can be uniform
-    for all query points, or vary across different query points if desired. Any
-    number of additional variances can be included in the calculation by passing
-    them when computing the implausibility.
-
-    Once implausibility is computed, the class can determine query points that
-    can be ruled out, as well as points that are not ruled out yet, based on the
-    threshold class attribute. See the methods ``get_RO``, ``get_NROY``, and
-    ``set_threshold`` for more details.
-
-    Example - implausibility computation for a 1D GP::
-
-        import math
-        import numpy as np
-        from HistoryMatching import HistoryMatching
-        from mogp_emulator import GaussianProcess
-
-        # Set up a gaussian process
-        x_training = np.array([[0.],[10.],[20.],[30.],[43.],[50.]])
-        def get_y_training(x):
-            n_points = len(x)
-            f = np.zeros(n_points)
-            for i in range(n_points):
-                f[i] = np.sin(2*math.pi*x[i] / 50)
-            return f
-        y_training = get_y_training(x_training)
-
-        gp = GaussianProcess(x_training, y_training)
-        np.random.seed(47)
-        gp.learn_hyperparameters()
-
-        # Define the observation to which to compare
-        obs = [-0.8, 0.0004]
-
-        # Coordinates to predict
-        n_rand = 2000
-        coords = np.sort(np.random.rand(n_rand),axis=0)*(56)-3
-        coords = coords[:,None]
-
-        # Generate Expectations
-        expectations = gp.predict(coords)
-
-        # Calculate implausibility
-        hm = HistoryMatching(obs=obs, expectations=expectations)
-        I = hm.get_implausibility()
-
-
-    """
-
-    def __init__(self, gp=None, obs=None, coords=None, expectations=None, threshold=3.):
-        r"""
-        Create a new instance of history matching.
-
-        The primary function of this method is to initialise the ``self.gp``,
-        ``.obs``, ``.coords``, ``.expectations``, ``.ndim``, ``.ncoords``,
-        ``.threshold``, ``.I``, ``.NROY``, and ``.RO`` variables as placeholders.
-        The optional keyword arguments can also be used to set user-defined values for
-        various quantities in a single call to ``HistoryMatching()``. Setting sufficient
-        numbers of these to allow the number of dimensions and/or number of coordinates to
-        be computed also causes these to be set at this stage.
-
-        :param gp: (Optional) ``GaussianProcess`` object used to make predictions in
-                   history matching. Optional, can instead provide predictions directly
-                   via the ``expectations`` argument. Default is ``None``
-        :type gp: GaussianProcess or None
-        :param obs: (Optional) Observations against which the predictions will be
-                    compared. If provided, must either be a float (assumes no
-                    uncertainty in the observations) or a list of two floats
-                    holding the observation and its variance. Required for history
-                    matching, but can be provided when calling ``get_implausibility``.
-                    Default is ``None``.
-        :type obs: float, list, or None
-        :param coords: (Optional) Inputs at which the emulator values will be computed
-                       to compare the GP output to the observations. If provided, must
-                       be a numpy array matching the input for the GP (must be a 2D
-                       array with dimensions ``(n, D)`` where ``n`` is the number of
-                       points to be considered and ``D`` is the number of parameters
-                       for the GP). Only required if ``expectations`` is not provided.
-                       Default is ``None``.
-        :type coords: ndarray
-        :param expectations: (Optional) Tuple of 3 numpy arrays or 2 numpy arrays and
-                             ``None`` of the form expected from GP predictions. The first
-                             array must hold the predicted mean at all query points, and
-                             the second array must hold the variances from the GP predictions
-                             at all query points. The third is not used, so can simply be a
-                             dummy array or ``None``. Can instead provide a GP object and the
-                             points to query to have the predictions made internally. Default
-                             is ``None``.
-        :type expectations: tuple holding 3 ndarrays
-        """
-
-        # Place-holder values for user-defined quantities that can be passed in
-        # as args.
-        self.gp = None
-        self.obs = None
-        self.coords = None
-        self.expectations = None
-
-        # Place-holder or default values for optional or derived quantities.
-        self.ndim = None
-        self.ncoords = None
-        self.threshold = None
-        self.I = None
-        self.NROY = None
-        self.RO = None
-
-        # If suitable keyword arguments are provided, update the relevent
-        # variable(s)
-        if self.check_gp(gp):
-            self.set_gp(gp)
-        if self.check_obs(obs):
-            self.set_obs(obs)
-        if self.check_coords(coords):
-            self.set_coords(coords)
-        if self.check_expectations(expectations):
-            self.set_expectations(expectations)
-        if self.check_threshold(threshold):
-            self.set_threshold(threshold)
-
-        self.update()
-
-
-    def _select_expectations(self):
-        r"""
-        Determine the emulator predictions to use in HistoryMatching
-
-        This method queries the internal state of the HistoryMatching object to get
-        the expectations to use in the history matching calculation. It either
-        uses the expectations that were directly provided, or makes predictions
-        at the query points using the provided GP. If both methods are set
-        simultaneously, the method raises an exception.
-
-        :returns: Expectations to use in history matching. These are either provided
-                  explicitly, or made internally using the GP and query points.
-        :rtype: tuple of 3 ndarrays or 2 ndarrays and None
-        """
-        # Check that we have exactly 1 valid combination of parameters
-        UseCoordGP = False
-        UseExpectations = False
-        if (self.check_coords(self.coords) and self.check_gp(self.gp)):
-            UseCoordGP = True
-        if self.check_expectations(self.expectations):
-            UseExpectations = True
-        if UseCoordGP and UseExpectations:
-            raise ValueError("Multiple valid parameter combinations are set. Previously set " +
-                             "parameters can be removed by setting them to None")
-        if not UseCoordGP and not UseExpectations:
-            raise ValueError("Expectations are not provided, nor is a GP and coordinates." +
-                             "must set one in order to perform History Matching")
-
-        # Confirm that the ncoords parameter is set
-        if self.ncoords is None:
-            raise ValueError("ncoords is not set despite a valid parameter combination "+
-                             "being found.")
-
-        # From this point on, the calculation works off the self.expectations value.
-        # If this is not set, calculate it from the other parameters
-        if UseCoordGP:
-            expectations = self.gp.predict(self.coords)
-        else:
-            expectations = self.expectations
-
-        return expectations
-
-    def get_implausibility(self, discrepancy = 0.):
-        r"""
-        Compute Implausibility measure for all query points
-
-        Carries out the implausibility calculation given by:
-
-        .. math::
-            {I_i(\bar{x_0}) = \frac{|z_i - E(f_i(\bar{x_0}))|}
-            {\sqrt{Var\left[z_i - E(f_i(\bar{x_0}))\right]}}}
-
-        to return an implausibility value (a number of standard deviations between the
-        emulator mean and the observation)for each of the provided coordinates.
-
-        Requires that the observation parameter is set, and that at least one of the
-        following conditions are met:
-
-          a) The coords and gp parameters are set
-          b) The expectations parameter is set
-
-        Note that using the GP internally assumes that the standard prediction settings
-        will be used when making predictions. If the user wishes to have more control
-        over the prediction method (i.e. make the predictions from MCMC samples),
-        the user should explicitly pass ``expectations`` to the object.
-
-        An additional variance can be included that represents a general model
-        discrepancy that describes the prior beliefs regarding how well the model
-        matches reality. In practice, the model discrepancy is essential to have
-        predictions that are not overly confident, however it can be hard to estimate
-        (see further discussion in Brynjarsdottir and O\'Hagan, 2014).
-
-        As the implausibility calculation linearly sums variances, the result is
-        agnostic to the precise provenance of any of the included variances
-
-        :param discrepancy: Additional variance to be included in the implausibility
-                            calculation. Must be a non-negative float. Optional, default
-                            is ``0.``
-        :type discrepancy: float
-        :returns: Array holding implausibility metric for all query points accounting
-                  for all variances, ndarray of length ``(ncoords,)``
-        :rtype: ndarray
-        """
-
-        # Confirm that observation parameter is set
-        if not self.check_obs(self.obs):
-            raise ValueError("implausibility calculation requires that the observation value is " +
-                             "set. This can be done using the set_obs method.")
-
-        assert discrepancy >= 0., "Model discrepancy variance cannot be negative"
-
-        expectations = self._select_expectations()
-
-        # Compute implausibility for each expectation value
-        Vs = np.zeros(self.ncoords)
-        Vs += expectations[1]                         # variance on expectation
-        Vs += discrepancy                             # model discrepancy
-        Vs += self.obs[1]                             # variance on observation
-        self.I = (np.abs(self.obs[0] - expectations[0]) / np.sqrt(Vs))
-
-        return self.I
-
-
-    def get_NROY(self, discrepancy = 0.):
-        r"""
-        Return set of indices that are not yet ruled out
-
-        Returns a list of indices for ``self.I`` that correspond to entries that are not
-        yet ruled out. Points that are ruled out have an implausibility metric that
-        exceeds the threshold (can be set when initializing history matching or using
-        the ``set_threshold`` method). If the implausibility metric has not yet been
-        computed for the desired points, it is calculated. If a model discrepancy
-        is to be included, it can be passed here.
-
-        :param discrepancy: Additional variance to be included in the implausibility
-                            calculation. Must be a non-negative float. Optional, default
-                            is ``0.``
-        :type discrepancy: float
-        :returns: List of integer indices that have not yet been ruled out.
-        :rtype: list
-        """
-        if self.I is None:
-            self.get_implausibility(discrepancy)
-
-        self.NROY = list(np.where(self.I <= self.threshold)[0])
-
-        return self.NROY
-
-
-    def get_RO(self, discrepancy = 0.):
-        r"""
-        Return set of indices that have been ruled out
-
-        Returns a list of indices for ``self.I`` that correspond to entries that have
-        been ruled out. Points that are ruled out have an implausibility metric that
-        exceeds the threshold (can be set when initializing history matching or using
-        the ``set_threshold`` method). If the implausibility metric has not yet been
-        computed for the desired points, it is calculated. If a model discrepancy
-        is to be included, it can be passed here.
-
-        :param discrepancy: Additional variance to be included in the implausibility
-                            calculation. Must be a non-negative float. Optional, default
-                            is ``0.``
-        :type discrepancy: float
-        :returns: List of integer indices that have been ruled out.
-        :rtype: list
-        """
-        if self.I is None:
-            self.get_implausibility(discrepancy)
-
-        self.RO = list(np.where(self.I > self.threshold)[0])
-
-        return self.RO
-
-
-    def set_gp(self, gp):
-        r"""
-        Set the Gaussian Process to use with history matching
-
-        Sets the ``self.gp`` variable to the provided ``GaussianProcess`` argument.
-
-        :param gp: ``GaussianProcess`` object to use for history matching.
-        :type gp: GaussianProcess
-        :returns: None
-        """
-        if not self.check_gp(gp):
-            raise TypeError("bad input for set_gp - expects a GaussianProcess object.")
-        self.gp = gp
-
-
-    def set_obs(self, obs):
-        r"""
-        Set the observations to be used for history matching
-
-        Sets the ``self.obs`` variable to the provided ``obs`` argument. The object
-        must pass the ``check_obs`` requirements, meaning that it must be a
-        float (assumes that the observation has no error associated with it) or a
-        list containing two floats (representing an observation and its variance).
-        Note that the variance must be non-negative.
-
-        :param obs: Observations to be used for history matching. Must be a float
-                    or a list of two floats.
-        :type obs: float or list
-        :returns: None
-        """
-        if not self.check_obs(obs):
-            raise TypeError("bad input for set_obs")
-        if isinstance(obs, list):
-            if len(obs) == 1:
-                self.obs = [float(obs[0]), 0.]
-            else:
-                self.obs = [float(a) for a in obs]
-        else:
-            self.obs = [float(obs), 0.]
-
-
-    def set_coords(self, coords):
-        r"""
-        Set the query points to be used in history matching
-
-        Sets the ``self.coords`` variable to the provided query points ``coords`` if
-        it passes the ``check_coords`` requirements, or be ``None`` to remove a set
-        of existing query points. ``coords`` must be a numpy array matching the
-        inputs to the provided ``GaussianProcess`` object.
-
-        :param coords: Numpy array holding query points (array with shape ``(n, D)``,
-                       where ``n`` is the number of query points and ``D`` is the
-                       number of inputs to the emulator), or ``None``
-        :type coords: ndarray or None
-        :returns: None
-        """
-        # need to allow coords == None, as otherwise can't reset values
-        if not self.check_coords(coords) and (not coords is None):
-            raise TypeError("bad input for set_coords - expected coords in the form " +
-                            "of a list or 1D or 2D ndarray of numerical values")
-        if isinstance(coords, np.ndarray):
-            if len(coords.shape) == 1:
-                self.coords = np.reshape(coords, [-1, 1])
-            elif len(coords.shape) == 2:
-                self.coords = coords
-            else:
-                raise Exception("error in exception handling - an ndarray of >2 " +
-                                "dimensions somehow got through check_coords")
-        elif isinstance(coords, list):
-            self.coords = np.reshape(np.asarray(coords), [-1, 1])
-        elif coords is None:
-            self.coords = None
-        else:
-            raise TypeError("error in exception handling - an argument of illegal " +
-                            "type somehow got through check_coords")
-        self.update()
-
-
-    def set_expectations(self, expectations):
-        r"""
-        Set the expected output of the simulator to be used in history matching
-
-        Sets the ``self.expectations`` variable to the provided ``expectations``
-        argument if it passes the ``check_expectations`` requirements, or ``None``
-        can be used to remove an existing set of expectations. Expectations
-        must be a tuple of 3 numpy arrays, where the first holds the predicted
-        means for all query points and the second holds the predicted variances
-        for all query points. The third arrray is not used in the computation,
-        but is included as it is an expected output of the GP ``predict`` method.
-
-        :param expectations: GP predictions at all query points. Must be a tuple
-                             of 3 numpy arrays, or None to remove existing
-                             expectations.
-        :type expectations: tuple of 3 ndarrays or None
-        :returns: None
-        """
-        # need to allow expectations == None, as otherwise can't reset values
-        if not self.check_expectations(expectations) and (not expectations is None):
-            raise TypeError("bad input for set_expectations - expected a Tuple " +
-                            "of 3 ndarrays.")
-        self.expectations = expectations
-        self.update()
-
-
-    def set_threshold(self, threshold):
-        r"""
-        Set the threshold value for history matching
-
-        Sets the ``self.threshold variable`` to the provided threshold argument
-        if it  passes the ``check_threshold`` requirements. The threshold must
-        be a non-negative float.
-
-        :param threshold: New value for threshold, must be a non-negative float.
-        :type threshold: float
-        :returns: None
-        """
-        if not self.check_threshold(threshold):
-            raise TypeError("bad input for set_expectations - expected a float")
-        self.threshold = float(threshold)
-
-
-    def status(self):
-        r"""
-        Prints a summary of the current status of the class object.
-
-        Prints a summary of the current status of the class object, including the
-        values stored in ``self.gp``, ``.obs``, ``.ndim``, ``.ncoords``, and
-        ``.threshold``, and the shapes/sizes ofvalues stored in ``self.obs``,
-        ``.coords``, ``.expectations``, ``.I``, ``.NROY``, and ``.RO.``
-        No inputs, no return value.
-
-        :returns: None
-        """
-
-        print(str(self))
-
-
-    def check_gp(self, gp):
-        r"""
-        Checks if the provided argument is consistent with expectations for a GP.
-
-        Returns a boolean that is True if the provided quantity is consistent with
-        the requirements for a gaussian process, i.e. is of type ``GaussianProcess``.
-
-        :param gp: Input GP object to be checked.
-        :type gp: GaussianProcess
-        :returns: Boolean indicating if provided object is a ``GaussianProcess``
-        :rtype: bool
-        """
-        if gp is None: return False
-        if isinstance(gp, GaussianProcess):
-            return True
-        return False
-
-
-    def check_obs(self, obs):
-        r"""
-        Checks if the provided argument is consistent with expectations for
-        observations.
-
-        Returns a boolean that is ``True`` if the provided quantity is consistent with
-        the requirements for the observation quantity, i.e. is a numerical value
-        that can be converted to a float, or a list of up to two such values. Also
-        checks if the provided variance is non-negative.
-
-        :param obs: Input for observations to be checked
-        :type obs: float or list-like
-        :returns: Boolean indicating if provided observations are acceptable
-        :rtype: bool
-        """
-
-        if obs is None:
-            return False
-        if isinstance(obs, list) or isinstance(obs, tuple) or isinstance(obs, np.ndarray):
-            if len(obs) > 2:
-                raise ValueError("bad input type for HistoryMatching - the specified " +
-                                 "observation parameter cannot contain more than 2 entries "+
-                                 "(value, [variance])")
-            if len(obs) <= 2:
-                try:
-                    test = [float(a) for a in obs]
-                except TypeError:
-                    raise TypeError("bad input type for HistoryMatching - the specified " +
-                                    "observation parameter must contain only numerical " +
-                                    "values")
-            if len(obs) == 2:
-                assert float(obs[1]) >= 0., "variance in observations cannot be negative"
-            return True
-        else:
-            try:
-                test = float(obs)
-            except (TypeError, ValueError):
-                raise TypeError("bad input type for HistoryMatching - the specified " +
-                                "observation parameter must contain only numerical values")
-            return True
-        return False
-
-
-    def check_coords(self, coords):
-        r"""
-        Checks if the provided argument is consistent with expectations for
-        coordinates.
-
-        Returns a boolean that is ``True`` if the provided quantity is consistent with
-        the requirements for the coordinates quantity, i.e. a ndarray of
-        fewer than 3 dimensions.
-
-        :param coords: Input to check for consistency with coordinates.
-        :type coords: ndarray
-        :returns: Boolean indicating if coords is consistent
-        :rtype: bool
-        """
-
-        if coords is None:
-            return False
-
-        if isinstance(coords, np.ndarray):
-            if len(coords.shape) <= 2:
-                return True
-        return False
-
-
-    def check_expectations(self, expectations):
-        r"""
-        Checks if the provided argument is consistent with expectations for
-        Gaussian Process Expectations.
-
-        Returns a boolean that is ``True`` if the provided quantity is consistent with
-        the output of the predict method of a GaussianProcess object, i.e. that it
-        is a ``GaussianProcess.PredictResult`` object with mean and variance defined.
-
-        :param expectations: Input to check for consistency with expectations
-        :type expectations: tuple of 3 numpy arrays or 2 numpy arrays and None
-        :returns: Boolean indicating if expectations is consistent
-        :rtype: bool
-        """
-        if expectations is None:
-            return False
-        if not isinstance(expectations, PredictResult):
-            return False
-        if not all((isinstance(expectations[0], np.ndarray),
-                    isinstance(expectations[1], np.ndarray),
-                    (isinstance(expectations[2], np.ndarray) or
-                     expectations[2] is None))):
-            raise TypeError("bad input type for HistoryMatching - expected expectation " +
-                            "values in the form of a PredictResults object with mean" +
-                            "and uncertainty set.")
-        if not expectations[0].shape == expectations[1].shape:
-            raise ValueError("bad input for HistoryMatching - mean and variance " +
-                             "expectations do not match")
-        assert np.all(expectations[1] >= 0.), "all variances must be non-negative"
-        return True
-
-
-    def check_threshold(self, threshold):
-        r"""
-        Check value of threshold
-
-        Checks if the provided argument is consistent with expectations for a
-        threshold value.
-
-        Returns a boolean that is ``True`` if the provided quantity is consistent with
-        the requirements for a threshold value, i.e. that it is a non-negative numerical
-        value.
-
-        :param threshold: threshold to be tested
-        :type threshold: float
-        :returns:  Boolean indicating if the provided argument is consistent with
-                   a threshold
-        :rtype: bool
-        """
-        if threshold is None:
-            return False
-        try:
-            test = float(threshold)
-            assert test >= 0., "threshold must be non-negative"
-            return True
-        except TypeError:
-            return False
-
-    def update(self):
-        r"""
-        Update History Matching object
-
-        Checks that sufficient information exists to compute ``ndim`` and/or ``ncoords``
-        and, if so, computes these values. This also serves to update these values
-        if the information on which they are based is changed. No inputs, no return
-        value.
-
-        :returns: None
-        """
-
-        if self.check_coords(self.coords):
-            self.ndim = self.coords.shape[1]
-            self.ncoords = self.coords.shape[0]
-        elif self.check_expectations(self.expectations):
-            self.ncoords = self.expectations[0].shape[0]
-
-
-    def __str__(self):
-        r"""
-        Returns string representation of ``HistoryMatching`` object
-
-        Returns a string representation of a ``HistoryMatching`` object,
-        used when printing information to the interpreter or when the
-        ``status`` method is called.
-
-        :returns: string representation of the HistoryMatching object.
-        :rtype: str
-        """
-
-        if self.coords is None:
-            coord_str = None
-        else:
-            coord_str = self.coords.shape
-
-        if self.expectations is None:
-            exp_str = None
-        else:
-            exp_str = (str(type(self.expectations)) + " of len " +
-                       str(len(self.expectations)) + " containing arrays of shape: " +
-                       str(self.expectations[0].shape))
-
-        if self.I is None:
-            I_str = None
-        elif isinstance(self.I, np.ndarray):
-            I_str = (str(type(self.I)) + " of shape " + str(self.I.shape))
-
-        if self.NROY is None:
-            NROY_str = None
-        elif isinstance(self.NROY, list):
-            NROY_str = (str(type(self.NROY)) + " of length " + len(self.NROY))
-
-        if self.RO is None:
-            RO_str = None
-        elif isinstance(self.RO, list):
-            RO_str = (str(type(self.RO)) + " of length " + len(self.RO))
-
-        return ("History Matching tools created with:\n" +
-                "Gaussian Process: {}\n" +
-                "Observations: {}\n" +
-                "Coords: {}\n" +
-                "Expectations: {}\n" +
-                "No. of Input Dimensions: {}\n" +
-                "No. of Descrete Expectation Values: {}\n" +
-                "I_threshold: {}\n" +
-                "I: {}\n" +
-                "NROY: {}\n" +
-                "RO: {}\n").format(self.gp, self.obs, coord_str, exp_str,
-                                   self.ndim, self.ncoords, self.threshold,
-                                   I_str, NROY_str, RO_str)
+import numpy as np
+from mogp_emulator.GaussianProcess import GaussianProcessBase, PredictResult
+
+class HistoryMatching(object):
+    r"""
+    Class containing tools to implement history matching between the outputs of a
+    gaussian process and an observation.
+
+    Primary usage is the ``get_implausibility`` method, which calculates the
+    implausibility metric (a number of standard deviations between the emulator mean
+    and the observation) for a number of query points:
+
+    .. math::
+        {I_i(\bar{x_0}) = \frac{|z_i - E(f_i(\bar{x_0}))|}
+        {\sqrt{Var\left[z_i - E(f_i(\bar{x_0}))\right]}}}
+
+    The code allows a number of ways to specify variances, and all are summed to
+    compute the final variance that determines the implausibility measure.
+    Variances on the observation itself can be provided with the observations,
+    while variances arising from the GP prediction are included in the emulator
+    expectations. Additional variances can be included to account for model
+    discrepancy when computing the implausibility metric, and these can be uniform
+    for all query points, or vary across different query points if desired. Any
+    number of additional variances can be included in the calculation by passing
+    them when computing the implausibility.
+
+    Once implausibility is computed, the class can determine query points that
+    can be ruled out, as well as points that are not ruled out yet, based on the
+    threshold class attribute. See the methods ``get_RO``, ``get_NROY``, and
+    ``set_threshold`` for more details.
+
+    Example - implausibility computation for a 1D GP::
+
+        import math
+        import numpy as np
+        from HistoryMatching import HistoryMatching
+        from mogp_emulator import GaussianProcess
+
+        # Set up a gaussian process
+        x_training = np.array([[0.],[10.],[20.],[30.],[43.],[50.]])
+        def get_y_training(x):
+            n_points = len(x)
+            f = np.zeros(n_points)
+            for i in range(n_points):
+                f[i] = np.sin(2*math.pi*x[i] / 50)
+            return f
+        y_training = get_y_training(x_training)
+
+        gp = GaussianProcess(x_training, y_training)
+        np.random.seed(47)
+        gp.learn_hyperparameters()
+
+        # Define the observation to which to compare
+        obs = [-0.8, 0.0004]
+
+        # Coordinates to predict
+        n_rand = 2000
+        coords = np.sort(np.random.rand(n_rand),axis=0)*(56)-3
+        coords = coords[:,None]
+
+        # Generate Expectations
+        expectations = gp.predict(coords)
+
+        # Calculate implausibility
+        hm = HistoryMatching(obs=obs, expectations=expectations)
+        I = hm.get_implausibility()
+
+
+    """
+
+    def __init__(self, gp=None, obs=None, coords=None, expectations=None, threshold=3.):
+        r"""
+        Create a new instance of history matching.
+
+        The primary function of this method is to initialise the ``self.gp``,
+        ``.obs``, ``.coords``, ``.expectations``, ``.ndim``, ``.ncoords``,
+        ``.threshold``, ``.I``, ``.NROY``, and ``.RO`` variables as placeholders.
+        The optional keyword arguments can also be used to set user-defined values for
+        various quantities in a single call to ``HistoryMatching()``. Setting sufficient
+        numbers of these to allow the number of dimensions and/or number of coordinates to
+        be computed also causes these to be set at this stage.
+
+        :param gp: (Optional) ``GaussianProcessBase`` object used to make predictions in
+                   history matching. Optional, can instead provide predictions directly
+                   via the ``expectations`` argument. Default is ``None``
+        :type gp: GaussianProcessBase or None
+        :param obs: (Optional) Observations against which the predictions will be
+                    compared. If provided, must either be a float (assumes no
+                    uncertainty in the observations) or a list of two floats
+                    holding the observation and its variance. Required for history
+                    matching, but can be provided when calling ``get_implausibility``.
+                    Default is ``None``.
+        :type obs: float, list, or None
+        :param coords: (Optional) Inputs at which the emulator values will be computed
+                       to compare the GP output to the observations. If provided, must
+                       be a numpy array matching the input for the GP (must be a 2D
+                       array with dimensions ``(n, D)`` where ``n`` is the number of
+                       points to be considered and ``D`` is the number of parameters
+                       for the GP). Only required if ``expectations`` is not provided.
+                       Default is ``None``.
+        :type coords: ndarray
+        :param expectations: (Optional) Tuple of 3 numpy arrays or 2 numpy arrays and
+                             ``None`` of the form expected from GP predictions. The first
+                             array must hold the predicted mean at all query points, and
+                             the second array must hold the variances from the GP predictions
+                             at all query points. The third is not used, so can simply be a
+                             dummy array or ``None``. Can instead provide a GP object and the
+                             points to query to have the predictions made internally. Default
+                             is ``None``.
+        :type expectations: tuple holding 3 ndarrays
+        """
+
+        # Place-holder values for user-defined quantities that can be passed in
+        # as args.
+        self.gp = None
+        self.obs = None
+        self.coords = None
+        self.expectations = None
+
+        # Place-holder or default values for optional or derived quantities.
+        self.ndim = None
+        self.ncoords = None
+        self.threshold = None
+        self.I = None
+        self.NROY = None
+        self.RO = None
+
+        # If suitable keyword arguments are provided, update the relevent
+        # variable(s)
+        if self.check_gp(gp):
+            self.set_gp(gp)
+        if self.check_obs(obs):
+            self.set_obs(obs)
+        if self.check_coords(coords):
+            self.set_coords(coords)
+        if self.check_expectations(expectations):
+            self.set_expectations(expectations)
+        if self.check_threshold(threshold):
+            self.set_threshold(threshold)
+
+        self.update()
+
+
+    def _select_expectations(self):
+        r"""
+        Determine the emulator predictions to use in HistoryMatching
+
+        This method queries the internal state of the HistoryMatching object to get
+        the expectations to use in the history matching calculation. It either
+        uses the expectations that were directly provided, or makes predictions
+        at the query points using the provided GP. If both methods are set
+        simultaneously, the method raises an exception.
+
+        :returns: Expectations to use in history matching. These are either provided
+                  explicitly, or made internally using the GP and query points.
+        :rtype: tuple of 3 ndarrays or 2 ndarrays and None
+        """
+        # Check that we have exactly 1 valid combination of parameters
+        UseCoordGP = False
+        UseExpectations = False
+        if (self.check_coords(self.coords) and self.check_gp(self.gp)):
+            UseCoordGP = True
+        if self.check_expectations(self.expectations):
+            UseExpectations = True
+        if UseCoordGP and UseExpectations:
+            raise ValueError("Multiple valid parameter combinations are set. Previously set " +
+                             "parameters can be removed by setting them to None")
+        if not UseCoordGP and not UseExpectations:
+            raise ValueError("Expectations are not provided, nor is a GP and coordinates." +
+                             "must set one in order to perform History Matching")
+
+        # Confirm that the ncoords parameter is set
+        if self.ncoords is None:
+            raise ValueError("ncoords is not set despite a valid parameter combination "+
+                             "being found.")
+
+        # From this point on, the calculation works off the self.expectations value.
+        # If this is not set, calculate it from the other parameters
+        if UseCoordGP:
+            expectations = self.gp.predict(self.coords)
+        else:
+            expectations = self.expectations
+
+        return expectations
+
+    def get_implausibility(self, discrepancy = 0.):
+        r"""
+        Compute Implausibility measure for all query points
+
+        Carries out the implausibility calculation given by:
+
+        .. math::
+            {I_i(\bar{x_0}) = \frac{|z_i - E(f_i(\bar{x_0}))|}
+            {\sqrt{Var\left[z_i - E(f_i(\bar{x_0}))\right]}}}
+
+        to return an implausibility value (a number of standard deviations between the
+        emulator mean and the observation)for each of the provided coordinates.
+
+        Requires that the observation parameter is set, and that at least one of the
+        following conditions are met:
+
+          a) The coords and gp parameters are set
+          b) The expectations parameter is set
+
+        Note that using the GP internally assumes that the standard prediction settings
+        will be used when making predictions. If the user wishes to have more control
+        over the prediction method (i.e. make the predictions from MCMC samples),
+        the user should explicitly pass ``expectations`` to the object.
+
+        An additional variance can be included that represents a general model
+        discrepancy that describes the prior beliefs regarding how well the model
+        matches reality. In practice, the model discrepancy is essential to have
+        predictions that are not overly confident, however it can be hard to estimate
+        (see further discussion in Brynjarsdottir and O\'Hagan, 2014).
+
+        As the implausibility calculation linearly sums variances, the result is
+        agnostic to the precise provenance of any of the included variances
+
+        :param discrepancy: Additional variance to be included in the implausibility
+                            calculation. Must be a non-negative float. Optional, default
+                            is ``0.``
+        :type discrepancy: float
+        :returns: Array holding implausibility metric for all query points accounting
+                  for all variances, ndarray of length ``(ncoords,)``
+        :rtype: ndarray
+        """
+
+        # Confirm that observation parameter is set
+        if not self.check_obs(self.obs):
+            raise ValueError("implausibility calculation requires that the observation value is " +
+                             "set. This can be done using the set_obs method.")
+
+        assert discrepancy >= 0., "Model discrepancy variance cannot be negative"
+
+        expectations = self._select_expectations()
+
+        # Compute implausibility for each expectation value
+        Vs = np.zeros(self.ncoords)
+        Vs += expectations[1]                         # variance on expectation
+        Vs += discrepancy                             # model discrepancy
+        Vs += self.obs[1]                             # variance on observation
+        self.I = (np.abs(self.obs[0] - expectations[0]) / np.sqrt(Vs))
+
+        return self.I
+
+
+    def get_NROY(self, discrepancy = 0.):
+        r"""
+        Return set of indices that are not yet ruled out
+
+        Returns a list of indices for ``self.I`` that correspond to entries that are not
+        yet ruled out. Points that are ruled out have an implausibility metric that
+        exceeds the threshold (can be set when initializing history matching or using
+        the ``set_threshold`` method). If the implausibility metric has not yet been
+        computed for the desired points, it is calculated. If a model discrepancy
+        is to be included, it can be passed here.
+
+        :param discrepancy: Additional variance to be included in the implausibility
+                            calculation. Must be a non-negative float. Optional, default
+                            is ``0.``
+        :type discrepancy: float
+        :returns: List of integer indices that have not yet been ruled out.
+        :rtype: list
+        """
+        if self.I is None:
+            self.get_implausibility(discrepancy)
+
+        self.NROY = list(np.where(self.I <= self.threshold)[0])
+
+        return self.NROY
+
+
+    def get_RO(self, discrepancy = 0.):
+        r"""
+        Return set of indices that have been ruled out
+
+        Returns a list of indices for ``self.I`` that correspond to entries that have
+        been ruled out. Points that are ruled out have an implausibility metric that
+        exceeds the threshold (can be set when initializing history matching or using
+        the ``set_threshold`` method). If the implausibility metric has not yet been
+        computed for the desired points, it is calculated. If a model discrepancy
+        is to be included, it can be passed here.
+
+        :param discrepancy: Additional variance to be included in the implausibility
+                            calculation. Must be a non-negative float. Optional, default
+                            is ``0.``
+        :type discrepancy: float
+        :returns: List of integer indices that have been ruled out.
+        :rtype: list
+        """
+        if self.I is None:
+            self.get_implausibility(discrepancy)
+
+        self.RO = list(np.where(self.I > self.threshold)[0])
+
+        return self.RO
+
+
+    def set_gp(self, gp):
+        r"""
+        Set the Gaussian Process to use with history matching
+
+        Sets the ``self.gp`` variable to the provided ``GaussianProcess`` argument.
+
+        :param gp: ``GaussianProcess`` object to use for history matching.
+        :type gp: GaussianProcess
+        :returns: None
+        """
+        if not self.check_gp(gp):
+            raise TypeError("bad input for set_gp - expects a GaussianProcess object.")
+        self.gp = gp
+
+
+    def set_obs(self, obs):
+        r"""
+        Set the observations to be used for history matching
+
+        Sets the ``self.obs`` variable to the provided ``obs`` argument. The object
+        must pass the ``check_obs`` requirements, meaning that it must be a
+        float (assumes that the observation has no error associated with it) or a
+        list containing two floats (representing an observation and its variance).
+        Note that the variance must be non-negative.
+
+        :param obs: Observations to be used for history matching. Must be a float
+                    or a list of two floats.
+        :type obs: float or list
+        :returns: None
+        """
+        if not self.check_obs(obs):
+            raise TypeError("bad input for set_obs")
+        if isinstance(obs, list):
+            if len(obs) == 1:
+                self.obs = [float(obs[0]), 0.]
+            else:
+                self.obs = [float(a) for a in obs]
+        else:
+            self.obs = [float(obs), 0.]
+
+
+    def set_coords(self, coords):
+        r"""
+        Set the query points to be used in history matching
+
+        Sets the ``self.coords`` variable to the provided query points ``coords`` if
+        it passes the ``check_coords`` requirements, or be ``None`` to remove a set
+        of existing query points. ``coords`` must be a numpy array matching the
+        inputs to the provided ``GaussianProcess`` object.
+
+        :param coords: Numpy array holding query points (array with shape ``(n, D)``,
+                       where ``n`` is the number of query points and ``D`` is the
+                       number of inputs to the emulator), or ``None``
+        :type coords: ndarray or None
+        :returns: None
+        """
+        # need to allow coords == None, as otherwise can't reset values
+        if not self.check_coords(coords) and (not coords is None):
+            raise TypeError("bad input for set_coords - expected coords in the form " +
+                            "of a list or 1D or 2D ndarray of numerical values")
+        if isinstance(coords, np.ndarray):
+            if len(coords.shape) == 1:
+                self.coords = np.reshape(coords, [-1, 1])
+            elif len(coords.shape) == 2:
+                self.coords = coords
+            else:
+                raise Exception("error in exception handling - an ndarray of >2 " +
+                                "dimensions somehow got through check_coords")
+        elif isinstance(coords, list):
+            self.coords = np.reshape(np.asarray(coords), [-1, 1])
+        elif coords is None:
+            self.coords = None
+        else:
+            raise TypeError("error in exception handling - an argument of illegal " +
+                            "type somehow got through check_coords")
+        self.update()
+
+
+    def set_expectations(self, expectations):
+        r"""
+        Set the expected output of the simulator to be used in history matching
+
+        Sets the ``self.expectations`` variable to the provided ``expectations``
+        argument if it passes the ``check_expectations`` requirements, or ``None``
+        can be used to remove an existing set of expectations. Expectations
+        must be a tuple of 3 numpy arrays, where the first holds the predicted
+        means for all query points and the second holds the predicted variances
+        for all query points. The third arrray is not used in the computation,
+        but is included as it is an expected output of the GP ``predict`` method.
+
+        :param expectations: GP predictions at all query points. Must be a tuple
+                             of 3 numpy arrays, or None to remove existing
+                             expectations.
+        :type expectations: tuple of 3 ndarrays or None
+        :returns: None
+        """
+        # need to allow expectations == None, as otherwise can't reset values
+        if not self.check_expectations(expectations) and (not expectations is None):
+            raise TypeError("bad input for set_expectations - expected a Tuple " +
+                            "of 3 ndarrays.")
+        self.expectations = expectations
+        self.update()
+
+
+    def set_threshold(self, threshold):
+        r"""
+        Set the threshold value for history matching
+
+        Sets the ``self.threshold variable`` to the provided threshold argument
+        if it  passes the ``check_threshold`` requirements. The threshold must
+        be a non-negative float.
+
+        :param threshold: New value for threshold, must be a non-negative float.
+        :type threshold: float
+        :returns: None
+        """
+        if not self.check_threshold(threshold):
+            raise TypeError("bad input for set_expectations - expected a float")
+        self.threshold = float(threshold)
+
+
+    def status(self):
+        r"""
+        Prints a summary of the current status of the class object.
+
+        Prints a summary of the current status of the class object, including the
+        values stored in ``self.gp``, ``.obs``, ``.ndim``, ``.ncoords``, and
+        ``.threshold``, and the shapes/sizes ofvalues stored in ``self.obs``,
+        ``.coords``, ``.expectations``, ``.I``, ``.NROY``, and ``.RO.``
+        No inputs, no return value.
+
+        :returns: None
+        """
+
+        print(str(self))
+
+
+    def check_gp(self, gp):
+        r"""
+        Checks if the provided argument is consistent with expectations for a GP.
+
+        Returns a boolean that is True if the provided quantity is consistent with
+        the requirements for a gaussian process, i.e. is of type ``GaussianProcessBase``.
+
+        :param gp: Input GP object to be checked.
+        :type gp: GaussianProcessBase
+        :returns: Boolean indicating if provided object is a ``GaussianProcess``
+        :rtype: bool
+        """
+        return isinstance(gp, GaussianProcessBase)
+
+
+    def check_obs(self, obs):
+        r"""
+        Checks if the provided argument is consistent with expectations for
+        observations.
+
+        Returns a boolean that is ``True`` if the provided quantity is consistent with
+        the requirements for the observation quantity, i.e. is a numerical value
+        that can be converted to a float, or a list of up to two such values. Also
+        checks if the provided variance is non-negative.
+
+        :param obs: Input for observations to be checked
+        :type obs: float or list-like
+        :returns: Boolean indicating if provided observations are acceptable
+        :rtype: bool
+        """
+
+        if obs is None:
+            return False
+        if isinstance(obs, list) or isinstance(obs, tuple) or isinstance(obs, np.ndarray):
+            if len(obs) > 2:
+                raise ValueError("bad input type for HistoryMatching - the specified " +
+                                 "observation parameter cannot contain more than 2 entries "+
+                                 "(value, [variance])")
+            if len(obs) <= 2:
+                try:
+                    test = [float(a) for a in obs]
+                except TypeError:
+                    raise TypeError("bad input type for HistoryMatching - the specified " +
+                                    "observation parameter must contain only numerical " +
+                                    "values")
+            if len(obs) == 2:
+                assert float(obs[1]) >= 0., "variance in observations cannot be negative"
+            return True
+        else:
+            try:
+                test = float(obs)
+            except (TypeError, ValueError):
+                raise TypeError("bad input type for HistoryMatching - the specified " +
+                                "observation parameter must contain only numerical values")
+            return True
+        return False
+
+
+    def check_coords(self, coords):
+        r"""
+        Checks if the provided argument is consistent with expectations for
+        coordinates.
+
+        Returns a boolean that is ``True`` if the provided quantity is consistent with
+        the requirements for the coordinates quantity, i.e. a ndarray of
+        fewer than 3 dimensions.
+
+        :param coords: Input to check for consistency with coordinates.
+        :type coords: ndarray
+        :returns: Boolean indicating if coords is consistent
+        :rtype: bool
+        """
+
+        if coords is None:
+            return False
+
+        if isinstance(coords, np.ndarray):
+            if len(coords.shape) <= 2:
+                return True
+        return False
+
+
+    def check_expectations(self, expectations):
+        r"""
+        Checks if the provided argument is consistent with expectations for
+        Gaussian Process Expectations.
+
+        Returns a boolean that is ``True`` if the provided quantity is consistent with
+        the output of the predict method of a GaussianProcess object, i.e. that it
+        is a ``GaussianProcess.PredictResult`` object with mean and variance defined.
+
+        :param expectations: Input to check for consistency with expectations
+        :type expectations: tuple of 3 numpy arrays or 2 numpy arrays and None
+        :returns: Boolean indicating if expectations is consistent
+        :rtype: bool
+        """
+        if expectations is None:
+            return False
+        if not isinstance(expectations, PredictResult):
+            return False
+        if not all((isinstance(expectations[0], np.ndarray),
+                    isinstance(expectations[1], np.ndarray),
+                    (isinstance(expectations[2], np.ndarray) or
+                     expectations[2] is None))):
+            raise TypeError("bad input type for HistoryMatching - expected expectation " +
+                            "values in the form of a PredictResults object with mean" +
+                            "and uncertainty set.")
+        if not expectations[0].shape == expectations[1].shape:
+            raise ValueError("bad input for HistoryMatching - mean and variance " +
+                             "expectations do not match")
+        assert np.all(expectations[1] >= 0.), "all variances must be non-negative"
+        return True
+
+
+    def check_threshold(self, threshold):
+        r"""
+        Check value of threshold
+
+        Checks if the provided argument is consistent with expectations for a
+        threshold value.
+
+        Returns a boolean that is ``True`` if the provided quantity is consistent with
+        the requirements for a threshold value, i.e. that it is a non-negative numerical
+        value.
+
+        :param threshold: threshold to be tested
+        :type threshold: float
+        :returns:  Boolean indicating if the provided argument is consistent with
+                   a threshold
+        :rtype: bool
+        """
+        if threshold is None:
+            return False
+        try:
+            test = float(threshold)
+            assert test >= 0., "threshold must be non-negative"
+            return True
+        except TypeError:
+            return False
+
+    def update(self):
+        r"""
+        Update History Matching object
+
+        Checks that sufficient information exists to compute ``ndim`` and/or ``ncoords``
+        and, if so, computes these values. This also serves to update these values
+        if the information on which they are based is changed. No inputs, no return
+        value.
+
+        :returns: None
+        """
+
+        if self.check_coords(self.coords):
+            self.ndim = self.coords.shape[1]
+            self.ncoords = self.coords.shape[0]
+        elif self.check_expectations(self.expectations):
+            self.ncoords = self.expectations[0].shape[0]
+
+
+    def __str__(self):
+        r"""
+        Returns string representation of ``HistoryMatching`` object
+
+        Returns a string representation of a ``HistoryMatching`` object,
+        used when printing information to the interpreter or when the
+        ``status`` method is called.
+
+        :returns: string representation of the HistoryMatching object.
+        :rtype: str
+        """
+
+        if self.coords is None:
+            coord_str = None
+        else:
+            coord_str = self.coords.shape
+
+        if self.expectations is None:
+            exp_str = None
+        else:
+            exp_str = (str(type(self.expectations)) + " of len " +
+                       str(len(self.expectations)) + " containing arrays of shape: " +
+                       str(self.expectations[0].shape))
+
+        if self.I is None:
+            I_str = None
+        elif isinstance(self.I, np.ndarray):
+            I_str = (str(type(self.I)) + " of shape " + str(self.I.shape))
+
+        if self.NROY is None:
+            NROY_str = None
+        elif isinstance(self.NROY, list):
+            NROY_str = (str(type(self.NROY)) + " of length " + len(self.NROY))
+
+        if self.RO is None:
+            RO_str = None
+        elif isinstance(self.RO, list):
+            RO_str = (str(type(self.RO)) + " of length " + len(self.RO))
+
+        return ("History Matching tools created with:\n" +
+                "Gaussian Process: {}\n" +
+                "Observations: {}\n" +
+                "Coords: {}\n" +
+                "Expectations: {}\n" +
+                "No. of Input Dimensions: {}\n" +
+                "No. of Descrete Expectation Values: {}\n" +
+                "I_threshold: {}\n" +
+                "I: {}\n" +
+                "NROY: {}\n" +
+                "RO: {}\n").format(self.gp, self.obs, coord_str, exp_str,
+                                   self.ndim, self.ncoords, self.threshold,
+                                   I_str, NROY_str, RO_str)
diff --git a/mogp_emulator/Kernel.py b/mogp_emulator/Kernel.py
index b9386ee0..1a6547dc 100644
--- a/mogp_emulator/Kernel.py
+++ b/mogp_emulator/Kernel.py
@@ -30,7 +30,7 @@ class Kernel(object):
     Note that the Kernel object just collates all of the methods together; the class itself
     does not hold any information on the data point or hyperparamters, which are passed
     directly to the appropriate methods. Thus, no information needs to be provided when
-    creating a new ``Kernal`` instance.
+    creating a new ``Kernel`` instance.
     """
     def __str__(self):
         r"""
@@ -116,10 +116,12 @@ def calc_r(self, x1, x2, params):
         Calculate distance between all pairs of points
 
         This method computes the scaled Euclidean distance between all pairs of points
-        in ``x1`` and ``x2``. Each component distance is multiplied by the corresponding
-        hyperparameter prior to summing and taking the square root. For example, if
+        in ``x1`` and ``x2``. Each component distance is multiplied by the exponential of
+        the corresponding hyperparameter, prior to summing and taking the square root.
+        For example, if
         ``x1 = [1.]``, ``x2`` = [2.], and ``params = [2., 2.]`` then ``calc_r`` would
-        return :math:`{\sqrt{2(1 - 2)^2}=\sqrt{2}}` as an array with shape ``(1,1)``.
+        return :math:`{\sqrt{exp(2)*(1 - 2)^2}=\sqrt{exp(2)}}`
+        as an array with shape ``(1,1)``.
 
         :param x1: First input array. Must be a 1-D or 2-D array, with the length of
                    the last dimension matching the last dimension of ``x2`` and
@@ -723,5 +725,3 @@ def __str__(self):
         :rtype: str
         """
         return "Matern 5/2 Kernel"
-
-
diff --git a/mogp_emulator/LibGPGPU.py b/mogp_emulator/LibGPGPU.py
new file mode 100644
index 00000000..939629a3
--- /dev/null
+++ b/mogp_emulator/LibGPGPU.py
@@ -0,0 +1,14 @@
+"""
+Wrapper for the optional GPU interface library, libgpgpu.so
+"""
+
+HAVE_LIBGPGPU = False
+try:
+    from libgpgpu import *
+    HAVE_LIBGPGPU = True
+except ModuleNotFoundError:
+    pass
+
+
+def gpu_usable():
+    return HAVE_LIBGPGPU and have_compatible_device()
diff --git a/mogp_emulator/MCMC.py b/mogp_emulator/MCMC.py
deleted file mode 100644
index 79225adf..00000000
--- a/mogp_emulator/MCMC.py
+++ /dev/null
@@ -1,261 +0,0 @@
-import numpy as np
-from numpy.linalg import LinAlgError
-from inspect import signature
-import warnings
-
-def MH_proposal(current_params, step_sizes):
-    """
-    Propose an MCMC step using a Metropolis-Hastings method
-    
-    Proposes the next point in an MCMC sampler using the Metropolis-Hastings method.
-    Inputs are the current point and a covariance matrix. The next point is drawn from
-    a multivariate normal distribution centered around the current point. The covariance
-    matrix must be a 2D ``n`` by ``n`` array, where ``n`` is the number of parameters, and
-    must be positive definite. Returns a 1D array of length ``n`` holding the new proposed
-    parameter values.
-    
-    :param current_params: Current value of the parameters. Must be a 1D array.
-    :type current_params: ndarray
-    :param step_sizes: Covariance matrix from which steps are drawn. Must be a 2D array
-                       with both dimensions the same length as ``current_params``, and
-                       must be positive definite.
-    :type step_sizes: ndarray
-    :returns: New value of parameters, a 1D array with the same length as ``current_params``
-    :rtype: ndarray
-    """
-    
-    current_params = np.array(current_params)
-    step_sizes = np.array(step_sizes)
-    
-    assert len(current_params.shape) == 1, "current parameters must be a 1D array"
-    assert len(step_sizes.shape) == 2, "step sizes must be a 2D array"
-    assert step_sizes.shape[0] == step_sizes.shape[1], "step sizes must be a square array"
-    assert len(current_params) == step_sizes.shape[0], "length of current parameters must match length of step sizes"
-    assert np.all(np.diag(step_sizes) > 0.), "step sizes must be a positive definite matrix"
-    
-    return np.random.multivariate_normal(mean=current_params, cov = step_sizes)
-
-
-def MCMC_step(loglikelihood, current_params, step_sizes, loglike_sign = 1.):
-    """
-    Method to take a weak prior Metropolis-Hastings MCMC step
-    
-    Take a Metropolis-Hastings MCMC step with given log-likelihood function and step sizes.
-    Method uses a multivariate normal distribution centered around the current parameter
-    values from which the next point is drawn, and evaluates the log-likelihood for both
-    points. If the next point has a larger log-likelihood, the step is always accepted,
-    and if the log-likelihood is less for the proposed step it is accepted with a
-    probability based on the difference between the two. If the method encounters an
-    error when evalutaing the log-likelihood, the step is rejected. Returns the next point
-    and a boolean indicating whether or not the step was accepted.
-    
-    An optional parameter ``loglike_sign`` can be passed that must be a float with the
-    value +/- 1. This is multiplied by the log-likelihood and thus allows methods that
-    compute the negative log-likelihood to be used in this routine. If values other than
-    +/- 1 are passed, the method will raise an error.
-    
-    :param loglikelihood: Log-likelihood function to be used in the MCMC step. Must be
-                          callable and must accept a single argument, which is the array
-                          holding the parameters. If this function computes the negative
-                          log-likelihood, pass ``loglike_sign = -1.`` to the function as well.
-    :type loglikelihood: function or other callable
-    :param current_params: Current value of the parameters. Must be a 1D array.
-    :type current_params: ndarray
-    :param step_sizes: Covariance matrix from which steps are drawn. Must be a 2D array
-                       with both dimensions the same length as ``current_params``, and
-                       must be positive definite.
-    :type step_sizes: ndarray
-    :param loglike_sign: Sign for the log-likelihood function. If the provided
-                         ``loglikelihood`` function computes the negative log-likelihood,
-                         pass ``-1.`` for this parameter. Optional, default value is ``1.``
-    :type loglike_sign: float
-    :returns: Proposed next point and whether or not the point is accepted as a tuple.
-              The first return item is the next point (as a 1D array with the same length\
-              as ``current_params``) and the second item is a boolean indicating whether
-              or not the step was accepted.
-    :rtype: tuple containing a ndarray and a bool
-    """
-    
-    assert callable(loglikelihood), "loglikelihood must be a callable function"
-    assert len(signature(loglikelihood).parameters) == 1
-    assert loglike_sign == 1. or loglike_sign == -1., "loglikelihood sign must be +/- 1"
-    
-    next_point = MH_proposal(current_params, step_sizes)
-
-    try:
-        H = loglike_sign*(-loglikelihood(current_params) + loglikelihood(next_point))
-    except (FloatingPointError, AssertionError, LinAlgError):
-        H = np.nan
-
-    if H >= np.log(np.random.random()) and np.isfinite(H):
-        accept = True
-    else:
-        accept = False
-
-    return next_point, accept
-
-def sample_MCMC(loglikelihood, start, step_sizes, n_samples = 1000, thin = 0, loglike_sign = 1.):
-    """
-    Draw MCMC samples for a given log-likelihood function with weak priors
-    
-    Compute an MCMC chain for a given log-likelihood function with weak priors. Function
-    requires the log-likelihood function, the starting point for the MCMC chain, and an
-    array describing the step sizes. Optional parameters are the number of steps to take,
-    how the thin the samples, and a sign for the log-likelihood function.
-    
-    The log-likelihood function must be a function or other callable that accepts a single
-    argument, which is a 1D array holding the current parameter values. The starting point
-    must be a 1D array holding the starting parameter values, which must match the length of
-    the input to the log-likelihood function. The step sizes must be a 2D array with each
-    dimension having the same length as the number of parameters, and must be positive
-    definite. The step size array is used as the covariance matrix for a multivariate
-    normal distribution, from which steps are drawn.
-    
-    Optional parameters are the number of steps to be taken (must be a positive integer).
-    Note that if the chain is thinned, the number of points in the final MCMC chain will
-    differ from the number of steps taken.
-    
-    Thinning may be specified with a non-negative integer. If a positive integer is
-    given, the chain will be thinned by only keeping every ``thin`` steps. Note that
-    ``thin = 1`` means that the chain will not be thinned. If ``thin = 0`` is given
-    (the default value), the chain will automatically be thinned by computing the
-    autocorrelation of the chain for each parameter separately and estimating the value
-    needed to eliminate correlations in the chain. If the autothinning method fails
-    (usually occurrs if the posterior is multimodal), the chain will not be thinned
-    and a warning will be given. More details on the autothinning procedure are
-    described in the corresponding function.
-    
-    An optional parameter ``loglike_sign`` can be passed that must be a float with the
-    value +/- 1. This is multiplied by the log-likelihood and thus allows methods that
-    compute the negative log-likelihood to be used in this routine.
-    
-    Returns the final thinned MCMC chain (a 2D array, where the first dimension indicates
-    the different samples and the second dimension indicates the different parameters),
-    an array holding all rejected steps (also a 2D array like the MCMC chain, useful for
-    diagnosing problems with convergence), the fraction of steps that are accepted (also
-    useful for diagnosing problems with convergence), and the first lag autocorrelation of
-    the thinned MCMC chain.
-    
-    :param loglikelihood: Log-likelihood function to be used in the MCMC step. Must be
-                          callable and must accept a single argument, which is the array
-                          holding the parameters. If this function computes the negative
-                          log-likelihood, pass ``loglike_sign = -1.`` to the function as well.
-    :type loglikelihood: function or other callable
-    :param start: Starting value of the parameters. Must be a 1D array.
-    :type start: ndarray
-    :param step_sizes: Covariance matrix from which steps are drawn. Must be a 2D array
-                       with both dimensions the same length as ``current_params``, and
-                       must be positive definite.
-    :type step_sizes: ndarray
-    :param n_samples: Number of steps to be taken. Must be a positive integer. Optional,
-                      default value is 1000. Note that if the chain is thinned, the
-                      final MCMC chain will be shorter than this.
-    :type n_samples: int
-    :param thin: Integer describing how to thin the MCMC chain. A positive integer
-                 indicates manual thinning by keeping every ``thin`` steps (note
-                 that ``thin = 1`` means the chain will not be thinned). A value
-                 of ``0`` will attempt to autothin the chain.
-    :type thin: int
-    :param loglike_sign: Sign for the log-likelihood function. If the provided
-                         ``loglikelihood`` function computes the negative log-likelihood,
-                         pass ``-1.`` for this parameter. Optional, default value is ``1.``
-    :type loglike_sign: float
-    :returns: MCMC chain (2D array), array of rejected points (2D array), acceptance rate
-              (float), and first lag autocorrelation of the thinned MCMC chain (float)
-    :rtype: tuple containing (ndarray, ndarray, float, float)
-    """
-
-    n_samples = int(n_samples)
-    thin = int(thin)
-
-    assert n_samples > 0, "number of samples must be a positive integer"
-    assert thin >= 0, "thin must be a non-negative integer"
-    
-    start = np.array(start)
-    
-    assert start.ndim == 1, "starting point must be a 1d array"
-    n_params = len(start)
-
-    samples = np.zeros((n_samples, n_params))
-    samples[0] = start
-    rejected = []
-
-    for i in range(n_samples - 1):
-        next_point, accept = MCMC_step(loglikelihood, samples[i], step_sizes, loglike_sign)
-        if accept:
-            samples[i+1] = np.copy(next_point)
-        else:
-            samples[i+1] = np.copy(samples[i])
-            rejected.append(np.array(next_point))
-        
-    acceptance = float(n_samples - len(rejected))/float(n_samples)
-
-    if thin == 0:
-        thin_freq = autothin_samples(samples)
-    else:
-        thin_freq = thin
-    
-    thinned = samples[::thin_freq]
-
-    first_lag = np.zeros(n_params)
-
-    if n_samples > 1:
-        for i in range(n_params):
-            autocorr = np.correlate(thinned[:,i]-np.mean(thinned[:,i]), thinned[:,i]-np.mean(thinned[:,i]), mode="full")
-            if np.max(autocorr) > 0.:
-                first_lag[i] = autocorr[np.argmax(autocorr)+1]/np.max(autocorr)
-
-    return thinned, np.array(rejected), acceptance, first_lag
-
-def autothin_samples(signal):
-    """
-    Automatically estimate thinning needed to obtain uncorrelated samples
-    
-    This function attempts to estimate the thinning needed to obtain uncorrelated samples in an
-    MCMC chain. For each separate parameter, the function computes the autocorrelation and
-    estimates the lag needed to obtain uncorrelated samples. This is done by recognizing that
-    the standard deviation of the autocorrelation of a random signal scales inversely with the
-    square root of the number of samples, so when the autocorrelation drops below three times
-    this, we use this as a guess to when the signal is uncorrelated. The maximum lag across all
-    parameters is returned. If the chain contains fewer than 10 points, or the autocorrelation
-    never drops below the target value, the method gives a warning and
-    returns 1.
-    
-    :param signal: MCMC chain to be thinned. Must be a 1D or 2D array. If 2D, the first dimension
-                   indicates the MCMC samples, and the second dimension indicates the different
-                   parameter values. If 1D, the array is assumed to hold samples of a single
-                   parameter, and the array is reshaped to have a singleton second dimension.
-    :type signal: ndarray
-    :returns: Estimated stride needed to thin the samples to obtain uncorrelated samples. If
-              the method does not succeed, gives a warning and returns ``1``.
-    :rtype: int
-    """
-    
-    signal = np.array(signal)
-    if signal.ndim == 1:
-        signal = np.reshape(signal, (len(signal), 1))
-
-    assert signal.ndim == 2, "input signal to be thinned must be a 1d or 2d array"
-    
-    n_samples, n_params = signal.shape
-    
-    maxthin = 0
-    
-    if n_samples >= 10:
-        for i in range(n_params):
-            autocorr = np.correlate(signal[:,i]-np.mean(signal[:,i]), signal[:,i]-np.mean(signal[:,i]), mode="full")
-            start = np.argmax(autocorr)
-            if np.max(autocorr) > 0.:
-                autocorr = autocorr/np.max(autocorr)
-            for j in range(1, len(autocorr) - start):
-                if np.abs(autocorr[start + j]) < 3./np.sqrt(float(n_samples)):
-                    if j > maxthin:
-                        maxthin = j
-                    break
-    
-    if maxthin == 0:
-        warnings.warn("automatic thinning failed, posterior distribution may be multimodal")
-        maxthin = 1
-        
-    return maxthin
-    
\ No newline at end of file
diff --git a/mogp_emulator/MultiOutputGP.py b/mogp_emulator/MultiOutputGP.py
index f5512f18..bb5fca4b 100644
--- a/mogp_emulator/MultiOutputGP.py
+++ b/mogp_emulator/MultiOutputGP.py
@@ -1,38 +1,55 @@
 from multiprocessing import Pool
 import platform
 import numpy as np
-from mogp_emulator.GaussianProcess import GaussianProcess, PredictResult
+from mogp_emulator.GaussianProcess import (
+    GaussianProcessBase,
+    GaussianProcess,
+    PredictResult
+)
+from mogp_emulator.GaussianProcessGPU import GaussianProcessGPU
 from mogp_emulator.MeanFunction import MeanBase
 from mogp_emulator.Kernel import Kernel, SquaredExponential, Matern52
 from mogp_emulator.Priors import Prior
 
+
 class MultiOutputGP(object):
-    """
-    Implementation of a multiple-output Gaussian Process Emulator.
+    """Implementation of a multiple-output Gaussian Process Emulator.
 
-    Essentially a parallelized wrapper for the predict method. To fit in parallel, use the fit_GP_MAP
-    routine
+    Essentially a parallelized wrapper for the predict method. To fit
+    in parallel, use the ``fit_GP_MAP`` routine
 
-    Required arguments are ``inputs`` and ``targets``, both of which must be numpy arrays. ``inputs``
-    can be 1D or 2D (if 1D, assumes second axis has length 1). ``targets`` can be 1D or 2D (if 2D,
+    Required arguments are ``inputs`` and ``targets``, both of which
+    must be numpy arrays. ``inputs`` can be 1D or 2D (if 1D, assumes
+    second axis has length 1). ``targets`` can be 1D or 2D (if 2D,
     assumes a single emulator and the first axis has length 1).
 
-    Optional arguments specify how each individual emulator is constructed, including the mean
-    function, kernel, priors, and how to handle the nugget. Each argument can take values allowed
-    by the base ``GaussianProcess`` class, in which case all emulators are assumed to use the
-    same value. Any of these arguments can alternatively be a list of values with length matching
-    the number of emulators to set those values individually.
+    Optional arguments specify how each individual emulator is
+    constructed, including the mean function, kernel, priors, and how
+    to handle the nugget. Each argument can take values allowed by the
+    base ``GaussianProcess`` class, in which case all emulators are
+    assumed to use the same value. Any of these arguments can
+    alternatively be a list of values with length matching the number
+    of emulators to set those values individually.
+
+    Additional keyword arguments include ``inputdict``, and
+    ``use_patsy``, which control how strings are parsed to mean
+    functions, if using.
 
-    Additional keyword arguments include ``inputdict``, and ``use_patsy``, which control how strings
-    are parsed to mean functions, if using.
     """
 
     def __init__(self, inputs, targets, mean=None, kernel=SquaredExponential(), priors=None,
-                 nugget="adaptive", inputdict={}, use_patsy=True):
+                 nugget="adaptive", inputdict={}, use_patsy=True, use_gpu=False):
         """
         Create a new multi-output GP Emulator
         """
 
+        # if use_gpu is selected, check whether we found the GPU .so file, and raise error if not
+        self.use_gpu = use_gpu
+        if self.use_gpu:
+            self.GPClass = GaussianProcessGPU
+        else:
+            self.GPClass = GaussianProcess
+
         # check input types and shapes, reshape as appropriate for the case of a single emulator
         inputs = np.array(inputs)
         targets = np.array(targets)
@@ -67,7 +84,7 @@ def __init__(self, inputs, targets, mean=None, kernel=SquaredExponential(), prio
         if issubclass(type(kernel), Kernel):
             kernel = self.n_emulators*[kernel]
 
-        assert isinstance(kernel, list), "kernel must be a Kernal subclass or a list of Kernel subclasses"
+        assert isinstance(kernel, list), "kernel must be a Kernel subclass or a list of Kernel subclasses"
         assert len(kernel) == self.n_emulators
 
         if priors is None:
@@ -88,76 +105,120 @@ def __init__(self, inputs, targets, mean=None, kernel=SquaredExponential(), prio
         assert isinstance(nugget, list), "nugget must be a string, float, or a list of strings and floats"
         assert len(nugget) == self.n_emulators
 
-        self.emulators = [ GaussianProcess(inputs, single_target, m, k, p, n, inputdict, use_patsy)
+        self.emulators = [ self.GPClass(inputs, single_target, m, k, p, n, inputdict, use_patsy)
                            for (single_target, m, k, p, n) in zip(targets, mean, kernel, priors, nugget)]
 
 
-    def predict(self, testing, unc=True, deriv=True, include_nugget=True, processes=None):
-        """
-        Make a prediction for a set of input vectors
-
-        Makes predictions for each of the emulators on a given set of input vectors. The
-        input vectors must be passed as a ``(n_predict, D)`` or ``(D,)`` shaped array-like
-        object, where ``n_predict`` is the number of different prediction points under
-        consideration and ``D`` is the number of inputs to the emulator. If the prediction
-        inputs array has shape ``(D,)``, then the method assumes ``n_predict == 1``.
-        The prediction points are passed to each emulator and the predictions are collected
-        into an ``(n_emulators, n_predict)`` shaped numpy array as the first return value
-        from the method.
-
-        Optionally, the emulator can also calculate the uncertainties in the predictions
-        (as a variance) and the derivatives with respect to each input parameter. If the
-        uncertainties are computed, they are returned as the second output from the method
-        as an ``(n_emulators, n_predict)`` shaped numpy array. If the derivatives are
-        computed, they are returned as the third output from the method as an
-        ``(n_emulators, n_predict, D)`` shaped numpy array. Finally, if uncertainties
-        are computed, the ``include_nugget`` flag determines if the uncertainties should
-        include the nugget. By default, this is set to ``True``.
-
-        As with the fitting, this computation can be done independently for each emulator
-        and thus can be done in parallel.
-
-        :param testing: Array-like object holding the points where predictions will be made.
-                        Must have shape ``(n_predict, D)`` or ``(D,)`` (for a single prediction)
+    def predict(self, testing, unc=True, deriv=True, include_nugget=True,
+                allow_not_fit=False, processes=None):
+        """Make a prediction for a set of input vectors
+
+        Makes predictions for each of the emulators on a given set of
+        input vectors. The input vectors must be passed as a
+        ``(n_predict, D)`` or ``(D,)`` shaped array-like object, where
+        ``n_predict`` is the number of different prediction points
+        under consideration and ``D`` is the number of inputs to the
+        emulator. If the prediction inputs array has shape ``(D,)``,
+        then the method assumes ``n_predict == 1``.  The prediction
+        points are passed to each emulator and the predictions are
+        collected into an ``(n_emulators, n_predict)`` shaped numpy
+        array as the first return value from the method.
+
+        Optionally, the emulator can also calculate the uncertainties
+        in the predictions (as a variance) and the derivatives with
+        respect to each input parameter. If the uncertainties are
+        computed, they are returned as the second output from the
+        method as an ``(n_emulators, n_predict)`` shaped numpy
+        array. If the derivatives are computed, they are returned as
+        the third output from the method as an ``(n_emulators,
+        n_predict, D)`` shaped numpy array. Finally, if uncertainties
+        are computed, the ``include_nugget`` flag determines if the
+        uncertainties should include the nugget. By default, this is
+        set to ``True``.
+
+        The ``allow_not_fit`` flag determines how the object handles
+        any emulators that do not have fit hyperparameter values
+        (because fitting presumably failed). By default,
+        ``allow_not_fit=False`` and the method will raise an error
+        if any emulators are not fit. Passing ``allow_not_fit=True``
+        will override this and ``NaN`` will be returned from any
+        emulators that have not been fit.
+
+        As with the fitting, this computation can be done
+        independently for each emulator and thus can be done in
+        parallel.
+
+        :param testing: Array-like object holding the points where
+                        predictions will be made.  Must have shape
+                        ``(n_predict, D)`` or ``(D,)`` (for a single
+                        prediction)
         :type testing: ndarray
-        :param unc: (optional) Flag indicating if the uncertainties are to be computed.
-                    If ``False`` the method returns ``None`` in place of the uncertainty
+        :param unc: (optional) Flag indicating if the uncertainties
+                    are to be computed.  If ``False`` the method
+                    returns ``None`` in place of the uncertainty
                     array. Default value is ``True``.
         :type unc: bool
-        :param deriv: (optional) Flag indicating if the derivatives are to be computed.
-                      If ``False`` the method returns ``None`` in place of the derivative
+        :param deriv: (optional) Flag indicating if the derivatives
+                      are to be computed.  If ``False`` the method
+                      returns ``None`` in place of the derivative
                       array. Default value is ``True``.
         :type deriv: bool
-        :param include_nugget: (optional) Flag indicating if the nugget should be included
-                               in the predictive variance. Only relevant if ``unc = True``.
-                               Default is ``True``.
+        :param include_nugget: (optional) Flag indicating if the
+                                nugget should be included in the
+                                predictive variance. Only relevant if
+                                ``unc = True``.  Default is ``True``.
         :type include_nugget: bool
-        :param processes: (optional) Number of processes to use when making the predictions.
-                          Must be a positive integer or ``None`` to use the number of
-                          processors on the computer (default is ``None``)
+        :param allow_not_fit: (optional) Flag that allows predictions
+                              to be made even if not all emulators have
+                              been fit. Default is ``False`` which
+                              will raise an error if any unfitted
+                              emulators are present.
+        :type allow_not_fit: bool
+        :param processes: (optional) Number of processes to use when
+                          making the predictions.  Must be a positive
+                          integer or ``None`` to use the number of
+                          processors on the computer (default is
+                          ``None``)
         :type processes: int or None
-        :returns: Tuple of numpy arrays holding the predictions, uncertainties, and derivatives,
-                  respectively. Predictions and uncertainties have shape ``(n_emulators, n_predict)``
-                  while the derivatives have shape ``(n_emulators, n_predict, D)``. If
-                  the ``do_unc`` or ``do_deriv`` flags are set to ``False``, then those arrays
-                  are replaced by ``None``.
-        :rtype: tuple
+        :returns: ``PredictResult`` object holding numpy arrays
+                  containing the predictions, uncertainties, and
+                  derivatives, respectively. Predictions and
+                  uncertainties have shape ``(n_emulators,
+                  n_predict)`` while the derivatives have shape
+                  ``(n_emulators, n_predict, D)``. If the ``do_unc``
+                  or ``do_deriv`` flags are set to ``False``, then
+                  those arrays are replaced by ``None``.
+        :rtype: PredictResult
+
         """
 
         testing = np.array(testing)
-        if testing.shape == (self.D,):
-            testing = np.reshape(testing, (1, self.D))
-        assert len(testing.shape) == 2, "testing must be a 2D array"
-        assert testing.shape[1] == self.D, "second dimension of testing must be the same as the number of input parameters"
+        if self.D == 1 and testing.ndim == 1:
+            testing = np.reshape(testing, (-1, 1))
+        elif testing.ndim == 1:
+            testing = np.reshape(testing, (1, len(testing)))
+        assert testing.ndim == 2, "testing must be a 2D array"
+
+        n_testing, D = np.shape(testing)
+        assert D == self.D, "second dimension of testing must be the same as the number of input parameters"
+
         if not processes is None:
             processes = int(processes)
             assert processes > 0, "number of processes must be a positive integer"
 
-        if platform.system() == "Windows":
-            predict_vals = [GaussianProcess.predict(gp, testing, unc, deriv, include_nugget) for gp in self.emulators]
+        if allow_not_fit:
+            predict_method = _gp_predict_default_NaN
+        else:
+            predict_method = self.GPClass.predict
+
+        if platform.system() == "Windows" or self.use_gpu:
+            predict_vals = [predict_method(gp, testing, unc, deriv, include_nugget)
+                            for gp in self.emulators]
         else:
             with Pool(processes) as p:
-                predict_vals = p.starmap(GaussianProcess.predict, [(gp, testing, unc, deriv, include_nugget) for gp in self.emulators])
+                predict_vals = p.starmap(predict_method,
+                                         [(gp, testing, unc, deriv, include_nugget)
+                                          for gp in self.emulators])
 
         # repackage predictions into numpy arrays
 
@@ -170,24 +231,123 @@ def predict(self, testing, unc=True, deriv=True, include_nugget=True, processes=
 
         return PredictResult(mean=predict_unpacked, unc=unc_unpacked, deriv=deriv_unpacked)
 
-    def __call__(self, testing):
+    def __call__(self, testing, process=None):
+        """Interface to predict means by calling the object
+
+        A MultiOutputGP object is callable, which makes predictions of
+        the mean only for a given set of inputs. Works similarly to
+        the same method of the base GP class.
         """
-        Interface to predict means by calling the object
 
-        A MultiOutputGP object is callable, which makes predictions of the mean only
-        for a given set of inputs. Works similarly to the same method of the base GP
-        class. Predictions are made in parallel using the number of available processors.
+        return self.predict(testing, unc=False, deriv=False, processes=processes)[0]
+
+
+    def get_indices_fit(self):
+        """Returns the indices of the emulators that have been fit
+
+        When a ``MultiOutputGP`` class is initialized, none of the
+        emulators are fit. Fitting is done by passing the object to an
+        external fitting routine, which modifies the object to fit the
+        hyperparameters. Any emulators where the fitting fails are
+        returned to the initialized state, and this method is used to
+        determine the indices of the emulators that have succesfully
+        been fit.
+
+        Returns a list of non-negative integers indicating the
+        indices of the emulators that have been fit.
+
+        :returns: List of integer indicies indicating the emulators
+                  that have been fit. If no emulators have been
+                  fit, returns an empty list.
+        :rtype: list of int
+
         """
 
-        return self.predict(testing, unc=False, deriv=False, processes=None)[0]
+        return [idx for (failed_fit, idx)
+                in zip([em.theta is None for em in self.emulators],
+                       list(range(len(self.emulators)))) if not failed_fit]
+
+    def get_indices_not_fit(self):
+        """Returns the indices of the emulators that have not been fit
+
+        When a ``MultiOutputGP`` class is initialized, none of the
+        emulators are fit. Fitting is done by passing the object to an
+        external fitting routine, which modifies the object to fit the
+        hyperparameters. Any emulators where the fitting fails are
+        returned to the initialized state, and this method is used to
+        determine the indices of the emulators that have not been fit.
+
+        Returns a list of non-negative integers indicating the
+        indices of the emulators that have not been fit.
+
+        :returns: List of integer indicies indicating the emulators
+                  that have not been fit. If all emulators have been
+                  fit, returns an empty list.
+        :rtype: list of int
 
-    def __str__(self):
         """
-        Returns a string representation of the model
 
-        :returns: A string representation of the model (indicates number of sub-components
-                  and array shapes)
-        :rtype: str
+        return [idx for (failed_fit, idx)
+                in zip([em.theta is None for em in self.emulators],
+                       list(range(len(self.emulators)))) if failed_fit]
+
+    def get_emulators_fit(self):
+        """Returns the emulators that have been fit
+
+        When a ``MultiOutputGP`` class is initialized, none of the
+        emulators are fit. Fitting is done by passing the object to an
+        external fitting routine, which modifies the object to fit the
+        hyperparameters. Any emulators where the fitting fails are
+        returned to the initialized state, and this method is used to
+        obtain a list of emulators that have successfully been fit.
+
+        Returns a list of ``GaussianProcess`` objects which have been
+        fit (i.e. those which have a current valid set of
+        hyperparameters).
+
+        :returns: List of ``GaussianProcess`` objects indicating the
+                  emulators that have been fit. If no emulators
+                  have been fit, returns an empty list.
+        :rtype: list of ``GaussianProcess`` objects
+
+        """
+
+        return [gpem for (failed_fit, gpem)
+                in zip([em.theta is None for em in self.emulators],
+                       self.emulators) if not failed_fit]
+
+    def get_emulators_not_fit(self):
+        """Returns the indices of the emulators that have not been fit
+
+        When a ``MultiOutputGP`` class is initialized, none of the
+        emulators are fit. Fitting is done by passing the object to an
+        external fitting routine, which modifies the object to fit the
+        hyperparameters. Any emulators where the fitting fails are
+        returned to the initialized state, and this method is used to
+        obtain a list of emulators that have not been fit.
+
+        Returns a list of ``GaussianProcess`` objects which have
+        not been fit (i.e. those which do not have a current set of
+        hyperparameters).
+
+        :returns: List of ``GaussianProcess`` objects indicating the
+                  emulators that have not been fit. If all emulators
+                  have been fit, returns an empty list.
+        :rtype: list of ``GaussianProcess`` objects
+
+        """
+
+        return [gpem for (failed_fit, gpem)
+                in zip([em.theta is None for em in self.emulators],
+                       self.emulators) if failed_fit]
+
+
+    def __str__(self):
+        """Returns a string representation of the model
+
+        :returns: A string representation of the model (indicates
+                  number of sub-components and array shapes) :rtype:
+                  str
         """
 
         return ("Multi-Output Gaussian Process with:\n"+
@@ -195,3 +355,70 @@ def __str__(self):
                  str(self.n)+" training examples\n"+
                  str(self.D)+" input variables")
 
+
+def _gp_predict_default_NaN(gp, testing, unc, deriv, include_nugget):
+    """Prediction method for GPs that defaults to NaN for unfit GPs
+
+    Wrapper function for the ``GaussianProcess`` predict method that
+    returns NaN if the GP is not fit. Allows ``MultiOutputGP`` objects
+    that do not have all emulators fit to still return predictions
+    with unfit emulator predictions replaced with NaN.
+
+    The first argument to this function is the GP that will be used
+    for prediction. All other arguments are the same as the
+    arguments for the ``predict`` method of ``GaussianProcess``.
+
+    :param gp: The ``GaussianProcess`` object (or related class) for
+               which predictions will be made.
+    :type gp: GaussianProcess
+    :param testing: Array-like object holding the points where
+                    predictions will be made.  Must have shape
+                    ``(n_predict, D)`` or ``(D,)`` (for a single
+                    prediction)
+    :type testing: ndarray
+    :param unc: (optional) Flag indicating if the uncertainties
+                are to be computed.  If ``False`` the method
+                returns ``None`` in place of the uncertainty
+                array. Default value is ``True``.
+    :type unc: bool
+    :param deriv: (optional) Flag indicating if the derivatives
+                  are to be computed.  If ``False`` the method
+                  returns ``None`` in place of the derivative
+                  array. Default value is ``True``.
+    :type deriv: bool
+    :param include_nugget: (optional) Flag indicating if the
+                           nugget should be included in the
+                           predictive variance. Only relevant if
+                           ``unc = True``.  Default is ``True``.
+    :type include_nugget: bool
+    :returns: Tuple of numpy arrays holding the predictions,
+              uncertainties, and derivatives,
+              respectively. Predictions and uncertainties have
+              shape ``(n_predict,)`` while the derivatives have
+              shape ``(n_predict, D)``. If the ``unc`` or
+              ``deriv`` flags are set to ``False``, then those
+              arrays are replaced by ``None``.
+    :rtype: tuple
+    """
+
+    assert isinstance(gp, GaussianProcessBase)
+
+    try:
+        return gp.predict(testing, unc, deriv, include_nugget)
+    except ValueError:
+
+        n_predict = testing.shape[0]
+
+        if unc:
+            unc_array = np.array([np.nan]*n_predict)
+        else:
+            unc_array = None
+
+        if deriv:
+            deriv_array = np.reshape(np.array([np.nan]*n_predict*gp.D),
+                                     (n_predict, gp.D))
+        else:
+            deriv_array = None
+
+        return PredictResult(mean =np.array([np.nan]*n_predict),
+                             unc=unc_array, deriv=deriv_array)
diff --git a/mogp_emulator/__init__.py b/mogp_emulator/__init__.py
index 370cffc5..b46c6790 100644
--- a/mogp_emulator/__init__.py
+++ b/mogp_emulator/__init__.py
@@ -1,6 +1,11 @@
 from .version import version as __version__
 
 from .GaussianProcess import GaussianProcess
+try:
+    from .GaussianProcessGPU import GaussianProcessGPU
+    from .GaussianProcessGPU import GPUUnavailableError
+except:
+    pass
 from .MultiOutputGP import MultiOutputGP
 from .fitting import fit_GP_MAP
 from .MeanFunction import MeanFunction
diff --git a/mogp_emulator/benchmarks/Makefile b/mogp_emulator/benchmarks/Makefile
index a82606ad..0240fe97 100644
--- a/mogp_emulator/benchmarks/Makefile
+++ b/mogp_emulator/benchmarks/Makefile
@@ -7,14 +7,15 @@
 help : Makefile
 	@sed -n 's/^##//p' $<
 
-## all        : Run unit tests and benchmarks
+## all        : Run all benchmarks
 .PHONY: all
 all: benchmarks
 
 ## benchmarks : Run the Branin, Rosenbrock (convergence), Tsunami (performance),
-##            : MICE, and History Matching benchmarks
+##            : Pivoted Cholesky, MICE, dimension reduction,
+##            : and History Matching benchmarks
 .PHONY: benchmarks
-benchmarks: branin tsunami rosenbrock mice mcmc gKDR histmatch
+benchmarks: branin tsunami rosenbrock pivot mice gKDR histmatch
 
 ## branin     : Run the 2D Branin tests of convergence. Produces the plots
 ##            : branin_2d_error.png and branin_2d_unc.png
@@ -41,16 +42,16 @@ mice: mice_error.png mice_unc.png
 .PHONY: histmatch
 histmatch: histmatch_1D.png histmatch_2D.png
 
-## mcmc       : Run the MCMC sampling benchmark
-##            : Produces the plot MCMC_histogram.png
-.PHONY: mcmc
-mcmc: MCMC_histogram.png
-
 ## gKDR       : Run the dimension reduction benchmark
 ##            : Produces the plot benchmark_kdr_GP_loss.pdf
 .PHONY: gKDR
 gKDR: benchmark_kdr_GP_loss.pdf
 
+## pivot      : Run the pivoted Cholesky decomposition benchmark
+##            : Produces the plots pivot_error.png and pivot_unc.png
+.PHONY: pivot
+pivot: pivot_error.png pivot_unc.png
+
 # commands above are shortcuts to the targets below
 
 branin_2d_unc.png branin_2d_error.png: benchmark_branin.py
@@ -71,5 +72,5 @@ histmatch_1D.png histmatch_2D.png: benchmark_historymatching.py
 benchmark_kdr_GP_loss.pdf benchmark_kdr_GP_loss.npy: benchmark_kdr_GP.py
 	python3 benchmark_kdr_GP.py
 
-MCMC_histogram.png: benchmark_MCMC.py
-	python3 benchmark_MCMC.py
+pivot_error.png pivot_unc.png: benchmark_pivot.py
+	python3 benchmark_pivot.py
diff --git a/mogp_emulator/benchmarks/benchmark_pivot.py b/mogp_emulator/benchmarks/benchmark_pivot.py
new file mode 100644
index 00000000..e92a1227
--- /dev/null
+++ b/mogp_emulator/benchmarks/benchmark_pivot.py
@@ -0,0 +1,194 @@
+'''This benchmark performs convergence tests using the pivoted
+Cholesky routines applied to the 2D Branin function. Details of the 2D
+Branin function can be found at
+https://www.sfu.ca/~ssurjano/branin.html. The code samples the Branin
+function using an increasing number of points, with a duplicate point
+added to make the matrix singular. When pivoting is not used, the
+algorithm is stabilized by adding a nugget term to the diagonal of the
+covariance matrix.  This degrades the performance of the emulator
+globally, despite the fact that the problem arises from a local
+problem in fitting the emulator. Pivoting ignores points that are too
+close to one another, ensuring that there is no loss of performance as
+the number of points increases.
+
+Note that this benchmark only covers relatively small designs. Tests
+have revealed that there are some stability issues when applying
+pivoting to larger numbers of inputs -- this appears to be due to the
+minimization algorithm, perhaps due to the fact that pivoting computes
+the inverse of a slightly different matrix which may influence the
+fitting algorithm performance. Care should thus be taken to examine
+the resulting performance when applying pivoting in practice. Future
+versions may implement other approaches to ensure that pivoting gives
+stable performance on a wide variety of input data.
+
+'''
+
+import numpy as np
+from mogp_emulator import GaussianProcess, fit_GP_MAP
+from mogp_emulator import MonteCarloDesign, LatinHypercubeDesign
+from scipy.stats import uniform
+try:
+    import matplotlib.pyplot as plt
+    makeplots = True
+except ImportError:
+    makeplots = False
+
+def branin_2d(x):
+    "2D Branin function, see https://www.sfu.ca/~ssurjano/branin.html for more information"
+    if np.array(x).shape == (2,):
+        x1, x2 = x
+    else:
+        assert len(np.array(x).shape) == 2
+        assert np.array(x).shape[1] == 2
+        x1 = x[:,0]
+        x2 = x[:,1]
+    a, b, c, r, s, t = 1., 5.1/4./np.pi**2, 5./np.pi, 6., 10., 1./8./np.pi
+    return a*(x2 - b*x1**2 + c*x1 - r)**2 + s*(1. - t)*np.cos(x1) + s
+
+
+f = branin_2d
+n_dim = 2
+design_space = [uniform(loc = -5., scale = 15.).ppf, uniform(loc = 0., scale = 15.).ppf]
+simulations = [5, 10, 15, 20, 25, 30]
+
+def generate_input_data(n_simulations, method = "random"):
+    "Generate random points x1 and x2 for evaluating the multivalued 2D Branin function"
+    
+    n_simulations = int(n_simulations)
+    assert(n_simulations > 0)
+    assert method == "random" or method == "lhd"
+    
+    if method == "random":
+        ed = MonteCarloDesign(design_space)
+    elif method == "lhd":
+        ed = LatinHypercubeDesign(design_space)
+    inputs = ed.sample(n_simulations)
+    return inputs
+    
+def generate_target_data(inputs):
+    "Generate target data for multivalued emulator benchmark"
+    
+    inputs = np.array(inputs)
+    assert len(inputs.shape) == 2
+    assert inputs.shape[1] == n_dim
+    n_simulations = inputs.shape[0]
+    
+    targets = f(inputs)
+        
+    return targets
+
+def generate_training_data(n_simulations):
+    "Generate n_simulations input data and add a duplicate point to make matrix singular"
+    
+    inputs = generate_input_data(n_simulations, method = "lhd")
+    targets = generate_target_data(inputs)
+    
+    inputs_new = np.zeros((inputs.shape[0] + 1, inputs.shape[1]))
+    targets_new = np.zeros(targets.shape[0] + 1)
+    
+    inputs_new[:-1, :] = np.copy(inputs)
+    targets_new[:-1] = np.copy(targets)
+    
+    inputs_new[-1,:] = np.copy(inputs[0,:])
+    targets_new[-1] = np.copy(targets[0])
+    
+    return inputs_new, targets_new
+    
+def generate_test_data(n_testing):
+    "Generate n_testing points for testing the accuracy of an emulator"
+    
+    testing = generate_input_data(n_testing, method = "random")
+    test_targets = generate_target_data(testing)
+    
+    return testing, test_targets
+    
+def run_model(n_simulations, n_testing):
+    "Generate training data, fit emulator, and test model accuracy on random points, returning RMSE"
+    
+    print('fitting GPs')
+    
+    # run LHD model
+    inputs, targets = generate_training_data(n_simulations)
+    
+    gp = GaussianProcess(inputs, targets, nugget="adaptive")
+    gp = fit_GP_MAP(gp)
+    
+    print("fitting pivoted GP")
+    
+    gp_pivot = GaussianProcess(inputs, targets, nugget="pivot")
+    gp_pivot = fit_GP_MAP(gp_pivot)
+    
+    print("making predictions")
+
+    testing, test_targets = generate_test_data(n_testing)
+    
+    norm_const = np.max(test_targets)-np.min(test_targets)
+    
+    test_vals_pivot, unc_pivot, deriv = gp_pivot.predict(testing, deriv = False, unc = True)
+    test_vals, unc, deriv = gp.predict(testing, deriv = False, unc = True)
+    
+    return (np.sqrt(np.sum((test_vals - test_targets)**2)/float(n_testing))/norm_const,
+            np.sqrt(np.sum(unc**2)/float(n_testing))/norm_const**2,
+            np.sqrt(np.sum((test_vals_pivot - test_targets)**2)/float(n_testing))/norm_const,
+            np.sqrt(np.sum(unc_pivot**2)/float(n_testing))/norm_const**2)
+
+def plot_model_errors(simulation_list, error, unc, error_pivot, unc_pivot, n_testing):
+    "Makes plot showing accuracy of emulator as a function of n_simulations"
+    
+    plt.figure(figsize=(4,3))
+    plt.semilogy(simulation_list, error_pivot,'-o', label = 'Pivot')
+    plt.semilogy(simulation_list, error,'-x', label = 'Nugget')
+    plt.xlabel('Number of design points')
+    plt.ylabel('Average prediction RMSE')
+    plt.legend()
+    plt.title('Error for '+str(n_testing)+' predictions')
+    plt.savefig('pivot_error.png',bbox_inches='tight')
+    
+    plt.figure(figsize=(4,3))
+    plt.semilogy(simulation_list, unc_pivot,'-o', label = "Pivot")
+    plt.semilogy(simulation_list, unc,'-x', label = 'Nugget')
+    plt.xlabel('Number of design points')
+    plt.ylabel('Average prediction variance')
+    plt.legend()
+    plt.title('Uncertainty for '+str(n_testing)+' predictions')
+    plt.savefig('pivot_unc.png',bbox_inches='tight')
+    
+def run_all_models(n_testing, simulation_list, n_iter = 10):
+    "Runs all models, printing out results and optionally making plots"
+    
+    n_simtrials = len(simulation_list)
+    
+    errors = np.zeros((n_simtrials, n_iter))
+    uncs = np.zeros((n_simtrials, n_iter))
+    errors_pivot = np.zeros((n_simtrials, n_iter))
+    uncs_pivot = np.zeros((n_simtrials, n_iter))
+    
+    for iteration in range(n_iter):
+        for sim_index in range(n_simtrials):
+            print(sim_index, iteration)
+            (errors[sim_index, iteration],
+             uncs[sim_index, iteration],
+             errors_pivot[sim_index, iteration],
+             uncs_pivot[sim_index, iteration]) = run_model(simulation_list[sim_index], n_testing)
+    
+    error = np.mean(errors, axis = -1)
+    unc = np.mean(uncs, axis = -1)
+    error_pivot = np.mean(errors_pivot, axis = -1)
+    unc_pivot = np.mean(uncs_pivot, axis = -1)
+    
+    print("\n")
+    print("Convergence test results:")
+    print("Num. design points  RMSE Nugget         RMSE Pivot")
+    for sim, err, pivot_err in zip(simulation_list, error, error_pivot):
+        print('{:19} {:19} {:19}'.format(str(sim), str(err), str(pivot_err)))
+        
+    print("\n")
+    print("Num. design points  Variance Nugget     Variance Pivot")
+    for sim, un, pivot_un in zip(simulation_list, unc, unc_pivot):
+        print('{:19} {:19} {:19}'.format(str(sim), str(un), str(pivot_un)))
+    
+    if makeplots:
+        plot_model_errors(simulation_list, error, unc, error_pivot, unc_pivot, n_testing)
+    
+if __name__ == '__main__':
+    run_all_models(100, [int(x) for x in simulations], n_iter = 10)
diff --git a/mogp_emulator/demos/gp_demo_gpu.py b/mogp_emulator/demos/gp_demo_gpu.py
new file mode 100644
index 00000000..edc6c773
--- /dev/null
+++ b/mogp_emulator/demos/gp_demo_gpu.py
@@ -0,0 +1,65 @@
+import numpy as np
+import mogp_emulator
+from projectile import simulator, print_results
+
+# GP example using the projectile demo on a GPU
+
+# To run this demo you must be on a machine with an Nvidia GPU, and with
+# CUDA libraries available.  There are also dependencies on eigen and pybind11
+# If you are working on a managed cluster, these may be available via commands
+#
+# module load cuda/11.2
+# module load py-pybind11-2.2.4-gcc-5.4.0-tdtz6iq
+# module load gcc/7
+# module load eigen
+#
+# You should then be able to compile the cuda code at the same time as installing the mogp_emulator package, by doing (from the main mogp_emulator/ directory:
+# pip install .
+# (note that if you don't have write access to the global directory
+# (e.g. if you are on a cluster such as CSD3), you should add the
+# `--user` flag to this command)
+
+
+# define some common variables
+
+n_samples = 20
+n_preds = 10
+
+# Experimental design -- requires a list of parameter bounds if you would like to use
+# uniform distributions. If you want to use different distributions, you
+# can use any of the standard distributions available in scipy to create
+# the appropriate ppf function (the inverse of the cumulative distribution).
+# Internally, the code creates the design on the unit hypercube and then uses
+# the distribution to map from [0,1] to the real parameter space.
+
+ed = mogp_emulator.LatinHypercubeDesign([(-5., 1.), (0., 1000.)])
+
+# sample space
+
+inputs = ed.sample(n_samples)
+
+# run simulation
+
+targets = np.array([simulator(p) for p in inputs])
+
+###################################################################################
+
+# Basic example -- fit GP using MLE and Squared Exponential Kernel and predict
+
+print("Example: Basic GP")
+
+# create GP and then fit using MLE
+# the only difference between this and the standard CPU implementation
+# is to use the GaussianProcessGPU class rather than GaussianProcess.
+
+gp = mogp_emulator.GaussianProcessGPU(inputs, targets)
+
+gp = mogp_emulator.fit_GP_MAP(gp)
+
+# create 20 target points to predict
+
+predict_points = ed.sample(n_preds)
+
+means, variances, derivs = gp.predict(predict_points)
+
+print_results(predict_points, means)
diff --git a/mogp_emulator/demos/historymatch_demos.py b/mogp_emulator/demos/historymatch_demos.py
new file mode 100644
index 00000000..89925720
--- /dev/null
+++ b/mogp_emulator/demos/historymatch_demos.py
@@ -0,0 +1,100 @@
+import mogp_emulator
+import numpy as np
+
+# simple History Matching example
+
+# simulator function -- needs to take a single input and output a single number
+
+def f(x):
+    return np.exp(-np.sum((x-2.)**2, axis = -1)/2.)
+
+# Experimental design -- requires a list of parameter bounds if you would like to use
+# uniform distributions. If you want to use different distributions, you
+# can use any of the standard distributions available in scipy to create
+# the appropriate ppf function (the inverse of the cumulative distribution).
+# Internally, the code creates the design on the unit hypercube and then uses
+# the distribution to map from [0,1] to the real parameter space.
+
+ed = mogp_emulator.LatinHypercubeDesign([(0., 5.), (0., 5.)])
+
+# sample space, use many samples to ensure we get a good emulator
+
+inputs = ed.sample(50)
+
+# run simulation
+
+targets = np.array([f(p) for p in inputs])
+
+# Example observational data is a single number plus an uncertainty.
+# In this case we use a number close to 1, which should have a corresponding
+# input close to (2,2) after performing history matching
+
+###################################################################################
+
+# First step -- fit GP using MLE and Squared Exponential Kernel
+
+gp = mogp_emulator.GaussianProcess(inputs, targets)
+
+gp = mogp_emulator.fit_GP_MAP(gp)
+
+###################################################################################
+
+# First Example: Use HistoryMatching class to make the predictions
+
+print("Example 1: Make predictions with HistoryMatching object")
+
+# create HistoryMatching object, set threshold to be low to make printed output
+# easier to read
+
+threshold = 0.01
+hm = mogp_emulator.HistoryMatching(threshold=threshold)
+
+# For this example, we set the observations, GP, and the coordinates
+# observations is either a single float (the value) or two floats (value and
+# uncertainty as a variance)
+
+obs = [1., 0.08]
+hm.set_obs(obs)
+hm.set_gp(gp)
+
+# set coordinates of GP object where we will test if the points can plausbily
+# explain the data here we use our existing experimental design, but sample
+# 10000 points
+
+coords = ed.sample(10000)
+hm.set_coords(coords)
+
+# calculate implausibility metric
+
+implaus = hm.get_implausibility()
+
+# print points that we have not ruled out yet:
+
+for p, im in zip(coords[hm.get_NROY()], implaus[hm.get_NROY()]):
+    print("Sample point: {}      Implausibility: {}".format(p, im))
+
+###################################################################################
+
+# Second Example: Pass external GP predictions and add model discrepancy
+
+print("Example 2: External Predictions and Model Discrepancy")
+
+# use gp to make predictions on 10000 new points externally
+
+coords = ed.sample(10000)
+
+expectations = gp.predict(coords)
+
+# now create HistoryMatching object with these new parameters
+
+hm_extern = mogp_emulator.HistoryMatching(obs=obs, expectations=expectations,
+                                          threshold=threshold)
+
+# calculate implausibility, adding a model discrepancy (as a variance)
+
+implaus_extern = hm_extern.get_implausibility(0.1)
+
+# print points that we have not ruled out yet:
+
+for p, im in zip(coords[hm_extern.get_NROY()], implaus_extern[hm_extern.get_NROY()]):
+    print("Sample point: {}      Implausibility: {}".format(p, im))
diff --git a/mogp_emulator/demos/kdr_demos.py b/mogp_emulator/demos/kdr_demos.py
new file mode 100644
index 00000000..24c94333
--- /dev/null
+++ b/mogp_emulator/demos/kdr_demos.py
@@ -0,0 +1,81 @@
+import mogp_emulator
+import numpy as np
+
+# simple Dimension Reduction examples
+
+# simulator function -- returns a single "important" dimension from
+# at least 4 inputs
+
+def f(x):
+    return (x[0]-x[1]+2.*x[3])/3.
+
+# Experimental design -- create a design with 5 input parameters
+# all uniformly distributed over [0,1].
+
+ed = mogp_emulator.LatinHypercubeDesign(5)
+
+# sample space
+
+inputs = ed.sample(100)
+
+# run simulation
+
+targets = np.array([f(p) for p in inputs])
+
+###################################################################################
+
+# First example -- dimension reduction given a specified number of dimensions
+# (note that in real life, we do not know that the underlying simulation only
+# has a single dimension)
+
+print("Example 1: Basic Dimension Reduction")
+
+# create DR object with a single reduced dimension (K = 1)
+
+dr = mogp_emulator.gKDR(inputs, targets, K=1)
+
+# use it to create GP
+
+gp = mogp_emulator.fit_GP_MAP(dr(inputs), targets)
+
+# create 5 target points to predict
+
+predict_points = ed.sample(5)
+predict_actual = np.array([f(p) for p in predict_points])
+
+means = gp(dr(predict_points))
+
+for pp, m, a in zip(predict_points, means, predict_actual):
+    print("Target point: {} Predicted mean: {} Actual mean: {}".format(pp, m, a))
+
+###################################################################################
+
+# Second Example: Estimate dimensions from data
+
+print("Example 2: Estimate the number of dimensions from the data")
+
+# Use the tune_parameters method to use cross validation to create DR object
+# Note this is more realistic than the above as it does not know the
+# number of dimensions in advance
+
+dr_tuned, loss = mogp_emulator.gKDR.tune_parameters(inputs, targets,
+                                                    mogp_emulator.fit_GP_MAP,
+                                                    cXs=[3.], cYs=[3.])
+
+# Get number of inferred dimensions (usually gives 2)
+
+print("Number of inferred dimensions is {}".format(dr_tuned.K))
+
+# use object to create GP
+
+gp_tuned = mogp_emulator.fit_GP_MAP(dr_tuned(inputs), targets)
+
+# create 10 target points to predict
+
+predict_points = ed.sample(5)
+predict_actual = np.array([f(p) for p in predict_points])
+
+means = gp_tuned(dr_tuned(predict_points))
+
+for pp, m, a in zip(predict_points, means, predict_actual):
+    print("Target point: {} Predicted mean: {} Actual mean: {}".format(pp, m, a))
diff --git a/mogp_emulator/fitting.py b/mogp_emulator/fitting.py
index 09cbb5c0..09cf2a37 100644
--- a/mogp_emulator/fitting.py
+++ b/mogp_emulator/fitting.py
@@ -2,83 +2,149 @@
 import scipy.stats
 from scipy.linalg import LinAlgError
 from scipy.optimize import minimize
-from mogp_emulator.GaussianProcess import GaussianProcess
-from mogp_emulator.MultiOutputGP import MultiOutputGP
 from multiprocessing import Pool
 from functools import partial
 import platform
 
-def fit_GP_MAP(*args, n_tries=15, theta0=None, method="L-BFGS-B", **kwargs):
-    """
-    Fit one or more Gaussian Processes by attempting to minimize the negative log-posterior
-
-    Fits the hyperparameters of one or more Gaussian Processes by attempting to minimize
-    the negative log-posterior multiple times from a given starting location and using
-    a particular minimization method. The best result found among all of the attempts is
-    returned, unless all attempts to fit the parameters result in an error (see below).
-
-    The arguments to the method can either be an existing ``GaussianProcess`` or
-    ``MultiOutputGP`` instance, or a list of arguments to be passed to the ``__init__``
-    method of ``GaussianProcess`` or ``MultiOutputGP`` if more than one output is detected.
-    Keyword arguments for creating a new ``GaussianProcess`` or ``MultiOutputGP`` object can
-    either be passed as part of the ``*args`` list or as keywords (if present in ``**kwargs``, they
-    will be extracted and passed separately to the ``__init__`` method).
-
-    If the method encounters an overflow (this can result because the parameter values stored are
-    the logarithm of the actual hyperparameters to enforce positivity) or a linear algebra error
-    (occurs when the covariance matrix cannot be inverted, even with additional noise added along
-    the diagonal if adaptive noise was selected), the iteration is skipped. If all attempts to find
-    optimal hyperparameters result in an error, then the method raises an exception.
-
-    The ``theta0`` parameter is the point at which the first iteration will start. If more than
-    one attempt is made, subsequent attempts will use random starting points. If you are fitting
-    Multiple Outputs, then this argument can take any of the following forms: (1) None (random
-    start points for all emulators), (2) a list of numpy arrays or ``NoneTypes`` with length
-    ``n_emulators``, (3) a numpy array of shape ``(n_params,)`` or ``(n_emulators, n_params)``
-    which with either use the same start point for all emulators or the specified start
-    point for all emulators. Note that if you us a numpy array, all emulators must have the
-    same number of parameters, while using a list allows more flexibility.
-
-    The user can specify the details of the minimization method, using any of the gradient-based
-    optimizers available in ``scipy.optimize.minimize``. Any additional parameters beyond the method
-    specification can be passed as keyword arguments.
-
-    The function returns a fit ``GaussianProcess`` or ``MultiOutputGP`` instance, either the original
-    one passed to the function, or the new one created from the included arguments.
-
-    :param ``*args``: Either a single ``GaussianProcess`` or ``MultiOutputGP`` instance,
-                      or arguments to be passed to the ``__init__`` method when creating a new
-                      ``GaussianProcess`` or ``MultiOutputGP`` instance.
-    :param n_tries: Number of attempts to minimize the negative log-posterior function.
-                    Must be a positive integer (optional, default is 15)
+from mogp_emulator.GaussianProcess import GaussianProcessBase, GaussianProcess
+from mogp_emulator.MultiOutputGP import MultiOutputGP
+
+
+def fit_GP_MAP(*args, n_tries=15, theta0=None, method="L-BFGS-B",
+               skip_failures=True, refit=False, **kwargs):
+    """Fit one or more Gaussian Processes by attempting to minimize the
+    negative log-posterior
+
+    Fits the hyperparameters of one or more Gaussian Processes by
+    attempting to minimize the negative log-posterior multiple times
+    from a given starting location and using a particular minimization
+    method. The best result found among all of the attempts is
+    returned, unless all attempts to fit the parameters result in an
+    error (see below).
+
+    The arguments to the method can either be an existing
+    ``GaussianProcess`` or ``MultiOutputGP`` instance, or a list of
+    arguments to be passed to the ``__init__`` method of
+    ``GaussianProcess`` or ``MultiOutputGP`` if more than one output
+    is detected.  Keyword arguments for creating a new
+    ``GaussianProcess`` or ``MultiOutputGP`` object can either be
+    passed as part of the ``*args`` list or as keywords (if present in
+    ``**kwargs``, they will be extracted and passed separately to the
+    ``__init__`` method).
+
+    If the method encounters an overflow (this can result because the
+    parameter values stored are the logarithm of the actual
+    hyperparameters to enforce positivity) or a linear algebra error
+    (occurs when the covariance matrix cannot be inverted, even with
+    additional noise added along the diagonal if adaptive noise was
+    selected), the iteration is skipped. If all attempts to find
+    optimal hyperparameters result in an error, then the emulator is
+    skipped and the parameters are reset to ``None``. By default, a
+    warning will be printed to the console if this occurs for
+    ``MultiOutputGP`` fitting, while an error will be raise for
+    ``GaussianProcess`` fitting. This behavior can be changed to raise
+    an error for ``MultiOutputGP`` fitting by passing the kwarg
+    ``skip_failures=False``. This default behavior is chosen because
+    ``MultiOutputGP`` fitting is often done in a situation where human
+    review of all fit emulators is not possible, so the fitting
+    routine skips over failures and then flags those that failed for
+    further review.
+
+    For ``MultiOutputGP`` fitting, by default the routine assumes that
+    only GPs that currently do not have hyperparameters fit need to be
+    fit. This behavior is controlled by the ``refit`` keyword, which
+    is ``False`` by default. To fit all emulators regardless of their
+    current fitting status, pass ``refit=True``. The ``refit``
+    argument has no effect on fitting of single ``GaussianProcess``
+    objects -- standard ``GaussianProcess`` objects will be fit
+    regardless of the current value of the hyperparameters.
+
+    The ``theta0`` parameter is the point at which the first iteration
+    will start. If more than one attempt is made, subsequent attempts
+    will use random starting points. If you are fitting Multiple
+    Outputs, then this argument can take any of the following forms:
+    (1) None (random start points for all emulators), (2) a list of
+    numpy arrays or ``NoneTypes`` with length ``n_emulators``, (3) a
+    numpy array of shape ``(n_params,)`` or ``(n_emulators,
+    n_params)`` which with either use the same start point for all
+    emulators or the specified start point for all emulators. Note
+    that if you us a numpy array, all emulators must have the same
+    number of parameters, while using a list allows more flexibility.
+
+    The user can specify the details of the minimization method, using
+    any of the gradient-based optimizers available in
+    ``scipy.optimize.minimize``. Any additional parameters beyond the
+    method specification can be passed as keyword arguments.
+
+    The function returns a fit ``GaussianProcess`` or
+    ``MultiOutputGP`` instance, either the original one passed to the
+    function, or the new one created from the included arguments.
+
+    :param ``*args``: Either a single ``GaussianProcess`` or
+                      ``MultiOutputGP`` instance, or arguments to be
+                      passed to the ``__init__`` method when creating
+                      a new ``GaussianProcess`` or ``MultiOutputGP``
+                      instance.
+    :param n_tries: Number of attempts to minimize the negative
+                    log-posterior function.  Must be a positive
+                    integer (optional, default is 15)
     :type n_tries: int
-    :param theta0: Initial starting point for the first iteration. If present, must be
-                   array-like with shape ``(n_params,)`` based on the specific
-                   ``GaussianProcess`` being fit. If a ``MultiOutputGP`` is being fit
-                   it must be a list of length ``n_emulators`` with each entry as either
-                   ``None`` or a numpy array of shape ``(n_params,)``, or a numpy array
-                   with shape ``(n_emulators, n_params)`` (note that if the various emulators
-                   have different numbers of parameters, the numpy array option will not work).
-                   If ``None`` is given, then a random value is chosen. (Default is ``None``)
+    :param theta0: Initial starting point for the first iteration. If
+                   present, must be array-like with shape
+                   ``(n_params,)`` based on the specific
+                   ``GaussianProcess`` being fit. If a
+                   ``MultiOutputGP`` is being fit it must be a list of
+                   length ``n_emulators`` with each entry as either
+                   ``None`` or a numpy array of shape ``(n_params,)``,
+                   or a numpy array with shape ``(n_emulators,
+                   n_params)`` (note that if the various emulators
+                   have different numbers of parameters, the numpy
+                   array option will not work).  If ``None`` is given,
+                   then a random value is chosen. (Default is
+                   ``None``)
     :type theta0: None or ndarray
-    :param method: Minimization method to be used. Can be any gradient-based optimization
-                   method available in ``scipy.optimize.minimize``. (Default is ``'L-BFGS-B'``)
+    :param method: Minimization method to be used. Can be any
+                   gradient-based optimization method available in
+                   ``scipy.optimize.minimize``. (Default is
+                   ``'L-BFGS-B'``)
     :type method: str
-    :param ``**kwargs``: Additional keyword arguments to be passed to ``GaussianProcess.__init__``,
-                         ``MultiOutputGP.__init__``, or the minimization routine. Relevant parameters
-                         for the GP classes are automatically split out from those used in the
-                         minimization function. See available parameters in the corresponding functions
-                         for details.
+    :param skip_failures: Boolean controlling how to handle failures
+                          in ``MultiOutputGP`` fitting. If set to
+                          ``True``, emulator fits will fail silently
+                          without raising an error and provide
+                          information on the emulators that failed
+                          and the end of fitting. If ``False``, any
+                          failed fit will raise a ``RuntimeError``.
+                          Has no effect on fitting a single
+                          ``GaussianProcess``, which will always
+                          raise an error. Optional, default is
+                          ``True``.
+    :type skip_failures: bool
+    :param refit: Boolean indicating if previously fit emulators
+                  for ``MultiOutputGP`` objects should be fit again.
+                  Optional, default is ``False``. Has no effect on
+                  ``GaussianProcess`` fitting, which will be fit
+                  irrespective of the current hyperparameter values.
+    :type refit: bool
+    :param ``**kwargs``: Additional keyword arguments to be passed to
+                         ``GaussianProcess.__init__``,
+                         ``MultiOutputGP.__init__``, or the
+                         minimization routine. Relevant parameters for
+                         the GP classes are automatically split out
+                         from those used in the minimization
+                         function. See available parameters in the
+                         corresponding functions for details.
     :returns: Fit GP or Multi-Output GP instance
     :rtype: GaussianProcess or MultiOutputGP
+
     """
 
     if len(args) == 1:
         gp = args[0]
         if isinstance(gp, MultiOutputGP):
-            return _fit_MOGP_MAP(gp, n_tries, theta0, method, **kwargs)
-        elif isinstance(gp, GaussianProcess):
-            return _fit_single_GP_MAP(gp, n_tries, theta0, method, **kwargs)
+            gp = _fit_MOGP_MAP(gp, n_tries, theta0, method, refit, **kwargs)
+        elif isinstance(gp, GaussianProcessBase):
+            gp = _fit_single_GP_MAP(gp, n_tries, theta0, method, **kwargs)
         else:
             raise TypeError("single arg to fit_GP_MAP must be a GaussianProcess or MultiOutputGP instance")
     elif len(args) < 2:
@@ -91,21 +157,39 @@ def fit_GP_MAP(*args, n_tries=15, theta0=None, method="L-BFGS-B", **kwargs):
                 del kwargs[key]
         try:
             gp = GaussianProcess(*args, **gp_kwargs)
-            return _fit_single_GP_MAP(gp, n_tries, theta0, method, **kwargs)
+            gp = _fit_single_GP_MAP(gp, n_tries, theta0, method, **kwargs)
         except AssertionError:
-            gp = MultiOutputGP(*args, **gp_kwargs)
-            return _fit_MOGP_MAP(gp, n_tries, theta0, method, **kwargs)
+            try:
+                gp = MultiOutputGP(*args, **gp_kwargs)
+                gp =  _fit_MOGP_MAP(gp, n_tries, theta0, method, **kwargs)
+            except AssertionError:
+                raise ValueError("Bad values for *args in fit_GP_MAP")
+
+    if isinstance(gp, GaussianProcessBase):
+        if gp.theta is None:
+            raise RuntimeError("GP fitting failed")
+    else:
+        if len(gp.get_indices_not_fit()) > 0:
+            failure_string = "Fitting failed for emulators {}".format(
+                    gp.get_indices_not_fit())
+            if skip_failures:
+                print(failure_string)
+            else:
+                raise RuntimeError(failure_string)
+
+    return gp
+
 
 def _fit_single_GP_MAP(gp, n_tries=15, theta0=None, method='L-BFGS-B', **kwargs):
-    """
-    Fit hyperparameters using MAP for a single GP
+    """Fit hyperparameters using MAP for a single GP
 
-    Returns a single GP object that has its hyperparameters fit to the MAP value.
-    Accepts keyword arguments passed to scipy's minimization routine.
-    """
+    Returns a single GP object that has its hyperparameters fit to the
+    MAP value.  Accepts keyword arguments passed to scipy's
+    minimization routine.
 
-    assert isinstance(gp, GaussianProcess)
+    """
 
+    assert isinstance(gp, GaussianProcessBase)
     n_tries = int(n_tries)
     assert n_tries > 0, "number of attempts must be positive"
 
@@ -136,12 +220,13 @@ def _fit_single_GP_MAP(gp, n_tries=15, theta0=None, method='L-BFGS-B', **kwargs)
             print("Floating point error in optimization routine, skipping this iteration")
 
     if len(logpost_values) == 0:
-        raise RuntimeError("Minimization routine failed to return a value")
-
-    logpost_values = np.array(logpost_values)
-    idx = np.argmin(logpost_values)
+        print("Minimization routine failed to return a value")
+        gp.theta = None
+    else:
+        logpost_values = np.array(logpost_values)
+        idx = np.argmin(logpost_values)
 
-    gp.fit(theta_values[idx])
+        gp.fit(theta_values[idx])
 
     return gp
 
@@ -150,16 +235,23 @@ def _fit_single_GP_MAP_bound(gp, theta0, n_tries, method, **kwargs):
 
     return _fit_single_GP_MAP(gp, n_tries=n_tries, theta0=theta0, method=method, **kwargs)
 
-def _fit_MOGP_MAP(gp, n_tries=15, theta0=None, method='L-BFGS-B', **kwargs):
-    """
-    Fit hyperparameters using MAP for multiple GPs in parallel
+def _fit_MOGP_MAP(gp, n_tries=15, theta0=None, method='L-BFGS-B',
+                  refit=False, **kwargs):
+    """Fit hyperparameters using MAP for multiple GPs in parallel
 
-    Uses Python Multiprocessing to fit GPs in parallel by calling the above routine for a single
-    GP for each of the emulators in the MOGP class. Returns a MultiOutputGP object where all
-    emulators have been fit to the MAP value.
+    Uses Python Multiprocessing to fit GPs in parallel by calling the
+    above routine for a single GP for each of the emulators in the
+    MOGP class. Returns a MultiOutputGP object where all emulators
+    have been fit to the MAP value.
+
+    Routine only fits GPs that have not previously been fit (indicated
+    by the hyperparameters being set to ``None`` by default. This can
+    be overridden by passing ``refit=True``.
+
+    Accepts a ``processes`` argument (integer or None) as a keyword to
+    control the number of subprocesses used to fit the individual GPs
+    in parallel. Must be positive. Default is ``None``.
 
-    Accepts a ``processes`` argument (integer or None) as a keyword to control the number of
-    subprocesses used to fit the individual GPs in parallel. Must be positive. Default is ``None``.
     """
 
     assert isinstance(gp, MultiOutputGP)
@@ -190,16 +282,25 @@ def _fit_MOGP_MAP(gp, n_tries=15, theta0=None, method='L-BFGS-B', **kwargs):
 
     n_tries = int(n_tries)
 
-    # partial(fit_GP_MAP, )
+    if refit:
+        emulators_to_fit = gp.emulators
+        indices_to_fit = list(range(len(gp.emulators)))
+        thetavals = theta0
+    else:
+        indices_to_fit = gp.get_indices_not_fit()
+        emulators_to_fit = gp.get_emulators_not_fit()
+        thetavals = [ theta0[idx] for idx in indices_to_fit]
+
+    if platform.system() == "Windows" or gp.use_gpu:
 
-    if platform.system() == "Windows":
         fit_MOGP = [fit_GP_MAP(emulator, n_tries=n_tries, theta0=t0, method=method, **kwargs)
-                    for (emulator, t0) in zip(gp.emulators, theta0)]
+                    for (emulator, t0) in zip(emulators_to_fit, thetavals)]
     else:
         with Pool(processes) as p:
             fit_MOGP = p.starmap(partial(_fit_single_GP_MAP_bound, n_tries=n_tries, method=method, **kwargs),
-                                 [(emulator, t0) for (emulator, t0) in zip(gp.emulators, theta0)])
+                                 [(emulator, t0) for (emulator, t0) in zip(emulators_to_fit, thetavals)])
 
-    gp.emulators = fit_MOGP
+    for (idx, em) in zip(indices_to_fit, fit_MOGP):
+        gp.emulators[idx] = em
 
-    return gp
\ No newline at end of file
+    return gp
diff --git a/mogp_emulator/linalg/__init__.py b/mogp_emulator/linalg/__init__.py
index e69de29b..77449de0 100644
--- a/mogp_emulator/linalg/__init__.py
+++ b/mogp_emulator/linalg/__init__.py
@@ -0,0 +1,1 @@
+from mogp_emulator.linalg.cholesky import jit_cholesky, pivot_cholesky, pivot_cho_solve
diff --git a/mogp_emulator/linalg/cholesky.py b/mogp_emulator/linalg/cholesky.py
index 5eeca19c..621c02b0 100644
--- a/mogp_emulator/linalg/cholesky.py
+++ b/mogp_emulator/linalg/cholesky.py
@@ -1,8 +1,8 @@
 import numpy as np
 from scipy import linalg
-from scipy.linalg import lapack
+from scipy.linalg import lapack, cho_solve
 
-def check_cholesky_inputs(A):
+def _check_cholesky_inputs(A):
     """
     Check inputs to cholesky routines
 
@@ -56,7 +56,7 @@ def jit_cholesky(A, maxtries = 5):
     :rtype: tuple containing an ndarray and a float
     """
 
-    A = check_cholesky_inputs(A)
+    A = _check_cholesky_inputs(A)
     assert int(maxtries) > 0, "maxtries must be a positive integer"
 
     A = np.ascontiguousarray(A)
@@ -78,3 +78,111 @@ def jit_cholesky(A, maxtries = 5):
         raise linalg.LinAlgError("not positive definite, even with jitter.")
 
     return L, jitter
+
+def pivot_cholesky(A):
+    """
+    Pivoted cholesky decomposition routine
+
+    Performs a pivoted Cholesky decomposition on a square, potentially
+    singular (i.e. can have collinear rows) covariance matrix. Rows
+    and columns are interchanged such that the diagonal entries of the
+    factorized matrix decrease from the first entry to the last entry.
+    Any collinear rows are skipped, and the diagonal entries for those
+    rows are instead replaced by a decreasing entry. This allows the
+    factorized matrix to still be used to compute the log determinant
+    in the same way, which is required to compute the marginal log
+    likelihood/posterior in the GP fitting.
+
+    Returns the factorized matrix, and an ordered array of integer
+    indices indicating the pivoting order. Raises an error if the
+    decomposition fails.
+
+    :param A: The matrix to be inverted as an array of shape ``(n,n)``.
+              Must be a symmetric matrix (though can be singular).
+    :type A: ndarray
+    :returns: Lower-triangular factored matrix (shape ``(n,n)`` an an array of
+              integers indicating the pivoting order needed to produce the
+              factorization.
+    :rtype: tuple containing an ndarray of shape `(n,n)` of floats and a
+            ndarray of shape `(n,)` of integers.
+    """
+
+    A = _check_cholesky_inputs(A)
+
+    A = np.ascontiguousarray(A)
+    L, P, rank, info = lapack.dpstrf(A, lower = 1)
+    L = np.tril(L)
+
+    if info < 0:
+        raise linalg.LinAlgError("Illegal value in covariance matrix")
+
+    n = A.shape[0]
+
+    idx = np.arange(rank, n)
+    divs = np.cumprod(np.arange(rank+1, n+1, dtype=np.float64))
+    L[idx, idx] = L[rank-1, rank-1]/divs
+
+    return L, P-1
+
+def _pivot_transpose(P):
+    """
+    Invert a pivot matrix by taking its transpose
+
+    Inverts the pivot matrix by taking its transpose. Since the pivot
+    matrix is represented by an array of integers(to make swapping the
+    rows more simple by using numpy indexing), this is done by
+    creating the equivalent array of integers representing the
+    transpose.  Input must be a list of non-negative integers, where
+    each integer up to the length of the array appears exactly
+    once. Otherwise this function will raise an exception.
+
+    :param P: Input pivot matrix, represented as an array of non-negative
+              integers.
+    :type P: ndarray containing integers
+    :returns: Transpose of the pivot matrix, represented as an array of
+              non-negative integers.
+    :rtype: ndarray containing integers.
+
+    """
+
+    assert np.array(P).ndim == 1, "pivot matrix must be a list of integers"
+
+    try:
+        return np.array([np.where(P == idx)[0][0] for idx in range(len(P))], dtype=np.int32)
+    except IndexError:
+        raise ValueError("Bad values for pivot matrix input to pivot_transpose")
+
+
+def pivot_cho_solve(L, P, b):
+    """Solve a Linear System factorized using Pivoted Cholesky Decomposition
+
+    Solve a system :math:`{Ax = b}` where the matrix has been factorized using the
+    `pivot_cholesky` function. Can also solve a system which has been
+    factorized using the regular Cholesky decomposition routine if
+    `P` is the set of integers from 0 to the length of the linear system.
+    The routine rearranges the order of the RHS based on the pivoting
+    order that was used, and then rearranges back to the original
+    ordering of the RHS when returning the array.
+
+    :param L: Factorized :math:`{A}` square matrix using
+              pivoting. Assumes lower triangular factorization is
+              used.
+    :type L: ndarray
+    :param P: Pivot matrix, expressed as a list or 1D array of
+              integers that is the same length as `L`. Must contain
+              only nonzero integers as entries, with each number up to
+              the length of the array appearing exactly once.
+    :type P: list or ndarray
+    :param b: Right hand side to be solved. Can be any array that
+              satisfies the rules of the scipy `cho_solve` routine.
+    :type b: ndarray
+    :returns: Solution to the appropriate linear system as a ndarray.
+    :rtype: ndarray
+    """
+
+    assert len(P) == L.shape[0], "Length of pivot matrix must match linear system"
+
+    try:
+        return cho_solve((L, True), b[P])[_pivot_transpose(P)]
+    except (IndexError, ValueError):
+        raise ValueError("Bad values for pivot matrix in pivot_cho_solve")
diff --git a/mogp_gpu/Makefile b/mogp_gpu/Makefile
new file mode 100644
index 00000000..181ff225
--- /dev/null
+++ b/mogp_gpu/Makefile
@@ -0,0 +1,56 @@
+# Makefile for standalone C++ tests of CUDA functionality.
+# Note that this Makefile is NOT used when building mogp_emulator.
+# (The logic in setup.py that builds the Extension takes care of that).
+# The targets in this Makefile are mainly standalone tests of low-level functionality.
+#
+# If running on CSD3, run the following commands to ensure dependencies are in place:
+#
+# module load anaconda  #(not needed here, but needed for mogp_emulator)
+# module load py-pybind11-2.2.4-gcc-5.4.0-tdtz6iq #(only needed for mogp_emulator)
+# module load cuda/11.2
+# module load gcc/7
+# module load eigen
+#
+
+CXX=g++
+CXXFLAGS=-std=c++14 -O3
+NVCCFLAGS=--compiler-options -Wall,-Wextra -arch=sm_60 --generate-code arch=compute_60,code=sm_60 --generate-code arch=compute_37,code=sm_37
+EXTRAFLAGS=-Xcompiler -fPIC
+
+CUDA_INC=/usr/local/cuda/include
+PYBIND_INC=$(shell python3 -m pybind11 --includes)
+
+all: lib/libgpgpu.so
+
+clean:
+	rm -rf obj lib bin
+
+lib/libgpgpu.so:  src/gp_gpu.cu obj/cov_gpu.o obj/util.o
+	mkdir -p lib
+	nvcc $(CXXFLAGS) $(NVCCFLAGS) $(EXTRAFLAGS) -shared $(PYBIND_INC) $< obj/cov_gpu.o obj/util.o -o $@ -lcusolver -lcublas -lcudart
+
+obj/%.o: src/%.cu src/%.hpp
+	mkdir -p obj
+	nvcc $(CXXFLAGS) $(NVCCFLAGS) $(EXTRAFLAGS) -c $< -o $@
+
+bin/test/gpu_basic: test/gpu_basic.cu  obj/util.o
+	mkdir -p bin/test
+	nvcc $(CXXFLAGS) $(NVCCFLAGS) $<  obj/util.o -o $@
+
+bin/test/gpu_kernel: test/gpu_kernel.cu  obj/kernel.o obj/util.o
+	mkdir -p bin/test
+	nvcc $(CXXFLAGS) $(NVCCFLAGS) $<  obj/kernel.o obj/util.o -o $@
+
+bin/test/gpu_test_utils: test/gpu_test_utils.cu obj/util.o
+	mkdir -p bin/test
+	nvcc $(CXXFLAGS) $(NVCCFLAGS) $< obj/util.o -o $@
+
+bin/test/gpu_cholesky: test/gpu_cholesky.cu
+	mkdir -p bin/test
+	nvcc $(CXXFLAGS) $(NVCCFLAGS) -lcusolver -lcublas $< -o $@
+
+bin/test/gpu_meanfunc: test/gpu_meanfunc.cu obj/util.o
+	mkdir -p bin/test
+	nvcc $(CXXFLAGS) $(NVCCFLAGS) $<  obj/util.o -o $@
+
+.PHONY: all clean
diff --git a/mogp_gpu/src/gp_gpu.cu b/mogp_gpu/src/gp_gpu.cu
new file mode 100644
index 00000000..f2571a30
--- /dev/null
+++ b/mogp_gpu/src/gp_gpu.cu
@@ -0,0 +1,302 @@
+#include "gp_gpu.hpp"
+#include "pybind11/pybind11.h"
+#include "pybind11/stl.h"
+#include "pybind11/eigen.h"
+
+namespace py = pybind11;
+
+
+PYBIND11_MODULE(libgpgpu, m) {
+    py::class_<DenseGP_GPU>(m, "DenseGP_GPU")
+      .def(py::init<mat_ref, vec_ref, unsigned int, BaseMeanFunc*, kernel_type>())
+
+////////////////////////////////////////
+        .def("n", &DenseGP_GPU::get_n,
+             "Returns the number of training examples used for the emulator")
+
+////////////////////////////////////////
+        .def("D", &DenseGP_GPU::get_D,
+             "Returns the dimension of the emulator inputs")
+
+////////////////////////////////////////
+        .def("inputs", &DenseGP_GPU::get_inputs,
+             "Returns the array of inputs points used by the emulator, with "
+             "shape ``(n, D)``")
+
+////////////////////////////////////////
+        .def("targets", &DenseGP_GPU::get_targets,
+             "Returns the array of targets used by the emulator, with shape "
+             "``(n,)``")
+
+////////////////////////////////////////
+        .def("n_params", &DenseGP_GPU::get_n_params,
+             "Returns the number of hyperparameters required for the emulator.")
+
+////////////////////////////////////////
+        .def("theta_fit_status", &DenseGP_GPU::get_theta_fit_status,
+             "Has the emulator been fit?  If `false`, "
+             ":func:``GaussianProcessGPU.theta`` will be ``None``")
+
+////////////////////////////////////////
+        .def("reset_theta_fit_status", &DenseGP_GPU::reset_theta_fit_status,
+             "Marks this emulator as having not been fit: "
+             ":func:`theta_fit_status` will be `false`")
+
+////////////////////////////////////////
+        .def("get_theta", &DenseGP_GPU::get_theta,
+             "Returns the emulator hyperparameters")
+
+////////////////////////////////////////
+        .def("predict", &DenseGP_GPU::predict,
+             R"(Single point predictive mean.
+
+:param testing: The input point
+
+:returns: The predicted mean
+
+Currently unused by :class`GaussianProcessGPU`, but useful for testing)",
+             py::arg("testing"))
+
+////////////////////////////////////////
+        .def("predict_variance", &DenseGP_GPU::predict_variance,
+             R"(Single point combined predictive mean and variance.
+
+:param testing: The input point to predict, with shape `(Nbatch, D)`
+                (`Nbatch` is determined from the shape of the input).
+                `Nbatch` must be less than `testing_size`.
+:param var: (Output) The variance prediction is written to `var[0]`
+
+:returns: The predicted mean
+
+Currently unused by :class`GaussianProcessGPU`, but useful for testing.
+)",
+             py::arg("testing"),
+             py::arg("var"))
+
+////////////////////////////////////////
+        .def("predict_batch", &DenseGP_GPU::predict_batch,
+             R"(Batched predictive means.
+
+:param testing: The input point to predict, with shape `(Nbatch, D)`
+                (`Nbatch` is determined from the shape of the input).
+                `Nbatch` must be less than `testing_size`.
+:param result: (Output) The predicted mean at each `testing` point.
+               Shape `(Nbatch, D)`.
+
+:returns: None
+)",
+             py::arg("testing"),
+             py::arg("result"))
+
+////////////////////////////////////////
+        .def("predict_variance_batch", &DenseGP_GPU::predict_variance_batch,
+             R"(Batched predictive means and variances.
+
+:param testing: The input point to predict, with shape `(Nbatch, D)`
+                (`Nbatch` is determined from the shape of the input).
+                `Nbatch` must be less than `testing_size`.
+:param mean: (Output) The predicted mean at each `testing` point.
+             Shape `(Nbatch, D)`.
+:param var: (Output) The predicted variance at each `testing` point.
+            Shape `(Nbatch, D)`.
+
+:returns: None
+)",
+             py::arg("testing"),
+             py::arg("mean"),
+             py::arg("var"))
+
+////////////////////////////////////////
+        .def("predict_deriv", &DenseGP_GPU::predict_deriv,
+             R"(Batched prediction of derivatives.
+
+:param testing: The input point to predict, with shape `(Nbatch, D)`
+                (`Nbatch` is determined from the shape of the input).
+                `Nbatch` must be less than `testing_size`.
+:param result: (Output) The predicted gradient of the mean with
+               respect to the inputs.  Shape `(Nbatch, D)`.
+
+:returns: None
+)",
+             py::arg("testing"),
+             py::arg("result"))
+
+////////////////////////////////////////
+
+        .def("fit", &DenseGP_GPU::fit,
+             R"(Sets theta and the nugget parameters, and updates the internal
+state of the emulator accordingly.
+
+:param theta: Values for the hyperparameters (length ``n_params``)
+:param nugget: The interpretation of the nugget: adaptive, fixed or fit
+:param nugget_size: The (initial) nugget value
+
+:returns: None
+)",
+             py::arg("theta"),
+             py::arg("nugget"),
+             py::arg("nugget_size")
+            )
+
+////////////////////////////////////////
+        .def("get_K", &DenseGP_GPU::get_K,
+             R"(The current covariance matrix
+
+:param K_h: (Output) The matrix, of shape ``(n, n)``.
+
+:returns: None
+)",
+             py::arg("K_h"))
+
+////////////////////////////////////////
+        .def("get_invQ", &DenseGP_GPU::get_invQ,
+             R"(The inverse of the current covariance matrix
+
+:param invQ_h: (Output) The matrix, of shape ``(n, n)``.
+
+:returns: None
+)",
+             py::arg("invQ_h"))
+
+////////////////////////////////////////
+        .def("get_invQt", &DenseGP_GPU::get_invQt,
+             R"(The inverse covariance matrix times the target vector.
+
+:param invQt_h: (Output) The resulting vector, of shape ``(n,)``
+
+:returns: None
+)",
+             py::arg("invQt_h"))
+
+////////////////////////////////////////
+        .def("get_logpost", &DenseGP_GPU::get_logpost,
+	     "Return the log posterior"
+            )
+
+////////////////////////////////////////
+        .def("get_jitter", &DenseGP_GPU::get_jitter,
+             "The value of jitter used for an adaptive nugget.")
+
+////////////////////////////////////////
+        .def("get_cholesky_lower", &DenseGP_GPU::get_cholesky_lower,
+             R"(Return the (lower) Cholesky factor of the covariance matrix.
+
+This is the output of (cu)BLAS dpotrf.  The lower triangular entries correspond
+to the factor; the upper triangular entries contain junk values.
+
+:param result: (Output) the resulting Cholesky factor.  Shape ``(n,n)``.
+               Lower triangular entries set.
+
+:returns: None
+ )",
+             py::arg("result"))
+
+
+////////////////////////////////////////
+        .def("logpost_deriv", &DenseGP_GPU::logpost_deriv,
+             R"(Calculate the partial derivatives of the negative log-posterior
+likelihood of the hyperparameters, from the current state of the emulator.)
+
+:param result: (Output) the resulting derivatives, shape ``(D+1,)``
+
+:returns: `None`
+)",
+             py::arg("result"));
+
+////////////////////////////////////////
+    py::class_<SquaredExponentialKernel>(m, "SquaredExponentialKernel")
+      .def(py::init<>())
+        .def("kernel_f", &SquaredExponentialKernel::kernel_f,
+             "Calculate the covariance matrix")
+        .def("kernel_deriv", &SquaredExponentialKernel::kernel_deriv,
+             "Calculate the derivative of the covariance matrix wrt hyperparameters")
+        .def("kernel_inputderiv", &SquaredExponentialKernel::kernel_inputderiv,
+	     "Derivative of covariance matrix wrt inputs");
+
+////////////////////////////////////////
+    py::class_<Matern52Kernel>(m, "Matern52Kernel")
+      .def(py::init<>())
+        .def("kernel_f", &Matern52Kernel::kernel_f,
+             "Calculate the covariance matrix")
+        .def("kernel_deriv", &Matern52Kernel::kernel_deriv,
+             "Calculate the derivative of the covariance matrix wrt hyperparameters")
+        .def("kernel_inputderiv", &Matern52Kernel::kernel_inputderiv,
+	     "Derivative of covariance matrix wrt inputs");
+
+////////////////////////////////////////
+    py::class_<BaseMeanFunc>(m, "BaseMeanFunc");
+
+////////////////////////////////////////
+    py::class_<ConstMeanFunc, BaseMeanFunc>(m, "ConstMeanFunc")
+      .def(py::init<>())
+        .def("mean_f", &ConstMeanFunc::mean_f,
+             "Evaluate the mean function at input points")
+        .def("mean_deriv", &ConstMeanFunc::mean_deriv,
+             "Calculate the derivative of the mean function wrt hyperparameters")
+        .def("mean_inputderiv", &ConstMeanFunc::mean_inputderiv,
+	     "Derivative of mean function wrt inputs")
+        .def("get_n_params", &ConstMeanFunc::get_n_params,
+	     "Number of parameters of mean function");
+
+////////////////////////////////////////
+    py::class_<PolyMeanFunc, BaseMeanFunc>(m, "PolyMeanFunc")
+      .def(py::init<std::vector< std::pair<int, int> > >())
+        .def("mean_f", &PolyMeanFunc::mean_f,
+             "Evaluate the mean function at input points")
+        .def("mean_deriv", &PolyMeanFunc::mean_deriv,
+             "Calculate the derivative of the mean function wrt hyperparameters")
+        .def("mean_inputderiv", &PolyMeanFunc::mean_inputderiv,
+	     "Derivative of mean function wrt inputs")
+        .def("get_n_params", &PolyMeanFunc::get_n_params,
+	     "Number of parameters of mean function");
+
+////////////////////////////////////////
+    py::class_<FixedMeanFunc, BaseMeanFunc>(m, "FixedMeanFunc")
+      .def(py::init<REAL>())
+        .def("mean_f", &FixedMeanFunc::mean_f,
+             "Evaluate the mean function at input points")
+        .def("mean_deriv", &FixedMeanFunc::mean_deriv,
+             "Calculate the derivative of the mean function wrt hyperparameters")
+        .def("mean_inputderiv", &FixedMeanFunc::mean_inputderiv,
+	     "Derivative of mean function wrt inputs")
+        .def("get_n_params", &FixedMeanFunc::get_n_params,
+	     "Number of parameters of mean function");
+
+////////////////////////////////////////
+    py::class_<ZeroMeanFunc, BaseMeanFunc>(m, "ZeroMeanFunc")
+      .def(py::init<>())
+        .def("mean_f", &ZeroMeanFunc::mean_f,
+             "Evaluate the mean function at input points")
+        .def("mean_deriv", &ZeroMeanFunc::mean_deriv,
+             "Calculate the derivative of the mean function wrt hyperparameters")
+        .def("mean_inputderiv", &ZeroMeanFunc::mean_inputderiv,
+	     "Derivative of mean function wrt inputs")
+        .def("get_n_params", &ZeroMeanFunc::get_n_params,
+	     "Number of parameters of mean function");
+
+    py::enum_<kernel_type>(m, "kernel_type")
+        .value("SquaredExponential", SQUARED_EXPONENTIAL)
+        .value("Matern52", MATERN52);
+
+    py::enum_<nugget_type>(m, "nugget_type")
+        .value("adaptive", NUG_ADAPTIVE)
+        .value("fixed", NUG_FIXED)
+        .value("fit", NUG_FIT);
+
+    m.def("have_compatible_device", &have_compatible_device);
+
+    m.doc() = R"(
+The libgpgpu library
+--------------------
+
+.. currentmodule:: libgpgpu
+
+The members of this class are
+
+.. autoclass:: libgpgpu.DenseGP_GPU
+    :members:
+
+)";
+
+    // Include have_compatible_device in the docstring
+}
diff --git a/mogp_gpu/src/gp_gpu.hpp b/mogp_gpu/src/gp_gpu.hpp
new file mode 100644
index 00000000..52387449
--- /dev/null
+++ b/mogp_gpu/src/gp_gpu.hpp
@@ -0,0 +1,743 @@
+#ifndef GP_GPU_HPP
+#define GP_GPU_HPP
+
+/*
+This file contains the C++ implementation of the Gaussian Process class.
+(The corresponding .cu file contains the pybind11 bindings allowing the functions
+to be called from Python).
+
+The key methods of the DenseGP_GPU class are:
+  fit:  update the theta hyperparameters
+  predict_batch:  make a prediction on a set of testing points
+  predict_variance_batch: make a prediction on a set of testing points, including variance
+  predict_deriv: get the derivative of prediction on a set of testing points.
+These in turn use CUDA kernels defined in the file cov_gpu.cu
+*/
+
+#include <iostream>
+
+#include <algorithm>
+#include <string>
+#include <sstream>
+#include <assert.h>
+#include <stdexcept>
+
+#include "util.hpp"
+#include "kernel.hpp"
+#include "meanfunc.hpp"
+
+
+
+
+//////////////////////////////////////////////////////////////////////////////////////
+////////////////////////// The Gaussian Process  class ///////////////////////////////
+//////////////////////////////////////////////////////////////////////////////////////
+
+// By convention, members ending "_d" are allocated on the device
+class DenseGP_GPU {
+    // batch size (for allocation)
+    unsigned int testing_size;
+
+    // Number of training points, dimension of input
+    unsigned int n, D;
+
+    // inputs
+    mat inputs;
+
+    // targets
+    vec targets;
+
+    // what Kernel are we using?
+    kernel_type kern_type;
+
+    // pointer to the Kernel
+    BaseKernel* kernel;
+
+    // pointer to the Mean Function
+    BaseMeanFunc* meanfunc;
+
+    // is the nugget adapted, fixed, or fitted?
+    nugget_type nug_type;
+
+    // flag for whether we have fit hyperparameters
+    bool theta_fitted;
+
+    // handle for CUBLAS calls
+    cublasHandle_t cublasHandle;
+
+    // handle for cuSOLVER calls
+    cusolverDnHandle_t cusolverHandle;
+
+    // covariance matrix on device
+    thrust::device_vector<REAL> K_d;
+
+    // inverse covariance matrix on device
+    thrust::device_vector<REAL> invQ_d;
+
+    // lower triangular Cholesky factor on device
+    thrust::device_vector<REAL> chol_lower_d;
+
+    // log determinant of covariance matrix (on host)
+    double logdetC;
+
+    // adaptive nugget jitter
+    double jitter;
+
+    // kernel hyperparameters on the device
+    thrust::device_vector<REAL> theta_d;
+
+    // mean function hyperparameters
+    vec meanfunc_params;
+
+    // inputs, on the device, row major order
+    thrust::device_vector<REAL> inputs_d;
+
+    // targets, on the device
+    thrust::device_vector<REAL> targets_d;
+
+    // precomputed product, used in the prediction
+    thrust::device_vector<REAL> invQt_d;
+
+    // preallocated work array (length n) on the CUDA device
+    thrust::device_vector<REAL> work_d;
+
+    // device vector for derivative of mean function
+    thrust::device_vector<REAL> meanfunc_deriv_d;
+
+    // buffer for Cholesky factorization
+    thrust::device_vector<REAL> potrf_buffer_d;
+
+    // buffer for cub::DeviceReduce::Sum
+    thrust::device_vector<REAL> sum_buffer_d;
+
+    // size of sum_buffer_d
+    size_t sum_buffer_size_bytes;
+
+    // more work arrays
+    thrust::device_vector<REAL> testing_d, work_mat_d, result_d, kappa_d, invQk_d;
+
+public:
+    int get_n(void) const
+    {
+        return n;
+    }
+
+    int get_D(void) const
+    {
+        return D;
+    }
+
+    mat get_inputs(void) const
+    {
+        return inputs;
+    }
+
+    vec get_targets(void) const
+    {
+        return targets;
+    }
+
+    int get_n_params(void) const
+    {
+        return D + 2;
+    }
+
+    void get_theta(vec_ref theta)
+    {
+      // mean function params then kernel params
+        int param_switch_index = meanfunc_params.rows();
+	theta.block(0,0,param_switch_index,1) = meanfunc_params;
+        thrust::copy(theta_d.begin(), theta_d.end(), theta.data()+param_switch_index);
+    }
+
+    double get_jitter(void) const
+    {
+        return jitter;
+    }
+
+    // make a single prediction (mainly for testing - most use-cases will use predict_batch or predict_deriv_batch)
+    double predict(mat_ref testing)
+    {
+        // On entry: the number of points to predict (number of rows
+        // of testing) is assumed to be D
+
+        thrust::device_vector<REAL> testing_d(testing.data(), testing.data() + D);
+        kernel->cov_all_gpu(dev_ptr(work_d), n, D, dev_ptr(testing_d), dev_ptr(inputs_d),
+			    dev_ptr(theta_d));
+
+        REAL result = std::numeric_limits<double>::quiet_NaN();
+        CUBLASDOT(cublasHandle, n, dev_ptr(work_d), 1, dev_ptr(invQt_d), 1,
+                  &result);
+	// evaluate the mean function and add to the result
+        vec meanfunc_vals = meanfunc->mean_f(testing, meanfunc_params);
+        return double(result + meanfunc_vals(0));
+    }
+
+    // variance of a single prediction (mainly for testing - most use-cases will use predict_variance_batch)
+    double predict_variance(mat_ref testing, vec_ref var)
+    {
+        if (var.size() < testing.rows()) {
+            throw std::runtime_error("predict_variance: the result buffer passed was "
+                                     "too small to hold the variance");
+        }
+
+        // value prediction
+        thrust::device_vector<REAL> testing_d(testing.data(), testing.data() + D);
+        kernel->cov_all_gpu(dev_ptr(work_d), n, D, dev_ptr(testing_d), dev_ptr(inputs_d),
+                    dev_ptr(theta_d));
+
+        REAL result = std::numeric_limits<double>::quiet_NaN();
+        CUBLASDOT(cublasHandle, n, dev_ptr(work_d), 1, dev_ptr(invQt_d), 1,
+                  &result);
+
+        // variance prediction
+        thrust::device_vector<REAL> kappa_d(1);
+        kernel->cov_val_gpu(dev_ptr(kappa_d), D, dev_ptr(testing_d), dev_ptr(testing_d),
+                    dev_ptr(theta_d));
+
+        double zero(0.0);
+        double one(1.0);
+        thrust::device_vector<REAL> invQk_d(n);
+
+        cublasDgemv(cublasHandle, CUBLAS_OP_N, n, n, &one, dev_ptr(invQ_d), n,
+                    dev_ptr(work_d), 1, &zero, dev_ptr(invQk_d), 1);
+
+        CUBLASDOT(cublasHandle, n, dev_ptr(work_d), 1, dev_ptr(invQt_d),
+                  1, &result);
+        CUBLASDOT(cublasHandle, n, dev_ptr(work_d), 1, dev_ptr(invQk_d),
+                  1, var.data());
+
+        double kappa;
+        thrust::copy(kappa_d.begin(), kappa_d.end(), &kappa);
+
+        cudaDeviceSynchronize();
+
+        var = kappa - var.array();
+	// evaluate the mean function and add to the result
+	vec meanfunc_vals = meanfunc->mean_f(testing, meanfunc_params);
+        return REAL(result + meanfunc_vals(0));
+    }
+
+    // Use the GP emulator to calculate a prediction on testing points, without calculating variance or derivative
+    void predict_batch(mat_ref testing, vec_ref result)
+    {
+        // Assumes on entry that testing has shape (Nbatch, D), and that result
+        // contains space for Nbatch result values
+
+        int Nbatch = testing.rows();
+
+        if (result.size() < testing.rows()) {
+            throw std::runtime_error("predict_batch: the result buffer passed was "
+                                     "too small to hold the result");
+        }
+
+        if (Nbatch > testing_size) {
+            throw std::runtime_error("predict_variance_batch: More test points were passed "
+                                     "than the maximum batch size");
+        }
+
+        REAL zero(0.0);
+        REAL one(1.0);
+        thrust::device_vector<REAL> testing_d(testing.data(), testing.data() + Nbatch * D);
+        thrust::device_vector<REAL> result_d(Nbatch);
+
+        kernel->cov_batch_gpu(dev_ptr(work_mat_d), Nbatch, n, D,
+			      dev_ptr(testing_d), dev_ptr(inputs_d), dev_ptr(theta_d));
+
+        cublasStatus_t status =
+            cublasDgemv(cublasHandle, CUBLAS_OP_N, Nbatch, n, &one,
+                        dev_ptr(work_mat_d), Nbatch, dev_ptr(invQt_d), 1, &zero,
+                        dev_ptr(result_d), 1);
+
+        cudaDeviceSynchronize();
+
+        thrust::copy(result_d.begin(), result_d.end(), result.data());
+
+	// evaluate the mean function and add to the result
+	vec meanfunc_vals = meanfunc->mean_f(testing, meanfunc_params);
+	result += meanfunc_vals;
+    }
+
+    // Use the GP emulator to calculate a prediction on testing points, also calculating variance
+    void predict_variance_batch(mat_ref testing, vec_ref mean, vec_ref var)
+    {
+        REAL zero(0.0);
+        REAL one(1.0);
+        REAL minus_one(-1.0);
+        int Nbatch = testing.rows();
+
+        if (var.size() < testing.rows() || mean.size() < testing.rows()) {
+            throw std::runtime_error("predict_variance_batch: The result buffer passed was "
+                                     "too small to hold the variance");
+        }
+
+        if (Nbatch > testing_size) {
+            throw std::runtime_error("predict_variance_batch: More test points were passed "
+                                     "than the maximum batch size");
+        }
+
+
+        thrust::device_vector<REAL> testing_d(testing.data(), testing.data() + Nbatch * D);
+        thrust::device_vector<REAL> mean_d(Nbatch);
+
+        // compute predictive means for the batch
+        kernel->cov_batch_gpu(dev_ptr(work_mat_d), Nbatch, n, D, dev_ptr(testing_d),
+                      dev_ptr(inputs_d), dev_ptr(theta_d));
+
+        cublasStatus_t status =
+            cublasDgemv(cublasHandle, CUBLAS_OP_N, Nbatch, n, &one,
+                        dev_ptr(work_mat_d), Nbatch, dev_ptr(invQt_d), 1, &zero,
+                        dev_ptr(mean_d), 1);
+
+        // compute predictive variances for the batch
+        kernel->cov_diag_gpu(dev_ptr(kappa_d), Nbatch, D, dev_ptr(testing_d),
+			     dev_ptr(testing_d), dev_ptr(theta_d));
+
+        cublasDgemm(cublasHandle, CUBLAS_OP_N, CUBLAS_OP_T,
+                    n,
+                    Nbatch,
+                    n,
+                    &one,
+                    dev_ptr(invQ_d), n,
+                    dev_ptr(work_mat_d), Nbatch,
+                    &zero,
+                    dev_ptr(invQk_d), n);
+
+        // result accumulated into 'kappa'
+        status = cublasDgemmStridedBatched(
+            cublasHandle, CUBLAS_OP_N, CUBLAS_OP_N,
+            1, // m
+            1, // n
+            n, // k
+            // A (m x k), B (k x n), C (m x n)
+            &minus_one, // alpha
+            dev_ptr(work_mat_d), Nbatch, 1, // A, lda, strideA
+            dev_ptr(invQk_d), n, n, // B, ldb, strideB (= covariances "k")
+            &one,
+            dev_ptr(kappa_d), 1, 1, // C, ldc, strideC
+            Nbatch);
+
+        cudaDeviceSynchronize();
+
+        // copy back means
+        thrust::copy(mean_d.begin(), mean_d.end(), mean.data());
+	// evaluate the mean function and add to the mean
+	vec meanfunc_vals = meanfunc->mean_f(testing, meanfunc_params);
+	mean += meanfunc_vals;
+        // copy back variances
+        thrust::copy(kappa_d.begin(), kappa_d.begin() + Nbatch, var.data());
+    }
+
+    // Use the GP emulator to calculate derivative of  prediction on testing points
+    void predict_deriv(mat_ref testing, mat_ref result)
+    {
+        int Nbatch = testing.rows();
+
+        if (result.rows() < testing.rows() || result.cols() != D) {
+            throw std::runtime_error("predict_deriv: the result buffer passed was "
+                                     "the wrong shape to hold the result");
+        }
+
+        if (Nbatch > testing_size) {
+            throw std::runtime_error("predict_variance_batch: More test points were passed "
+                                     "than the maximum batch size");
+        }
+
+        REAL zero(0.0);
+        REAL one(1.0);
+        thrust::device_vector<REAL> testing_d(testing.data(), testing.data() + Nbatch * D);
+        thrust::device_vector<REAL> result_d(Nbatch*D);
+
+        kernel->cov_deriv_x_batch_gpu(dev_ptr(work_mat_d), D, Nbatch, n,
+				      dev_ptr(testing_d), dev_ptr(inputs_d), dev_ptr(theta_d));
+
+        cublasStatus_t status =
+            cublasDgemv(cublasHandle, CUBLAS_OP_N,
+                        D * Nbatch, n, // nrows, ncols
+                        &one, dev_ptr(work_mat_d), D * Nbatch, // alpha, A, lda
+                        dev_ptr(invQt_d), 1, // x, incx
+                        &zero, dev_ptr(result_d), 1); // beta, y, incy
+
+        cudaDeviceSynchronize();
+
+        thrust::copy(result_d.begin(), result_d.end(), result.data());
+
+	// evaluate deriv of meanfunc wrt test points, and add to result
+	mat meanfunc_inputderiv = meanfunc->mean_inputderiv(testing, meanfunc_params);
+	result += meanfunc_inputderiv.transpose();
+    }
+
+    // perform Cholesky factorization of the matrix currently stored in work_mat_d
+    int calc_cholesky_factors(void)
+    {
+        // On entry: assumes that work_mat_d contains the inverse covariance matrix invQ_d
+
+        thrust::device_vector<int> info_d(1);
+        int info_h;
+        cusolverStatus_t status;
+
+	// compute Cholesky factors
+	status = cusolverDnDpotrf(cusolverHandle, CUBLAS_FILL_MODE_LOWER, n,
+				  dev_ptr(work_mat_d), n, dev_ptr(potrf_buffer_d),
+				  potrf_buffer_d.size(), dev_ptr(info_d));
+
+	thrust::copy(info_d.begin(), info_d.end(), &info_h);
+
+	if (status != CUSOLVER_STATUS_SUCCESS || info_h < 0) {
+            std::string msg;
+            std::stringstream smsg(msg);
+            smsg << "Error in potrf: return code " << status << ", info " << info_h;
+            throw std::runtime_error(smsg.str());
+
+	}
+	return info_h;
+    }
+
+    // return the lower triangular matrix (actually it will be the whole matrix
+    // but only the lower triangle will be correct!)
+    void get_cholesky_lower(mat_ref result)
+    {
+        thrust::copy(chol_lower_d.begin(), chol_lower_d.end(), result.data());
+    }
+
+    // Update the hyperparameters, and K, invQ and invQt which depend on them
+    void fit(vec_ref theta, nugget_type nugget, double nugget_size=0.0)
+    {
+
+        nug_type = nugget;
+	int param_switch_index = meanfunc->get_n_params(inputs);
+	meanfunc_params = theta.block(0,0,param_switch_index,1);
+
+	// evaluate the mean function at the input values and subtract from targets
+	vec new_targets = targets - meanfunc->mean_f(inputs, meanfunc_params);
+	thrust::copy(new_targets.data(), new_targets.data() + n, targets_d.begin());
+
+	// copy all the parameters _after_ those for the mean function to the device vector theta_d
+        thrust::copy(theta.data() + param_switch_index,
+		     theta.data() + D + 2 + param_switch_index,
+		     theta_d.begin());
+
+	// calculate covariance matrix, put result into K_d
+        kernel->cov_batch_gpu(dev_ptr(K_d), n, n, D, dev_ptr(inputs_d),
+			      dev_ptr(inputs_d), dev_ptr(theta_d));
+	/// copy the covariance matrix K_d into work_mat_d
+	thrust::copy(K_d.begin(), K_d.end(), work_mat_d.begin());
+
+        thrust::device_vector<int> info_d(1);
+        int info_h;
+        cusolverStatus_t status;
+	int factorisation_status;
+
+	// for adaptive nugget start with a nugget of zero and increase by small amount
+	// until we find a value where factorization succeeds.
+	if (nug_type == NUG_ADAPTIVE) {
+	    double mean_diag;
+	    const int max_tries = 5;
+	    int itry = 0;
+	    while (itry < max_tries) {
+		// K_d holds the covariance matrix - work with a copy
+		// in work_mat_d, in case the factorization fails
+		thrust::copy(K_d.begin(), K_d.end(), work_mat_d.begin());
+
+		// if the first attempt at factorization failed, add a
+		// small term to the diagonal, increasing each iteration
+		// until the factorization succeeds
+		if (itry >= 1) {
+		    if (itry == 1) {
+			// find mean of (absolute) diagonal elements (diagonal
+			// elements should all be positive)
+			cublasDasum(cublasHandle, n, dev_ptr(K_d), n+1, &mean_diag);
+			mean_diag /= n;
+			jitter = 1e-6 * mean_diag;
+		    }
+		    add_diagonal(n, jitter, dev_ptr(work_mat_d));
+		}
+
+		factorisation_status = calc_cholesky_factors();
+		if (factorisation_status == 0) {
+                    break;
+		}
+		jitter *= 10;
+		itry++;
+	    }
+
+	    // if none of the factorization attempts succeeded:
+	    if (itry == max_tries) {
+		std::string msg;
+		std::stringstream smsg(msg);
+		smsg << "All attempts at factorization failed. Last return code " << factorisation_status;
+		throw std::runtime_error(smsg.str());
+	    }
+	// for fixed nugget, add "nugget_size" to the diagonal of the matrix.
+	} else if (nug_type == NUG_FIXED) {
+	    add_diagonal(n, nugget_size, dev_ptr(work_mat_d));
+
+	    factorisation_status = calc_cholesky_factors();
+	    if (factorisation_status != 0) {
+                throw std::runtime_error("Unable to factorize matrix using fixed nugget");
+	    }
+
+	} else if (nug_type == NUG_FIT) {
+	    // set to exp(last-element-of-theta)
+	    jitter = exp( theta(theta.size()-1) );
+
+	    add_diagonal(n, jitter, dev_ptr(work_mat_d));
+	    factorisation_status = calc_cholesky_factors();
+	    if (factorisation_status != 0) {
+                throw std::runtime_error("Unable to factorize matrix using fitted nugget");
+	    }
+	} else throw std::runtime_error("Unrecognized nugget_type");
+
+
+        // get the inverse covariance matrix invQ by solving the system of linear eqns
+	//    work_mat_d . invQ_d = I
+	// where work_mat_d is holding the current covariance matrix.
+
+        identity_device(n, dev_ptr(invQ_d));
+
+        status = cusolverDnDpotrs(cusolverHandle, CUBLAS_FILL_MODE_LOWER, n, n,
+                                  dev_ptr(work_mat_d), n, dev_ptr(invQ_d), n,
+                                  dev_ptr(info_d));
+
+        thrust::copy(info_d.begin(), info_d.end(), &info_h);
+        check_cusolver_status(status, info_h);
+
+
+        // invQt - product of inverse covariance matrix with the target values
+        thrust::copy(targets_d.begin(), targets_d.end(), invQt_d.begin());
+        status = cusolverDnDpotrs(cusolverHandle, CUBLAS_FILL_MODE_LOWER, n, 1,
+                                  dev_ptr(work_mat_d), n, dev_ptr(invQt_d), n,
+                                  dev_ptr(info_d));
+
+        thrust::copy(info_d.begin(), info_d.end(), &info_h);
+        check_cusolver_status(status, info_h);
+
+        // logdetC - sum the log of the diagonal elements of the Cholesky factor of covariance matrix (in work_mat_d)
+        thrust::device_vector<double> logdetC_d(1);
+
+        sum_log_diag(n, dev_ptr(work_mat_d), dev_ptr(logdetC_d), dev_ptr(sum_buffer_d), sum_buffer_size_bytes);
+
+        thrust::copy(logdetC_d.begin(), logdetC_d.end(), &logdetC);
+
+	//copy work_mat_d into the lower triangular Cholesky factor
+	thrust::copy(work_mat_d.begin(), work_mat_d.begin()+n*n, chol_lower_d.begin());
+
+	//set the flag to say we have fitted theta
+	theta_fitted = true;
+    }
+
+    bool get_theta_fit_status(void)
+    {
+        return theta_fitted;
+    }
+
+    void reset_theta_fit_status(void)
+    {
+        theta_fitted = false;
+    }
+
+    void get_K(mat_ref K_h)
+    {
+        thrust::copy(K_d.begin(), K_d.end(), K_h.data());
+    }
+
+    void get_invQ(mat_ref invQ_h)
+    {
+        thrust::copy(invQ_d.begin(), invQ_d.end(), invQ_h.data());
+    }
+
+    void get_invQt(mat_ref invQt_h)
+    {
+        thrust::copy(invQt_d.begin(), invQt_d.end(), invQt_h.data());
+    }
+
+    double get_logpost(void)
+    {
+        double result;
+        CUBLASDOT(cublasHandle, n, dev_ptr(targets_d), 1, dev_ptr(invQt_d), 1,
+                  &result);
+
+        result += logdetC + n * log(2.0 * M_PI);
+
+        return 0.5 * result;
+    }
+
+    void logpost_deriv(vec_ref result)
+    {
+        double zero(0.0);
+        double one(1.0);
+        double half(0.5);
+        double m_half(-0.5);
+	double minusone(-1.0);
+
+        const int Ntheta = D + 1;
+
+        // Compute
+        //   \pderiv{C_{jk}}{theta_i}
+        //
+        // The length of work_mat_d is n * testing_size
+        // The derivative above has  n * n * Ntheta components
+        // The following assumes that testing_size > Ntheta * n
+        kernel->cov_deriv_theta_batch_gpu(dev_ptr(work_mat_d),
+					  D, n, n,
+					  dev_ptr(inputs_d), dev_ptr(inputs_d),
+					  dev_ptr(theta_d));
+
+        // Compute
+        //   \deriv{logpost}{theta_i}
+        //     =   0.5 Tr(inv(C)*\pderiv{C}{\theta_i}           (Term 1)
+        //       - 0.5 (invQ_ts)^T \pderiv{C}{\theta_i} invQt  (Term 2)
+
+        // Working memory on device:
+        //   result_d:
+        //       accumulated result
+        //   work_mat_d (until Term 2 step 1):
+        //       covariance derivatives
+        //   work_mat_d (after Term 2 step 1):
+        //       temporary product in term 2 (okay to overwrite derivatives at this point)
+
+        // ***** Term 1 *****
+        // Treat 'jk' as a new 1-d index in C_{jk} and \pderiv{C_{jk}}{\theta}, called 'l' below.
+        // That is:
+        //    R_i = \pderiv{vec(C)_l}{\theta_i} vec(inv(C))_l
+        //
+        // Compute with gemv (y = \alpha A x + \beta y)
+        cublasDgemv(cublasHandle, CUBLAS_OP_T, // handle, op (transpose)
+                    n * n, Ntheta, // nrows, ncols (N.B. column-major order!)
+                    &half, dev_ptr(work_mat_d), n * n, // alpha, A, lda
+                    dev_ptr(invQ_d), 1, // x, incx
+                    &zero, // beta
+                    dev_ptr(result_d), 1); // y, incy
+
+        // ***** Term 2 *****
+        // First step:
+        //   S_ij = \pderiv{C_{jk}}{\theta_i} invQt_k
+        //
+        // Treat \pderiv{C_{jk}}{\theta_i} as (n * Ntheta) * n, treating 'ij' as a single index
+        // and take mat-vec product with invQt:
+        cublasDgemv(cublasHandle, CUBLAS_OP_T, // handle, op (transpose)
+                    n, n * Ntheta, // nrows, ncols (N.B. column-major order!)
+                    &one, dev_ptr(work_mat_d), n, // alpha, A, lda
+                    dev_ptr(invQt_d), 1, // x, incx
+                    &zero, // beta
+                    dev_ptr(work_mat_d), 1); // y, incy
+
+        // Second step:
+        //   R_i += 0.5 S_ij invQt_j
+        cublasDgemv(cublasHandle, CUBLAS_OP_T, // handle, op (transpose)
+                    n, Ntheta, // nrows, ncols (N.B. column-major order!)
+                    &m_half, dev_ptr(work_mat_d), n, // alpha, A, lda
+                    dev_ptr(invQt_d), 1, // x, incx
+                    &one, // beta
+                    dev_ptr(result_d), 1); // y, incy
+
+	// first elements of result (up to meanfunc_nparam) will be from meanfunc,
+	// then the next (D+1) from the Kernel.
+	int meanfunc_nparam = meanfunc_params.rows();
+	if (meanfunc_nparam > 0) { // only copy data to device and do calculation if we need to.
+	  mat meanfunc_deriv = meanfunc->mean_deriv(inputs, meanfunc_params);
+	  thrust::copy(meanfunc_deriv.data(), meanfunc_deriv.data() + (n * meanfunc_nparam), meanfunc_deriv_d.begin());
+	  // product of meanfunc_deriv with invQt
+	  cublasDgemv(cublasHandle, CUBLAS_OP_T, // handle, op (transpose)
+		      n, n * meanfunc_nparam, // nrows, ncols (N.B. column-major order!)
+                    &minusone, dev_ptr(meanfunc_deriv_d), n, // alpha, A, lda
+                    dev_ptr(invQt_d), 1, // x, incx
+                    &zero, // beta
+                    dev_ptr(meanfunc_deriv_d), 1); // y, incy
+	  thrust::copy(meanfunc_deriv_d.begin(), meanfunc_deriv_d.begin() + (meanfunc_nparam), result.data());
+	}
+        thrust::copy(result_d.begin(), result_d.begin() + Ntheta, result.data()+(meanfunc_nparam));
+
+	// fitted nugget - last element of theta is log(nugget)
+	// partial deriv is 0.5*nugget*(trace(invQ) - invQt.invQt)
+	if (nug_type == NUG_FIT) {
+
+	  // trace of invQ
+	  REAL tr_invQ = 0.;
+	  thrust::device_vector<double> tr_invQ_d(1);
+	  trace(n, dev_ptr(invQ_d), dev_ptr(tr_invQ_d), dev_ptr(sum_buffer_d), sum_buffer_size_bytes);
+	  thrust::copy(tr_invQ_d.begin(), tr_invQ_d.end(), &tr_invQ);
+	  // invQt dot invQt
+	  REAL invQtSq = std::numeric_limits<double>::quiet_NaN();
+	  CUBLASDOT(cublasHandle, n, dev_ptr(invQt_d), 1, dev_ptr(invQt_d), 1,
+		    &invQtSq);
+
+	  // set the last element of result, putting it all together
+	  result(result.size()-1) = 0.5 * jitter * (tr_invQ - invQtSq);
+	}
+    }
+
+    // constructor
+  DenseGP_GPU(mat_ref inputs_,
+	      vec_ref targets_,
+	      unsigned int testing_size_,
+	      BaseMeanFunc* mean_,
+	      kernel_type kern_=SQUARED_EXPONENTIAL)
+        : testing_size(testing_size_)
+        , n(inputs_.rows())
+        , D(inputs_.cols())
+        , K_d(n * n, 0.0)
+        , invQ_d(n * n, 0.0)
+        , chol_lower_d(n * n, 0.0)
+        , theta_d(D + 2)
+	, meanfunc_params(1) // resize later if necessary
+        , inputs(inputs_)
+        , targets(targets_)
+	, kern_type(kern_)
+        , kernel(0)
+	, meanfunc(mean_)
+	, nug_type(NUG_ADAPTIVE)
+        , theta_fitted(false)
+        , inputs_d(inputs_.data(), inputs_.data() + D * n)
+        , targets_d(targets_.data(), targets_.data() + n)
+        , logdetC(0.0)
+        , jitter(0.0)
+        , invQt_d(n, 0.0)
+        , testing_d(D * testing_size, 0.0)
+        , work_d(n, 0.0)
+        , work_mat_d(n * testing_size, 0.0)
+        , sum_buffer_size_bytes(0)
+        , result_d(testing_size, 0.0)
+        , kappa_d(testing_size, 0.0)
+        , invQk_d(testing_size * n, 0.0)
+    {
+
+        cublasStatus_t cublas_status = cublasCreate(&cublasHandle);
+        if (cublas_status != CUBLAS_STATUS_SUCCESS)
+        {
+            throw std::runtime_error("CUBLAS initialization error\n");
+        }
+        cusolverStatus_t cusolver_status = cusolverDnCreate(&cusolverHandle);
+        if (cusolver_status != CUSOLVER_STATUS_SUCCESS)
+        {
+            throw std::runtime_error("cuSolver initialization error\n");
+        }
+
+        // CUBLAS potrf workspace
+        int potrfBufferSize;
+        cusolverDnDpotrf_bufferSize(cusolverHandle, CUBLAS_FILL_MODE_LOWER,
+                                    n, dev_ptr(potrf_buffer_d), n,
+                                    &potrfBufferSize);
+        potrf_buffer_d.resize(potrfBufferSize);
+
+        // The following call determines the size of the cub::DeviceReduce::Sum workspace:
+        // sum_buffer_size_bytes is 0 before this call, and the size of sum_buffer_d afterwards.
+        // The end iterators are supplied but are not used.
+        cub::DeviceReduce::Sum(dev_ptr(sum_buffer_d), sum_buffer_size_bytes,
+                               sum_buffer_d.end(), sum_buffer_d.end(), n);
+        sum_buffer_d.resize(sum_buffer_size_bytes);
+
+	if (kern_type == SQUARED_EXPONENTIAL) {
+	  kernel = new SquaredExponentialKernel();
+	} else if (kern_type == MATERN52) {
+	  kernel = new Matern52Kernel();
+	} else throw std::runtime_error("Unrecognized kernel type\n");
+
+	// resize meanfunc_params vector here
+	meanfunc_params.resize(meanfunc->get_n_params(inputs),1);
+	// resize the device vector that will store derivative of mean function
+	meanfunc_deriv_d.resize(meanfunc->get_n_params(inputs) * inputs.rows());
+    }
+};
+
+#endif
diff --git a/mogp_gpu/src/kernel.cu b/mogp_gpu/src/kernel.cu
new file mode 100644
index 00000000..637845e2
--- /dev/null
+++ b/mogp_gpu/src/kernel.cu
@@ -0,0 +1,408 @@
+#include "kernel.hpp"
+#include <stdio.h>
+#include <iostream>
+
+/// Device functions, and global functions (also, confusingly called "kernels") preceded by
+/// "sqexp" for the SquaredExponential kernel, and "mat" for the Matern52 kernel.
+
+
+// Covariance device function
+__device__ REAL sqexp_cov_val_d(int Ninput, REAL *x_d, REAL *y_d, REAL *theta_d)
+{
+    REAL s = 0.0;
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        REAL d_i = x_d[i] - y_d[i];
+        s += d_i * d_i * exp(theta_d[i]);
+    }
+    return exp(-0.5 * s + theta_d[Ninput]);
+}
+
+////////////////////
+__global__ void sqexp_cov_val_kernel(REAL *result_d, int Ninput, REAL *x_d,
+                               REAL *y_d, REAL *theta_d)
+{
+    *result_d = sqexp_cov_val_d(Ninput, x_d, y_d, theta_d);
+}
+
+
+////////////////////
+__global__ void sqexp_cov_diag_kernel(REAL *result_d, int N, int Ninput, REAL *xnew_d,
+                                REAL *xs, REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    if (i < N) {
+        result_d[i] = sqexp_cov_val_d(Ninput, xnew_d + Ninput * i, xs + Ninput * i,
+                                theta_d);
+    }
+}
+
+
+
+////////////////////
+__global__ void sqexp_cov_all_kernel(REAL *result_d, int N, int Ninput, REAL *xnew_d,
+                               REAL *xs_d, REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    if (i < N) {
+        result_d[i] = sqexp_cov_val_d(Ninput, xnew_d, xs_d + Ninput * i, theta_d);
+    }
+}
+
+
+
+////////////////////
+__global__ void sqexp_cov_batch_kernel(REAL *result_d, int Nnew, int N, int Ninput,
+                                 REAL *xsnew_d, REAL *xs_d, REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    int j = blockIdx.y * blockDim.y + threadIdx.y;
+    if (i < N && j < Nnew)
+    {
+        result_d[j + Nnew * i] =
+            sqexp_cov_val_d(Ninput, xsnew_d + Ninput * j, xs_d + Ninput * i, theta_d);
+    }
+}
+
+
+////////////////////
+__device__ void sqexp_cov_deriv_x(REAL *result_d, int Ninput,
+			    const REAL *x_d, const REAL *y_d,
+			    const REAL *theta_d)
+{
+    REAL s = 0.0;
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        REAL d_i = x_d[i] - y_d[i];
+        REAL a = d_i * exp(theta_d[i]);
+	result_d[i] = a;
+        s += d_i * a;
+    }
+    REAL c = -exp(-0.5*s + theta_d[Ninput]);
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        result_d[i] *= c;
+    }
+}
+
+__global__ void sqexp_cov_deriv_x_batch_kernel(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    int j = blockIdx.y * blockDim.y + threadIdx.y;
+    if (i < Nx && j < Ny)
+    {
+        sqexp_cov_deriv_x(result_d + Ninput * (Nx * j + i),
+		    Ninput, xs_d + Ninput * i, ys_d + Ninput * j,
+		    theta_d);
+    }
+}
+
+
+
+
+////////////////////
+__device__ void sqexp_cov_deriv_theta(REAL *result_d, int result_stride, int Ninput,
+                          const REAL *x_d, const REAL *y_d,
+                          const REAL *theta_d)
+{
+    REAL s = 0.0;
+    REAL exp_thetaN = exp(theta_d[Ninput]);
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        REAL d_i = x_d[i] - y_d[i];
+        REAL a = d_i * d_i * exp(theta_d[i]);
+        s += a;
+        result_d[i * result_stride] = -0.5 * a;
+    }
+    s = exp_thetaN * exp(-0.5 * s);
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        result_d[i * result_stride] *= s;
+    }
+    result_d[Ninput * result_stride] = s;
+}
+
+__global__ void sqexp_cov_deriv_theta_batch_kernel(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    int j = blockIdx.y * blockDim.y + threadIdx.y;
+    const int result_stride = Nx * Ny;
+    if (i < Nx && j < Ny)
+    {
+        sqexp_cov_deriv_theta(result_d + Ny * i + j,
+			      result_stride,
+			      Ninput, xs_d + Ninput * j, ys_d + Ninput * i,
+			      theta_d);
+    }
+}
+
+
+// Implementation of the SquaredExponentialKernel
+
+void SquaredExponentialKernel::cov_val_gpu(REAL *result_d, int Ninput, REAL *x_d, REAL *y_d,
+                 REAL *theta_d)
+{
+    sqexp_cov_val_kernel<<<1,1>>>(result_d, Ninput, x_d, y_d, theta_d);
+}
+
+void SquaredExponentialKernel::cov_all_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+                 REAL *theta_d)
+{
+    const int threads_per_block = 256;
+    sqexp_cov_all_kernel<<<10, threads_per_block>>>(
+        result_d, N, Ninput, xnew_d, xs_d, theta_d);
+}
+
+
+void SquaredExponentialKernel::cov_diag_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+                  REAL *theta_d)
+{
+    const int threads_per_block = 256;
+    sqexp_cov_diag_kernel<<<10, threads_per_block>>>(
+        result_d, N, Ninput, xnew_d, xs_d, theta_d);
+}
+
+
+void SquaredExponentialKernel::cov_batch_gpu(REAL *result_d, int Nnew, int N, int Ninput, REAL *xsnew_d,
+                   REAL *xs_d, REAL *theta_d)
+{
+    dim3 threads_per_block(8, 32);
+    dim3 blocks(250, 625);
+    sqexp_cov_batch_kernel<<<blocks, threads_per_block>>>(
+	result_d, Nnew, N, Ninput, xsnew_d, xs_d, theta_d
+							  );
+}
+
+
+void SquaredExponentialKernel::cov_deriv_x_batch_gpu(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    const int Bx = 16, By = 16;
+    dim3 threads_per_block(Bx, By);
+    dim3 blocks((Nx + Bx - 1)/Bx, (Ny + By - 1)/By);
+    sqexp_cov_deriv_x_batch_kernel<<<blocks, threads_per_block>>>(
+        result_d, Ninput, Nx, Ny, xs_d, ys_d, theta_d);
+}
+
+
+void SquaredExponentialKernel::cov_deriv_theta_batch_gpu(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    const int Bx = 16, By = 16;
+    dim3 threads_per_block(Bx, By);
+    dim3 blocks((Nx + Bx - 1)/Bx, (Ny + By - 1)/By);
+    sqexp_cov_deriv_theta_batch_kernel<<<blocks, threads_per_block>>>(
+        result_d, Ninput, Nx, Ny, xs_d, ys_d, theta_d);
+}
+
+
+////////////////////////////////////////////////////////////////////////////////////
+///////////////////////////////// Matern52 /////////////////////////////////////////
+////////////////////////////////////////////////////////////////////////////////////
+
+// Covariance device function
+__device__ REAL mat52_cov_val_d(int Ninput, REAL *x_d, REAL *y_d, REAL *theta_d)
+{
+    REAL s = 0.0;
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        REAL d_i = x_d[i] - y_d[i];
+        s += d_i * d_i * exp(theta_d[i]);
+    }
+    REAL r = sqrt(s);
+    return (1 + r*sqrt(5.) + (5./3.)*s) *  exp(theta_d[Ninput] - sqrt(5.)*r);
+}
+
+////////////////////
+__global__ void mat52_cov_val_kernel(REAL *result_d, int Ninput, REAL *x_d,
+                               REAL *y_d, REAL *theta_d)
+{
+    *result_d = mat52_cov_val_d(Ninput, x_d, y_d, theta_d);
+}
+
+////////////////////
+__global__ void mat52_cov_diag_kernel(REAL *result_d, int N, int Ninput, REAL *xnew_d,
+                                REAL *xs, REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    if (i < N) {
+        result_d[i] = mat52_cov_val_d(Ninput, xnew_d + Ninput * i, xs + Ninput * i,
+				    theta_d);
+    }
+}
+
+////////////////////
+__global__ void mat52_cov_all_kernel(REAL *result_d, int N, int Ninput, REAL *xnew_d,
+				   REAL *xs_d, REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    if (i < N) {
+        result_d[i] = mat52_cov_val_d(Ninput, xnew_d, xs_d + Ninput * i, theta_d);
+    }
+}
+
+////////////////////
+__global__ void mat52_cov_batch_kernel(REAL *result_d, int Nnew, int N, int Ninput,
+				     REAL *xsnew_d, REAL *xs_d, REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    int j = blockIdx.y * blockDim.y + threadIdx.y;
+    if (i < N && j < Nnew)
+    {
+        result_d[j + Nnew * i] =
+            mat52_cov_val_d(Ninput, xsnew_d + Ninput * j, xs_d + Ninput * i, theta_d);
+    }
+}
+
+////////////////////
+__device__ void mat52_cov_deriv_x(REAL *result_d, int Ninput,
+			    const REAL *x_d, const REAL *y_d,
+			    const REAL *theta_d)
+{
+  // dK/dx = dr/dx * dK/dr
+
+  //drdx
+    REAL s = 0.0;
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        REAL d_i = x_d[i] - y_d[i];
+        REAL a = d_i * exp(theta_d[i]);
+	result_d[i] = a;
+        s += d_i * a;
+    }
+    REAL r = sqrt(s);
+    if (r > 0.) {
+      REAL dkdr = -5./3.*r*(1+sqrt(5.)*r)*exp(-1.*sqrt(5.)*r);
+      dkdr *= exp(theta_d[Ninput]);
+      for (unsigned int i=0; i < Ninput; i++)
+	{
+	  result_d[i] = dkdr * result_d[i] / r;
+	}
+    }
+}
+
+__global__ void mat52_cov_deriv_x_batch_kernel(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    int j = blockIdx.y * blockDim.y + threadIdx.y;
+    if (i < Nx && j < Ny)
+    {
+        mat52_cov_deriv_x(result_d + Ninput * (Nx * j + i),
+		    Ninput, xs_d + Ninput * i, ys_d + Ninput * j,
+		    theta_d);
+    }
+}
+
+////////////////////
+__device__ void mat52_cov_deriv_theta(REAL *result_d, int result_stride, int Ninput,
+                          const REAL *x_d, const REAL *y_d,
+                          const REAL *theta_d)
+{
+
+    //dK/dtheta = exp(thetaN) * dK/dr * dr/dtheta
+    REAL exp_thetaN = exp(theta_d[Ninput]);
+    REAL s = 0.0;
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+        REAL d_i = x_d[i] - y_d[i];
+        REAL a = d_i * d_i * exp(theta_d[i]);
+	result_d[i * result_stride] = 0.5 * a;
+        s += a;
+    }
+    REAL r = sqrt(s);
+    REAL dkdr = 0.;
+    if (r > 0.) {
+      dkdr = -5./3.*r*(1+sqrt(5.)*r)*exp(-1.*sqrt(5.)*r);
+      dkdr *= exp_thetaN;
+    }
+
+    for (unsigned int i=0; i < Ninput; i++)
+    {
+      if (r > 0.) result_d[i * result_stride] = dkdr * result_d[i * result_stride] / r;
+    }
+    result_d[Ninput * result_stride] = (1 + r*sqrt(5.) + (5./3.)*s) *  exp(theta_d[Ninput] - sqrt(5.)*r);
+}
+
+__global__ void mat52_cov_deriv_theta_batch_kernel(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    int i = blockIdx.x * blockDim.x + threadIdx.x;
+    int j = blockIdx.y * blockDim.y + threadIdx.y;
+    const int result_stride = Nx * Ny;
+    if (i < Nx && j < Ny)
+    {
+        mat52_cov_deriv_theta(result_d + Ny * i + j,
+			      result_stride,
+			      Ninput, xs_d + Ninput * j, ys_d + Ninput * i,
+			      theta_d);
+    }
+}
+
+/// implementation of Matern52 kernel
+
+void Matern52Kernel::cov_val_gpu(REAL *result_d, int Ninput, REAL *x_d, REAL *y_d,
+                 REAL *theta_d)
+{
+    mat52_cov_val_kernel<<<1,1>>>(result_d, Ninput, x_d, y_d, theta_d);
+}
+
+void Matern52Kernel::cov_all_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+                 REAL *theta_d)
+{
+    const int threads_per_block = 256;
+    mat52_cov_all_kernel<<<10, threads_per_block>>>(
+        result_d, N, Ninput, xnew_d, xs_d, theta_d);
+}
+
+
+void Matern52Kernel::cov_diag_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+                  REAL *theta_d)
+{
+    const int threads_per_block = 256;
+    mat52_cov_diag_kernel<<<10, threads_per_block>>>(
+        result_d, N, Ninput, xnew_d, xs_d, theta_d);
+}
+
+
+void Matern52Kernel::cov_batch_gpu(REAL *result_d, int Nnew, int N, int Ninput, REAL *xsnew_d,
+                   REAL *xs_d, REAL *theta_d)
+{
+    dim3 threads_per_block(8, 32);
+    dim3 blocks(250, 625);
+    mat52_cov_batch_kernel<<<blocks, threads_per_block>>>(
+	result_d, Nnew, N, Ninput, xsnew_d, xs_d, theta_d
+							  );
+}
+
+
+void Matern52Kernel::cov_deriv_x_batch_gpu(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    const int Bx = 16, By = 16;
+    dim3 threads_per_block(Bx, By);
+    dim3 blocks((Nx + Bx - 1)/Bx, (Ny + By - 1)/By);
+    mat52_cov_deriv_x_batch_kernel<<<blocks, threads_per_block>>>(
+        result_d, Ninput, Nx, Ny, xs_d, ys_d, theta_d);
+}
+
+
+void Matern52Kernel::cov_deriv_theta_batch_gpu(
+    REAL *result_d, int Ninput, int Nx, int Ny, const REAL *xs_d,
+    const REAL *ys_d, const REAL *theta_d)
+{
+    const int Bx = 16, By = 16;
+    dim3 threads_per_block(Bx, By);
+    dim3 blocks((Nx + Bx - 1)/Bx, (Ny + By - 1)/By);
+    mat52_cov_deriv_theta_batch_kernel<<<blocks, threads_per_block>>>(
+        result_d, Ninput, Nx, Ny, xs_d, ys_d, theta_d);
+}
\ No newline at end of file
diff --git a/mogp_gpu/src/kernel.hpp b/mogp_gpu/src/kernel.hpp
new file mode 100644
index 00000000..69cd31f7
--- /dev/null
+++ b/mogp_gpu/src/kernel.hpp
@@ -0,0 +1,179 @@
+#ifndef KERNEL_HPP
+#define KERNEL_HPP
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+
+#include "types.hpp"
+
+#define CUBLASDOT cublasDdot
+#define CUBLASGEMV cublasDgemv
+
+
+class BaseKernel {
+
+public:
+
+  virtual ~BaseKernel(){};
+
+  // Performs a single evaluation of the covariance function, for inputs
+  // x and y; that is, Cov(x(:), y(:); theta).
+  virtual void cov_val_gpu(REAL *result_d, int Ninput, REAL *x_d, REAL *y_d,
+			   REAL *theta_d) = 0;
+
+// Computes the vector k_i = Cov(xnew(:), xs(i,:); theta).
+  virtual void cov_all_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+			   REAL *theta_d) = 0;
+
+  // Computes the vector k_i = Cov(xnew(i,:), xs(i,:); theta).
+  virtual void cov_diag_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+		    REAL *theta_d) = 0;
+
+  // Computes the submatrix K_ij = Cov(xsnew(i,:), xs(j,:); theta)
+  virtual void cov_batch_gpu(REAL *result_d, int Nnew, int N, int Ninput, REAL *xsnew_d,
+			     REAL *xs_d, REAL *theta_d) = 0;
+
+  // Computes the matrix (dC/dtheta)(xs(i,:), ys(j,:); theta)
+  virtual void cov_deriv_theta_batch_gpu(REAL *result_d, int Ninput, int Nx, int Ny,
+					 const REAL *xs_d, const REAL *ys_d,
+					 const REAL *theta_d) = 0;
+
+  // Computes the matrix (dC/dx)(xs(i,:), ys(j,:); theta)
+  virtual void cov_deriv_x_batch_gpu(REAL *result_d, int Ninput, int Nx, int Ny,
+				     const REAL *xs_d, const REAL *ys_d,
+				     const REAL *theta_d) = 0;
+
+  // For pybind11, implement method to replicate python interface for evaluating cov matrix
+  mat kernel_f(mat x1, mat x2, vec params) {
+    int N = x1.rows();
+    int Nnew = x2.rows();
+    int D = x1.cols();
+    assert (x2.cols() == D);
+    thrust::device_vector<REAL> result_d(N*Nnew);
+    thrust::device_vector<REAL> x1_d(x1.data(), x1.data()+N*D);
+    thrust::device_vector<REAL> x2_d(x2.data(), x2.data()+Nnew*D);
+    thrust::device_vector<REAL> params_d(params.data(), params.data()+D+1);
+
+    cov_batch_gpu(dev_ptr(result_d), Nnew, N, D, dev_ptr(x2_d), dev_ptr(x1_d), dev_ptr(params_d));
+    mat result(N,Nnew);
+    thrust::copy(result_d.begin(), result_d.end(), result.data());
+    return result;
+  }
+
+  // For pybind11, implement method to replicate python interface deriv of
+  // cov matrix wrt first set of inputs
+  // NOTE - this will return values as a 1D array of size N*Nnew*D, unlike
+  // the pure Python version that will return a 3D array of size [D, N, Nnew]
+  vec kernel_inputderiv(mat x1, mat x2, vec params) {
+    int N = x1.rows();
+    int Nnew = x2.rows();
+    int D = x1.cols();
+    assert (x2.cols() == D);
+    thrust::device_vector<REAL> result_d(N * Nnew * D);
+    thrust::device_vector<REAL> x1_d(x1.data(), x1.data()+N*D);
+    thrust::device_vector<REAL> x2_d(x2.data(), x2.data()+Nnew*D);
+    thrust::device_vector<REAL> params_d(params.data(), params.data()+D+1);
+    cov_deriv_x_batch_gpu(dev_ptr(result_d), D, N, Nnew,
+			      dev_ptr(x1_d), dev_ptr(x2_d), dev_ptr(params_d));
+    vec result(N * Nnew * D);
+    thrust::copy(result_d.begin(), result_d.end(), result.data());
+    return result;
+  }
+
+  // For pybind11, implement method to replicate python interface deriv of
+  //  covariance matrix wrt theta.
+  // NOTE - this will return values as a 1D array of size N*Nnew*n_params, unlike
+  // the pure Python version that will return a 3D array of size [n_params, N, Nnew]
+  vec kernel_deriv(mat x1, mat x2, vec params) {
+    int N = x1.rows();
+    int Nnew = x2.rows();
+    int D = x1.cols();
+    assert (x2.cols() == D);
+    int n_params = D+1;
+    thrust::device_vector<REAL> result_d(N * Nnew * n_params);
+    thrust::device_vector<REAL> x1_d(x1.data(), x1.data()+N*D);
+    thrust::device_vector<REAL> x2_d(x2.data(), x2.data()+Nnew*D);
+    thrust::device_vector<REAL> params_d(params.data(), params.data()+n_params);
+    cov_deriv_theta_batch_gpu(dev_ptr(result_d), D, N, Nnew,
+			      dev_ptr(x1_d), dev_ptr(x2_d), dev_ptr(params_d));
+    vec result(N*Nnew*n_params);
+    thrust::copy(result_d.begin(), result_d.end(), result.data());
+    return result;
+  }
+
+};
+
+
+class SquaredExponentialKernel : public BaseKernel {
+public:
+
+  // Squared exponential covariance function
+
+  // Performs a single evaluation of the covariance function, for inputs
+  // x and y; that is, Cov(x(:), y(:); theta).
+  void cov_val_gpu(REAL *result_d, int Ninput, REAL *x_d, REAL *y_d,
+		   REAL *theta_d);
+
+  // Computes the vector k_i = Cov(xnew(:), xs(i,:); theta).
+  void cov_all_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+		   REAL *theta_d);
+
+  // Computes the vector k_i = Cov(xnew(i,:), xs(i,:); theta).
+  void cov_diag_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+		    REAL *theta_d);
+
+  // Computes the submatrix K_ij = Cov(xsnew(i,:), xs(j,:); theta)
+  void cov_batch_gpu(REAL *result_d, int Nnew, int N, int Ninput, REAL *xsnew_d,
+		     REAL *xs_d, REAL *theta_d);
+
+  // Computes the matrix (dC/dtheta)(xs(i,:), ys(j,:); theta)
+  void cov_deriv_theta_batch_gpu(REAL *result_d, int Ninput, int Nx, int Ny,
+				 const REAL *xs_d, const REAL *ys_d,
+				 const REAL *theta_d);
+
+  // Computes the matrix (dC/dx)(xs(i,:), ys(j,:); theta)
+  void cov_deriv_x_batch_gpu(REAL *result_d, int Ninput, int Nx, int Ny,
+			     const REAL *xs_d, const REAL *ys_d,
+			     const REAL *theta_d);
+
+};
+
+
+class Matern52Kernel : public BaseKernel {
+public:
+
+  // Squared exponential covariance function
+
+  // Performs a single evaluation of the covariance function, for inputs
+  // x and y; that is, Cov(x(:), y(:); theta).
+  void cov_val_gpu(REAL *result_d, int Ninput, REAL *x_d, REAL *y_d,
+		   REAL *theta_d);
+
+  // Computes the vector k_i = Cov(xnew(:), xs(i,:); theta).
+  void cov_all_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+		   REAL *theta_d);
+
+  // Computes the vector k_i = Cov(xnew(i,:), xs(i,:); theta).
+  void cov_diag_gpu(REAL *result_d, int N, int Ninput, REAL *xnew_d, REAL *xs_d,
+		    REAL *theta_d);
+
+  // Computes the submatrix K_ij = Cov(xsnew(i,:), xs(j,:); theta)
+  void cov_batch_gpu(REAL *result_d, int Nnew, int N, int Ninput, REAL *xsnew_d,
+		     REAL *xs_d, REAL *theta_d);
+
+  // Computes the matrix (dC/dtheta)(xs(i,:), ys(j,:); theta)
+  void cov_deriv_theta_batch_gpu(REAL *result_d, int Ninput, int Nx, int Ny,
+				 const REAL *xs_d, const REAL *ys_d,
+				 const REAL *theta_d);
+
+  // Computes the matrix (dC/dx)(xs(i,:), ys(j,:); theta)
+  void cov_deriv_x_batch_gpu(REAL *result_d, int Ninput, int Nx, int Ny,
+			     const REAL *xs_d, const REAL *ys_d,
+			     const REAL *theta_d);
+
+};
+
+#endif
diff --git a/mogp_gpu/src/meanfunc.hpp b/mogp_gpu/src/meanfunc.hpp
new file mode 100644
index 00000000..a7d098ad
--- /dev/null
+++ b/mogp_gpu/src/meanfunc.hpp
@@ -0,0 +1,258 @@
+#ifndef MEANFUNC_HPP
+#define MEANFUNC_HPP
+
+#include <string>
+#include <vector>
+#include <utility>
+
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+
+#include "types.hpp"
+
+
+class BaseMeanFunc {
+
+public:
+
+  virtual ~BaseMeanFunc(){};
+
+  // Performs a single evaluation of the mean function at specific inputs
+  virtual vec mean_f(mat_ref xs,
+		     vec_ref params) = 0;
+
+  //Derivative wrt theta of the mean function at specific inputs
+  virtual mat mean_deriv(mat_ref xs,
+			 vec_ref params) = 0;
+
+
+  //Derivative wrt x of the mean function at specific inputs
+  virtual mat mean_inputderiv(mat_ref xs,
+			      vec_ref params) = 0;
+
+  // Return the number of parameters in the function.
+  virtual int get_n_params(mat_ref xs) = 0;
+
+};
+
+class ZeroMeanFunc : public BaseMeanFunc {
+
+public:
+
+  // Zero mean function
+
+  ZeroMeanFunc() {};
+
+  virtual ~ZeroMeanFunc() {};
+
+  inline virtual int get_n_params(mat_ref xs) { return 0; }
+
+
+  inline virtual vec mean_f(mat_ref xs,
+			    vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    vec result = vec::Constant(xs.rows(),1, 0.);
+    return result;
+  }
+
+  inline virtual mat mean_deriv(mat_ref xs,
+				vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    mat result = mat::Constant(xs.rows(),1, 0.);
+    return result;
+  }
+
+  inline virtual mat mean_inputderiv(mat_ref xs,
+				     vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    mat result = mat::Constant(xs.cols(),xs.rows(), 0.);
+    return result;
+  }
+
+};
+
+class FixedMeanFunc : public BaseMeanFunc {
+
+public:
+
+  // Fixed mean function
+
+  FixedMeanFunc(REAL value_): value(value_) {};
+
+  virtual ~FixedMeanFunc() {} ;
+
+  inline virtual int get_n_params(mat_ref xs) { return 0; }
+
+
+  inline virtual vec mean_f(mat_ref xs,
+			    vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    vec result = vec::Constant(xs.rows(),1, value);
+    return result;
+  }
+
+  inline virtual mat mean_deriv(mat_ref xs,
+				vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    mat result = mat::Constant(xs.rows(),1, 0.);
+    return result;
+  }
+
+  inline virtual mat mean_inputderiv(mat_ref xs,
+				     vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    mat result = mat::Constant(xs.cols(),xs.rows(), 0.);
+    return result;
+  }
+
+private:
+  REAL value;
+
+};
+
+
+class ConstMeanFunc : public BaseMeanFunc {
+
+public:
+
+  // Constant mean function
+
+  ConstMeanFunc() {};
+
+  virtual ~ConstMeanFunc() {};
+
+  inline virtual int get_n_params(mat_ref xs) { return 1; }
+
+
+  inline virtual vec mean_f(mat_ref xs,
+			    vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    vec result = vec::Constant(xs.rows(),1,params(0));
+    return result;
+  }
+
+  inline virtual mat mean_deriv(mat_ref xs,
+				vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    mat result = mat::Constant(xs.rows(),1, 1.);
+    return result;
+  }
+
+  inline virtual mat mean_inputderiv(mat_ref xs,
+				     vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length "+ std::to_string(get_n_params(xs)));
+    mat result = mat::Constant(xs.cols(),xs.rows(), 0.);
+    return result;
+  }
+
+
+};
+
+
+class PolyMeanFunc : public BaseMeanFunc {
+
+public:
+
+  // Polynomial mean function
+
+  // constructor takes a vector of pair<int, int>.  In each pair,
+  // the first number represents the dimension index of the input, and
+  // the second is the power to which that will be raised.
+  // So, for example, if we have 1D input, and we want a 2nd order polynomial, we would have
+  // <0,1>,<0,2>, corresponding to "theta_0 + theta_1 * x[0] + theta_2 * x[0]^2"
+  // If we have 2D input, and we want a function linear in both dimensions, we would have
+  // <0,1>, <1,1>, corresponding to "theta_0 + theta_1 * x[0] + theta_2 * x[1]"
+  // The number of parameters is always 1 greater than the number of pairs in the vector,
+  // as we allow for a const term as well as the coefficients multiplying these inputs/powers.
+  // The const term will always be the first of the parameters.
+  PolyMeanFunc(std::vector<std::pair<int,int> > dp) : dims_powers(dp) {};
+
+  virtual ~PolyMeanFunc() {};
+
+  inline virtual int get_n_params(mat_ref xs) { return dims_powers.size() + 1; }
+
+  inline virtual vec mean_f(mat_ref xs,
+			    vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length " +  std::to_string(get_n_params(xs)));
+    vec result(xs.rows());
+    for (unsigned int i=0; i< xs.rows(); ++i) {
+      // have the const term as the first parameter
+      REAL val = params(0);
+      for (unsigned int j=0; j< dims_powers.size(); ++j) {
+	int dim_index = dims_powers[j].first;
+	int power = dims_powers[j].second;
+	if (dim_index > xs.cols()-1) throw std::runtime_error("Dimension index must be less than "+ std::to_string(xs.cols()));
+	val += params(j+1) * pow(xs(i,dim_index), power);
+      }
+
+      result(i) = val;
+    }
+    return result;
+  }
+
+  inline virtual mat mean_deriv(mat_ref xs,
+				vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length " + std::to_string(get_n_params(xs)));
+    mat result(params.rows(), xs.rows());
+    for (unsigned int i=0; i< xs.rows(); ++i) {
+      // deriv wrt the first param (the const term) is always 1.
+      result(0,i) = 1.;
+      // for the rest of the rows, evaluate x[index[j]]^pow[j]
+      for (unsigned int j=0; j< dims_powers.size(); ++j) {
+	int dim_index = dims_powers[j].first;
+	int power = dims_powers[j].second;
+	if (dim_index > xs.cols()-1) throw std::runtime_error("Dimension index must be less than D");
+	result(j+1,i) = pow(xs(i,dim_index), power);
+      }
+    }
+    return result;
+  }
+
+  inline virtual mat mean_inputderiv(mat_ref xs,
+				     vec_ref params) {
+    if ( params.rows() != get_n_params(xs))
+      throw std::runtime_error("Expected params list of length " + std::to_string(get_n_params(xs)));
+    mat result = mat::Constant(xs.cols(),xs.rows(), 0.);
+    // loop through all inputs
+    for (unsigned int i=0; i< xs.rows(); ++i) {
+      // loop through all dimensions of the input
+      for (unsigned int j=0; j< xs.cols(); ++j) {
+	REAL val = 0.;
+	// loop through all terms of the formula
+	for (unsigned int k=0; k< dims_powers.size(); ++k) {
+	  int dim_index = dims_powers[k].first;
+	  int power = dims_powers[k].second;
+	  if (dim_index > xs.cols()-1) throw std::runtime_error("Dimension index must be less than D");
+	  // if this term deals with dimension j, differentiate and add to val
+	  if (dim_index == j) {
+	    val += params(k+1) * power * pow(xs(i,j),(power-1));
+	  }
+	}
+	result(j,i) = val;
+      }
+    }
+    return result;
+  }
+
+private:
+  std::vector<std::pair<int, int> > dims_powers;
+
+};
+
+
+#endif
diff --git a/mogp_gpu/src/strided_range.hpp b/mogp_gpu/src/strided_range.hpp
new file mode 100644
index 00000000..626ce4fb
--- /dev/null
+++ b/mogp_gpu/src/strided_range.hpp
@@ -0,0 +1,88 @@
+#ifndef STRIDED_RANGE_HPP
+#define STRIDED_RANGE_HPP
+
+#include <thrust/iterator/counting_iterator.h>
+#include <thrust/iterator/transform_iterator.h>
+#include <thrust/iterator/permutation_iterator.h>
+#include <thrust/functional.h>
+#include <thrust/device_vector.h>
+#include <iostream>
+
+// ----------------------------------------
+// The `strided_range` class, from a portion of the file found at:
+//
+//      https://github.com/thrust/thrust/blob/master/examples/strided_range.cu
+//
+//  Copyright NVIDIA Corporation
+//
+//  Licensed under the Apache License, Version 2.0 (the "License")
+//
+//  You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+//  Unless required by applicable law or agreed to in writing, software
+//  distributed under the License is distributed on an "AS IS" BASIS,
+//  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+//  See the License for the specific language governing permissions and
+//  limitations under the License.
+// 
+template <typename Iterator>
+class strided_range
+{
+    public:
+
+    typedef typename thrust::iterator_difference<Iterator>::type difference_type;
+
+    struct stride_functor : public thrust::unary_function<difference_type,difference_type>
+    {
+        difference_type stride;
+
+        stride_functor(difference_type stride)
+            : stride(stride) {}
+
+        __host__ __device__
+        difference_type operator()(const difference_type& i) const
+        { 
+            return stride * i;
+        }
+    };
+
+    typedef typename thrust::counting_iterator<difference_type>                   CountingIterator;
+    typedef typename thrust::transform_iterator<stride_functor, CountingIterator> TransformIterator;
+    typedef typename thrust::permutation_iterator<Iterator,TransformIterator>     PermutationIterator;
+
+    // type of the strided_range iterator
+    typedef PermutationIterator iterator;
+
+    // construct strided_range for the range [first,last)
+    strided_range(Iterator first, Iterator last, difference_type stride)
+        : first(first), last(last), stride(stride) {}
+   
+    iterator begin(void) const
+    {
+        return PermutationIterator(first, TransformIterator(CountingIterator(0), stride_functor(stride)));
+    }
+
+    iterator end(void) const
+    {
+        return begin() + ((last - first) + (stride - 1)) / stride;
+    }
+    
+    protected:
+    Iterator first;
+    Iterator last;
+    difference_type stride;
+};
+
+// ----------------------------------------
+
+template <typename Iterator>
+strided_range<Iterator> make_strided_range(
+    Iterator first, Iterator last,
+    typename thrust::iterator_difference<Iterator>::type stride)
+{
+    return strided_range<Iterator>(first, last, stride);
+}
+
+#endif // STRIDED_RANGE_HPP
diff --git a/mogp_gpu/src/types.hpp b/mogp_gpu/src/types.hpp
new file mode 100644
index 00000000..27db46d1
--- /dev/null
+++ b/mogp_gpu/src/types.hpp
@@ -0,0 +1,34 @@
+#ifndef TYPES_HPP
+#define TYPES_HPP
+
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <Eigen/Dense>
+
+// ----------------------------------------
+// ------- Some useful typedefs for vectors and matrices
+
+typedef double REAL;
+typedef int obs_kind;
+typedef typename Eigen::Matrix<REAL, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor> mat;
+typedef typename Eigen::Ref<Eigen::Matrix<REAL, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor> > mat_ref;
+typedef typename Eigen::Matrix<REAL, Eigen::Dynamic, 1> vec;
+typedef typename Eigen::Ref<Eigen::Matrix<REAL, Eigen::Dynamic, 1> > vec_ref;
+
+/// Extract the raw pointer from a device vector
+template <typename T>
+T *dev_ptr(thrust::device_vector<T>& dv)
+{
+    return dv.data().get();
+}
+
+// ----------------------------------------
+// enum to allow the python code to select the type of "nugget"
+enum nugget_type {NUG_ADAPTIVE, NUG_FIT, NUG_FIXED};
+
+// enum to allow python code to select Kernel function
+enum kernel_type {SQUARED_EXPONENTIAL, MATERN52};
+
+#endif
diff --git a/mogp_gpu/src/util.cu b/mogp_gpu/src/util.cu
new file mode 100644
index 00000000..1f1fab8d
--- /dev/null
+++ b/mogp_gpu/src/util.cu
@@ -0,0 +1,57 @@
+#include "util.hpp"
+
+// Create an NxN Identity matrix A on the device
+__global__ void identity_kernel(int N, double *A)
+{
+    int i = blockDim.x*blockIdx.x + threadIdx.x;
+    int j = blockDim.y*blockIdx.y + threadIdx.y;
+    if (i < N && j < N) A[i + N * j] = (i == j)?1.0:0.0;
+}
+
+void identity_device(int N, double *A)
+{
+    const int Nx = 8, Ny = 8;
+    dim3 threads_per_block(Nx, Ny);
+    dim3 blocks((N + Nx - 1)/Nx, (N + Ny - 1)/Ny);
+    identity_kernel<<<blocks, threads_per_block>>>(N, A);
+}
+
+// Add b to the diagonal of NxN matrix A
+__global__ void add_diagonal_kernel(int N, double b, double *A)
+{
+    int i = blockDim.x*blockIdx.x + threadIdx.x;
+    int j = blockDim.y*blockIdx.y + threadIdx.y;
+    if (i < N && j < N && i == j) A[i + N * j] += b;
+}
+
+void add_diagonal(int N, double b, double *A)
+{
+    const int Nx = 8, Ny = 8;
+    dim3 threads_per_block(Nx, Ny);
+    dim3 blocks((N + Nx - 1)/Nx, (N + Ny - 1)/Ny);
+    add_diagonal_kernel<<<blocks, threads_per_block>>>(N, b, A);
+}
+
+// CUDA kernel for summing log diagonal elements of a matrix,
+// using cub::DeviceReduce
+
+struct LogSq : public thrust::unary_function<double, double>
+{
+    __host__ __device__ double operator()(double x) const { return 2.0 * log(x); }
+};
+
+void sum_log_diag(int N, double *A, double *result, double *work, size_t work_size)
+{
+    auto transform_it = thrust::make_transform_iterator(A, LogSq());
+    auto sr = make_strided_range(transform_it, transform_it + N*N, N+1);
+
+    cub::DeviceReduce::Sum(work, work_size, sr.begin(), result, N);
+}
+
+// Implementation of trace using cub::DeviceReduce
+
+void trace(int N, double *A, double *result, double *work, size_t work_size)
+{
+    auto sr = make_strided_range(A, A + N*N, N+1);
+    cub::DeviceReduce::Sum(work, work_size, sr.begin(), result, N);
+}
\ No newline at end of file
diff --git a/mogp_gpu/src/util.hpp b/mogp_gpu/src/util.hpp
new file mode 100644
index 00000000..e6753598
--- /dev/null
+++ b/mogp_gpu/src/util.hpp
@@ -0,0 +1,57 @@
+#ifndef UTIL_HPP
+#define UTIL_HPP
+
+#include <cuda_runtime.h>
+#include <cublas_v2.h>
+#include <cusolverDn.h>
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+#include <thrust/copy.h>
+#include <cub/cub.cuh>
+
+#include <Eigen/Dense>
+
+#include "types.hpp"
+#include "strided_range.hpp"
+
+#define WARP_SIZE 32
+#define FULL_MASK 0xffffffff
+
+// ----------------------------------------------
+// ----   Utility functions ---------------------
+
+
+/// Fail if a recent cusolver call did not succeed
+inline void check_cusolver_status(cusolverStatus_t status, int info_h)
+{
+    if (status || info_h) {
+	std::string msg;
+	std::stringstream smsg(msg);
+	smsg << "Error in potrf: return code " << status << ", info " << info_h;
+	throw std::runtime_error(smsg.str());
+    }
+}
+
+/// Can a usable CUDA capable device be found?
+inline bool have_compatible_device(void)
+{
+    int device_count = 0;
+    cudaError_t err = cudaGetDeviceCount(&device_count);
+
+    return (err == cudaSuccess && device_count > 0);
+
+}
+// ----------------------------------------------
+
+
+void identity_device(int N, double *A);
+void add_diagonal(int N, double b, double *A);
+void sum_log_diag(int N, double *A, double *result, double *work, size_t work_size);
+void trace(int N, double *A, double *result, double *work, size_t work_size);
+
+
+#endif
diff --git a/setup.py b/setup.py
index 1192b95d..115934d5 100644
--- a/setup.py
+++ b/setup.py
@@ -1,8 +1,14 @@
+import os
 import setuptools
 
+from distutils.extension import Extension
+from distutils.command.build_ext import build_ext
+
+import numpy as np
+
 # version information
 MAJOR = 0
-MINOR = 4
+MINOR = 5
 MICRO = 0
 PRERELEASE = 0
 ISRELEASED = True
@@ -29,6 +35,95 @@ def write_version_file(version):
 with open("README.md", "r") as fh:
     long_description = fh.read()
 
+
+def get_cuda_config():
+    """
+    Find the nvcc executable in the path, and
+    return a dictionary with keys 'home', 'nvcc', 'include', 'lib64'.
+    """
+    path = os.environ["PATH"]
+    exe = "nvcc"
+    binpath = None
+    for location in path.split(os.pathsep):
+        if os.path.exists(os.path.join(location, exe)):
+            binpath = os.path.abspath(os.path.join(location, exe))
+            break
+    if not binpath:
+        print("Unable to find 'nvcc' in path")
+        return {}
+    home = os.path.dirname(os.path.dirname(binpath))
+    include = os.path.join(home, "include")
+    lib64 = os.path.join(home, "lib64")
+    return {
+        "home": home,
+        "nvcc": binpath,
+        "include": include,
+        "lib64": lib64
+    }
+
+
+def customize_compiler_for_nvcc(self):
+    """
+    Taken from SO
+    https://stackoverflow.com/questions/10034325/can-python-distutils-compile-cuda-code
+    by Robert T. McGibbon.
+    Modify the _compile of distutils extension builder.
+    """
+    # add .cu to list of extensions compile knows about
+    self.src_extensions += ['.cu',".hpp"]
+    # save references to original methods
+    default_compiler_so = self.compiler_so
+    super = self._compile
+    # redefine _compile method.
+    def _compile(obj, src, ext, cc_args, extra_postargs, pp_opts):
+        if os.path.splitext(src)[1] == '.cu':
+            # use nvcc for .cu files
+            cuda_config = get_cuda_config()
+            if not "nvcc" in cuda_config.keys():
+                return
+            self.set_executable('compiler_so', cuda_config['nvcc'])
+            # use subset of extra_postargs
+            postargs = extra_postargs['nvcc']
+        else:
+            postargs = extra_postargs['gcc']
+
+        super(obj, src, ext, cc_args, postargs, pp_opts)
+        # reset default compiler_so
+        self.compiler_so = default_compiler_so
+    # replace the class default _compile method with this one
+    self._compile = _compile
+
+# Run the custom compiler
+class custom_build_ext(build_ext):
+    def build_extensions(self):
+        customize_compiler_for_nvcc(self.compiler)
+        build_ext.build_extensions(self)
+
+cuda_config = get_cuda_config()
+
+# we only want to add the mogp_gpu extension if we have cuda compiler
+ext_modules = []
+if len(cuda_config) > 0:
+    import pybind11
+    pybind_include = pybind11.get_include()
+    numpy_include = np.get_include()
+    ext = Extension("libgpgpu",
+                    sources=["mogp_gpu/src/gp_gpu.cu",
+                             "mogp_gpu/src/kernel.cu",
+                             "mogp_gpu/src/util.cu"],
+                    library_dirs=[cuda_config["lib64"]],
+                    libraries=["cudart","cublas","cusolver"],
+                    runtime_library_dirs=[cuda_config["lib64"]],
+                    extra_compile_args={"gcc":["-std=c++14"],
+                                    "nvcc": ["--compiler-options",
+                                             "-O3,-Wall,-shared,-std=c++14,-fPIC",
+                                             "-arch=sm_60",
+                                             "--generate-code","arch=compute_37,code=sm_37",
+                                             "--generate-code","arch=compute_60,code=sm_60"
+                                    ]},
+                    include_dirs=[numpy_include, pybind_include, cuda_config["include"],"mogp_gpu/src"])
+    ext_modules.append(ext)
+
 setuptools.setup(name='mogp_emulator',
       version=version,
       description='Tool for Multi-Output Gaussian Process Emulators',
@@ -43,5 +138,8 @@ def write_version_file(version):
       author='Alan Turing Institute Research Engineering Group',
       author_email='edaub@turing.ac.uk',
       packages=setuptools.find_packages(),
-      license=['MIT'],
-      install_requires=['numpy', 'scipy'])
+      license='MIT',
+      ext_modules=ext_modules,
+      cmdclass={"build_ext": custom_build_ext},
+      install_requires=['numpy', 'scipy'],
+      zip_safe=False)


Test patch:
diff --git a/mogp_emulator/tests/test_GPUMeanFunction.py b/mogp_emulator/tests/test_GPUMeanFunction.py
new file mode 100644
index 00000000..3d1d7585
--- /dev/null
+++ b/mogp_emulator/tests/test_GPUMeanFunction.py
@@ -0,0 +1,131 @@
+import numpy as np
+import pytest
+from numpy.testing import assert_allclose
+from ..MeanFunction import MeanFunction
+from .. import LibGPGPU
+
+from ..GaussianProcessGPU import parse_meanfunc_formula
+
+GPU_NOT_FOUND_MSG = "A compatible GPU could not be found or the GPU library (libgpgpu) could not be loaded"
+
+@pytest.fixture
+def x():
+    return np.array([[1., 2., 3.], [4., 5., 6.]])
+
+@pytest.fixture
+def params():
+    return np.array([1., 2., 4.])
+
+@pytest.fixture
+def zeroparams():
+    return np.zeros(0)
+
+@pytest.fixture
+def oneparams():
+    return np.ones(1)
+
+@pytest.fixture
+def dx():
+    return 1.e-6
+
+@pytest.mark.skipif(not LibGPGPU.gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_ZeroMean(x, params):
+    "test zero mean function works as expected"
+    mf = LibGPGPU.ZeroMeanFunc()
+    assert isinstance(mf, LibGPGPU.BaseMeanFunc)
+    assert mf.get_n_params(x) == 0
+    assert_allclose(mf.mean_f(x,np.array([])), np.zeros(2))
+    assert_allclose(mf.mean_deriv(x,np.array([])), np.zeros((2,1)))
+    assert_allclose(mf.mean_inputderiv(x,np.array([])), np.zeros((3,2)))
+
+
+@pytest.mark.skipif(not LibGPGPU.gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_FixedMean(x, params):
+    "test fixed mean function works as expected"
+    mf = LibGPGPU.FixedMeanFunc(22.2)
+    assert isinstance(mf, LibGPGPU.BaseMeanFunc)
+    assert mf.get_n_params(x) == 0
+    assert_allclose(mf.mean_f(x,np.array([])), np.array([22.2,22.2]))
+    assert_allclose(mf.mean_deriv(x,np.array([])), np.zeros((2,1)))
+    assert_allclose(mf.mean_inputderiv(x,np.array([])), np.zeros((3,2)))
+
+
+@pytest.mark.skipif(not LibGPGPU.gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_ConstMean(x, params):
+    "test const mean function works as expected"
+    mf = LibGPGPU.ConstMeanFunc()
+    assert isinstance(mf, LibGPGPU.BaseMeanFunc)
+    assert mf.get_n_params(x) == 1
+    param = np.array([7.])
+    assert_allclose(mf.mean_f(x, param), np.array([7.,7.]))
+    assert_allclose(mf.mean_deriv(x, param), np.ones((2,1)))
+    assert_allclose(mf.mean_inputderiv(x, param), np.zeros((3,2)))
+
+
+@pytest.mark.skipif(not LibGPGPU.gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_PolyMean(x, params):
+    "test polynomial mean function works as expected"
+
+    # constructor takes list of lists, where inner list is a pair [index, power]
+    mf = LibGPGPU.PolyMeanFunc([[0,1],[1,1],[2,1]]) # linear in all three dimensions
+    assert isinstance(mf, LibGPGPU.BaseMeanFunc)
+    assert mf.get_n_params(x) == 4
+    param = np.array([1.,2.,3.,4.])
+    expected = []
+    for xi in x:
+        expected.append(param[0]+param[1]*xi[0]+param[2]*xi[1]+param[3]*xi[2])
+    assert_allclose(mf.mean_f(x, param), np.array(expected))
+    deriv_result = mf.mean_deriv(x, param)
+    assert_allclose(deriv_result[0], np.ones(2))
+    for i in range(1, 4):
+        assert_allclose(deriv_result[i], np.array([x[0][i-1],x[1][i-1]]))
+    inputderiv_result = mf.mean_inputderiv(x, param)
+    for i in range(3):
+        assert_allclose(inputderiv_result[i], np.array((param[i+1],param[i+1])))
+
+
+@pytest.mark.skipif(not LibGPGPU.gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_WrongNumParams(x, params):
+    " test we get an informative RuntimeError if we provide the wrong number of parameters "
+    zmf = LibGPGPU.ZeroMeanFunc()
+    with pytest.raises(RuntimeError, match=r"Expected params list of length 0"):
+        zmf.mean_f(x,params)
+    fmf = LibGPGPU.FixedMeanFunc(5.)
+    with pytest.raises(RuntimeError, match=r"Expected params list of length 0"):
+        fmf.mean_f(x,params)
+    cmf = LibGPGPU.ConstMeanFunc()
+    with pytest.raises(RuntimeError, match=r"Expected params list of length 1"):
+        cmf.mean_f(x,params)
+    pmf = LibGPGPU.PolyMeanFunc([[0,1],[1,2],[1,3]])
+    with pytest.raises(RuntimeError, match=r"Expected params list of length 4"):
+        pmf.mean_f(x,params)
+
+
+@pytest.mark.skipif(not LibGPGPU.gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_PolyBadDim(x, params):
+    " test we get an informative RuntimeError if we have a polynomial term in a non-existent dimension "
+    mf = LibGPGPU.PolyMeanFunc([[0,1],[4,3]])  # dimension index 4 doesn't exist
+    with pytest.raises(RuntimeError, match=r"Dimension index must be less than 3"):
+        mf.mean_f(x,params)
+
+
+@pytest.mark.skipif(not LibGPGPU.gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_parse_formula(x):
+    mf = parse_meanfunc_formula("1.")
+    assert isinstance(mf, LibGPGPU.FixedMeanFunc)
+    mf = parse_meanfunc_formula("c")
+    assert isinstance(mf, LibGPGPU.ConstMeanFunc)
+    mf = parse_meanfunc_formula("c+c*x[0]")
+    assert isinstance(mf, LibGPGPU.PolyMeanFunc)
+    assert mf.get_n_params(x) == 2
+    mf = parse_meanfunc_formula("c+c*x[0]+c*x[0]^2+c*x[1]")
+    assert isinstance(mf, LibGPGPU.PolyMeanFunc)
+    assert mf.get_n_params(x) == 4
+    # try without the initial const term (should be the same
+    mf = parse_meanfunc_formula("c*x[0]+c*x[0]^2+c*x[1]")
+    assert isinstance(mf, LibGPGPU.PolyMeanFunc)
+    assert mf.get_n_params(x) == 4
+    # try with a cross term x[0]*x[1]
+    with pytest.raises(NotImplementedError,
+                       match=r"Cross terms, e.g. x\[0\]\*x\[1\] not implemented"):
+        mf = parse_meanfunc_formula("c+x[0]*x[1]")
diff --git a/mogp_emulator/tests/test_GaussianProcess.py b/mogp_emulator/tests/test_GaussianProcess.py
index 29140eda..fb13bb56 100644
--- a/mogp_emulator/tests/test_GaussianProcess.py
+++ b/mogp_emulator/tests/test_GaussianProcess.py
@@ -2,12 +2,16 @@
 import numpy as np
 import pytest
 from numpy.testing import assert_allclose
+from ..LibGPGPU import gpu_usable
 from ..GaussianProcess import GaussianProcess, PredictResult
+from ..GaussianProcessGPU import GaussianProcessGPU
 from ..MeanFunction import ConstantMean, LinearMean, MeanFunction
 from ..Kernel import SquaredExponential, Matern52
 from ..Priors import NormalPrior, GammaPrior, InvGammaPrior
 from scipy import linalg
 
+GPU_NOT_FOUND_MSG = "A compatible GPU could not be found or the GPU library (libgpgpu) could not be loaded"
+
 @pytest.fixture
 def x():
     return np.array([[1., 2., 3.], [4., 5., 6.]])
@@ -48,6 +52,30 @@ def test_GaussianProcess_init(x, y):
 
     assert str(gp.mean) == "c + c*x[0]"
 
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_init(x, y):
+    "Test function for correct functioning of the init method of GaussianProcess"
+
+    gp = GaussianProcessGPU(x, y)
+    assert_allclose(x, gp.inputs)
+    assert_allclose(y, gp.targets)
+    assert gp.D == 3
+    assert gp.n == 2
+    assert gp.nugget == None
+    assert gp.nugget_type == "adaptive"
+    gp = GaussianProcessGPU(y, y)
+    assert gp.inputs.shape == (2, 1)
+
+    gp = GaussianProcessGPU(x, y, nugget=1.e-12)
+    assert_allclose(gp.nugget, 1.e-12)
+    from ..LibGPGPU import kernel_type
+    gp = GaussianProcessGPU(x, y, kernel="SquaredExponential")
+    assert isinstance(gp.kernel_type, kernel_type)
+    assert gp.kernel_type is kernel_type.SquaredExponential
+    assert isinstance(gp.kernel, SquaredExponential)
+
+
 def test_GP_init_failures(x, y):
     "Tests that GaussianProcess fails correctly with bad inputs"
 
@@ -72,6 +100,33 @@ def test_GP_init_failures(x, y):
     with pytest.raises(ValueError):
         gp = GaussianProcess(x, y, nugget="a")
 
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GPGPU_init_failures(x, y):
+    "Tests that GaussianProcessGPU fails correctly with bad inputs"
+
+    with pytest.raises(AssertionError):
+        gp = GaussianProcessGPU(np.ones((2, 2, 2)), y)
+
+    with pytest.raises(AssertionError):
+        gp = GaussianProcessGPU(x, x)
+
+    with pytest.raises(AssertionError):
+        gp = GaussianProcessGPU(np.ones((2, 3)), np.ones(3))
+
+    with pytest.raises(ValueError):
+        gp = GaussianProcessGPU(x, y, mean=1)
+
+    with pytest.raises(ValueError):
+        gp = GaussianProcessGPU(x, y, kernel="blah")
+
+    with pytest.raises(ValueError):
+        gp = GaussianProcessGPU(x, y, kernel=1)
+
+    with pytest.raises(ValueError):
+        gp = GaussianProcessGPU(x, y, nugget="a")
+
+
 def test_GaussianProcess_n_params(x, y):
     "test the get_n_params method of GaussianProcess"
 
@@ -81,6 +136,15 @@ def test_GaussianProcess_n_params(x, y):
     gp = GaussianProcess(x, y, mean="x[0]")
     assert gp.n_params == 2 + x.shape[1] + 2
 
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_n_params(x, y):
+    "test the get_n_params method of GaussianProcessGPU"
+
+    gp = GaussianProcessGPU(x, y)
+    assert gp.n_params == x.shape[1] + 2
+
+
 def test_GaussianProcess_nugget(x, y):
     "Tests the get_nugget method of GaussianProcess"
 
@@ -100,6 +164,40 @@ def test_GaussianProcess_nugget(x, y):
     assert_allclose(gp.nugget, 0.)
     assert gp.nugget_type == "fixed"
 
+    gp.nugget = "pivot"
+    assert gp.nugget is None
+    assert gp.nugget_type == "pivot"
+
+    with pytest.raises(TypeError):
+        gp.nugget = [1]
+
+    with pytest.raises(ValueError):
+        gp.nugget = "blah"
+
+    with pytest.raises(ValueError):
+        gp.nugget = -1.
+
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_nugget(x, y):
+    "Tests the get_nugget method of GaussianProcessGPU"
+
+    gp = GaussianProcessGPU(x, y)
+    assert gp.nugget is None
+    assert gp.nugget_type == "adaptive"
+
+    gp.nugget = "fit"
+    assert gp.nugget is None
+    assert gp.nugget_type == "fit"
+
+    gp.nugget = 1.
+    assert_allclose(gp.nugget, 1.)
+    assert gp.nugget_type == "fixed"
+
+    gp.nugget = 0
+    assert_allclose(gp.nugget, 0.)
+    assert gp.nugget_type == "fixed"
+
     with pytest.raises(TypeError):
         gp.nugget = [1]
 
@@ -111,6 +209,7 @@ def test_GaussianProcess_nugget(x, y):
 
 
 @pytest.mark.parametrize("mean,nugget,sn", [(None, 0., 1.), (None, "adaptive", 0.),
+                                            (None, "pivot", 0.),
                                             (MeanFunction("x[0]"), "fit", np.log(1.e-6))])
 def test_GaussianProcess_theta(x, y, mean, nugget, sn):
     "test the theta property of GaussianProcess (effectively the same as fit)"
@@ -126,7 +225,7 @@ def test_GaussianProcess_theta(x, y, mean, nugget, sn):
 
     switch = gp.mean.get_n_params(x)
 
-    if nugget == "adaptive" or nugget == 0.:
+    if nugget == "adaptive" or nugget == 0. or nugget == "pivot":
         assert gp.nugget == 0.
         noise = 0.
     else:
@@ -145,6 +244,59 @@ def test_GaussianProcess_theta(x, y, mean, nugget, sn):
     assert_allclose(invQt_expect, gp.invQt)
     assert_allclose(logpost_expect, gp.current_logpost)
 
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+@pytest.mark.parametrize("nugget,sn", [(0., 1.), ("adaptive", 0.)]) # ("fit", np.log(1.e-6))])
+def test_GaussianProcessGPU_theta(x, y, nugget, sn):
+    "test the theta property of GaussianProcess (effectively the same as fit)"
+
+    # zero mean, zero nugget
+
+    gp = GaussianProcessGPU(x, y, nugget=nugget)
+
+    theta = np.ones(gp.n_params)
+    theta[-1] = sn
+
+    gp.theta = theta
+
+    if nugget == "adaptive" or nugget == 0.:
+        assert gp.nugget == 0.
+        noise = 0.
+    else:
+        assert_allclose(gp.nugget, np.exp(sn))
+        noise = np.exp(sn)*np.eye(x.shape[0])
+    Q = gp.kernel.kernel_f(x, x, theta[:-1]) + noise
+
+    L_expect = np.linalg.cholesky(Q)
+    invQt_expect = np.linalg.solve(Q, y)
+    logpost_expect = 0.5*(np.log(np.linalg.det(Q)) +
+                          np.dot(y, invQt_expect) +
+                          gp.n*np.log(2.*np.pi))
+
+    assert_allclose(L_expect, gp.L)
+    assert_allclose(invQt_expect, gp.invQt)
+    assert_allclose(logpost_expect, gp.current_logpost)
+
+
+def test_GaussianProcess_theta_pivot():
+    "test that pivoting works as expected"
+
+    # input arrays are re-ordered such that pivoting should re-order the second to match the first
+
+    x1 = np.array([1., 4., 2.])
+    x2 = np.array([1., 2., 4.])
+    y1 = np.array([1., 1., 2.])
+    y2 = np.array([1., 2., 1.])
+
+    gp1 = GaussianProcess(x1, y1, nugget=0.)
+    gp2 = GaussianProcess(x2, y2, nugget="pivot")
+
+    gp1.theta = np.zeros(3)
+    gp2.theta = np.zeros(3)
+
+    assert_allclose(gp1.L, gp2.L)
+    assert_allclose(gp1.invQt, gp2.invQt[gp2.P])
+    assert np.array_equal(gp2.P, [0, 2, 1])
+
 def test_GaussianProcess_priors(x, y):
     "test that priors are set properly"
 
@@ -192,6 +344,7 @@ def test_GaussianProcess_priors(x, y):
         GaussianProcess(x, y, priors=priors)
 
 @pytest.mark.parametrize("mean,nugget,sn", [(None, 0., 1.), (None, "adaptive", 0.),
+                                            (None, "pivot", 0.),
                                             (MeanFunction("x[0]"), "fit", np.log(1.e-6))])
 def test_GaussianProcess_fit_logposterior(x, y, mean, nugget, sn):
     "test the fit and logposterior methods of GaussianProcess"
@@ -207,7 +360,7 @@ def test_GaussianProcess_fit_logposterior(x, y, mean, nugget, sn):
 
     switch = gp.mean.get_n_params(x)
 
-    if nugget == "adaptive" or nugget == 0.:
+    if nugget == "adaptive" or nugget == 0. or nugget == "pivot":
         assert gp.nugget == 0.
         noise = 0.
     else:
@@ -227,6 +380,7 @@ def test_GaussianProcess_fit_logposterior(x, y, mean, nugget, sn):
     assert_allclose(logpost_expect, gp.current_logpost)
     assert_allclose(logpost_expect, gp.logposterior(theta))
 
+
 def test_GaussianProcess_logposterior(x, y):
     "test logposterior method of GaussianProcess"
 
@@ -252,11 +406,38 @@ def test_GaussianProcess_logposterior(x, y):
     assert_allclose(invQt_expect, gp.invQt)
     assert_allclose(logpost_expect, gp.current_logpost)
 
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_logposterior(x, y):
+    "test logposterior method of GaussianProcessGPU"
+
+    # logposterior already tested, but check that parameters are re-fit if changed
+
+    gp = GaussianProcessGPU(x, y, nugget = 0.)
+
+    theta = np.ones(gp.n_params)
+    gp.fit(theta)
+
+    theta = np.zeros(gp.n_params)
+
+    Q = gp.kernel.kernel_f(x, x, theta[:-1])
+
+    L_expect = np.linalg.cholesky(Q)
+    invQt_expect = np.linalg.solve(Q, y)
+    logpost_expect = 0.5*(np.log(np.linalg.det(Q)) +
+                          np.dot(y, invQt_expect) +
+                          gp.n*np.log(2.*np.pi))
+
+    assert_allclose(logpost_expect, gp.logposterior(theta))
+    assert_allclose(gp.L, L_expect)
+    assert_allclose(invQt_expect, gp.invQt)
+    assert_allclose(logpost_expect, gp.current_logpost)
+
 @pytest.fixture
 def dx():
     return 1.e-6
 
 @pytest.mark.parametrize("mean,nugget,sn", [(None, 0., 1.), (None, "adaptive", 1.),
+                                            (None, "pivot", 1.),
                                             ("x[0]", "fit", np.log(1.e-6))])
 def test_GaussianProcess_logpost_deriv(x, y, dx, mean, nugget, sn):
     "test logposterior derivatives for GaussianProcess via finite differences"
@@ -276,7 +457,31 @@ def test_GaussianProcess_logpost_deriv(x, y, dx, mean, nugget, sn):
 
     assert_allclose(deriv, gp.logpost_deriv(theta), atol=1.e-7, rtol=1.e-5)
 
-@pytest.mark.parametrize("mean,nugget,sn", [(None, 0., 1.), ("x[0]", "fit", np.log(1.e-6))])
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+@pytest.mark.parametrize("nugget,sn", [(0., 1.), ("adaptive", 1.),
+                                            ("fit", np.log(1.e-6))])
+def test_GaussianProcessGPU_logpost_deriv(x, y, dx, nugget, sn):
+    "test logposterior derivatives for GaussianProcessGPU via finite differences"
+
+    gp = GaussianProcessGPU(x, y, nugget=nugget)
+
+    n = gp.n_params
+    theta = np.ones(n)
+    theta[-1] = sn
+
+    deriv = np.zeros(n)
+
+    for i in range(n):
+        dx_array = np.zeros(n)
+        dx_array[i] = dx
+        deriv[i] = (gp.logposterior(theta) - gp.logposterior(theta - dx_array))/dx
+
+    assert_allclose(deriv, gp.logpost_deriv(theta), atol=1.e-7, rtol=1.e-5)
+
+@pytest.mark.parametrize("mean,nugget,sn", [(None, 0., 1.),
+                                            (None, "pivot", 1.), ("x[0]", "fit", np.log(1.e-6))])
+
+
 def test_GaussianProcess_logpost_hessian(x, y, dx, mean, nugget, sn):
     "test the hessian method of GaussianProcess with finite differences"
 
@@ -470,6 +675,95 @@ def test_GaussianProcess_predict(x, y, dx):
 
     assert_allclose(gp(x_test), mu_expect)
 
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_predict(x, y, dx):
+    "test the predict method of GaussianProcessGPU"
+
+    # zero mean
+
+    gp = GaussianProcessGPU(x, y, nugget=0.)
+    theta = np.ones(gp.n_params)
+
+    gp.fit(theta)
+
+    x_test = np.array([[2., 3., 4.]])
+
+    mu, var, deriv = gp.predict(x_test)
+
+    K = gp.kernel.kernel_f(x, x, theta[:-1])
+    Ktest = gp.kernel.kernel_f(x_test, x, theta[:-1])
+
+    mu_expect = np.dot(Ktest, gp.invQt)
+    var_expect = np.exp(theta[-2]) - np.diag(np.dot(Ktest, np.linalg.solve(K, Ktest.T)))
+
+    D = gp.D
+
+    deriv_expect = np.zeros((1, D))
+
+    for i in range(D):
+        dx_array = np.zeros(D)
+        dx_array[i] = dx
+        deriv_expect[0, i] = (gp.predict(x_test)[0] - gp.predict(x_test - dx_array)[0])/dx
+
+    assert_allclose(mu, mu_expect)
+    assert_allclose(var, var_expect)
+    assert_allclose(deriv, deriv_expect, atol=1.e-7, rtol=1.e-7)
+
+    # check that reshaping works as expected
+
+    x_test = np.array([2., 3., 4.])
+
+    mu, var, deriv = gp.predict(x_test)
+
+    assert_allclose(mu, mu_expect)
+    assert_allclose(var, var_expect)
+    assert_allclose(deriv, deriv_expect, atol=1.e-7, rtol=1.e-7)
+
+    # check that with 1D input data can handle 1D prediction data correctly
+
+    gp = GaussianProcessGPU(y, y, nugget=0.)
+
+    gp.fit(np.ones(gp.n_params))
+
+    n_predict = 51
+    mu, var, deriv = gp.predict(np.linspace(0., 1., n_predict))
+
+    assert mu.shape == (n_predict,)
+    assert var.shape == (n_predict,)
+    assert deriv.shape == (n_predict, 1)
+
+    # check unc and deriv flags work
+
+    _, var, deriv = gp.predict(x_test, unc=False, deriv=False)
+
+    assert var is None
+    assert deriv is None
+
+    # check that the returned PredictResult works correctly
+    gp = GaussianProcessGPU(x, y, nugget=0.)
+    theta = np.ones(gp.n_params)
+    gp.fit(theta)
+    x_test = np.array([[2., 3., 4.]])
+
+    pr = gp.predict(x_test)
+
+    assert_allclose(pr.mean, mu_expect)
+    assert_allclose(pr.unc, var_expect)
+    assert_allclose(pr.deriv, deriv_expect, atol=1.e-7, rtol=1.e-7)
+
+    assert_allclose(pr['mean'], mu_expect)
+    assert_allclose(pr['unc'], var_expect)
+    assert_allclose(pr['deriv'], deriv_expect, atol=1.e-7, rtol=1.e-7)
+
+    assert_allclose(pr[0], mu_expect)
+    assert_allclose(pr[1], var_expect)
+    assert_allclose(pr[2], deriv_expect, atol=1.e-7, rtol=1.e-7)
+
+    # check that calling gp is equivalent to predicting
+
+    assert_allclose(gp(x_test), mu_expect)
+
+
 def test_GaussianProcess_predict_nugget(x, y):
     "test that the nugget works correctly when making predictions"
 
@@ -494,6 +788,59 @@ def test_GaussianProcess_predict_nugget(x, y):
 
     assert_allclose(preds.unc, var_expect, atol=1.e-7)
 
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_predict_nugget(x, y):
+    "test that the nugget works correctly when making predictions"
+
+    nugget = 1.e0
+
+    gp = GaussianProcessGPU(x, y, nugget=nugget)
+    theta = np.ones(gp.n_params)
+
+    gp.fit(theta)
+
+    preds = gp.predict(x)
+
+    K = gp.kernel.kernel_f(x, x, theta[:-1])
+
+    var_expect = np.exp(theta[-2]) + nugget - np.diag(np.dot(K, np.linalg.solve(K + np.eye(gp.n)*nugget, K)))
+
+    assert_allclose(preds.unc, var_expect, atol=1.e-7)
+
+    preds = gp.predict(x, include_nugget=False)
+
+    var_expect = np.exp(theta[-2]) - np.diag(np.dot(K, np.linalg.solve(K + np.eye(gp.n)*nugget, K)))
+
+    assert_allclose(preds.unc, var_expect, atol=1.e-7)
+
+
+def test_GaussianProcess_predict_pivot():
+    "test that pivoting gives same predictions as standard version"
+
+    # input arrays are re-ordered such that pivoting should re-order the second to match the first
+
+    x1 = np.array([1., 4., 2.])
+    x2 = np.array([1., 2., 4.])
+    y1 = np.array([1., 1., 2.])
+    y2 = np.array([1., 2., 1.])
+
+    gp1 = GaussianProcess(x1, y1, nugget=0.)
+    gp2 = GaussianProcess(x2, y2, nugget="pivot")
+
+    gp1.theta = np.zeros(3)
+    gp2.theta = np.zeros(3)
+
+    xpred = np.linspace(0., 5.)
+
+    mean1, var1, deriv1 = gp1.predict(xpred)
+    mean2, var2, deriv2 = gp2.predict(xpred)
+
+    assert_allclose(mean1, mean2)
+    assert_allclose(var1, var2)
+    assert_allclose(deriv1, deriv2)
+
+
 def test_GaussianProcess_predict_variance():
     "confirm that caching factorized matrix produces stable variance predictions"
 
@@ -513,6 +860,7 @@ def test_GaussianProcess_predict_variance():
 
     assert_allclose(np.zeros(101), var, atol = 1.e-3)
 
+
 def test_GaussianProcess_predict_failures(x, y):
     "test situations where predict method of GaussianProcess should fail"
 
@@ -530,8 +878,35 @@ def test_GaussianProcess_predict_failures(x, y):
     with pytest.raises(AssertionError):
         gp.predict(np.array([[2., 4.]]))
 
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_predict_failures(x, y):
+    "test situations where predict method of GaussianProcessGPU should fail"
+
+    gp = GaussianProcessGPU(x, y)
+
+    with pytest.raises(ValueError):
+        gp.predict(np.array([2., 3., 4.]))
+
+    theta = np.ones(gp.n_params)
+    gp.fit(theta)
+
+    with pytest.raises(AssertionError):
+        gp.predict(np.ones((2, 2, 2)))
+
+    with pytest.raises(AssertionError):
+        gp.predict(np.array([[2., 4.]]))
+
 def test_GaussianProcess_str(x, y):
     "Test function for string method"
 
     gp = GaussianProcess(x, y)
-    assert (str(gp) == "Gaussian Process with {} training examples and {} input variables".format(x.shape[0], x.shape[1]))
\ No newline at end of file
+    assert (str(gp) == "Gaussian Process with {} training examples and {} input variables".format(x.shape[0], x.shape[1]))
+
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_GaussianProcessGPU_str(x, y):
+    "Test function for string method"
+
+    gp = GaussianProcessGPU(x, y)
+    assert (str(gp) == "Gaussian Process with {} training examples and {} input variables".format(x.shape[0], x.shape[1]))
diff --git a/mogp_emulator/tests/test_HistoryMatching.py b/mogp_emulator/tests/test_HistoryMatching.py
index 7b4dd062..eff883ea 100644
--- a/mogp_emulator/tests/test_HistoryMatching.py
+++ b/mogp_emulator/tests/test_HistoryMatching.py
@@ -1,505 +1,651 @@
-import numpy as np
-from numpy.testing import assert_allclose
-import pytest
-from ..HistoryMatching import HistoryMatching
-from ..fitting import fit_GP_MAP
-from ..GaussianProcess import GaussianProcess, PredictResult
-
-def get_y_simulated_1D(x):
-    n_points = len(x)
-    f = np.zeros(n_points)
-    for i in range(n_points):
-        f[i] = np.sin(2.*np.pi*x[i] / 50.)
-    return f
-
-def test_sanity_checks():
-    "test basic functioning of HistoryMatching"
-
-    # Create a gaussian process
-    x_training = np.array([
-        [0.],
-        [10.],
-        [20.],
-        [30.],
-        [43.],
-        [50.]
-    ])
-
-    y_training = get_y_simulated_1D(x_training)
-
-    np.random.seed(47)
-    gp = fit_GP_MAP(x_training, y_training)
-
-    # Define observation and implausibility threshold
-    obs = [-0.8, 0.0004]
-
-    # Coords to predict
-    n_rand = 2000
-    x_predict_min = -3
-    x_predict_max = 53
-    x_predict = np.random.rand(n_rand)
-    x_predict = np.sort(x_predict, axis=0)
-    x_predict *= (x_predict_max - x_predict_min)
-    x_predict += x_predict_min
-    x_predict = x_predict[:, None]
-
-    coords = x_predict
-
-    expectations = gp.predict(coords)
-
-    # Create History Matching Instance
-    print("---TEST INPUTS---")
-    print("No Args")
-    hm = HistoryMatching()
-    hm.status()
-
-    print("Obs Only a - list")
-    hm = HistoryMatching(obs=obs)
-    hm.status()
-
-    print("Obs only b - single-element list")
-    hm = HistoryMatching(obs=[3.])
-    hm.status()
-
-    print("Obs only c - single-value")
-    hm = HistoryMatching(obs=3.)
-    hm.status()
-
-    print("gp Only")
-    hm = HistoryMatching(gp=gp)
-    hm.status()
-
-    print("Coords only a - 2d ndarray")
-    hm = HistoryMatching(coords=coords)
-    hm.status()
-
-    print("Coords only b - 1d ndarray")
-    hm = HistoryMatching(coords=np.random.rand(n_rand))
-    hm.status()
-
-    print("Coords only c - list")
-    hm = HistoryMatching(coords=[a for a in range(n_rand)])
-    hm.status()
-
-    print("Expectation only")
-    hm = HistoryMatching(expectations=expectations)
-    hm.status()
-
-    print("Threshold Only")
-    hm = HistoryMatching(threshold=3.)
-    hm.status()
-
-
-    print("---TEST ASSIGNMENT---")
-    print("Assign gp")
-    hm = HistoryMatching(obs)
-    hm.status()
-    hm.set_gp(gp)
-    hm.status()
-
-    print("Assign Obs")
-    hm = HistoryMatching(gp)
-    hm.status()
-    hm.set_obs(obs)
-    hm.status()
-
-    print("Assign Coords")
-    hm = HistoryMatching()
-    hm.status()
-    hm.set_coords(coords)
-    hm.status()
-
-    print("Assign Expectations")
-    hm = HistoryMatching()
-    hm.status()
-    hm.set_expectations(expectations)
-    hm.status()
-
-    print("Assign Threshold")
-    hm = HistoryMatching()
-    hm.status()
-    hm.set_threshold(3.)
-    hm.status()
-
-
-    print("---TEST IMPLAUSABILIY---")
-    print("implausibility test a - no vars")
-    hm = HistoryMatching(obs=obs, gp=gp, coords=coords)
-    I = hm.get_implausibility()
-
-    print("implausibility test b - single value")
-    hm = HistoryMatching(obs=obs, gp=gp, coords=coords)
-    I = hm.get_implausibility(7.)
-
-def test_HistoryMatching_init():
-    "test the init method of HistoryMatching"
-
-    # basic functionality
-
-    hm = HistoryMatching()
-
-    assert hm.gp == None
-    assert hm.obs == None
-    assert hm.coords == None
-    assert hm.expectations == None
-    assert hm.ndim == None
-    assert hm.ncoords == None
-    assert_allclose(hm.threshold, 3.)
-    assert hm.I == None
-    assert hm.NROY == None
-    assert hm.RO == None
-
-    # with gp/coords, obs, and threshold
-
-    gp = GaussianProcess(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
-    coords = np.array([[0.2], [0.4]])
-    hm = HistoryMatching(gp=gp, obs=1., coords=coords, expectations=None, threshold=5.)
-
-    assert hm.gp == gp
-    assert_allclose(hm.obs, [1., 0.])
-    assert_allclose(hm.coords, coords)
-    assert hm.expectations is None
-    assert hm.ndim == 1
-    assert hm.ncoords == len(coords)
-    assert_allclose(hm.threshold, 5.)
-    assert hm.I == None
-    assert hm.NROY == None
-    assert hm.RO == None
-
-    # with obs, expectations, coords, and threshold
-
-    expectations = PredictResult(mean=np.array([1.]), unc=np.array([0.2]), deriv=np.array([[0.1]]))
-    hm = HistoryMatching(gp=None, obs=[1., 0.1], coords=None, expectations=expectations,
-                         threshold=5.)
-
-    assert hm.gp == None
-    assert_allclose(hm.obs, [1., 0.1])
-    assert hm.coords == None
-    for a in range(3):
-        assert_allclose(hm.expectations[a], expectations[a])
-    assert hm.ndim == None
-    assert hm.ncoords == len(expectations[0])
-    assert_allclose(hm.threshold, 5.)
-    assert hm.I == None
-    assert hm.NROY == None
-    assert hm.RO == None
-
-
-def test_HistoryMatching_select_expectations():
-    "test the select_expectations method of HistoryMatching"
-
-    # correct functionality
-
-    expectations = PredictResult(mean=np.array([2., 10.]), unc=np.array([0., 0.]),
-                                 deriv=np.array([[1., 2.]]))
-    hm = HistoryMatching(obs=[1., 1.], expectations=expectations)
-
-    expectations_new = hm._select_expectations()
-
-    for a in range(3):
-        assert_allclose(expectations[a], expectations_new[a])
-
-    np.random.seed(57483)
-    gp = fit_GP_MAP(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
-    coords = np.array([[0.1], [1.]])
-    obs = [1., 0.01]
-    expectations = gp.predict(coords)
-
-    hm = HistoryMatching(gp=gp, obs=obs, coords=coords)
-
-    expectations_new = hm._select_expectations()
-
-    for a, b in zip(expectations, expectations_new):
-        assert_allclose(a, b)
-
-    # ncoords somehow not set
-
-    hm.ncoords = None
-    with pytest.raises(ValueError):
-        hm._select_expectations()
-
-    # both coords and expectations set
-
-    hm = HistoryMatching(gp=gp, obs=obs, coords=coords, expectations=expectations)
-
-    with pytest.raises(ValueError):
-        hm._select_expectations()
-
-    # if no expectations provided, fails
-
-    hm = HistoryMatching(obs=obs)
-
-    with pytest.raises(ValueError):
-        hm._select_expectations()
-
-def test_HistoryMatching_get_implausibility():
-    "test the get_implausibility method of HistoryMatching"
-
-    # correct functionality
-
-    expectations = PredictResult(mean=np.array([2., 10.]), unc=np.array([0., 0.]), deriv=np.array([[1., 2.]]))
-    hm = HistoryMatching(obs=[1., 1.], expectations=expectations)
-    I = hm.get_implausibility()
-
-    assert_allclose(I, [1., 9.])
-    assert_allclose(hm.I, [1., 9.])
-
-    I = hm.get_implausibility(1.)
-
-    assert_allclose(I, [1./np.sqrt(2.), 9./np.sqrt(2.)])
-    assert_allclose(hm.I, [1./np.sqrt(2.), 9./np.sqrt(2.)])
-
-    np.random.seed(57483)
-    gp = fit_GP_MAP(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
-    coords = np.array([[0.1], [1.]])
-    obs = [1., 0.01]
-    mean, unc, _ = gp.predict(coords)
-    I_exp = np.abs(mean - obs[0])/np.sqrt(unc + obs[1])
-
-    hm = HistoryMatching(gp=gp, obs=obs, coords=coords)
-    I = hm.get_implausibility()
-
-    assert_allclose(I, I_exp)
-    assert_allclose(hm.I, I_exp)
-
-    # no observations
-
-    hm = HistoryMatching(expectations=expectations)
-
-    with pytest.raises(ValueError):
-        hm.get_implausibility()
-
-    # negative variance for model discrepancy
-
-    hm = HistoryMatching(obs=[1., 1.], expectations=expectations)
-
-    with pytest.raises(AssertionError):
-        hm.get_implausibility(-1.)
-
-def test_HistoryMatching_get_NROY():
-    "test the get_NROY method of HistoryMatching"
-
-    # correct functionality
-
-    hm = HistoryMatching(obs=[1., 1.], expectations=PredictResult(mean=np.array([2., 10.]),
-                                                                  unc=np.array([0., 0.]),
-                                                                  deriv=np.array([[1., 2.]])))
-    hm.get_implausibility()
-
-    NROY = hm.get_NROY()
-
-    assert NROY == [0]
-
-    hm = HistoryMatching(obs=[1., 0.], expectations=PredictResult(mean=np.array([2., 10.]),
-                                                                  unc=np.array([0., 0.]),
-                                                                  deriv=np.array([[1., 2.]])))
-
-    NROY = hm.get_NROY(1.)
-
-    assert NROY == [0]
-
-def test_HistoryMatching_get_RO():
-    "test the get_RO method of HistoryMatching"
-
-    # correct functionality
-
-    hm = HistoryMatching(obs=[1., 1.], expectations=PredictResult(mean=np.array([2., 10.]),
-                                                                  unc=np.array([0., 0.]),
-                                                                  deriv=np.array([[1., 2.]])))
-    hm.get_implausibility()
-
-    RO = hm.get_RO()
-
-    hm = HistoryMatching(obs=[1., 0.], expectations=PredictResult(mean=np.array([2., 10.]),
-                                                                  unc=np.array([0., 0.]),
-                                                                  deriv=np.array([[1., 2.]])))
-
-    RO = hm.get_RO(1.)
-
-    assert RO == [1]
-
-def test_HistoryMatching_set_gp():
-    'test the set_gp method of HistoryMatching'
-
-    # correct functionality
-
-    gp = GaussianProcess(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
-
-    hm = HistoryMatching()
-    hm.set_gp(gp)
-
-    assert hm.gp == gp
-
-    # bad type for GP
-
-    with pytest.raises(TypeError):
-        hm.set_gp(gp=1.)
-
-def test_HistoryMatching_set_obs():
-    'test the set_obs method of HistoryMatching'
-
-    # correct functionality
-
-    obs = [1., 0.1]
-
-    hm = HistoryMatching()
-    hm.set_obs(obs)
-
-    assert_allclose(hm.obs, obs)
-
-    obs = 1.
-    hm.set_obs(obs)
-
-    assert_allclose(hm.obs, [obs, 0.])
-
-    # wrong number of values for obs
-
-    with pytest.raises(ValueError):
-        hm.set_obs([1., 2., 3.])
-
-    # negative variance
-
-    with pytest.raises(AssertionError):
-        hm.set_obs([1., -1.])
-
-    # bad values for obs
-
-    with pytest.raises(TypeError):
-        hm.set_obs("abc")
-
-def test_HistoryMatching_set_coords():
-    "test the set_coords method of HistoryMatching"
-
-    # correct functionality
-
-    coords = np.array([[1.]])
-
-    hm = HistoryMatching()
-    hm.set_coords(coords)
-
-    assert_allclose(hm.coords, coords)
-
-    coords = np.array([1.])
-
-    hm.set_coords(coords)
-
-    assert_allclose(hm.coords, np.reshape(coords, (1, 1)))
-
-    # confirm can reset coords
-
-    hm.set_coords(None)
-
-    assert hm.coords == None
-
-def test_HistoryMatching_set_expectations():
-    "test the set_expectations method of HistoryMatching"
-
-    # correct functionality
-
-    expectations = PredictResult(mean=np.array([0.]), unc=np.array([0.1]), deriv=np.array([[2.]]))
-
-    hm = HistoryMatching()
-    hm.set_expectations(expectations)
-
-    for a in range(3):
-        assert_allclose(hm.expectations[a], expectations[a])
-
-    # confirm ability to reset expectations
-
-    hm.set_expectations(None)
-    assert hm.expectations == None
-
-    # verify derivatives can be None
-
-    expectations = PredictResult(mean=np.array([0.]), unc=np.array([0.1]), deriv=None)
-
-    hm = HistoryMatching()
-    hm.set_expectations(expectations)
-
-    for a in range(2):
-        assert_allclose(hm.expectations[a], expectations[a])
-    assert hm.expectations[2] is None
-
-    # negative variance
-
-    expectations = PredictResult(mean=np.array([0.]), unc=np.array([-0.1]), deriv=np.array([[2.]]))
-
-    with pytest.raises(AssertionError):
-        hm.set_expectations(expectations)
-
-    # bad values for expectations
-
-    expectations = PredictResult(mean=np.array([0.1]), unc=1., deriv=1.)
-    hm = HistoryMatching()
-
-    with pytest.raises(TypeError):
-        hm.set_expectations(expectations)
-
-    expectations = PredictResult(mean=np.array([0.1]), unc=np.array([1., 2.]), deriv=1.)
-
-    with pytest.raises(TypeError):
-        hm.set_expectations(expectations)
-
-    expectations = PredictResult(mean=np.array([1.]), unc=np.array([1., 2.]), deriv=None)
-
-    with pytest.raises(ValueError):
-        hm.set_expectations(expectations)
-
-
-def test_HistoryMatching_set_threshold():
-    "test the set_threshold method of HistoryMatching"
-
-    # correct functionality
-
-    hm = HistoryMatching()
-
-    hm.set_threshold(1.)
-
-    assert_allclose(hm.threshold, 1.)
-
-    # threshold cannot be negative
-
-    with pytest.raises(AssertionError):
-        hm.set_threshold(-1.)
-
-def test_HistoryMatching_update():
-    "test the update method of HistoryMatching"
-
-    # correct functionality
-
-    hm = HistoryMatching()
-
-    hm.coords = np.array([[1.]])
-
-    hm.update()
-
-    assert hm.ncoords == 1
-    assert hm.ndim == 1
-
-    hm = HistoryMatching()
-
-    hm.expectations = PredictResult(mean=np.array([0.]), unc=np.array([0.1]), deriv=np.array([[2.]]))
-    hm.update()
-
-    assert hm.ncoords == 1
-
-def test_HistoryMatching_str():
-    "test the str method of HistoryMatching"
-
-    # correct functionality
-
-    hm = HistoryMatching()
-
-    str_exp = ("History Matching tools created with:\n" +
-               "Gaussian Process: None\n" +
-               "Observations: None\n" +
-               "Coords: None\n" +
-               "Expectations: None\n" +
-               "No. of Input Dimensions: None\n" +
-               "No. of Descrete Expectation Values: None\n" +
-               "I_threshold: {}\n" +
-               "I: None\n" +
-               "NROY: None\n" +
-               "RO: None\n").format(3.)
-
-    assert str(hm) == str_exp
+import numpy as np
+from numpy.testing import assert_allclose
+import pytest
+from ..HistoryMatching import HistoryMatching
+from ..fitting import fit_GP_MAP
+from ..GaussianProcess import GaussianProcess, PredictResult
+from ..LibGPGPU import gpu_usable
+from ..GaussianProcessGPU import GaussianProcessGPU
+
+GPU_NOT_FOUND_MSG = "A compatible GPU could not be found or the GPU library (libgpgpu) could not be loaded"
+
+def get_y_simulated_1D(x):
+    n_points = len(x)
+    f = np.zeros(n_points)
+    for i in range(n_points):
+        f[i] = np.sin(2.*np.pi*x[i] / 50.)
+    return f
+
+def test_sanity_checks():
+    "test basic functioning of HistoryMatching"
+
+    # Create a gaussian process
+    x_training = np.array([
+        [0.],
+        [10.],
+        [20.],
+        [30.],
+        [43.],
+        [50.]
+    ])
+
+    y_training = get_y_simulated_1D(x_training)
+
+    np.random.seed(47)
+    gp = fit_GP_MAP(x_training, y_training)
+
+    # Define observation and implausibility threshold
+    obs = [-0.8, 0.0004]
+
+    # Coords to predict
+    n_rand = 2000
+    x_predict_min = -3
+    x_predict_max = 53
+    x_predict = np.random.rand(n_rand)
+    x_predict = np.sort(x_predict, axis=0)
+    x_predict *= (x_predict_max - x_predict_min)
+    x_predict += x_predict_min
+    x_predict = x_predict[:, None]
+
+    coords = x_predict
+
+    expectations = gp.predict(coords)
+
+    # Create History Matching Instance
+    print("---TEST INPUTS---")
+    print("No Args")
+    hm = HistoryMatching()
+    hm.status()
+
+    print("Obs Only a - list")
+    hm = HistoryMatching(obs=obs)
+    hm.status()
+
+    print("Obs only b - single-element list")
+    hm = HistoryMatching(obs=[3.])
+    hm.status()
+
+    print("Obs only c - single-value")
+    hm = HistoryMatching(obs=3.)
+    hm.status()
+
+    print("gp Only")
+    hm = HistoryMatching(gp=gp)
+    hm.status()
+
+    print("Coords only a - 2d ndarray")
+    hm = HistoryMatching(coords=coords)
+    hm.status()
+
+    print("Coords only b - 1d ndarray")
+    hm = HistoryMatching(coords=np.random.rand(n_rand))
+    hm.status()
+
+    print("Coords only c - list")
+    hm = HistoryMatching(coords=[a for a in range(n_rand)])
+    hm.status()
+
+    print("Expectation only")
+    hm = HistoryMatching(expectations=expectations)
+    hm.status()
+
+    print("Threshold Only")
+    hm = HistoryMatching(threshold=3.)
+    hm.status()
+
+
+    print("---TEST ASSIGNMENT---")
+    print("Assign gp")
+    hm = HistoryMatching(obs)
+    hm.status()
+    hm.set_gp(gp)
+    hm.status()
+
+    print("Assign Obs")
+    hm = HistoryMatching(gp)
+    hm.status()
+    hm.set_obs(obs)
+    hm.status()
+
+    print("Assign Coords")
+    hm = HistoryMatching()
+    hm.status()
+    hm.set_coords(coords)
+    hm.status()
+
+    print("Assign Expectations")
+    hm = HistoryMatching()
+    hm.status()
+    hm.set_expectations(expectations)
+    hm.status()
+
+    print("Assign Threshold")
+    hm = HistoryMatching()
+    hm.status()
+    hm.set_threshold(3.)
+    hm.status()
+
+
+    print("---TEST IMPLAUSABILIY---")
+    print("implausibility test a - no vars")
+    hm = HistoryMatching(obs=obs, gp=gp, coords=coords)
+    I = hm.get_implausibility()
+
+    print("implausibility test b - single value")
+    hm = HistoryMatching(obs=obs, gp=gp, coords=coords)
+    I = hm.get_implausibility(7.)
+
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_sanity_checks_GPU():
+    "test basic functioning of HistoryMatching"
+
+    # Create a gaussian process
+    x_training = np.array([
+        [0.],
+        [10.],
+        [20.],
+        [30.],
+        [43.],
+        [50.]
+    ])
+
+    y_training = get_y_simulated_1D(x_training)
+
+    np.random.seed(47)
+    gp = GaussianProcessGPU(x_training, y_training)
+    gp = fit_GP_MAP(gp)
+
+    # Define observation and implausibility threshold
+    obs = [-0.8, 0.0004]
+
+    # Coords to predict
+    n_rand = 2000
+    x_predict_min = -3
+    x_predict_max = 53
+    x_predict = np.random.rand(n_rand)
+    x_predict = np.sort(x_predict, axis=0)
+    x_predict *= (x_predict_max - x_predict_min)
+    x_predict += x_predict_min
+    x_predict = x_predict[:, None]
+
+    coords = x_predict
+
+    expectations = gp.predict(coords)
+
+    # Create History Matching Instance
+    print("---TEST INPUTS---")
+    print("No Args")
+    hm = HistoryMatching()
+    hm.status()
+
+    print("Obs Only a - list")
+    hm = HistoryMatching(obs=obs)
+    hm.status()
+
+    print("Obs only b - single-element list")
+    hm = HistoryMatching(obs=[3.])
+    hm.status()
+
+    print("Obs only c - single-value")
+    hm = HistoryMatching(obs=3.)
+    hm.status()
+
+    print("gp Only")
+    hm = HistoryMatching(gp=gp)
+    hm.status()
+
+    print("Coords only a - 2d ndarray")
+    hm = HistoryMatching(coords=coords)
+    hm.status()
+
+    print("Coords only b - 1d ndarray")
+    hm = HistoryMatching(coords=np.random.rand(n_rand))
+    hm.status()
+
+    print("Coords only c - list")
+    hm = HistoryMatching(coords=[a for a in range(n_rand)])
+    hm.status()
+
+    print("Expectation only")
+    hm = HistoryMatching(expectations=expectations)
+    hm.status()
+
+    print("Threshold Only")
+    hm = HistoryMatching(threshold=3.)
+    hm.status()
+
+
+    print("---TEST ASSIGNMENT---")
+    print("Assign gp")
+    hm = HistoryMatching(obs)
+    hm.status()
+    hm.set_gp(gp)
+    hm.status()
+
+    print("Assign Obs")
+    hm = HistoryMatching(gp)
+    hm.status()
+    hm.set_obs(obs)
+    hm.status()
+
+    print("Assign Coords")
+    hm = HistoryMatching()
+    hm.status()
+    hm.set_coords(coords)
+    hm.status()
+
+    print("Assign Expectations")
+    hm = HistoryMatching()
+    hm.status()
+    hm.set_expectations(expectations)
+    hm.status()
+
+    print("Assign Threshold")
+    hm = HistoryMatching()
+    hm.status()
+    hm.set_threshold(3.)
+    hm.status()
+
+
+    print("---TEST IMPLAUSABILIY---")
+    print("implausibility test a - no vars")
+    hm = HistoryMatching(obs=obs, gp=gp, coords=coords)
+    I = hm.get_implausibility()
+
+    print("implausibility test b - single value")
+    hm = HistoryMatching(obs=obs, gp=gp, coords=coords)
+    I = hm.get_implausibility(7.)
+
+
+def test_HistoryMatching_init():
+    "test the init method of HistoryMatching"
+
+    # basic functionality
+
+    hm = HistoryMatching()
+
+    assert hm.gp == None
+    assert hm.obs == None
+    assert hm.coords == None
+    assert hm.expectations == None
+    assert hm.ndim == None
+    assert hm.ncoords == None
+    assert_allclose(hm.threshold, 3.)
+    assert hm.I == None
+    assert hm.NROY == None
+    assert hm.RO == None
+
+    # with gp/coords, obs, and threshold
+
+    gp = GaussianProcess(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
+    coords = np.array([[0.2], [0.4]])
+    hm = HistoryMatching(gp=gp, obs=1., coords=coords, expectations=None, threshold=5.)
+
+    assert hm.gp == gp
+    assert_allclose(hm.obs, [1., 0.])
+    assert_allclose(hm.coords, coords)
+    assert hm.expectations is None
+    assert hm.ndim == 1
+    assert hm.ncoords == len(coords)
+    assert_allclose(hm.threshold, 5.)
+    assert hm.I == None
+    assert hm.NROY == None
+    assert hm.RO == None
+
+    # with obs, expectations, coords, and threshold
+
+    expectations = PredictResult(mean=np.array([1.]), unc=np.array([0.2]), deriv=np.array([[0.1]]))
+    hm = HistoryMatching(gp=None, obs=[1., 0.1], coords=None, expectations=expectations,
+                         threshold=5.)
+
+    assert hm.gp == None
+    assert_allclose(hm.obs, [1., 0.1])
+    assert hm.coords == None
+    for a in range(3):
+        assert_allclose(hm.expectations[a], expectations[a])
+    assert hm.ndim == None
+    assert hm.ncoords == len(expectations[0])
+    assert_allclose(hm.threshold, 5.)
+    assert hm.I == None
+    assert hm.NROY == None
+    assert hm.RO == None
+
+
+def test_HistoryMatching_select_expectations():
+    "test the select_expectations method of HistoryMatching"
+
+    # correct functionality
+
+    expectations = PredictResult(mean=np.array([2., 10.]), unc=np.array([0., 0.]),
+                                 deriv=np.array([[1., 2.]]))
+    hm = HistoryMatching(obs=[1., 1.], expectations=expectations)
+
+    expectations_new = hm._select_expectations()
+
+    for a in range(3):
+        assert_allclose(expectations[a], expectations_new[a])
+
+    np.random.seed(57483)
+    gp = fit_GP_MAP(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
+    coords = np.array([[0.1], [1.]])
+    obs = [1., 0.01]
+    expectations = gp.predict(coords)
+
+    hm = HistoryMatching(gp=gp, obs=obs, coords=coords)
+
+    expectations_new = hm._select_expectations()
+
+    for a, b in zip(expectations, expectations_new):
+        assert_allclose(a, b)
+
+    # ncoords somehow not set
+
+    hm.ncoords = None
+    with pytest.raises(ValueError):
+        hm._select_expectations()
+
+    # both coords and expectations set
+
+    hm = HistoryMatching(gp=gp, obs=obs, coords=coords, expectations=expectations)
+
+    with pytest.raises(ValueError):
+        hm._select_expectations()
+
+    # if no expectations provided, fails
+
+    hm = HistoryMatching(obs=obs)
+
+    with pytest.raises(ValueError):
+        hm._select_expectations()
+
+def test_HistoryMatching_get_implausibility():
+    "test the get_implausibility method of HistoryMatching"
+
+    # correct functionality
+
+    expectations = PredictResult(mean=np.array([2., 10.]), unc=np.array([0., 0.]), deriv=np.array([[1., 2.]]))
+    hm = HistoryMatching(obs=[1., 1.], expectations=expectations)
+    I = hm.get_implausibility()
+
+    assert_allclose(I, [1., 9.])
+    assert_allclose(hm.I, [1., 9.])
+
+    I = hm.get_implausibility(1.)
+
+    assert_allclose(I, [1./np.sqrt(2.), 9./np.sqrt(2.)])
+    assert_allclose(hm.I, [1./np.sqrt(2.), 9./np.sqrt(2.)])
+
+    np.random.seed(57483)
+    gp = fit_GP_MAP(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
+    coords = np.array([[0.1], [1.]])
+    obs = [1., 0.01]
+    mean, unc, _ = gp.predict(coords)
+    I_exp = np.abs(mean - obs[0])/np.sqrt(unc + obs[1])
+
+    hm = HistoryMatching(gp=gp, obs=obs, coords=coords)
+    I = hm.get_implausibility()
+
+    assert_allclose(I, I_exp)
+    assert_allclose(hm.I, I_exp)
+
+    # no observations
+
+    hm = HistoryMatching(expectations=expectations)
+
+    with pytest.raises(ValueError):
+        hm.get_implausibility()
+
+    # negative variance for model discrepancy
+
+    hm = HistoryMatching(obs=[1., 1.], expectations=expectations)
+
+    with pytest.raises(AssertionError):
+        hm.get_implausibility(-1.)
+
+def test_HistoryMatching_get_NROY():
+    "test the get_NROY method of HistoryMatching"
+
+    # correct functionality
+
+    hm = HistoryMatching(obs=[1., 1.], expectations=PredictResult(mean=np.array([2., 10.]),
+                                                                  unc=np.array([0., 0.]),
+                                                                  deriv=np.array([[1., 2.]])))
+    hm.get_implausibility()
+
+    NROY = hm.get_NROY()
+
+    assert NROY == [0]
+
+    hm = HistoryMatching(obs=[1., 0.], expectations=PredictResult(mean=np.array([2., 10.]),
+                                                                  unc=np.array([0., 0.]),
+                                                                  deriv=np.array([[1., 2.]])))
+
+    NROY = hm.get_NROY(1.)
+
+    assert NROY == [0]
+
+def test_HistoryMatching_get_RO():
+    "test the get_RO method of HistoryMatching"
+
+    # correct functionality
+
+    hm = HistoryMatching(obs=[1., 1.], expectations=PredictResult(mean=np.array([2., 10.]),
+                                                                  unc=np.array([0., 0.]),
+                                                                  deriv=np.array([[1., 2.]])))
+    hm.get_implausibility()
+
+    RO = hm.get_RO()
+
+    hm = HistoryMatching(obs=[1., 0.], expectations=PredictResult(mean=np.array([2., 10.]),
+                                                                  unc=np.array([0., 0.]),
+                                                                  deriv=np.array([[1., 2.]])))
+
+    RO = hm.get_RO(1.)
+
+    assert RO == [1]
+
+def test_HistoryMatching_set_gp():
+    'test the set_gp method of HistoryMatching'
+
+    # correct functionality
+
+    gp = GaussianProcess(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
+
+    hm = HistoryMatching()
+    hm.set_gp(gp)
+
+    assert hm.gp == gp
+
+    # bad type for GP
+
+    with pytest.raises(TypeError):
+        hm.set_gp(gp=1.)
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_HistoryMatching_set_gpGPU():
+    'test the set_gp method of HistoryMatching'
+
+    # correct functionality
+
+    gp = GaussianProcessGPU(np.reshape(np.linspace(0., 1.), (-1, 1)), np.linspace(0., 1.))
+
+    hm = HistoryMatching()
+    hm.set_gp(gp)
+
+    assert hm.gp == gp
+
+    # bad type for GP
+
+    with pytest.raises(TypeError):
+        hm.set_gp(gp=1.)
+
+
+def test_HistoryMatching_set_obs():
+    'test the set_obs method of HistoryMatching'
+
+    # correct functionality
+
+    obs = [1., 0.1]
+
+    hm = HistoryMatching()
+    hm.set_obs(obs)
+
+    assert_allclose(hm.obs, obs)
+
+    obs = 1.
+    hm.set_obs(obs)
+
+    assert_allclose(hm.obs, [obs, 0.])
+
+    # wrong number of values for obs
+
+    with pytest.raises(ValueError):
+        hm.set_obs([1., 2., 3.])
+
+    # negative variance
+
+    with pytest.raises(AssertionError):
+        hm.set_obs([1., -1.])
+
+    # bad values for obs
+
+    with pytest.raises(TypeError):
+        hm.set_obs("abc")
+
+def test_HistoryMatching_set_coords():
+    "test the set_coords method of HistoryMatching"
+
+    # correct functionality
+
+    coords = np.array([[1.]])
+
+    hm = HistoryMatching()
+    hm.set_coords(coords)
+
+    assert_allclose(hm.coords, coords)
+
+    coords = np.array([1.])
+
+    hm.set_coords(coords)
+
+    assert_allclose(hm.coords, np.reshape(coords, (1, 1)))
+
+    # confirm can reset coords
+
+    hm.set_coords(None)
+
+    assert hm.coords == None
+
+def test_HistoryMatching_set_expectations():
+    "test the set_expectations method of HistoryMatching"
+
+    # correct functionality
+
+    expectations = PredictResult(mean=np.array([0.]), unc=np.array([0.1]), deriv=np.array([[2.]]))
+
+    hm = HistoryMatching()
+    hm.set_expectations(expectations)
+
+    for a in range(3):
+        assert_allclose(hm.expectations[a], expectations[a])
+
+    # confirm ability to reset expectations
+
+    hm.set_expectations(None)
+    assert hm.expectations == None
+
+    # verify derivatives can be None
+
+    expectations = PredictResult(mean=np.array([0.]), unc=np.array([0.1]), deriv=None)
+
+    hm = HistoryMatching()
+    hm.set_expectations(expectations)
+
+    for a in range(2):
+        assert_allclose(hm.expectations[a], expectations[a])
+    assert hm.expectations[2] is None
+
+    # negative variance
+
+    expectations = PredictResult(mean=np.array([0.]), unc=np.array([-0.1]), deriv=np.array([[2.]]))
+
+    with pytest.raises(AssertionError):
+        hm.set_expectations(expectations)
+
+    # bad values for expectations
+
+    expectations = PredictResult(mean=np.array([0.1]), unc=1., deriv=1.)
+    hm = HistoryMatching()
+
+    with pytest.raises(TypeError):
+        hm.set_expectations(expectations)
+
+    expectations = PredictResult(mean=np.array([0.1]), unc=np.array([1., 2.]), deriv=1.)
+
+    with pytest.raises(TypeError):
+        hm.set_expectations(expectations)
+
+    expectations = PredictResult(mean=np.array([1.]), unc=np.array([1., 2.]), deriv=None)
+
+    with pytest.raises(ValueError):
+        hm.set_expectations(expectations)
+
+
+def test_HistoryMatching_set_threshold():
+    "test the set_threshold method of HistoryMatching"
+
+    # correct functionality
+
+    hm = HistoryMatching()
+
+    hm.set_threshold(1.)
+
+    assert_allclose(hm.threshold, 1.)
+
+    # threshold cannot be negative
+
+    with pytest.raises(AssertionError):
+        hm.set_threshold(-1.)
+
+def test_HistoryMatching_update():
+    "test the update method of HistoryMatching"
+
+    # correct functionality
+
+    hm = HistoryMatching()
+
+    hm.coords = np.array([[1.]])
+
+    hm.update()
+
+    assert hm.ncoords == 1
+    assert hm.ndim == 1
+
+    hm = HistoryMatching()
+
+    hm.expectations = PredictResult(mean=np.array([0.]), unc=np.array([0.1]), deriv=np.array([[2.]]))
+    hm.update()
+
+    assert hm.ncoords == 1
+
+def test_HistoryMatching_str():
+    "test the str method of HistoryMatching"
+
+    # correct functionality
+
+    hm = HistoryMatching()
+
+    str_exp = ("History Matching tools created with:\n" +
+               "Gaussian Process: None\n" +
+               "Observations: None\n" +
+               "Coords: None\n" +
+               "Expectations: None\n" +
+               "No. of Input Dimensions: None\n" +
+               "No. of Descrete Expectation Values: None\n" +
+               "I_threshold: {}\n" +
+               "I: None\n" +
+               "NROY: None\n" +
+               "RO: None\n").format(3.)
+
+    assert str(hm) == str_exp
diff --git a/mogp_emulator/tests/test_MCMC.py b/mogp_emulator/tests/test_MCMC.py
deleted file mode 100644
index 9b282883..00000000
--- a/mogp_emulator/tests/test_MCMC.py
+++ /dev/null
@@ -1,162 +0,0 @@
-import numpy as np
-import pytest
-from numpy.testing import assert_allclose
-from ..MCMC import MH_proposal, MCMC_step, sample_MCMC, autothin_samples
-
-def test_MH_proposal():
-    "test the Metropolis-Hastings proposal distribution"
-    
-    np.random.seed(438)
-    
-    current_param = np.zeros(2)
-    cov = np.eye(2)
-    
-    assert_allclose(MH_proposal(current_param, cov), np.array([-0.392811129690724 ,  0.4478392215027822]))
-    
-    with pytest.raises(AssertionError):
-        MH_proposal(np.zeros((2,2)), cov)
-        
-    with pytest.raises(AssertionError):
-        MH_proposal(np.zeros(2), np.ones((2, 2, 2)))
-        
-    with pytest.raises(AssertionError):
-        MH_proposal(np.zeros(2), np.ones((2, 3)))
-        
-    with pytest.raises(AssertionError):
-        MH_proposal(np.zeros(3), np.ones((2, 2)))
-        
-    with pytest.raises(AssertionError):
-        MH_proposal(np.zeros(2), -1.*np.eye(2))
-
-def test_MCMC_step():
-    "test the MCMC step routine"
-    
-    def loglikelihood(x):
-        return 1.
-        
-    current_param = np.zeros(2)
-    cov = np.eye(2)
-    
-    next_point_expected = np.array([-0.392811129690724 ,  0.4478392215027822])
-    
-    np.random.seed(438)
-    
-    next_point, accept = MCMC_step(loglikelihood, current_param, cov)
-    
-    assert_allclose(next_point, next_point_expected)
-    assert accept
-    
-    def loglikelihood(x):
-        return 100.*np.sum(x)
-
-    np.random.seed(438)
-    
-    next_point, accept = MCMC_step(loglikelihood, current_param, cov, 1.)
-    
-    assert_allclose(next_point, next_point_expected)
-    assert accept
-    
-    np.random.seed(438)
-    
-    next_point, accept = MCMC_step(loglikelihood, current_param, cov, -1.)
-    
-    assert_allclose(next_point, next_point_expected)
-    assert not accept
-    
-    with pytest.raises(AssertionError):
-        MCMC_step(loglikelihood, current_param, cov, 0.)
-        
-    def loglikelihood(x, y):
-        return -1.
-    
-    with pytest.raises(AssertionError):
-        MCMC_step(loglikelihood, current_param, cov)
-        
-    with pytest.raises(AssertionError):
-        MCMC_step(2., current_param, cov)
-        
-def test_sample_MCMC():
-    "test MCMC sampling"
-    
-    def loglikelihood(x):
-        return 100.*np.sum(x)
-    
-    current_param = np.zeros(2)
-    cov = np.eye(2)
-    
-    samples_expected = np.array([[0., 0.],
-                                [-0.392811129690724 ,  0.4478392215027822],
-                                [-1.6681438506139665,  2.697940030289671 ],
-                                [-1.6681438506139665,  2.697940030289671 ]])
-    rejected_expected = np.array([[-3.4166078360234584,  2.5824123019889043]])
-    acceptance_expected = 0.75
-    first_lag_expected = np.array([0.2886356417500504, 0.2824351401835904])
-    
-    np.random.seed(438)
-    
-    with pytest.warns(Warning):
-        samples, rejected, acceptance, first_lag = sample_MCMC(loglikelihood, current_param, cov, n_samples = 4, thin = 0)
-    
-    assert_allclose(samples, samples_expected)
-    assert_allclose(rejected, rejected_expected)
-    assert_allclose(acceptance, acceptance_expected)
-    assert_allclose(first_lag, first_lag_expected)
-    
-    np.random.seed(438)
-    
-    samples, rejected, acceptance, first_lag = sample_MCMC(loglikelihood, current_param, cov, n_samples = 4, thin = 1)
-    
-    assert_allclose(samples, samples_expected)
-    assert_allclose(rejected, rejected_expected)
-    assert_allclose(acceptance, acceptance_expected)
-    assert_allclose(first_lag, first_lag_expected)
-    
-    samples_expected = np.array([[0., 0.],
-                                [-1.6681438506139665,  2.697940030289671 ]])
-    rejected_expected = np.array([[-3.4166078360234584,  2.5824123019889043]])
-    acceptance_expected = 0.75
-    first_lag_expected = np.array([-0.5, -0.5])
-    
-    np.random.seed(438)
-    
-    samples, rejected, acceptance, first_lag = sample_MCMC(loglikelihood, current_param, cov, n_samples = 4, thin = 2)
-    
-    assert_allclose(samples, samples_expected)
-    assert_allclose(rejected, rejected_expected)
-    assert_allclose(acceptance, acceptance_expected)
-    assert_allclose(first_lag, first_lag_expected)
-    
-    with pytest.raises(AssertionError):
-        sample_MCMC(loglikelihood, current_param, cov, n_samples = -1)
-        
-    with pytest.raises(AssertionError):
-        sample_MCMC(loglikelihood, current_param, cov, 1000, -1)
-        
-    with pytest.raises(AssertionError):
-        sample_MCMC(loglikelihood, np.zeros((2,3)), cov, 1000, -1)
-
-def test_autothin_samples():
-    "test the autothinning routine"
-    
-    a = np.zeros(100)
-    a[45:55] = 1.
-    
-    assert autothin_samples(a) == 7
-    
-    a = np.zeros((100, 2))
-    a[45:55, 0] = 1.
-    a[40:60, 1] = 1.
-    
-    assert autothin_samples(a) == 11
-    
-    a = np.zeros(9)
-    
-    with pytest.warns(Warning):
-        assert autothin_samples(a) == 1
-    
-    a = np.zeros((2, 2, 2))
-    
-    with pytest.raises(AssertionError):
-        autothin_samples(a)
-    
-
diff --git a/mogp_emulator/tests/test_MultiOutputGP.py b/mogp_emulator/tests/test_MultiOutputGP.py
index 601d63ee..0c54d995 100644
--- a/mogp_emulator/tests/test_MultiOutputGP.py
+++ b/mogp_emulator/tests/test_MultiOutputGP.py
@@ -3,10 +3,13 @@
 from numpy.testing import assert_allclose
 from ..GaussianProcess import GaussianProcess, PredictResult
 from ..MultiOutputGP import MultiOutputGP
+from ..LibGPGPU import gpu_usable
 from ..MeanFunction import ConstantMean, LinearMean, MeanFunction
 from ..Kernel import Matern52
 from scipy import linalg
 
+GPU_NOT_FOUND_MSG = "A compatible GPU could not be found or the GPU library (libgpgpu) could not be loaded"
+
 @pytest.fixture
 def x():
     return np.array([[1., 2., 3.], [4., 5., 6.]])
@@ -27,6 +30,20 @@ def test_MultiOutputGP_init(x, y):
     assert gp.n == 2
     assert gp.n_emulators == 2
 
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_MultiOutputGP_GPU_init(x, y):
+    "Test function for correct functioning of the init method of GaussianProcessGPU"
+
+    gp = MultiOutputGP(x, y, use_gpu=True)
+    assert_allclose(x, gp.emulators[0].inputs)
+    assert_allclose(x, gp.emulators[1].inputs)
+    assert_allclose(y[0], gp.emulators[0].targets)
+    assert_allclose(y[1], gp.emulators[1].targets)
+    assert gp.D == 3
+    assert gp.n == 2
+    assert gp.n_emulators == 2
+
 @pytest.fixture
 def dx():
     return 1.e-6
@@ -95,3 +112,160 @@ def test_MultiOutputGP_predict(x, y, dx):
         var_expect = np.exp(theta[-2]) - np.diag(np.dot(Ktest, np.linalg.solve(K, Ktest.T)))
 
         assert_allclose(var[i], var_expect)
+
+    # test behavior if not all emulators are fit
+
+    gp.emulators[1].theta = None
+
+    mu, var, deriv = gp.predict(x_test, allow_not_fit=True)
+    assert np.all(np.isnan(mu[1]))
+    assert np.all(np.isnan(var[1]))
+    assert np.all(np.isnan(deriv[1]))
+
+    mu, var, deriv = gp.predict(x_test, unc=False, deriv=False,
+                                allow_not_fit=True)
+    assert np.all(np.isnan(mu[1]))
+    assert var is None
+    assert deriv is None
+
+    with pytest.raises(ValueError):
+        gp.predict(x_test)
+
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_MultiOutputGP_GPU_predict(x, y, dx):
+    "test the predict method of GaussianProcess"
+
+    gp = MultiOutputGP(x, y, nugget=0., use_gpu=True)
+    theta = np.ones(gp.emulators[0].n_params)
+
+    gp.emulators[0].fit(theta)
+    gp.emulators[1].fit(theta)
+
+    x_test = np.array([[2., 3., 4.]])
+
+    mu, var, deriv = gp.predict(x_test)
+
+    for i in range(2):
+
+        K = gp.emulators[i].kernel.kernel_f(x, x, theta[:-1])
+        Ktest = gp.emulators[i].kernel.kernel_f(x_test, x, theta[:-1])
+
+        mu_expect = np.dot(Ktest, gp.emulators[i].invQt)
+        var_expect = np.exp(theta[-2]) - np.diag(np.dot(Ktest, np.linalg.solve(K, Ktest.T)))
+
+        D = gp.D
+
+        deriv_expect = np.zeros((1, D))
+
+        for j in range(D):
+            dx_array = np.zeros(D)
+            dx_array[j] = dx
+            deriv_expect[0, j] = (gp.emulators[i].predict(x_test)[0] - gp.emulators[i].predict(x_test - dx_array)[0])/dx
+
+        assert_allclose(mu[i], mu_expect)
+        assert_allclose(var[i], var_expect)
+        assert_allclose(deriv[i], deriv_expect, atol=1.e-6, rtol=1.e-6)
+
+    nugget = 1.
+    gp = MultiOutputGP(x, y, nugget=nugget, use_gpu=True)
+    theta = np.ones(gp.emulators[0].n_params)
+
+    gp.emulators[0].fit(theta)
+    gp.emulators[1].fit(theta)
+
+    x_test = np.array([[2., 3., 4.]])
+
+    mu, var, deriv = gp.predict(x_test)
+
+    for i in range(2):
+
+        K = gp.emulators[i].kernel.kernel_f(x, x, theta[:-1]) + np.eye(gp.emulators[i].n)*nugget
+        Ktest = gp.emulators[i].kernel.kernel_f(x_test, x, theta[:-1])
+
+        var_expect = np.exp(theta[-2]) + nugget - np.diag(np.dot(Ktest, np.linalg.solve(K, Ktest.T)))
+
+        assert_allclose(var[i], var_expect)
+
+    mu, var, deriv = gp.predict(x_test, include_nugget=False)
+
+    for i in range(2):
+
+        K = gp.emulators[i].kernel.kernel_f(x, x, theta[:-1]) + np.eye(gp.emulators[i].n)*nugget
+        Ktest = gp.emulators[i].kernel.kernel_f(x_test, x, theta[:-1])
+
+        var_expect = np.exp(theta[-2]) - np.diag(np.dot(Ktest, np.linalg.solve(K, Ktest.T)))
+
+        assert_allclose(var[i], var_expect)
+
+    gp.emulators[1].theta = None
+
+    mu, var, deriv = gp.predict(x_test, allow_not_fit=True)
+    assert np.all(np.isnan(mu[1]))
+    assert np.all(np.isnan(var[1]))
+    assert np.all(np.isnan(deriv[1]))
+
+    mu, var, deriv = gp.predict(x_test, unc=False, deriv=False, allow_not_fit=True)
+    assert np.all(np.isnan(mu[1]))
+    assert var is None
+    assert deriv is None
+
+    with pytest.raises(ValueError):
+        gp.predict(x_test, allow_not_fit=False)
+
+
+def test_MultiOutputGP_check(x, y):
+    """test the methods of MultiOutputGP that extracts GPs that have or
+    have not been fit (or their indices)
+    """
+
+    gp = MultiOutputGP(x, y, nugget=0.)
+    theta = np.ones(gp.emulators[0].n_params)
+
+    assert gp.get_indices_fit() == []
+    assert gp.get_indices_not_fit() == [0, 1]
+    assert len(gp.get_emulators_fit()) == 0
+    assert len(gp.get_emulators_not_fit()) == 2
+
+    gp.emulators[0].fit(theta)
+
+    assert gp.get_indices_fit() == [0]
+    assert gp.get_indices_not_fit() == [1]
+    assert len(gp.get_emulators_fit()) == 1
+    assert len(gp.get_emulators_not_fit()) == 1
+
+    gp.emulators[1].fit(theta)
+
+    assert gp.get_indices_fit() == [0, 1]
+    assert gp.get_indices_not_fit() == []
+    assert len(gp.get_emulators_fit()) == 2
+    assert len(gp.get_emulators_not_fit()) == 0
+
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_MultiOutputGP_GPU_check(x, y):
+    """test the methods of MultiOutputGP that extracts GPs that have or
+    have not been fit (or their indices)
+    """
+
+    gp = MultiOutputGP(x, y, nugget=0., use_gpu=True)
+    theta = np.ones(gp.emulators[0].n_params)
+
+    assert gp.get_indices_fit() == []
+    assert gp.get_indices_not_fit() == [0, 1]
+    assert len(gp.get_emulators_fit()) == 0
+    assert len(gp.get_emulators_not_fit()) == 2
+
+    gp.emulators[0].fit(theta)
+
+    assert gp.get_indices_fit() == [0]
+    assert gp.get_indices_not_fit() == [1]
+    assert len(gp.get_emulators_fit()) == 1
+    assert len(gp.get_emulators_not_fit()) == 1
+
+    gp.emulators[1].fit(theta)
+
+    assert gp.get_indices_fit() == [0, 1]
+    assert gp.get_indices_not_fit() == []
+    assert len(gp.get_emulators_fit()) == 2
+    assert len(gp.get_emulators_not_fit()) == 0
diff --git a/mogp_emulator/tests/test_SequentialDesign.py b/mogp_emulator/tests/test_SequentialDesign.py
index 1950a1cd..3aee2f0f 100644
--- a/mogp_emulator/tests/test_SequentialDesign.py
+++ b/mogp_emulator/tests/test_SequentialDesign.py
@@ -8,6 +8,9 @@
 from ..GaussianProcess import GaussianProcess
 from ..fitting import fit_GP_MAP
 from tempfile import TemporaryFile
+from ..LibGPGPU import gpu_usable
+
+GPU_NOT_FOUND_MSG = "A compatible GPU could not be found or the GPU library (libgpgpu) could not be loaded"
 
 def test_SequentialDesign_init():
     "test the init method of ExperimentalDesign"
@@ -859,6 +862,7 @@ def test_MICEDesign_estimate_next_target():
     with pytest.raises(AssertionError):
         md._estimate_next_target(np.zeros(2))
 
+
 def test_MICEDesign_MICE_criterion():
     "test the function to compute the MICE criterion"
 
@@ -927,4 +931,4 @@ def test_MICEFastGP():
         result = gp.fast_predict(-1)
 
     with pytest.raises(AssertionError):
-        result = gp.fast_predict(5)
\ No newline at end of file
+        result = gp.fast_predict(5)
diff --git a/mogp_emulator/tests/test_fitting.py b/mogp_emulator/tests/test_fitting.py
index 34239e8f..fd6318ec 100644
--- a/mogp_emulator/tests/test_fitting.py
+++ b/mogp_emulator/tests/test_fitting.py
@@ -2,12 +2,22 @@
 import pytest
 from numpy.testing import assert_allclose
 from ..GaussianProcess import GaussianProcess
+from ..GaussianProcessGPU import GaussianProcessGPU
+from ..LibGPGPU import gpu_usable
 from ..MultiOutputGP import MultiOutputGP
 from ..fitting import fit_GP_MAP, _fit_single_GP_MAP, _fit_MOGP_MAP
 
-def test_fit_GP_MAP():
+GPU_NOT_FOUND_MSG = "A compatible GPU could not be found or the GPU library (libgpgpu) could not be loaded"
+
+def minimize_mock(fun, x0, args=(), method=None, jac=None, hess=None, hessp=None, bounds=None, constraints=(), tol=None, callback=None, options=None):
+    return {"x": np.array([1.6, -2.1, -0.8]),
+            "fun": 0.}
+
+def test_fit_GP_MAP(monkeypatch):
     "test the fit_GP_MAP function"
 
+    monkeypatch.setattr("mogp_emulator.fitting.minimize", minimize_mock)
+
     # test correct basic functioning
 
     x = np.linspace(0., 1.)
@@ -15,9 +25,8 @@ def test_fit_GP_MAP():
 
     gp = GaussianProcess(x, y)
 
-    theta_exp = np.array([ 1.6030532031342832, -2.090511425471982 , -0.7803960307137198])
-    logpost_exp = -296.0297245831661
-    np.random.seed(4335)
+    theta_exp = np.array([ 1.6, -2.1 , -0.8])
+    logpost_exp = gp.logposterior(theta_exp)
 
     gp = fit_GP_MAP(gp)
 
@@ -27,13 +36,43 @@ def test_fit_GP_MAP():
 
     # same test, but pass args and kwargs rather than gp
 
-    np.random.seed(4335)
-
     gp = fit_GP_MAP(x, y, mean="0.", use_patsy=False, method="L-BFGS-B")
     assert isinstance(gp, GaussianProcess)
     assert_allclose(gp.theta, theta_exp)
     assert_allclose(gp.current_logpost, logpost_exp)
 
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_fit_GP_MAP_GPU(monkeypatch):
+    "test the fit_GP_MAP function"
+
+    monkeypatch.setattr("mogp_emulator.fitting.minimize", minimize_mock)
+
+    # test correct basic functioning
+
+    x = np.linspace(0., 1.)
+    y = x**2
+
+    gp = GaussianProcessGPU(x, y)
+
+    theta_exp = np.array([ 1.6, -2.1 , -0.8])
+    logpost_exp = gp.logposterior(theta_exp)
+
+    gp = fit_GP_MAP(gp)
+
+    assert isinstance(gp, GaussianProcessGPU)
+    assert_allclose(gp.theta, theta_exp)
+    assert_allclose(gp.current_logpost, logpost_exp)
+
+
+def test_fit_GP_MAP_failures():
+    "test failures of fit_GP_MAP"
+
+    x = np.linspace(0., 1.)
+    y = x**2
+
+    gp = GaussianProcess(x, y)
+
     # minimization fails
 
     with pytest.raises(RuntimeError):
@@ -41,6 +80,42 @@ def test_fit_GP_MAP():
 
     gp = GaussianProcess(x, y, nugget=0.)
 
+    with pytest.raises(RuntimeError):
+        fit_GP_MAP(gp, theta0=np.array([0., 0., 0.]), n_tries=1)
+
+    with pytest.raises(RuntimeError):
+        fit_GP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1)
+
+    # bad inputs
+
+    with pytest.raises(TypeError):
+        fit_GP_MAP(x)
+
+    with pytest.raises(TypeError):
+        fit_GP_MAP()
+
+    with pytest.raises(AssertionError):
+        fit_GP_MAP(gp, n_tries=-1)
+
+    with pytest.raises(AssertionError):
+        fit_GP_MAP(gp, theta0=np.ones(1))
+
+@pytest.mark.skipif(not gpu_usable(), reason=GPU_NOT_FOUND_MSG)
+def test_fit_GP_MAP_GPU_failures():
+    "test failures of fit_GP_MAP using GaussianProcessGPU"
+
+    x = np.linspace(0., 1.)
+    y = x**2
+
+    gp = GaussianProcessGPU(x, y)
+
+    # minimization fails
+
+    with pytest.raises(RuntimeError):
+        fit_GP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3))
+
+    gp = GaussianProcessGPU(x, y, nugget=0.)
+
     with pytest.raises(RuntimeError):
         fit_GP_MAP(gp, n_tries=1)
 
@@ -72,35 +147,21 @@ def test_fit_GP_MAP_MOGP():
 
     gp = MultiOutputGP(x, y)
 
-    theta_exp = np.zeros((2, 3))
-    theta_exp[0] = np.array([ 1.6030532031342832, -2.090511425471982 , -0.7803960307137198])
-    theta_exp[1] = np.array([ 1.414112951818647 , -0.5974688573393576,  0.6857536842773265])
-    logpost_exp = np.zeros(2)
-    logpost_exp[0] = -296.0297245831661
-    logpost_exp[1] = -250.06025683867367
     np.random.seed(4335)
 
-    gp = fit_GP_MAP(gp, processes=1)
+    gp = fit_GP_MAP(gp)
 
     assert isinstance(gp, MultiOutputGP)
-    for i in range(2):
-        assert_allclose(gp.emulators[i].theta, theta_exp[i])
-        assert_allclose(gp.emulators[i].current_logpost, logpost_exp[i])
-
-    # same test, but pass args and kwargs rather than gp
 
     np.random.seed(4335)
 
+    # same test, but pass args and kwargs rather than gp
+
     gp = fit_GP_MAP(x, y, mean="0.", use_patsy=False, method="L-BFGS-B", processes=1)
     assert isinstance(gp, MultiOutputGP)
-    for i in range(2):
-        assert_allclose(gp.emulators[i].theta, theta_exp[i])
-        assert_allclose(gp.emulators[i].current_logpost, logpost_exp[i])
 
     # pass processes argument
 
-    np.random.seed(4335)
-
     gp = fit_GP_MAP(x, y, mean="0.", use_patsy=False, method="L-BFGS-B", processes=1)
     assert isinstance(gp, MultiOutputGP)
 
@@ -112,18 +173,52 @@ def test_fit_GP_MAP_MOGP():
 
     gp = fit_GP_MAP(x, y, theta0=[None, np.zeros(3)])
 
+    # test re-fitting
+
+    gp = MultiOutputGP(x, y)
+    gp.emulators[0].fit(np.ones(3))
+
+    gp = fit_GP_MAP(gp, theta0=np.zeros(3), n_tries=1, refit=False)
+    assert_allclose(gp.emulators[0].theta, np.ones(3))
+    assert not gp.emulators[1].theta is None
+
+    gp = fit_GP_MAP(gp, theta0=np.zeros(3), n_tries=1, refit=True)
+    assert not np.allclose(gp.emulators[0].theta, np.ones(3))
+
+def test_fit_GP_MAP_MOGP_failures():
+    "test situations where mogp fitting should fail"
+
+    x = np.linspace(0., 1.)
+    y = np.zeros((2, 50))
+    y[0] = x**2
+    y[1] = 2. + x**3
+
+    gp = MultiOutputGP(x, y)
+
     # minimization fails
 
+    gp = fit_GP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3))
+    assert gp.get_indices_not_fit() == [0, 1]
+
     with pytest.raises(RuntimeError):
-        fit_GP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3))
+        gp = fit_GP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3),
+                        skip_failures=False, refit=True)
 
     gp = MultiOutputGP(x, y, nugget=0.)
 
+    gp = fit_GP_MAP(gp, theta0=np.array([0., 0., 0.]), n_tries=1)
+    assert gp.get_indices_not_fit() == [0, 1]
+
     with pytest.raises(RuntimeError):
-        fit_GP_MAP(gp, n_tries=1)
+        gp = fit_GP_MAP(gp, theta0=np.array([0., 0., 0.]), n_tries=1,
+                        skip_failures=False, refit=True)
+
+    gp = fit_GP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1)
+    assert gp.get_indices_not_fit() == [0, 1]
 
     with pytest.raises(RuntimeError):
-        fit_GP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1)
+        gp = fit_GP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1,
+                        skip_failures=False, refit=True)
 
     # bad inputs
 
@@ -154,17 +249,18 @@ def test_fit_GP_MAP_MOGP():
     with pytest.raises(AssertionError):
         fit_GP_MAP(gp, theta0=[np.zeros(1), np.zeros(3)], processes=1)
 
-def test_fit_single_GP_MAP():
+def test_fit_single_GP_MAP(monkeypatch):
     "test the method to run the minimization algorithm on a GP class"
 
+    monkeypatch.setattr("mogp_emulator.fitting.minimize", minimize_mock)
+
     x = np.linspace(0., 1.)
     y = x**2
 
     gp = GaussianProcess(x, y)
 
-    theta_exp = np.array([ 1.6030532031342832, -2.090511425471982 , -0.7803960307137198])
-    logpost_exp = -296.0297245831661
-    np.random.seed(4335)
+    theta_exp = np.array([ 1.6, -2.1 , -0.8])
+    logpost_exp = gp.logposterior(theta_exp)
 
     gp = _fit_single_GP_MAP(gp)
 
@@ -172,18 +268,27 @@ def test_fit_single_GP_MAP():
     assert_allclose(gp.theta, theta_exp)
     assert_allclose(gp.current_logpost, logpost_exp)
 
+
+def test_fit_single_GP_MAP_failures():
+    "test situation where fitting one emulator should fail"
+
+    x = np.linspace(0., 1.)
+    y = x**2
+
+    gp = GaussianProcess(x, y)
+
     # minimization fails
 
-    with pytest.raises(RuntimeError):
-        _fit_single_GP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3))
+    gp = _fit_single_GP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3))
+    assert gp.theta is None
 
     gp = GaussianProcess(x, y, nugget=0.)
 
-    with pytest.raises(RuntimeError):
-        _fit_single_GP_MAP(gp, n_tries=1)
+    gp = _fit_single_GP_MAP(gp, theta0 = np.array([0., 0., 0.]), n_tries=1)
+    assert gp.theta is None
 
-    with pytest.raises(RuntimeError):
-        _fit_single_GP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1)
+    gp =_fit_single_GP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1)
+    assert gp.theta is None
 
     # bad inputs
 
@@ -196,7 +301,7 @@ def test_fit_single_GP_MAP():
     with pytest.raises(AssertionError):
         fit_GP_MAP(gp, theta0=np.ones(1))
 
-def test_fit_MOGP_MAP_MOGP():
+def test_fit_MOGP_MAP():
     "test the fit_GP_MAP function with multiple outputs"
 
     x = np.linspace(0., 1.)
@@ -206,20 +311,11 @@ def test_fit_MOGP_MAP_MOGP():
 
     gp = MultiOutputGP(x, y)
 
-    theta_exp = np.zeros((2, 3))
-    theta_exp[0] = np.array([ 1.6030532031342832, -2.090511425471982 , -0.7803960307137198])
-    theta_exp[1] = np.array([ 1.414112951818647 , -0.5974688573393576,  0.6857536842773265])
-    logpost_exp = np.zeros(2)
-    logpost_exp[0] = -296.0297245831661
-    logpost_exp[1] = -250.06025683867367
     np.random.seed(4335)
 
-    gp = _fit_MOGP_MAP(gp, processes=1)
+    gp = _fit_MOGP_MAP(gp)
 
     assert isinstance(gp, MultiOutputGP)
-    for i in range(2):
-        assert_allclose(gp.emulators[i].theta, theta_exp[i])
-        assert_allclose(gp.emulators[i].current_logpost, logpost_exp[i])
 
     # pass processes argument
 
@@ -230,24 +326,47 @@ def test_fit_MOGP_MAP_MOGP():
 
     # pass various theta0 arguments
 
-    gp = fit_GP_MAP(gp, theta0=np.zeros(3))
+    gp = _fit_MOGP_MAP(gp, theta0=np.zeros(3))
+
+    gp = _fit_MOGP_MAP(gp, theta0=np.zeros((2, 3)))
+
+    gp = _fit_MOGP_MAP(gp, theta0=[None, np.zeros(3)])
+
+    # test re-fitting
 
-    gp = fit_GP_MAP(gp, theta0=np.zeros((2, 3)))
+    gp = MultiOutputGP(x, y)
+    gp.emulators[0].fit(np.ones(3))
+
+    gp = _fit_MOGP_MAP(gp, theta0=np.zeros(3), n_tries=1, refit=False)
+    assert_allclose(gp.emulators[0].theta, np.ones(3))
+    assert not gp.emulators[1].theta is None
+
+    gp = _fit_MOGP_MAP(gp, theta0=np.zeros(3), n_tries=1, refit=True)
+    assert not np.allclose(gp.emulators[0].theta, np.ones(3))
+
+
+def test_fit_MOGP_MAP_failures():
+    "test situations where fitting should fail"
+
+    x = np.linspace(0., 1.)
+    y = np.zeros((2, 50))
+    y[0] = x**2
+    y[1] = 2. + x**3
 
-    gp = fit_GP_MAP(gp, theta0=[None, np.zeros(3)])
+    gp = MultiOutputGP(x, y)
 
     # minimization fails
 
-    with pytest.raises(RuntimeError):
-        _fit_MOGP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3))
+    gp = _fit_MOGP_MAP(gp, n_tries=1, theta0=-10000.*np.ones(3))
+    assert gp.get_indices_not_fit() == [0, 1]
 
     gp = MultiOutputGP(x, y, nugget=0.)
 
-    with pytest.raises(RuntimeError):
-        _fit_MOGP_MAP(gp, n_tries=1)
+    gp = _fit_MOGP_MAP(gp, theta0=np.array([0., 0., 0.]), n_tries=1)
+    assert gp.get_indices_not_fit() == [0, 1]
 
-    with pytest.raises(RuntimeError):
-        _fit_MOGP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1)
+    gp = _fit_MOGP_MAP(gp, theta0 = np.array([800., 0., 0.]), n_tries=1)
+    assert gp.get_indices_not_fit() == [0, 1]
 
     # bad inputs
 
@@ -276,4 +395,4 @@ def test_fit_MOGP_MAP_MOGP():
         _fit_MOGP_MAP(gp, theta0=[np.zeros(1), None], processes=1)
 
     with pytest.raises(AssertionError):
-        _fit_MOGP_MAP(gp, processes=-1)
\ No newline at end of file
+        _fit_MOGP_MAP(gp, processes=-1)
diff --git a/mogp_emulator/tests/test_linalg.py b/mogp_emulator/tests/test_linalg.py
new file mode 100644
index 00000000..b17a1ba2
--- /dev/null
+++ b/mogp_emulator/tests/test_linalg.py
@@ -0,0 +1,124 @@
+import numpy as np
+import pytest
+from numpy.testing import assert_allclose
+from ..linalg.cholesky import jit_cholesky, _check_cholesky_inputs, pivot_cholesky, _pivot_transpose
+from ..linalg.cholesky import pivot_cho_solve
+from scipy import linalg
+
+def test_check_cholesky_inputs():
+    "Test function that checks inputs to cholesky decomposition routines"
+    
+    A = np.array([[2., 1.], [1., 2.]])
+    B = _check_cholesky_inputs(A)
+    
+    assert_allclose(A, B)
+
+    A = np.array([[1., 2.], [1., 2.]])
+    with pytest.raises(AssertionError):
+        _check_cholesky_inputs(A)
+    
+    A = np.array([1., 2.])
+    with pytest.raises(AssertionError):
+        _check_cholesky_inputs(A)
+        
+    A = np.array([[1., 2., 3.], [4., 5., 6.]])
+    with pytest.raises(AssertionError):
+        _check_cholesky_inputs(A)
+    
+    input_matrix = np.array([[-1., 2., 2.], [2., 3., 2.], [2., 2., -3.]])
+    with pytest.raises(linalg.LinAlgError):
+        _check_cholesky_inputs(input_matrix)
+
+def test_jit_cholesky():
+    "Tests the stabilized Cholesky decomposition routine"
+    
+    L_expected = np.array([[2., 0., 0.], [6., 1., 0.], [-8., 5., 3.]])
+    input_matrix = np.array([[4., 12., -16.], [12., 37., -43.], [-16., -43., 98.]])
+    L_actual, jitter = jit_cholesky(input_matrix)
+    assert_allclose(L_expected, L_actual)
+    assert_allclose(jitter, 0.)
+    
+    L_expected = np.array([[1.0000004999998751e+00, 0.0000000000000000e+00, 0.0000000000000000e+00],
+                         [9.9999950000037496e-01, 1.4142132088085626e-03, 0.0000000000000000e+00],
+                         [6.7379436301144941e-03, 4.7644444411381860e-06, 9.9997779980004420e-01]])
+    input_matrix = np.array([[1.                , 1.                , 0.0067379469990855],
+                             [1.                , 1.                , 0.0067379469990855],
+                             [0.0067379469990855, 0.0067379469990855, 1.                ]])
+    L_actual, jitter = jit_cholesky(input_matrix)
+    assert_allclose(L_expected, L_actual)
+    assert_allclose(jitter, 1.e-6)
+    
+    input_matrix = np.array([[1.e-6, 1., 0.], [1., 1., 1.], [0., 1., 1.e-10]])
+    with pytest.raises(linalg.LinAlgError):
+        jit_cholesky(input_matrix)
+
+def test_pivot_cholesky():
+    "Tests  pivoted cholesky decomposition routine"
+    
+    input_matrix = np.array([[4., 12., -16.], [12., 37., -43.], [-16., -43., 98.]])
+    input_matrix_copy = np.copy(input_matrix)
+    L_expected = np.array([[ 9.899494936611665 ,  0.                ,  0.                ],
+                           [-4.3436559415745055,  4.258245303082538 ,  0.                ],
+                           [-1.616244071283537 ,  1.1693999481734827,  0.1423336335961131]])
+    Piv_expected = np.array([2, 1, 0], dtype = np.int32)
+    
+    L_actual, Piv_actual = pivot_cholesky(input_matrix)
+    
+    assert_allclose(L_actual, L_expected)
+    assert np.array_equal(Piv_expected, Piv_actual)
+    assert_allclose(input_matrix, input_matrix_copy)
+    
+    input_matrix = np.array([[1., 1., 1.e-6], [1., 1., 1.e-6], [1.e-6, 1.e-6, 1.]])
+    input_matrix_copy = np.copy(input_matrix)
+    L_expected = np.array([[1.0000000000000000e+00, 0.0000000000000000e+00, 0.0000000000000000e+00],
+                           [9.9999999999999995e-07, 9.9999999999949996e-01, 0.0000000000000000e+00],
+                           [1.0000000000000000e+00, 0.0000000000000000e+00, 3.3333333333316667e-01]])
+    Piv_expected = np.array([0, 2, 1], dtype=np.int32)               
+    
+    L_actual, Piv_actual = pivot_cholesky(input_matrix)
+    
+    assert_allclose(L_actual, L_expected)
+    assert np.array_equal(Piv_expected, Piv_actual)
+    assert_allclose(input_matrix, input_matrix_copy)
+    
+def test_pivot_transpose():
+    "Test function to invert pivot matrix"
+    
+    P = np.array([0, 2, 1], dtype = np.int32)
+    
+    Piv = _pivot_transpose(P)
+    
+    np.array_equal(Piv, P)
+    
+    P = np.array([1, 2, 1], dtype = np.int32)
+    
+    with pytest.raises(ValueError):
+        _pivot_transpose(P)
+
+    P = np.array([[0, 1, 2], [2, 1, 0]], dtype=np.int32)
+
+    with pytest.raises(AssertionError):
+        _pivot_transpose(P)
+
+def test_pivot_cho_solve():
+    "test the cho_solve routine using pivoting"
+
+    A = np.array([[2., 1., 0.4], [1., 2., 0.2], [0.4, 0.2, 2.]])
+    b = np.array([2., 3., 1.])
+
+    L = np.linalg.cholesky(A)
+
+    x = linalg.cho_solve((L, True), b)
+
+    L_pivot, P = pivot_cholesky(A)
+
+    x_pivot = pivot_cho_solve(L_pivot, P, b)
+
+    assert_allclose(x, x_pivot)
+
+    with pytest.raises(AssertionError):
+        pivot_cho_solve(L_pivot, np.array([0, 2, 1, 1], dtype=np.int32), b)
+
+    with pytest.raises(ValueError):
+        pivot_cho_solve(L_pivot, np.array([0, 0, 1], dtype=np.int32), b)
+       
diff --git a/mogp_gpu/test/gpu_basic.cu b/mogp_gpu/test/gpu_basic.cu
new file mode 100644
index 00000000..5ec85868
--- /dev/null
+++ b/mogp_gpu/test/gpu_basic.cu
@@ -0,0 +1,117 @@
+#include <iostream>
+
+#include <vector>
+#include <algorithm>
+#include <string>
+#include <sstream>
+#include <assert.h>
+#include <stdexcept>
+#include <cuda_runtime.h>
+#include <cublas_v2.h>
+#include <cusolverDn.h>
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+#include <thrust/copy.h>
+
+#include "../src/gp_gpu.hpp"
+
+#define WARP_SIZE 32
+#define FULL_MASK 0xffffffff
+
+typedef double REAL;
+
+void test_device_vector_copy()
+{
+    const size_t N=5;
+    // std::vector<REAL> a{1.0, 2.0, 3.0, 4.0, 5.0};
+    const double a[] = {1.0, 2.0, 3.0, 4.0, 5.0};
+    thrust::device_vector<REAL> a_d(N);
+
+    std::vector<REAL> result(N);
+    thrust::device_vector<REAL> result_d(N);
+
+    thrust::copy(a, a + N, a_d.begin());
+
+    thrust::fill(result_d.begin(), result_d.end(), 0.0);
+    thrust::transform(a_d.begin(), a_d.end(), a_d.begin(), result_d.begin(), thrust::plus<float>());
+
+    thrust::copy(result_d.begin(), result_d.end(), result.begin());
+
+    for (int i=0; i<N; i++)
+        std::cout << result[i] << " ";
+    std::cout << "\n";
+}
+
+
+
+void test_sum_log_diag()
+{
+
+  size_t n=3;
+  std::vector<REAL> x{1.0, 2.0, 3.0, 4., 5., 6., 7., 8., 9.};
+  thrust::device_vector<REAL> x_d(x);
+  // determine the size of the buffer for cub::device reduce
+
+
+  // buffer for cub::DeviceReduce::Sum
+  thrust::device_vector<REAL> sum_buffer_d;
+
+    // size of sum_buffer_d
+  size_t sum_buffer_size_bytes;
+  // The following call determines the size of the cub::DeviceReduce::Sum workspace:
+  // sum_buffer_size_bytes is 0 before this call, and the size of sum_buffer_d afterwards.
+  // The end iterators are supplied but are not used.
+  cub::DeviceReduce::Sum(dev_ptr(sum_buffer_d), sum_buffer_size_bytes,
+			 sum_buffer_d.end(), sum_buffer_d.end(), n);
+  sum_buffer_d.resize(sum_buffer_size_bytes);
+
+  double result;
+  thrust::device_vector<double> result_d(1);
+  // call sum_log_diag
+  sum_log_diag(n, dev_ptr(x_d), dev_ptr(result_d), dev_ptr(sum_buffer_d), sum_buffer_size_bytes);
+  thrust::copy(result_d.begin(), result_d.end(), &result);
+
+  std::cout<<"Result of sum_log_diag is "<<result<<std::endl;
+}
+
+void test_trace()
+{
+
+  size_t n=3;
+  std::vector<REAL> x{1.0, 2.0, 3.0, 4., 5., 6., 7., 8., 9.};
+  thrust::device_vector<REAL> x_d(x);
+  // determine the size of the buffer for cub::device reduce
+
+  // buffer for cub::DeviceReduce::Sum
+  thrust::device_vector<REAL> sum_buffer_d;
+
+    // size of sum_buffer_d
+  size_t sum_buffer_size_bytes;
+  // The following call determines the size of the cub::DeviceReduce::Sum workspace:
+  // sum_buffer_size_bytes is 0 before this call, and the size of sum_buffer_d afterwards.
+  // The end iterators are supplied but are not used.
+  cub::DeviceReduce::Sum(dev_ptr(sum_buffer_d), sum_buffer_size_bytes,
+			 sum_buffer_d.end(), sum_buffer_d.end(), n);
+  sum_buffer_d.resize(sum_buffer_size_bytes);
+
+  double result;
+  thrust::device_vector<double> result_d(1);
+  // call trace
+  trace(n, dev_ptr(x_d), dev_ptr(result_d), dev_ptr(sum_buffer_d), sum_buffer_size_bytes);
+  thrust::copy(result_d.begin(), result_d.end(), &result);
+
+  std::cout<<"Result of trace is "<<result<<std::endl;
+}
+
+
+int main(void)
+{
+    test_device_vector_copy();
+    test_sum_log_diag();
+    test_trace();
+    return 0;
+}
diff --git a/mogp_gpu/test/gpu_cholesky.cu b/mogp_gpu/test/gpu_cholesky.cu
new file mode 100644
index 00000000..5e4edd69
--- /dev/null
+++ b/mogp_gpu/test/gpu_cholesky.cu
@@ -0,0 +1,117 @@
+/// Example of using cuSOLVER Cholesky decomposition, based on StackOverflow
+/// https://stackoverflow.com/questions/29196139/cholesky-decomposition-with-cuda
+
+#include<iostream>
+#include <vector>
+#include <algorithm>
+#include <string>
+#include <sstream>
+#include <assert.h>
+#include <stdexcept>
+
+#include <cuda_runtime.h>
+#include <cublas_v2.h>
+#include <cusolverDn.h>
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+#include <thrust/copy.h>
+
+#define WARP_SIZE 32
+#define FULL_MASK 0xffffffff
+
+typedef double REAL;
+
+template <typename T>
+T *dev_ptr(thrust::device_vector<T>& dv)
+{
+    return dv.data().get();
+}
+
+
+/************************************/
+/*
+   Create test matrix (from Wikipedia)
+      4  12 -16       2 0 0     2 6 -8
+    ( 12 37 -43 ) = ( 6 1 0 ) ( 0 1  5 )
+     -16 -43 98      -8 5 3     0 0 3
+ */
+/************************************/
+
+void setTest3x3Matrix(double * __restrict h_A) {
+  h_A[0] = 4.0; h_A[1] = 12.0; h_A[2] = -16.0;
+  h_A[3] = 12.0; h_A[4] = 37.; h_A[5] = -43.0;
+  h_A[6] = 16.0; h_A[7] = -43.0; h_A[8] = 98.0;
+}
+
+
+void testCholesky()
+{
+
+    // --- CUDA solver initialization
+    cusolverDnHandle_t solver_handle;
+    cusolverDnCreate(&solver_handle);
+
+    // --- CUBLAS initialization
+    cublasHandle_t cublas_handle;
+    cublasCreate(&cublas_handle);
+
+    /**********************
+    Use 3x3 example from wikipedia
+    ************************/
+
+    const int N = 3;
+    double *A_h = (double *)malloc(N * N * sizeof(double));
+    setTest3x3Matrix(A_h);
+
+    // copy to device
+    thrust::device_vector<REAL> A_d(N * N);
+
+    thrust::copy(A_h, A_h + N*N, A_d.begin());
+
+    /****************************************/
+    /* COMPUTING THE CHOLESKY DECOMPOSITION */
+    /****************************************/
+    // --- cuSOLVE input/output parameters/arrays
+    int work_size = 0;
+    int info_h;
+    thrust::device_vector<int> info_d(1);
+    //   int *devInfo;           cudaMalloc(&devInfo, sizeof(int));
+
+    // --- CUDA CHOLESKY initialization
+    cusolverDnDpotrf_bufferSize(solver_handle, CUBLAS_FILL_MODE_LOWER, N, dev_ptr(A_d), N, &work_size);
+
+    std::cout<<"Buffer size is "<<work_size<<std::endl;
+    // make the buffer and resize it to the right size
+    thrust::device_vector<REAL> potrf_buffer_d;
+    potrf_buffer_d.resize(work_size);
+
+    // --- CUDA POTRF execution
+    cusolverStatus_t status = cusolverDnDpotrf(solver_handle, CUBLAS_FILL_MODE_LOWER, N, dev_ptr(A_d), N, dev_ptr(potrf_buffer_d), work_size, dev_ptr(info_d));
+
+    // check the status
+    thrust::copy(info_d.begin(), info_d.end(), &info_h);
+
+    if (status != CUSOLVER_STATUS_SUCCESS || info_h < 0) {
+      std::string msg;
+      std::stringstream smsg(msg);
+      smsg << "Error in potrf: return code " << status << ", info " << info_h;
+      throw std::runtime_error(smsg.str());
+
+    }
+
+    std::vector<double> expected = { 2.0, 6.0, -8.0, 12.0, 1.0, 5.0, 16.0, -43.0, 3.0};
+    for (unsigned int i=0; i< A_d.size(); ++i) {
+      std::cout<<"element "<<i<<" is "<<A_d[i]<<std::endl;
+      assert (A_d[i] == expected[i]);
+    }
+
+}
+
+int main() {
+  testCholesky();
+  return 0;
+}
\ No newline at end of file
diff --git a/mogp_gpu/test/gpu_kernel.cu b/mogp_gpu/test/gpu_kernel.cu
new file mode 100644
index 00000000..66c585a1
--- /dev/null
+++ b/mogp_gpu/test/gpu_kernel.cu
@@ -0,0 +1,138 @@
+#include <iostream>
+
+#include <vector>
+#include <algorithm>
+#include <string>
+#include <sstream>
+#include <assert.h>
+#include <stdexcept>
+#include <cuda_runtime.h>
+#include <cublas_v2.h>
+#include <cusolverDn.h>
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+#include <thrust/copy.h>
+
+#include "../src/kernel.hpp"
+#include "../src/gp_gpu.hpp"
+
+#define WARP_SIZE 32
+#define FULL_MASK 0xffffffff
+
+typedef double REAL;
+
+void test_cov(kernel_type kernel)
+{
+  BaseKernel* kern(0);
+  if (kernel == SQUARED_EXPONENTIAL)
+    kern = new SquaredExponentialKernel();
+  else if (kernel == MATERN52)
+    kern = new Matern52Kernel();
+  const size_t N=3;
+  const size_t Ninput=1;
+  const size_t Ntheta=Ninput+1;
+  std::vector<REAL> x{1.0, 2.0, 3.0};
+  std::vector<REAL> result(N*N);
+
+  thrust::device_vector<REAL> result_d(N*N, 0.0);
+  thrust::device_vector<REAL> x_d(x);
+  thrust::device_vector<REAL> theta_d(Ntheta, -1.0);
+  /// calculate the covariance matrix between the inputs x
+  kern->cov_batch_gpu(dev_ptr(result_d), N, N, Ninput, dev_ptr(x_d),
+                dev_ptr(x_d), dev_ptr(theta_d));
+
+  thrust::copy(result_d.begin(), result_d.end(), result.begin());
+  std::cout<<"Covariance matrix: "<<std::endl;
+  for (size_t i=0; i<result.size(); i++)
+    std::cout << result[i] << " ";
+  std::cout << "\n";
+  delete kern;
+}
+
+void test_cov_deriv(kernel_type kernel)
+{
+  BaseKernel* kern(0);
+    if (kernel == SQUARED_EXPONENTIAL)
+      kern = new SquaredExponentialKernel();
+    else if (kernel == MATERN52)
+      kern = new Matern52Kernel();
+    const size_t Ninput=3;
+    const size_t Ntheta=Ninput+1;
+    /// Test data - single points, 3 dimensions.
+    std::vector<REAL> x{1.0, 2.0, 3.0};
+    std::vector<REAL> y{4.0, 5.0, 6.0};
+
+    std::vector<REAL> result(Ntheta);
+
+    thrust::device_vector<REAL> result_d(Ntheta, 0.0);
+    thrust::device_vector<REAL> x_d(x);
+    thrust::device_vector<REAL> y_d(y);
+    thrust::device_vector<REAL> theta_d(Ntheta, -1.0);
+
+    // x, y
+
+    kern->cov_deriv_theta_batch_gpu(dev_ptr(result_d), Ninput, 1, 1, dev_ptr(x_d), dev_ptr(y_d), dev_ptr(theta_d));
+    thrust::copy(result_d.begin(), result_d.end(), result.begin());
+
+    for (size_t i=0; i<Ntheta; i++)
+        std::cout << result[i] << " ";
+    std::cout << "\n";
+
+    // x, x
+    kern->cov_deriv_theta_batch_gpu(dev_ptr(result_d), Ninput, 1, 1, dev_ptr(x_d), dev_ptr(x_d), dev_ptr(theta_d));
+    thrust::copy(result_d.begin(), result_d.end(), result.begin());
+
+    for (size_t i=0; i<Ntheta; i++)
+        std::cout << result[i] << " ";
+    std::cout << "\n";
+    delete kern;
+}
+
+void test_deriv_x(kernel_type kernel) {
+  BaseKernel* kern(0);
+  if (kernel == SQUARED_EXPONENTIAL)
+    kern = new SquaredExponentialKernel();
+  else if (kernel == MATERN52)
+    kern = new Matern52Kernel();
+  int Ninput = 2;
+  int N = 3;
+  mat x(N, Ninput);
+  // input x = [[1,2],[2,4],[3,6]]
+  for (int i=0; i<N; i++) {
+    int j = 2*(i+1);
+    x(i,0) = float(i+1);
+    x(i,1) = float(j);
+  }
+  std::cout<<" x: "<<x<<std::endl;
+  std::vector<REAL> result(N*N*Ninput);
+
+  thrust::device_vector<REAL> result_d(N*N*Ninput, 0.);
+  thrust::device_vector<REAL> x_d(x.data(), x.data() + Ninput * N);
+
+  std::vector<REAL> theta{-1.0, -1.0, -1.0};
+  thrust::device_vector<REAL> theta_d(theta);
+
+  kern->cov_deriv_x_batch_gpu(
+    dev_ptr(result_d), Ninput, N, N, dev_ptr(x_d), dev_ptr(x_d), dev_ptr(theta_d)
+  );
+  thrust::copy(result_d.begin(), result_d.end(), result.begin());
+  std::cout<<"drdx: "<<std::endl;
+  for (size_t i=0; i<result.size(); i++)
+    std::cout << result[i] << " ";
+  std::cout << "\n";
+  delete kern;
+}
+
+
+
+int main(void)
+{
+  test_cov(MATERN52);
+  test_cov_deriv(MATERN52);
+  test_deriv_x(MATERN52);
+  return 0;
+}
diff --git a/mogp_gpu/test/gpu_meanfunc.cu b/mogp_gpu/test/gpu_meanfunc.cu
new file mode 100644
index 00000000..78954d30
--- /dev/null
+++ b/mogp_gpu/test/gpu_meanfunc.cu
@@ -0,0 +1,93 @@
+#include <iostream>
+
+#include <vector>
+#include <algorithm>
+#include <string>
+#include <sstream>
+#include <assert.h>
+#include <stdexcept>
+#include <cuda_runtime.h>
+#include <cublas_v2.h>
+#include <cusolverDn.h>
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+#include <thrust/copy.h>
+
+#include "../src/meanfunc.hpp"
+
+typedef double REAL;
+
+void test_const_meanfunc()
+{
+    const size_t N=5;
+    vec x(5);
+    x << 1.0, 2.0, 3.0, 4.0, 5.0;
+
+    vec params(1);
+    params << 4.4;
+
+
+    ConstMeanFunc* mf = new ConstMeanFunc();
+    vec mean = mf->mean_f(x,params);
+    vec deriv = mf->mean_deriv(x,params);
+    vec inputderiv = mf->mean_inputderiv(x,params);
+
+    std::cout<<" mean: ";
+    for (int i=0; i<N; i++)
+        std::cout << mean [i] << " ";
+    std::cout << "\n deriv: ";
+    for (int i=0; i<N; i++)
+        std::cout << deriv [i] << " ";
+    std::cout << "\n inputderiv: ";
+    for (int i=0; i<N; i++)
+        std::cout << inputderiv [i] << " ";
+    std::cout << "\n";
+
+    delete mf;
+}
+
+void test_poly_meanfunc()
+{
+  // 1D case - 2nd order polynomial
+  // mean f should be 2.2 + 4.4*x + 3.3*x^2
+    const size_t N=5;
+    vec x(5);
+    x << 1.0, 2.0, 3.0, 4.0, 5.0;
+
+    vec params(3);
+    params << 4.4, 3,3, 2.2;
+
+    std::vector< std::pair<int, int> > dims_powers;
+    dims_powers.push_back(std::make_pair<int, int>(0,1));
+    dims_powers.push_back(std::make_pair<int, int>(0,2));
+    PolyMeanFunc* mf = new PolyMeanFunc(dims_powers);
+    vec mean = mf->mean_f(x,params);
+    vec deriv = mf->mean_deriv(x,params);
+    vec inputderiv = mf->mean_inputderiv(x,params);
+
+    std::cout<<" mean: ";
+    for (int i=0; i<N; i++)
+        std::cout << mean [i] << " ";
+    std::cout << "\n deriv: ";
+    for (int i=0; i<N; i++)
+        std::cout << deriv [i] << " ";
+    std::cout << "\n inputderiv: ";
+    for (int i=0; i<N; i++)
+        std::cout << inputderiv [i] << " ";
+    std::cout << "\n";
+
+    delete mf;
+}
+
+
+
+int main(void)
+{
+    test_const_meanfunc();
+    test_poly_meanfunc();
+    return 0;
+}
diff --git a/mogp_gpu/test/gpu_test_utils.cu b/mogp_gpu/test/gpu_test_utils.cu
new file mode 100644
index 00000000..f7207dcb
--- /dev/null
+++ b/mogp_gpu/test/gpu_test_utils.cu
@@ -0,0 +1,59 @@
+#include <iostream>
+
+#include <vector>
+#include <algorithm>
+#include <string>
+#include <sstream>
+#include <assert.h>
+#include <stdexcept>
+#include <cuda_runtime.h>
+#include <cublas_v2.h>
+#include <cusolverDn.h>
+#include <thrust/device_ptr.h>
+#include <thrust/device_malloc.h>
+#include <thrust/device_free.h>
+#include <thrust/device_vector.h>
+#include <thrust/functional.h>
+#include <thrust/transform_reduce.h>
+#include <thrust/copy.h>
+
+#include "../src/util.hpp"
+
+#define WARP_SIZE 32
+#define FULL_MASK 0xffffffff
+
+typedef double REAL;
+
+template <typename T>
+T *dev_ptr(thrust::device_vector<T>& dv)
+{
+    return dv.data().get();
+}
+
+
+void test_add_diagonal()
+{
+    const size_t N=3;
+    /// create 3x3 matrix and copy to device
+    const double a[] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0};
+    thrust::device_vector<REAL> a_d(N*N);
+    thrust::copy(a, a + N*N, a_d.begin());
+    // add b to the diagonal
+    const REAL b = 0.5;
+
+    add_diagonal(N, b, dev_ptr(a_d));
+    std::vector<double> expected = { 1.5, 2.0, 3.0, 4.0, 5.5, 6.0, 7.0, 8.0, 9.5};
+    for (unsigned int i=0; i< a_d.size(); ++i) {
+      std::cout<<"Element "<<i<<" is "<<a_d[i]<<std::endl;
+      assert (a_d[i] == expected[i]);
+    }
+}
+
+
+
+int main(void)
+{
+    test_add_diagonal();
+
+    return 0;
+}