High-dimensional Additive Gaussian Processes under Monotonicity Constraints
Abstract
We introduce an additive Gaussian process framework accounting for monotonicity constraints and scalable to high dimensions.
Our contributions are threefold.
First, we show that our framework enables to satisfy the constraints everywhere in the input space.
We also show that more general componentwise linear inequality constraints can be handled similarly, such as componentwise convexity.
Second, we propose the additive MaxMod algorithm for sequential dimension reduction.
By sequentially maximizing a squared-norm criterion, MaxMod identiﬁes the active input dimensions and reﬁnes the most important ones.
This criterion can be computed explicitly at a linear cost.
Finally, we provide open-source codes for our full framework.
We demonstrate the performance and scalability of the methodology in several synthetic examples with hundreds of dimensions under monotonicity constraints as well as on a real-world ﬂood application.
Introduction
The framework of additive and constrained Gaussian processes.
In high dimension, many statistical regression models are based on additive structures of the form:
[Equation]
Although such structures may lead to more “rigid” models than non-additive ones they result in simple frameworks that easily scale in high dimensions [1, 2].
Generalized additive models (GAMs) [1] and additive Gaussian processes (GPs) [3, 4] are the most common models in a wide range of applications.
The latter can also be seen as a generalization of GAMs that allow uncertainty quantiﬁcation.
As shown in [3, 4], additive GPs can signiﬁcantly improve modeling efﬁciency and have major advantages for interpretability.
Furthermore, in non-additive small-dimensional GP models, adding inequality constraints (in particular monotonicity which will be the main focus of this paper) leads to more realistic uncertainty quantiﬁcation in learning from real data [5–10].
Contributions.
Our contributions are threefold.
We combine the additive and constrained frameworks to propose an additive constrained GP (cGP) prior.
Our framework is based on a ﬁnite-dimensional representation involving one-dimensional knots for each active variable.
The corresponding mode predictor can be computed and posterior realizations can be sampled, both in a scalable way to high dimension.
We suggest the additive MaxMod algorithm for sequential dimension reduction.
At each step, MaxMod either adds a new variable to the model or inserts a knot for an active one.
This choice is made by maximizing the squared-norm modiﬁcation of the mode function, for which we supply exact expressions with linear complexity.
We provide open-source codes for our full framework.
We demonstrate the performance and scalability of our methodology with numerical examples in dimension up to 1000 as well as in a real-world application in dimension 37.
MaxMod identiﬁes the most important input variables, with data size as low as n = 2 d in dimension d .
It also yields accurate models satisfying the constraints everywhere on the input space.
36th Conference on Neural Information Processing Systems (NeurIPS 2022).
Range of applicability.
The computational bottleneck of cGPs is sampling from the posterior distribution.
Here, it boils down to sampling a constrained Gaussian vector with dimension equal to the number of knots.
This is done with Hamiltonian Monte Carlo (HMC) [11] which currently works with several hundreds of knots.
Notice that MaxMod enables to minimize this number of knots.
Our framework is illustrated with monotonicity constraints and can be directly applied to other componentwise constraints such as componentwise convexity.
These constraints should be linear and such that satisfying them on the knots is equivalent to satisfying them everywhere (see Section 3.2).
Related literature.
Additive GPs have been considered in [3, 4, 12–14], to name a few.
The beneﬁt of considering inequality constraints in (non-additive) GPs is demonstrated in [5–10, 15] and in application ﬁelds such as nuclear safety [5], geostatistics [6], tree distributions [7], econometrics [16], coastal ﬂooding [17], and nuclear physics [18].
Note that there is a bridge between cGPs and (frequentist) splines-based regression with reproducing kernel Hilbert spaces (RKHS) [19, 20].
Our cGP model and MaxMod algorithm are extensions of the works in [5, 6] and [21] (respectively) to the additive case.
Besides GPs, [22] provides a stochastic optimization framework with RKHS that yields regression with inequality constraints.
[23] suggests a non-parametric model for non-negative functions; a problem related to that of monotonicity constraints.
To the best of our knowledge, our framework is the ﬁrst that enables to satisfy the constraints (especially monotonicity) everywhere and to scale to high dimension (up to one thousand in our experiments).
In particular, the aforementioned cGP works are not applicable in these dimensions.
Furthermore, [22] does not satisfy the constraints everywhere, but only approximately and in expectation, and [23] does not directly address monotonicity constraints.
For instance, with increasing constraints in dimension one, our framework provides posterior sample functions h satisfying h ( u )  h ( v ) for all u  v .
In contrast, the derivative of h , h 0 ( u ) , would be guaranteed to be non-negative only at some points u in [15].
Also, h 0 ( u ) would only be guaranteed to be above some strictly negative threshold, in expectation in [22].
Note that for unconstrained GP regression, the likelihood and conditional distribution are explicit but their computation cost scales as O ( n 3 ) , where n is the size of the dataset.
An active strand of literature then focuses on providing efﬁcient numerical procedures [24, 25].
Because of the constraints, our work is complementary to this strand of literature.
Indeed, with cGPs, even if n is small, the numerical challenge of sampling from the posterior distribution remains, as discussed previously.
The main indicator of complexity is thus the number of knots, rather than the dataset size.
The references [24, 25] provide convergence guarantees.
At this stage, we do not provide convergence guarantees in our setting.
To reach that goal, a possibility would be to extend the convergence proof of MaxMod given in [21] from the non-additive to the additive case.
Two comments are in order on this.
First, the proof techniques in [21] are different from these in [24, 25] (and also from standard asymptotic statistics settings, for instance [26]).
Indeed, n is ﬁxed and the number of knots increases in [21], while n increases in [24, 25].
Second, extending the proof of [21] to the additive case is not straightforward, since the various spaces of (non-additive) functions and their closures in [21] would have to be appropriately redeﬁned for additive functions.
Note also that [21] relies on a convergence proof in [27], that would need to be extended from the non-additive to the additive setting.
Paper organization.
Section 2 describes the additive GP framework.
Sections 3 and 4 present our framework for additive cGPs and introduce the MaxMod algorithm.
Section 5 provides the numerical experiments.
Appendix A gathers the technical proofs as well as additional details and experiments.
Framework on additive Gaussian processes
In additive models, GP priors are placed here over the functions y 1 , . . .
, y d in (1) [3, 4].
For i = 1 , . . .
, d , let { Y i ( x i ); x i 2 [0 , 1] } be a zero-mean GP with covariance function (or kernel)
k i .
Taking Y 1 , . .
, Y d as independent GPs, the process { Y ( x ); x 2 [0 , 1] d } , with x = ( x 1 , . . .
, x d ) , that results from the addition of Y 1 , . .
, Y d , is also a GP and its kernel is given by
[Equation]
In regression tasks, we often train the GP Y to noisy data ( x  , y  ) {+
+}
   n .
We denote x (  ) i , for  = 1 , . . . , n and i = 1 , . . .
, d , the element corresponding to the i -th input of x  .
By considering additive xxb Y ( x 1 , x 2 ) xxb Y ( x 1 , x 2 ) xxb Y ( x 1 , x 2 )
Gaussian noises "
[Equation]
N  0 , ⌧ , with " 1 , · · · , " n assumed to be independent and independent of
Y , then the conditional process Y n ( x ) :
= Y ( x ) |{ Y ( x 1 ) + " 1 = y 1 , . . .
, Y ( x n ) + " n = y n } is GP-distributed with mean function and covariance function given by [3]
[Equation]
where y n =
[ y 1 , . . .
y n ] > , k ( x ) =
P di =1 k i ( x i ) with k i ( x i ) =
[ k i ( x i , x (1) i ) , . . .
, k i ( x i , x ( n ) i )] > , and K =
P di =1 K i with ( K i )  , ` = k i ( x (  ) i , x ( ` ) i ) for 1   , `  n .
The conditional mean µ and variance v ( · ) =
c ( · , · ) are used for predictions and prediction errors, respectively.
2D illustration.
Figure 1 shows the prediction of an additive GP modeling the function ( x 1 , x 2 ) 7! 4( x 1  0 .
5) 2 + 2 x 2 .
The GP is trained with a squared exponential (SE) kernel, k
( x , x 0 )
= P di
[-=1 σ 
-][Equation]
(  ( x i  x 0 i ) 2 / 2 ` 2 i ) and with ( x 1 , x 2 ) : (0 . 5 , 0) , (0 . 5 , 0 . 5) , (0 . 5 , 1) , (0 , 0 . 5) , (1 , 0 . 5)
The covariance parameters ✓ =
((  21 , ` 1 ) , (  22 , ` 2 )) , corresponding to the variance and length-scale parameters (respectively), and the noise variance ⌧ 2 are estimated via maximum likelihood [28].
Although the resulting GP does preserve the additive condition, from Figure 1 (left) we can observe that the quality of the prediction will depend on the availability of data.
In our example, we can observe that the GP model does not properly capture the convexity condition along x 1 .
Contributions on additive Gaussian processes under inequality constraints 3.1 Finite-dimensional Gaussian process for ﬁxed subdivisions
In order to satisfy the constraints everywhere (see the next subsection), we introduce the following ﬁnite-dimensional GP.
For i = 1 , . . .
, d we consider a one-dimensional subdivision S i (a ﬁnite subset of [0 , 1] composed of knots) with at least two knots at 0 and 1 .
Throughout Section 3, S i is ﬁxed, but its data-driven selection is studied in Section 4.
If the number of knots of S i is m i , then the total number of knots is given by m = m 1 + · · · + m d .
We let S =
( S 1 , . . .
, S d ) .
The ﬁnite-dimensional
GP, denoted by Y S ( x ) , is written, for x 2 [0 , 1] d ,
[Equation]
where ⇠ i,j = Y i ( t ( S i ) ( j ) ) with Y 1 , . . .
, Y d GPs as in Section 2, and with 0 =
t ( S i ) (1) < · · · < t ( S i ) ( m i ) =
the knots in S i .
We let t ( S i ) (0) =
1 and t ( S i ) ( m i +1)
= 2 by convention, and  i,j =  t ( S i )
( j  1) ,t ( S i ) ( j ) ,t ( S i ) ( j +1) :
R is the hat basis function centered at the knot t ( S i )
( j ) of S i .
That is, for u < v < w , we let  u,v,w ( t ) =>< >: 1 v  u ( t  u
[Equation]
for u  t  v,w  v ( w
[Equation]
) for v  t  w, for t 62 [ u, w ] .
1.00 x  ( x ) - -    0.
1.00 x
Y ( x ) - -    0.
1.00 x
Y ( x )
Observe from (3) that, since ⇠ i,j , for i = 1 , . . .
, d and j = 1 , . . .
, m i , are Gaussian distributed, then
Y i, S i is a GP with kernel given by
[Equation]
Moreover, Y S is a GP with kernel
[Equation]
Figure 2 shows random realizations from a standard GP and a ﬁnite-dimensional GP using a SE kernel with ✓ =
(  2 , ` ) =
We remark that we consider asymetric basis functions here, which is a novelty compared to [5, 6], that will allow MaxMod (Section 4) to promote non-equispaced designs.
The additive ﬁnite-dimensional kernel in (5) also seems not to appear in the existing literature.
3.2 Satisfying inequality constraints
We consider the componentwise constraints
Y i, S i 2 E i , i = 1 , . . .
, d , where E i is a one-dimensional function set.
In line with [5, 6], we assume that there are convex sets
C i ⇢ R m i such that
[Equation]
where ⇠ i =
[ ⇠ i, 1 , · · · , ⇠ i,m i ] > .
Furthermore, for i = 1 , . . .
, d , C i is itself composed by q i linear inequalities:
[Equation]
where the  ( i )  ,j ’s encode the linear operations and the l ( i )  ’s and u ( i )  ’s represent the lower and upper bounds.
We write the constraints in (7) as l i  ⇤ i c  u i .
Examples of such constraints are monotonicity and componentwise convexity.
For instance, for the case where the function y in (1) is non-decreasing with respect to each input, then E i is the set of non-decreasing functions on [0 , 1] and C i is given by
[Equation]
Hence in this case, Y S is monotonic on the entire [0 , 1] d if and only if (by deﬁnition)
each of its additive component Y i, S i is monotonic on [0 , 1] , which happens if and only if the m i  1 constraints (8) are satisﬁed.
We discuss the example of componentwise convexity in Appendix A.1.
3.3 Constrained GP predictions
We let ⌃ i
= k i ( S i , S i ) be the m i ⇥ m i covariance matrix of ⇠ i .
We consider x 1 , . . .
, x n 2 [0 , 1] d and write  i for the n ⇥ m i matrix with element ( a, b ) given by  i,b ( x a ) .
Then
[Equation]
[Equation] By considering noisy data ( x  , y  ) 1    n , we have the regression conditions Y n + " n = y n , where " n ⇠ N (0 , ⌧ 2 I n ) and is independent from ⇠ 1 , . . . , ⇠ d .
Then given the observations and the constraints, the maximum à posteriori (MAP) estimate, also called the mode function, is given by
[Equation]
[Equation]
The vector
[Equation]
] > with
[Equation]
b ⇠ i,m i ] > is the mode of the Gaussian distribution
N ( µ c , ⌃ c ) of the values at the knots conditionally to the observations and truncated from the constraints l 1  ⇤ 1 ⇠ 1  u 1 , . . .
, l d  ⇤ d ⇠ d  u d as in (7):
[Equation]
Above µ c = [ µ > c, 1 , . . . , µ > c,d ] > is the m ⇥ 1 vector with block i given by
[Equation]
and ( ⌃ c,i,j ) i,j is the m ⇥ m matrix with block ( i, j ) given by
[Equation]
The expressions for µ c and ⌃ c are obtained from the conditional formulas for Gaussian vectors [28].
Note that the matrix ⌃ c is not block-diagonal, that is one cannot compute the posterior mode along each dimension separately.
In Appendix A.2, based on the matrix inversion lemma [28, 29], efﬁcient implementations to speed-up the computation of µ c and ⌃ c are given when m ⌧ n .
Notice that the covariance parameters ✓ of the kernel in (5) are estimated via maximum likelihood once the (unconstrained) additive GP framework is established.
This is an intermediate step before solving the optimization problem in (11).
Hence, given µ c , ⌃ c , ⇤ 1 , . . .
, ⇤ d , l 1 , . . .
, l d , u 1 , . . . , u d , and ✓ , (11) is convex and can be solved via quadratic programming [5, 30].
To the best of our knowledge, the expression of the mode in (11) does not appear in the existing literature.
The cGP mode in (10) can be used as a point estimate of predictions.
Since trajectories of [ ⇠ > 1 , . . . , ⇠ > d ] > conditionally on the observations and constraints can be generated via HMC [11], they can be used for uncertainty quantiﬁcation.
Furthermore, the mean of the HMC samples can be used for prediction purposes.
Continuing the illustrative 2D example in Figure 1, we see the improvement brought by the mode function and the mean of conditional simulations, compared to the unconstrained additive GP model.
Additive MaxMod algorithm 4.1 Squared-norm criterion
Consider an additive cGP model that uses only a subset J ✓ { 1 , . . .
, d } of active variables, with cardinality |J | .
We write its (vector of) subdivisions as S =
( S i ; i 2 J ) .
Its mode function b
Y S is deﬁned similarly as in (10), from R |J| to R , by, for x = ( x i
[Equation]
i 2 J
[Equation]
[Equation] [Equation] Adding a new active variable i ? 62 J to J , and allocating it the base (minimal) subdivision S i ?
= { 0 , 1 } deﬁnes a new mode function b
Y S ,i ? : R |J| +1 ! R .
Adding a knot t 2 [0 , 1] to the subdivision
S i ? for i ?
2 J also deﬁnes a new mode function b
Y S ,i ? ,t : R |J| ! R .
Since a new variable or knot increases the computational cost of the model (optimization dimension for the mode computation and sampling dimension for generating conditional trajectories via HMC), it is key to quantify its contribution beneﬁt.
We measure this beneﬁt by the squared-norm modiﬁcation of the cGP mode
[Equation]
where in the ﬁrst case we see b
Y S as a function of |J | +1 variables that does not use variable i ? .
4.2 Analytic expressions of the squared-norm criterion
Adding a variable.
For a new variable i ? 62 J , the new mode function is
[Equation]
where ( e ⇠ i,j ) i,j and ( e ⇠ i ? ,j ) j follow from the optimization problem in dimension
P i 2 J m i + 2 corresponding to (10) and (11).
We let  i ? , 1 ( u ) = 1  u and  i ? , 2 ( u ) =
u for u 2 [0 , 1] .
Note that even though, for i 2 J and j = 1 , . . .
, m i , b
⇠ i,j and e ⇠ i,j correspond to the same basis function  i,j , they are not equal in general as the new mode is reestimated with two more knots, which can modify the coefﬁcients of all the knots.
Next, we provide an analytic expression of the integral in (13).
Proposition 1 (See proof in Appendix A.3) .
We have
I S ,i ?
[Equation]
where ⌘ i,j = b ⇠ i,j  e ⇠ i,j , ⌘ i ? = e ⇠ i ? , [Equation]
e ⇠ i
[Equation]
, 1 , ⇣ i ?
=
e
[Equation]
+
[Equation] E [Equation] j :=
R [Equation]
[Equation] j ( t ) dt and E ( S i ) j,j 0 :=
R 1 0  i,j ( t )  i,j 0 ( t ) dt with explicit expressions in Lemma 1 in Appendix A.3.
The matrices ( E ( S i ) j,j 0 ) 1  j,j 0  m i are 1-band and the computational cost is linear with respect to m =
P i 2 J m i .
Inserting a knot to a variable.
For a new t added to S i ?
with i ? 2 J , the new mode function is b Y S ,i ? , t
[Equation]
x ) = [Equation] i
[Equation] [Equation] ) , where e [Equation]
[Equation] e  [Equation] [Equation]
j for [Equation]
[Equation]
and e  i ? ,j is obtained from S i ? [ { t } as in Lemma 1.
As before, this follows from the optimization problem in dimension P i 2 J m i + 1 corresponding to (10) and (11).
Next, we provide an analytic expression of (14).
Proposition 2 (See proof in Appendix A.3) .
For i 2 J \{ i ? } , let e
S i =
S i .
Let e
S i ? = S i ? [ { t } .
Recall that the knots in S i ? are written
= t ( S i ? ) (1) < · · · < t ( S i ? ) ( m i ? )
= 1 . Let ⌫ 2 { 1 , . . .
, m i ?  1 } be such that t ( S i ? ) ( ⌫ ) <
t < t ( S i ? ) ( ⌫ +1) .
Then, with a linear computational cost with respect to e m =
{+P i 
 J e m+} i , we have
[Equation]
S i ) j,j 0  X i 2 J ✓ e m i X j = ¯ ⌘ i,j E ( e S i )
{+j ◆ + ✓ X i 
 J e m i X j = ¯ ⌘ i,j E ( e+}
S i ) j ◆ , where ¯ ⌘ i,j = ¯ ⇠ i,j  e ⇠ i,j , E ( e
S i ) j and E ( e
S i ) j,j 0 are as in Proposition 1, ¯ ⇠ i,j = b ⇠ i,j for i {+
+}
= i ? ,
¯ ⇠ i ? ,j = b ⇠ i ? ,j for j  ⌫ , ¯ ⇠ i ? ,j = b ⇠ i ? ,j  1 for j  ⌫ + 2 , and
[Equation]
4.3 The MaxMod algorithm
Algorithm 1 summarizes the routine of MaxMod.
When considering inserting a knot t to an active variable i , we promote space ﬁlling by adding a reward of the form  D ( t, S i ) , where S i is the current subdivision and where D ( t, S i ) is the smallest distance from t to an element of S i .
When adding a new variable, we add a ﬁxed reward  0 .
Both  and  0 are tuning parameters of the algorithm and allow to promote adding new variables over reﬁning existing ones with new knots, or conversely.
Step 3 in Algorithm 1 is performed by a grid search, and involves multiple optimizations for computing the new modes in (11), followed by applications of (14).
Step 4 yields a single
>
0 > d
[Equation] [Equation]
optimization for computing the new mode in (11), followed by an application of (13).
For each computation of a new mode, the covariance parameters of the kernels k i , for i 2 J , are estimated.
For faster implementations, the covariance parameters can be ﬁxed throughout each pass corresponding to a ﬁxed value of m (Steps 2 to 6) or can be re-estimated every T values of m , for some period
T 2 N .
Furthermore, steps 3 and 4 can be parallelized and computed in different clusters
Numerical experiments
Implementations of the additive cGP framework are based on the R package lineqGPR [31].
Experiments throughout this section are executed on an 11th Gen Intel(R) Core(TM) i5-1145G7 2.60GHz 1.50 GHz, 16 Gb RAM.
Both R codes and notebooks to reproduce some of the numerical results are available in the Github repository: https://github.com/anfelopera/lineqGPR .
As training data, we use random Latin hypercube designs (LHDs).
Note that using them is not necessary for the implementation of our methodology, but it is recommended when the user is able to design the experiments.
Indeed, [32] shows that LH sampling reduces estimation variances compared to simple random sampling, especially for additive functions.
Thus, using LHDs enables us to choose a minimal design size n = 2 d , corresponding to the number of parameters of the chosen additive GP kernel: an additive Matérn 5/2 kernel with one variance parameter  2 i and one characteristic-length parameter ` i per dimension [3].
We denote ✓ =
((  21 , ` 1 ) , · · · , (  2 d , ` d )) .
For other applications where n 2 [ d + 1 , 2 d [ , we may consider the kernel structure in [4] with only one global variance parameter, at the cost of more restricted cGP models.
Finally, in the examples that require a comparison with non-additive GP models, we use common GP settings: maximin LHDs and n = 10 d .
The larger value of n accounts for the fact that additivity is not a prior information of standard GPs.
We compare the quality of the GP predictions in terms of the Q 2 criterion on unobserved data.
It is deﬁned as Q 2 = 1  SMSE , where SMSE is the standardized mean squared error [28].
For noise-free data, Q 2 is equal to one if predictions are exactly equal to the test data and lower than one otherwise.
5.1 Additive Gaussian processes with monotonicity constraints 5.1.1 Monotonicity in 5D
We start by an example in small dimensions, in order to compare with non-additive constrained GPs which do not scale to high dimensions.
We thus consider the additive function given by
[Equation]
with x 2 [0 , 1] 5 .
Observe that y is non-decreasing with respect to all its input variables.
We evaluate y on a maximin LHD over [0 , 1] 5 at n = 50 locations using [33].
In this example, as explained in the introduction of the section, we have chosen a maximin LHD rather than a random LHD, and x 1 xy ( x 1 , x 2 , 0 , 0 , 0 ) x 1 xy ( x 1 , 0 , x 3 , 0 , 0 ) x 1 xy ( x 1 , 0 , 0 , x 4 , 0 ) x 1 xy ( x 1 , 0 , 0 , 0 , x 5 ) x 1 xb Y ( x 1 , x 2 , 0 , 0 , 0 ) x 1 xb Y ( x 1 , 0 , x 3 , 0 , 0 ) x 1 xb Y ( x 1 , 0 , 0 , x 4 , 0 ) x 1 xb Y ( x 1 , 0 , 0 , 0 , x 5 )
[Table]
[Figure]
[Table]
n = 10 d rather than n = 2 d , because we also consider non-additive GPs for which these settings are recommended.
We ﬁx 20 knots per dimension, leading to a total of m = 100 knots.
Figure 3 shows the additive cGP mean prediction under monotonicity constraints considering
HMC samples.
Our framework leads to improvements on both CPU time and quality of predictions compared to the non-additive cGP in [5].
Due to computational limitations, the non-additive cGP is performed with a reduced but tractable number of knots per dimension set to m 1 = m
= m
= 4 , m 3 = 2 and m 5 = 6 , for a total of m = 768 knots.
With this setup, and considering a factorial design with 11 test points per dimension, the additive cGP yields Q 2 = 99 .
8% , an absolute improvement of 1.3% compared to the non-additive cGP.
In term of the CPU times, the computation of the cGP mode and cGP mean using the additive framework are obtained in 6s and 0.8s (respectively), a signiﬁcant speed-up compared to the non-additive cGP that required 28.9s and 24.3 minutes.
5.1.2 Monotonicity in hundreds of dimensions
For testing the additive cGP in high dimensions, we consider the target function used in [21]:
[Equation]
(16) with x 2 [0 , 1] d .
This function exhibits decreasing growth rates as the index i increases.
For different values of d , we assess GPs with and without constraints.
We ﬁx ✓ =
(  2 i , ` i ) 1  i  d = (1 , 2) .
Although ✓ can be estimated our focus here is to assess both computational cost and quality of cGP predictors.
The cGP mean is obtained by averaging 10 3 HMC samples.
We set 5 knots per dimension.
Table 1 summarizes the CPU times and Q 2 values of the cGP predictors.
The Q 2 criterion is computed considering 10 5 evaluations of y based on a LHD ﬁxed for all the experiments.
Results are shown over 10 replicates with different random LHDs.
From Table 1, it can be observed that the cGPs lead
[Table]
to prediction improvements, with Q 2 (median) increments between 1.7-5.8%.
Although the cGP mean predictor provides Q 2 values above 88%, it becomes expensive when d  500 .
On the other hand, the cGP mode yields a trade-off between computational cost and quality of prediction.
5.2 MaxMod algorithm 5.2.1 Dimension reduction illustration
We test the capability of MaxMod to account for dimension reduction considering the function in (16).
In addition to ( x 1 , . . .
, x d ) , we include D  d virtual variables, indexed as ( x d +1 , . . .
, x D ) , which will compose the subset of inactive dimensions.
For any combination of D 2 { 10 , 20 } and d 2 { 2 , 3 , 5 } , we apply Algorithm 1 to ﬁnd the true d active dimensions.
For each value of D , we evaluate f at a maximin LHD with n = 10 D .
Similarly to Section 5.1.1 where we compare to non-additive GPs, we have used the common GP settings rather than the ones used here for additive
GPs (random LHD, n = 2 d ).
We compare the accuracy of two modes:
• b Y MaxMod : the mode resulting from the additive cGP and additive MaxMod.
• e Y MaxMod : the mode resulting from the non-additive cGP in [5] with equispaced onedimensional knots but where the number of knots per dimension is the same as for b Y MaxMod .
Table 2 shows that MaxMod correctly identiﬁes the d dimensions that are actually active where the most variable ones have been reﬁned with more knots.
There, MaxMod is considered to converge if the squared-norm criterion is smaller than ✏ = 5 ⇥ 10  4 .
In terms of the Q 2 criterion, b
Y MaxMod leads to more accurate results compared to e
Y MaxMod , with Q 2  99 .
7% in all the cases.
5.2.2 Real application: ﬂood study of the Vienne river
The database contains a ﬂood study conducted by the French multinational electric utility company
EDF in the Vienne river [34].
It is composed of N = 2 ⇥ 10 4 simulations of an output H representing the water level and 37 inputs depending on: a value of ﬂow upstream, data on the geometry of the bed, and Strickler friction coefﬁcients (see further details in Appendix A.5.1).
This database was generated as a random design, assuming that the inputs are independent and follow speciﬁc probability distributions.
In order to apply our framework, we use componentwise quantile transformations to make the inputs uniform.
Such transformations do not modify the additivity and monotonicity properties, but may increase non-linearities.
With expert knowledge, it is possible to identify that H is decreasing along the ﬁrst 24 input dimensions and increasing along dimension 37 .
The behavior along inputs 25 to 36 is not clear.
Although underlying functions modeling the ﬂood phenomena may be non additive, a previous sensitivity analysis in [34], done with the full database, has shown that the additive assumption is realistic here.
Moreover, it has shown that inputs 11, 35 and 37 explain most of the variance.
We aim at obtaining similar conclusions with a fraction of the simulation budget, using the information of additivity and monotonocity constraints with respect to dimensions 1 to 24 and 37.
In this experiment, MaxMod is applied for 10 replicates with random training sets and 10 iterations.
The settings described at the beginning of Section 5 are followed as much as possible.
As the database is ﬁxed, we cannot directly use a random LHD of size n = 2 d .
Thus, we pick the closest subset of size n (measured by the Euclidean norm).
The remaining data are used for testing the cGPs.
Figureshows the choice of MaxMod per iteration and for the 10 replicates (adding or reﬁning variables).
M a x
M o d
I t e r a t i o n reps
[Figure]
l og ( 1
Q 2 ) [Figure]
The ﬁrst three iterations of the algorithm show that MaxMod activates mainly dimensions 11, 35 and 37.
In the subsequent iterations, MaxMod either reﬁnes the these dimensions or activates dimensions 12 and 36.
In terms of the quality of predictions (Figure 4), MaxMod leads to Q 2 > 0 .
92 after the ﬁrst three iterations.
Similar results are obtained for n = 3 d and n = 4 d (see Appendix A.5.2).
MaxMod indeed activates dimensions 11, 35 and 37 in the ﬁrst three iterations and reﬁnes them mainly in subsequent iterations.
This allows to conclude that MaxMod correctly identiﬁes the most relevant input dimensions and that accurate predictions are obtained once those dimensions are activated.
Acknowledgments and Disclosure of Funding
This research was conducted within the frame of the ANR GAP Project (ANR-21-CE40-0007) and the consortium in Applied Mathematics CIROQUO, gathering partners in technological research and academia in the development of advanced methods for Computer Experiments (doi:10.5281/zenodo.6581217).
We thank Bertrand Iooss and EDF R&D/LNHE for providing the Mascaret test case and Sébastien Petit who has performed the computations on this model
[-= τ 
-]
[-, u m } and write its ordered knots as 
-]