Original: C HE PAN: C ONSTRAINED B LACK -B OX U NCERTAINTY M ODELLING WITH Q UANTILE R EGRESSION
Revised: C HE PAN: C ONSTRAINED B LACK -B OX U NCERTAINTY M ODELLING WITH Q UANTILE R EGRESSION

Original: Anonymous authors
Revised: authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Mostof the predictive systems currently in use do not report any useful information for auditing their associated uncertainty and evaluating the corresponding risk.
Revised: Most predictive systems currently in use do not report any useful information for auditing their associated uncertainty and evaluating the corresponding risk.

Original: Taking it for granted that their replacement may not be advisable in the short term, in this paper we propose a novel approach to modelling conﬁdence in such systems while preserving their predictions.
Revised: Taking it for granted that their replacement may not be advisable in the short term, in this paper we propose a novel approach to modelling conﬁdence in such systems while preserving their predictions.

Original: The method is based on the Chebyshev 
Revised: The method is based on the Chebyshev Polynomial

Original: Polynomial Approximation Network (ChePAN), a new way of modelling aleatoric uncertainty in a regression scenario.
Revised: Approximation Network (the ChePAN), a new way of modelling aleatoric uncertainty in a regression scenario.

Original: In the case addressed here, uncertainty is modelled by building conditional quantiles on top of the original pointwise forecasting system considered as a black box, i.e. without making assumptions about its internal structure.
Revised: In the case addressed here, uncertainty is modelled by building conditional quantiles on top of the original pointwise forecasting system considered as a black box, i.e. without making assumptions about its internal structure.

Original: Furthermore,  ChePAN allows users to consistently choose how to constrain any predicted quantile with respect to the original forecaster.
Revised: Furthermore, the ChePAN allows users to consistently choose how to constrain any predicted quantile with respect to the original forecaster.

Original: Experiments show that the proposed method scales to large size data sets and transfers the advantages of quantile regression to estimating black-box uncertainty.
Revised: Experiments show that the proposed method scales to large size data sets and transfers the advantages of quantile regression to estimating black-box uncertainty.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The present paper proposes a novel method for adding aleatoric uncertainty estimation to any pointwise predictive system currently in use.
Revised: The present paper proposes a novel method for adding aleatoric uncertainty estimation to any pointwise predictive system currently in use.

Original: Considering the system as a black box , i.e. avoiding any hypothesis about the internal structure of the system, the method offers a solution to the technical debt debate.
Revised: Considering the system as a black box , i.e. avoiding any hypothesis about the internal structure of the system, the method offers a solution to the technical debt debate.

Original: The concept of technical debt was introduced in 1992 to initiate a debate on the long-term costs incurred when moving quickly in software engineering (Sculley et al.
Revised: The concept of technical debt was introduced in 1992 to initiate a debate on the long-term costs incurred when moving quickly in software engineering (Sculley et al.

Original: ; Cunningham (1992)).
Revised: ; Cunningham (1992)).

Original: Speciﬁcally, most of the predictive systems currently in use have previously required much effort in terms of code development, documentation writing, unit test implementation, preparing dependencies or even their compliance with the appropriate regulations (e.g., medical (Ustun & Rudin (2016)) or ﬁnancial models (Rudin (2019)) may have to satisfy interpretability constraints).
Revised: Speciﬁcally, most of the predictive systems currently in use have previously required much effort in terms of code development, documentation writing, unit test implementation, preparing dependencies or even their compliance with the appropriate regulations (e.g., medical (Ustun & Rudin (2016)) or ﬁnancial models (Rudin (2019)) may have to satisfy interpretability constraints).

Original: However, once the system is being used with real-world problems, a new requirement can arise regarding the conﬁdence of its predictions when the cost of an erroneous prediction is high.
Revised: However, once the system is being used with real-world problems, a new requirement can arise regarding the conﬁdence of its predictions when the cost of an erroneous prediction is high.

Original: That being said, replacing the currently-in-use system may not be advisable in the short term.
Revised: That being said, replacing the currently-in-use system may not be advisable in the short term.

Original: To address this issue, x τ ... p t d −p t d −p tp t...
Revised: To address this issue, x τ ... p t d −p t d −p tp t...

Original: c d −c d −ccβ ...
Revised: c d −c d −ccβ ...

Original: C d − C d −
Revised: C d − C d −

Original: C C
Revised: C C

Original: P ( τ, x ; d )
Revised: P ( τ, x ; d )

Original: [Figure]
Revised: [Figure]

Original: the aim of this work is to report any information that is useful for auditing the system’s associated uncertainty without modifying its predictions.
Revised: the aim of this work is to report any information that is useful for auditing the system’s associated uncertainty without modifying its predictions.

Original: In general terms, sources of uncertainty can be understood by analysing the conditional members of this joint distribution: p p y, x q “ ş
Revised: In general terms, sources of uncertainty can be understood by analysing the conditional members of this joint distribution: p p y, x q “ ş

Original: M p p y | x , M q p p M | x q p p x q dM where M P M is the family (assumed non-ﬁnite) of models being considered.
Revised: M p p y | x , M q p p M | x q p p x q dM where M P M is the family (assumed non-ﬁnite) of models being considered.

Original: Not all methods developed to model uncertainty can be applied in the black-box scenario, since the main hypothesis is that the black box is a ﬁxed single model and unknown internally.
Revised: Not all methods developed to model uncertainty can be applied in the black-box scenario, since the main hypothesis is that the black box is a ﬁxed single model and unknown internally.

Original: Here, we refer speciﬁcally to those solutions that model epistemic uncertainty, which requires modelling p p M | x q .
Revised: Here, we refer speciﬁcally to those solutions that model epistemic uncertainty, which requires modelling p p M | x q .

Original: By epistemic, we mean that uncertainty which can derive from ignorance about the model, including, for example, ensemble models (Lakshminarayanan et al.
Revised: By epistemic, we mean that uncertainty which can derive from ignorance about the model, including, for example, ensemble models (Lakshminarayanan et al.

Original: Bayesian neural networks (Rasmussen (1996); Blundell et al.
Revised: Bayesian neural networks (Rasmussen (1996); Blundell et al.

Original: ; Hern ´ andez-Lobato & Adams (2015b); Teye et al.
Revised: ; Hern ´ andez-Lobato & Adams (2015b); Teye et al.

Original: or MC-Dropout (Gal & Ghahramani (2016)).
Revised: or MC-Dropout (Gal & Ghahramani (2016)).

Original: However, the black box could be a non-parametric predictive system or even a handcrafted rulebased system, as shown in Figure 1.
Revised: However, the black box could be a non-parametric predictive system or even a handcrafted rulebased system, as shown in Figure 1.

Original: Hence the reason for studying aleatoric uncertainty (Der Kiureghian & Ditlevsen (2009); Kendall & Gal (2017); Brando et al.
Revised: Hence the reason for studying aleatoric uncertainty (Der Kiureghian & Ditlevsen (2009); Kendall & Gal (2017); Brando et al.

Original: (2019)), which originates from the variability of possible correct answers given the same input data, p p y
Revised: (2019)), which originates from the variability of possible correct answers given the same input data, p p y

Original: | x q .
Revised: | x q .

Original: This type of uncertainty can be tackled by modelling the response variable distribution.
Revised: This type of uncertainty can be tackled by modelling the response variable distribution.

Original: For instance, imposing a conditional normal distribution where the location parameter is the black-box function and the corresponding scale parameter is learnt.
Revised: For instance, imposing a conditional normal distribution where the location parameter is the black-box function and the corresponding scale parameter is learnt.

Original: However, the more restricted the assumptions made about this distribution, the more difﬁcult it will be to model heterogeneous distributions.
Revised: However, the more restricted the assumptions made about this distribution, the more difﬁcult it will be to model heterogeneous distributions.

Original: One solution to this limitation is the type of regression analysis used in statistics and econometrics known as Quantile
Revised: One solution to this limitation is the type of regression analysis used in statistics and econometrics known as Quantile

Original: Regression (QR), which will provide a more comprehensive estimation.
Revised: Regression (QR), which will provide a more comprehensive estimation.

Original: Unlike classic regression methods, which only estimate a selected statistic such as the mean or the median, QR allows us to approximate any desired quantile.
Revised: Unlike classic regression methods, which only estimate a selected statistic such as the mean or the median, QR allows us to approximate any desired quantile.

Original: The main advantage of this method is that it allows conﬁdence intervals to be captured without having to make strong assumptions about the distribution function to be approximated.
Revised: The main advantage of this method is that it allows conﬁdence intervals to be captured without having to make strong assumptions about the distribution function to be approximated.

Original: Recently, several works (Dabney et al.
Revised: Recently, several works (Dabney et al.

Original: (2018a); Tagasovska & Lopez-Paz (2018);
Revised: (2018a); Tagasovska & Lopez-Paz (2018);

Original: Brando et al.
Revised: Brando et al.

Original: (2019)) have proposed a single deep learning model that implicitly learns all the quantiles at the same time, i.e. the model can be evaluated for any real value τ P r 0 , 1 s to give a pointwise estimation of any quantile value of the response variable.
Revised: (2019)) have proposed a single deep learning model that implicitly learns all the quantiles at the same time, i.e. the model can be evaluated for any real value τ P r 0 , 1 s to give a pointwise estimation of any quantile value of the response variable.

Original: Nevertheless, these QR solutions are not directly applicable to the uncertainty modelling of a black box because the predicted quantiles need to be linked to the black-box prediction in some way.
Revised: Nevertheless, these QR solutions are not directly applicable to the uncertainty modelling of a black box because the predicted quantiles need to be linked to the black-box prediction in some way.

Original: In the present paper, we propose a novel method for QR based on estimating the derivative of the ﬁnal function using a Chebyshev polynomial approximation to model the uncertainty of a blackbox system.
Revised: In the present paper, we propose a novel method for QR based on estimating the derivative of the ﬁnal function using a Chebyshev polynomial approximation to model the uncertainty of a blackbox system.

Original: Speciﬁcally, this method disentangles the estimation of a selected statistic β of the distribution p p y | x q from the estimation of the quantiles of p p
Revised: Speciﬁcally, this method disentangles the estimation of a selected statistic β of the distribution p p y | x q from the estimation of the quantiles of p p

Original: y
Revised: y

Original: | x q (shown in Figure 2).
Revised: | x q (shown in Figure 2).

Original: Hence, our method is not restricted to scenarios where we can jointly train both estimators, but can also be applied to pre-existing regression systems as a wrapper that produces the necessary information to evaluate aleatoric uncertainty.
Revised: Hence, our method is not restricted to scenarios where we can jointly train both estimators, but can also be applied to pre-existing regression systems as a wrapper that produces the necessary information to evaluate aleatoric uncertainty.

Original: Additionally, the proposed method scales to several real-world data sets.
Revised: Additionally, the proposed method scales to several real-world data sets.

Original: This paper is organised as follows.
Revised: This paper is organised as follows.

Original: Section 2 states the real-world motivation of the current research as well as the contribution it will be presented.
Revised: Section 2 states the real-world motivation of the current research as well as the contribution it will be presented.

Original: Section 3 introduces the problem of QR and reviews the classic approach to use with neural networks, showing how it cannot be applied directly to constrained black-box uncertainty modelling.
Revised: Section 3 introduces the problem of QR and reviews the classic approach to use with neural networks, showing how it cannot be applied directly to constrained black-box uncertainty modelling.

Original: Section 4 explores an approach for modelling the derivative of a function using neural networks.
Revised: Section 4 explores an approach for modelling the derivative of a function using neural networks.

Original: The two previous sections provide the baseline for developing our proposed model and its properties, which is presented in Section 5.
Revised: The two previous sections provide the baseline for developing our proposed model and its properties, which is presented in Section 5.

Original: And ﬁnally, in
Revised: And ﬁnally, in

Original: Section 6, we show how our model can be applied in large data sets and deﬁnes a new way of modelling the aleatoric uncertainty of a black box.
Revised: Section 6, we show how our model can be applied in large data sets and deﬁnes a new way of modelling the aleatoric uncertainty of a black box.

Original: The results are then summarised in the conclusion.
Revised: The results are then summarised in the conclusion.

Original:  R ESEARCH GOAL AND CONTRIBUTION
Revised:  R ESEARCH GOAL AND CONTRIBUTION

Original: The present article was motivated by a real-world need that appears in a pointwise regression forecasting system of a large company.
Revised: The present article was motivated by a real-world need that appears in a pointwise regression forecasting system of a large company.

Original: Due to the risk nature of the internal problem where it is applied, uncertainty modelling is important.
Revised: Due to the risk nature of the internal problem where it is applied, uncertainty modelling is important.

Original: However, similarly to the medical or ﬁnancial cases presented in the introduction, interpretability requirements were essential in deﬁning the model currently used by the company, which does not report conﬁdence any prediction made.
Revised: However, similarly to the medical or ﬁnancial cases presented in the introduction, interpretability requirements were essential in deﬁning the model currently used by the company, which does not report conﬁdence any prediction made.

Original: The need for this research arises in cases where the replacement of the aforementioned system is not advisable in the short term, despite the ongoing need for the uncertainty estimation of that system.
Revised: The need for this research arises in cases where the replacement of the aforementioned system is not advisable in the short term, despite the ongoing need for the uncertainty estimation of that system.

Original: Deﬁnition of
Revised: Deﬁnition of

Original: constrained
Revised: constrained

Original: black-
Revised: black-

Original: box uncertainty modelling
Revised: box uncertainty modelling

Original: From the probabilistic perspective, solving a regression problem involves determining a conditional density model, q p y | x q .
Revised: From the probabilistic perspective, solving a regression problem involves determining a conditional density model, q p y | x q .

Original: This model ﬁts an observed set of samples D “ p X, Y q “ (cid:32) p x i , y i q | x i P R D , y i P R ( n i “ 1 , which we assume to be sampled from an unknown distribution p p y | x q .
Revised: This model ﬁts an observed set of samples D “ p X, Y q “ (cid:32) p x i , y i q | x i P R D , y i P R ( n i “ 1 , which we assume to be sampled from an unknown distribution p p y | x q .

Original: i.e. the real data.
Revised: i.e. the real data.

Original: Given this context, the pointwise forecasting system mentioned above is a function, β : R D Ñ R , which tries to approximate a certain conditional summary statistic (a percentile or moment) of p p y | x q .
Revised: Given this context, the pointwise forecasting system mentioned above is a function, β : R D Ñ R , which tries to approximate a certain conditional summary statistic (a percentile or moment) of p p y | x q .

Original: Regarding the notation, we will call the “constraint” the known or assumed summary statistic that is approximated by β p x q (e.g. if β is reducing the mean square error, then it corresponds to the conditional mean.
Revised: Regarding the notation, we will call the “constraint” the known or assumed summary statistic that is approximated by β p x q (e.g. if β is reducing the mean square error, then it corresponds to the conditional mean.

Original: Otherwise, if it minimises the mean absolute error, it corresponds to the median).
Revised: Otherwise, if it minimises the mean absolute error, it corresponds to the median).

Original: Importantly, in the constrained black-box uncertainty modelling context, the mismatch between the real conditional statistic and the black box, β , becomes a new source of aleatoric uncertainty that is different from the one derived from the data.
Revised: Importantly, in the constrained black-box uncertainty modelling context, the mismatch between the real conditional statistic and the black box, β , becomes a new source of aleatoric uncertainty that is different from the one derived from the data.

Original: However, the way to model it continues to be by estimating p p y | x q .
Revised: However, the way to model it continues to be by estimating p p y | x q .

Original: Therefore, a poorly estimated β will impact the modelling of p p y | x q , given that we always force the constraint to be satisﬁed (as shown in Figure 3 of the Experiment section).
Revised: Therefore, a poorly estimated β will impact the modelling of p p y | x q , given that we always force the constraint to be satisﬁed (as shown in Figure 3 of the Experiment section).

Original: So far, we have attempted to highlight the fact that we do not have a strong hypothesis about the internals of this β function, we have only assumed that it approximates a certain statistic of p p y | x q .
Revised: So far, we have attempted to highlight the fact that we do not have a strong hypothesis about the internals of this β function, we have only assumed that it approximates a certain statistic of p p y | x q .

Original: Accordingly, we call this function the “constrained black box”.
Revised: Accordingly, we call this function the “constrained black box”.

Original: This ﬂexible assumption will enable us to consider several pointwise models as β , as shown in Figure 1.
Revised: This ﬂexible assumption will enable us to consider several pointwise models as β , as shown in Figure 1.

Original: The overall goal of the present article is, taking a pre-deﬁned black box β p x q that estimates a certain conditional summary statistic of p p y | x q , to model q p y | x q under the constraint that if we calculate the summary statistic of this predicted conditional distribution, it will correspond to β p x q .
Revised: The overall goal of the present article is, taking a pre-deﬁned black box β p x q that estimates a certain conditional summary statistic of p p y | x q , to model q p y | x q under the constraint that if we calculate the summary statistic of this predicted conditional distribution, it will correspond to β p x q .

Original: As mentioned in the Introduction, since we have a ﬁxed black box, we are unable to apply Bayesian techniques such as those that infer the distribution of parameters within the model, p p M | x q .
Revised: As mentioned in the Introduction, since we have a ﬁxed black box, we are unable to apply Bayesian techniques such as those that infer the distribution of parameters within the model, p p M | x q .

Original: In general, even though they are very common techniques in generic uncertainty modelling, no such epistemic uncertainty techniques can be applied in this context due to the limitation of only having a single ﬁxed model.
Revised: In general, even though they are very common techniques in generic uncertainty modelling, no such epistemic uncertainty techniques can be applied in this context due to the limitation of only having a single ﬁxed model.

Original: In addition, it should be noted that not all models that estimate p p y | x q can be used in the constrained black-box uncertainty modelling context.
Revised: In addition, it should be noted that not all models that estimate p p y | x q can be used in the constrained black-box uncertainty modelling context.

Original: To solve this problem, we require models that predict q p y | x q but also force the chosen conditional summary statistic of q p y | x q to have the same value as β p x q .
Revised: To solve this problem, we require models that predict q p y | x q but also force the chosen conditional summary statistic of q p y | x q to have the same value as β p x q .

Original: The main contribution of this work is to present a new approach that allows us not only to outperform other baseline models when tackling this problem, but also to decide which kind of constraint we wish to impose between β p x q and q p y | x q .
Revised: The main contribution of this work is to present a new approach that allows us not only to outperform other baseline models when tackling this problem, but also to decide which kind of constraint we wish to impose between β p x q and q p y | x q .

Original: The q p y | x q will be approximated using Quantile Regression (explained in Section 3) and the constraint will be created considering the integration constant of the q p y | x q derivative (shown in Section 5.1).
Revised: The q p y | x q will be approximated using Quantile Regression (explained in Section 3) and the constraint will be created considering the integration constant of the q p y | x q derivative (shown in Section 5.1).

Original:  C ONDITIONAL Q UANTILE R EGRESSION
Revised:  C ONDITIONAL Q UANTILE R EGRESSION

Original: In Quantile Regression (QR), we estimate q in a discrete manner by means of quantiles, which does not assume any typical parametric family distribution to the predicted p , i.e. it goes beyond central tendency or unimodality assumptions.
Revised: In Quantile Regression (QR), we estimate q in a discrete manner by means of quantiles, which does not assume any typical parametric family distribution to the predicted p , i.e. it goes beyond central tendency or unimodality assumptions.

Original: For each quantile value τ P r 0 , 1 s and each input value x P R D , the conditional quantile function will be f : r 0 , 1 s ˆ R D Ñ R .
Revised: For each quantile value τ P r 0 , 1 s and each input value x P R D , the conditional quantile function will be f : r 0 , 1 s ˆ R D Ñ R .

Original: In our case, we use deep learning as a generic function approximator (Hornik et al.
Revised: In our case, we use deep learning as a generic function approximator (Hornik et al.

Original: to build the model f , as we shall see later.
Revised: to build the model f , as we shall see later.

Original: Consequently, f is a parametric function that will be optimised by minimising the following loss function with respect to their weights w ,
Revised: Consequently, f is a parametric function that will be optimised by minimising the following loss function with respect to their weights w ,

Original: [Equation]
Revised: [Equation]

Original: where 1 r c s denotes the indicator function that veriﬁes the condition c .
Revised: where 1 r c s denotes the indicator function that veriﬁes the condition c .

Original: Equation 1 is an asymmetric convex loss function that penalises overestimation errors with weight τ and underestimation errors with weight 1 ´ τ .
Revised: Equation 1 is an asymmetric convex loss function that penalises overestimation errors with weight τ and underestimation errors with weight 1 ´ τ .

Original: Recently, different works (Dabney et al.
Revised: Recently, different works (Dabney et al.

Original: (2018b;a); Wen et al.
Revised: (2018b;a); Wen et al.

Original: (2017)) have proposed deep learning models that minimise a QR loss function similar to equation 1.
Revised: (2017)) have proposed deep learning models that minimise a QR loss function similar to Equation 1.

Original: For instance, in the ﬁeld of reinforcement learning, the Implicit Quantile Network (IQN) model was proposed (Dabney et al.
Revised: For instance, in the ﬁeld of reinforcement learning, the Implicit Quantile Network (IQN) model was proposed (Dabney et al.

Original: (2018a))
Revised: (2018a))

Original: and subsequently applied to solve regression problems as the Simultaneous Quantile Regression (SQR) model (Tagasovska & Lopez-Paz (2019)) or the IQN in (Brando et al.
Revised: and subsequently applied to solve regression problems as the Simultaneous Quantile Regression (SQR) model (Tagasovska & Lopez-Paz (2019)) or the IQN in (Brando et al.

Original: These models consist of a neural network ψ : r 0 , 1 s ˆ R D Ñ R such that it directly learns the function f that minimises equation 1, i.e. f “ ψ .
Revised: These models consist of a neural network ψ : r 0 , 1 s ˆ R D Ñ R such that it directly learns the function f that minimises Equation 1, i.e. f “ ψ .

Original: In order to optimise ψ for all possible τ values, these models pair up each input x with a sampled τ „ U p 0 , 1 q from a uniform distribution in each iteration of the stochastic gradient descent method.
Revised: In order to optimise ψ for all possible τ values, these models pair up each input x with a sampled τ „ U p 0 , 1 q from a uniform distribution in each iteration of the stochastic gradient descent method.

Original: Thus, the ﬁnal loss function is an expectation over τ of equation 1.
Revised: Thus, the ﬁnal loss function is an expectation over τ of Equation 1.

Original: However, these QR models cannot be applied to the constrained black-box scenario , given that they do not link their predicted quantiles with a pointwise forecasting system in a constrained way (Section 5.1).
Revised: However, these QR models cannot be applied to the constrained black-box scenario , given that they do not link their predicted quantiles with a pointwise forecasting system in a constrained way (Section 5.1).

Original: Other models, such as quantile forests, have a similar limitation.
Revised: Other models, such as quantile forests, have a similar limitation.

Original: In the next section, we introduce the other main part required to deﬁne our proposed method.
Revised: In the next section, we introduce the other main part required to deﬁne our proposed method.

Original:  M ODELLING THE D ERIVATIVE WITH A N EURAL N ETWORK
Revised:  M ODELLING THE D ERIVATIVE WITH A N EURAL N ETWORK

Original: Recently, a non-QR approach was proposed to build a monotonic function based on deep learning: the Unconstrained Monotonic Neural Network (UMNN) (Wehenkel & Louppe (2019)).
Revised: Recently, a non-QR approach was proposed to build a monotonic function based on deep learning: the Unconstrained Monotonic Neural Network (UMNN) (Wehenkel & Louppe (2019)).

Original: The UMNN estimates the derivative of a function by means of a neural network, φ , which has its output restricted to strictly positive values, i.e. approaching H p
Revised: The UMNN estimates the derivative of a function by means of a neural network, φ , which has its output restricted to strictly positive values, i.e. approaching H p

Original: z q such that
Revised: z q such that

Original: [Equation]
Revised: [Equation]

Original: Therefore, if the neural network φ p z q « B H B z p z q ą 0 , this is in fact a sufﬁcient condition to force
Revised: Therefore, if the neural network φ p z q « B H B z p z q ą 0 , this is in fact a sufﬁcient condition to force

Original: H p
Revised: H p

Original: z q to be monotone.
Revised: z q to be monotone.

Original: To compute the integral of B H B z , the UMNN approximates the integral of equation 2 using the
Revised: To compute the integral of B H B z , the UMNN approximates the integral of Equation 2 using the

Original: Clenshaw-Curtis quadrature, which has a closed expression.
Revised: Clenshaw-Curtis quadrature, which has a closed expression.

Original: The UMNN is designed to obtain a general monotonic function with respect to all the model inputs, z , but our interest is to build a partial monotonic function with respect to the quantile value, as we will explain hereafter.
Revised: The UMNN is designed to obtain a general monotonic function with respect to all the model inputs, z , but our interest is to build a partial monotonic function with respect to the quantile value, as we will explain hereafter.

Original: The partial monotonic function will be obtained using the Clenshaw-Curtis Network (CCN) model, which is an extension of the UMNN model introduced in Section A.3 of the Appendix.
Revised: The partial monotonic function will be obtained using the Clenshaw-Curtis Network (CCN) model, which is an extension of the UMNN model introduced in Section A.3 of the Appendix and an intermediate step we took to arrive at the main proposal of the current article.

Original: Although the CCN is not present in the literature, we have not included it in the main article because it cannot be applied to the constrained black-box uncertainty modelling scenario (as described in Section A.3).
Revised: Importantly, we have not included it in the main article because it cannot be applied to the constrained black-box uncertainty modelling scenario (as described in Section A.3).

Original: It does, however, constitute a natural step between the UMNN and the model we will present here, which is the main contribution of this article.

Original:  C HE PAN: THE C HEBYSHEV P OLYNOMIAL A PPROXIMATION N ETWORK 
Revised:  C HE PAN: THE C HEBYSHEV P OLYNOMIAL A PPROXIMATION N ETWORK In

Original: In this section, we will extend the UMNN to a model that is only monotonic with respect to the quantile input τ .
Revised: this section, we will extend the UMNN to a model that is only monotonic with respect to the quantile input τ .

Original: Moreover, we will exploit  that the quantiles domain is in r 0 , 1 s to provide an approach which is uniformly deﬁned in all  the interval.
Revised: Moreover, we will exploit the fact that the quantile domain is in r 0 , 1 s to provide an approach which is uniformly deﬁned over all of the interval.

Original: We call this approach the Chebyshev Polynomial Approximation Network (ChePAN), which allows us to transfers the advantages of quantile regression to the constrained uncertainty modelling of a black box.
Revised: We call this approach the Chebyshev Polynomial Approximation Network (ChePAN), which allows us to transfer the advantages of quantile regression to the constrained uncertainty modelling of a black box.

Original: As Figure 2 shows, the ChePAN contains a neural network φ : r 0 , 1 s ˆ R D Ñ R ` that only produces positive outputs and models the derivative of the ﬁnal function with respect to τ .
Revised: As Figure 2 shows, the ChePAN contains a neural network φ : r 0 , 1 s ˆ R D Ñ R ` that only produces positive outputs and models the derivative of the ﬁnal function with respect to τ .

Original: The goal is to optimise the neural networks φ p τ, x q by calculating the coefﬁcients of a truncated Chebyshev polynomial expansion p p τ, x ; d q of degree d with respect to τ .
Revised: The goal is to optimise the neural networks φ p τ, x q by calculating the coefﬁcients of a truncated Chebyshev polynomial expansion p p τ, x ; d q of degree d with respect to τ .

Original: That is, we will use the Chebyshev polynomials (described in Section A.1 of the Appendix) to give a representation of the neural network, φ , uniformly deﬁned in τ P r 0 , 1 s .
Revised: That is, we will use a Chebyshev polynomial (described in Section A.1 of the Appendix) to give a representation of the neural network, φ , uniformly deﬁned in τ P r 0 , 1 s .

Original: After that, we will use its properties to model the uncertainty of a black box in a constrained way (described in Section 5.1).
Revised: After that, we will use its properties to model the uncertainty of a black box in a constrained way (described in Section 5.1).

Original: Internally, the ChePAN considers a ﬁnite mesh of quantile values, called Chebyshev roots , t t k u d ´ 1 k “ 0 Ă r 0 , 1 s and deﬁned by
Revised: Internally, the ChePAN considers a ﬁnite mesh of quantile values, called Chebyshev roots , t t k u d ´ 1 k “ 0 Ă r 0 , 1 s and deﬁned by

Original: [Equation]
Revised: [Equation]

Original: The truncated Chebyshev expansion of a function can be interpreted as a linear transformation by a set of evaluations of φ at the roots, i.e. t φ p t k , x qu d ´ 1 k “ 0 .
Revised: The truncated Chebyshev expansion of a function can be interpreted as a linear transformation using a set of evaluations of φ at the roots, i.e. t φ p t k , x qu d ´ 1 k “ 0 .

Original: That linear transformations gives a vector of coefﬁcients, called Chebyshev coefﬁcients, and depending on x , i.e. t c k p x qu d ´ 1 k “ 0 , as illustrated by Figure 2.
Revised: This linear transformation gives a vector of coefﬁcients, which are known as Chebyshev coefﬁcients and depend on x , i.e. t c k p x qu d ´ 1 k “ 0 , as illustrated in Figure 2.

Original: The implementation of a linear transformation has, in general, a square complexity.
Revised: The implementation of a linear transformation generally has a square complexity.

Original: However, the transformation involved inthe Chebyshev coefﬁcients can be computed efﬁciently with a Θ p d log d q complexity.
Revised: However, the transformation involved in Chebyshev coefﬁcients can be computed efﬁciently with a Θ p d log d q complexity.

Original: In fact, the algorithm that speeds the computation is based on the Fast Fourier Transform (FFT) and known as the Discrete Cosine Transform of type-II (DCT-II) (discussed in Section A.1 of the Appendix).
Revised: In fact, the algorithm that speeds the computation is based on the Fast Fourier Transform (FFT) and known as the Discrete Cosine Transform of type-II (DCT-II) (discussed in Section A.1 of the Appendix).

Original: Once the Chebyshev coefﬁcients c k p x q have been computed, we can write them in a linear combination of Chebyshev polynomials T k p t q , i.e.,
Revised: Once the Chebyshev coefﬁcients c k p x q have been computed, we can write them in a linear combination of Chebyshev polynomials T k p t q , i.e.

Original: [Equation]
Revised: [Equation]

Original: where T k p t q are deﬁned recurrently as T 0 p t q “ 1 , T 1 p t q “ t , and T k ` 1 p t q “ 2 tT k p t q ´ T k ´ 1 p t q for k ě 1 .
Revised: where T k p t q are deﬁned recurrently as T 0 p t q “ 1 , T 1 p t q “ t , and T k ` 1 p t q “ 2 tT k p t q ´ T k ´ 1 p t q for k ě 1 .

Original: These polynomials T k do not need to be explicitly computed to evaluate p on a quantile (Clenshaw (1955)).
Revised: These polynomials T k do not need to be explicitly computed to evaluate p on a quantile (Clenshaw (1955)).

Original: Note that, given the construction of the coefﬁcients c k p x q , the p p t k , x ; d q is equal to φ p t k , x q at each of the root points t k .
Revised: Note that, given the construction of the coefﬁcients c k p x q , the p p t k , x ; d q is equal to φ p t k , x q at each of the root points t k .

Original: These equalities must be understood in terms of machine precision in the numerical representation system, classically „ 10 ´ 16 in double-precision or „ 10 ´ 8 in singleprecision arithmetic.
Revised: These equalities must be understood in terms of machine precision in the numerical representation system, classically „ 10 ´ 16 in double-precision or „ 10 ´ 8 in singleprecision arithmetic.

Original: In Figure 2, we denote this root evaluation step as p t k .
Revised: In Figure 2, we denote this root evaluation step as p t k .

Original: The ﬁnal goal is to provide P p τ, x ; d q so that it approximates the integral of p , that is ş [Equation] p p t, x ; d q dt .
Revised: The ﬁnal goal is to provide P p τ, x ; d q so that it approximates the integral of p , that is ş τ 0 p p t, x ; d q dt .

Original: In particular, the integral will also be the integral of the neural network φ ,
Revised: Speciﬁcally, the integral will also be the integral of the neural network φ ,

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Since φ p τ, x q is deﬁned to be positive for all τ P r 0 , 1 s , then P p τ, x ; d q will be an increasing function with respect to τ .
Revised: Since φ p τ, x q is deﬁned as positive for all τ P r 0 , 1 s , then P p τ, x ; d q will be an increasing function with respect to τ .

Original: Additionally, given that p p τ, x ; d q is a Chebyshev polynomial (deﬁned in equation 4), its integral w.r.t.
Revised: Additionally, given that p p τ, x ; d q is a Chebyshev polynomial (deﬁned in Equation 4), its integral w.r.t.

Original: τ is just the integral of the Chebyshev polynomial T k that corresponds to a new Chebyshev polynomial.
Revised: τ is simply the integral of the Chebyshev polynomial T k , which corresponds to a new Chebyshev polynomial.

Original: Using the recurrent deﬁnition of T k , we deduce the indeﬁnite integrals ż
Revised: Using the recurrent deﬁnition of T k , we deduce the indeﬁnite integrals ż

Original: [Equation]
Revised: [Equation]

Original: which leads to the conclusion that P can be given in terms of Chebyshev coefﬁcients as well.
Revised: which leads to the conclusion that P can be given in terms of Chebyshev coefﬁcients as well.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: where the coefﬁcients C k p x q have a recurrence expression in terms of a Toeplitz matrix (see Clenshaw (1955)).
Revised: where the coefﬁcients C k p x q have a recurrent expression in terms of a Toeplitz matrix (see Clenshaw (1955)).

Original: Indeed, by ordering the coefﬁcients of the integral in equation 4, we deduce that
Revised: Indeed, by ordering the coefﬁcients of the integral in Equation 4, we deduce that

Original: [Equation]
Revised: [Equation]

Original: and C 0 p x q depends on the constant of integration K p x q in equation 5 and the other coefﬁcient values in equation 7.
Revised: and C 0 p x q depends on the constant of integration K p x q in Equation 5 and the other coefﬁcient values in Equation 7.

Original: This freedom of the predicted τ in C 0 p x q allows us to impose a new condition, which becomes a uniform condition in all of the intervals r 0 , 1 s .
Revised: This freedom of the predicted τ in C 0 p x q allows us to impose a new condition, which becomes a uniform condition in all of the intervals r 0 , 1 s .

Original: In Section 5.1, we will discuss how to deﬁne the C 0 p x q depending on the black box desired.
Revised: In Section 5.1, we will discuss how to deﬁne the C 0 p x q depending on the black box desired.

Original: 5. A DDING  U NCERTAINTY E STIMATION TO A B LACK -B OX P REDICTION S YSTEM
Revised: 5. A DDING AN U NCERTAINTY E STIMATION TO A B LACK -B OX P REDICTION S YSTEM

Original: In this subsection, we tackle the constrained black-box uncertainty modelling problem introduced in Section 2.
Revised: In this subsection, we tackle the constrained black-box uncertainty modelling problem introduced in Section 2.

Original: The main assumption is that we have a pointwise predictive system, which we will refer to as β p x q and approximates a desired statistic such as the mean, median or a certain quantile of p p y
Revised: The main assumption is that we have a pointwise predictive system, which we will refer to as β p x q and approximates a desired statistic such as the mean, median or a certain quantile of p p y

Original: | x q , as shown in Figure 1.
Revised: | x q , as shown in Figure 1.

Original: It is not necessary for this system to be a deep learning model or even parametric.
Revised: It is not necessary for this system to be a deep learning model or even parametric.

Original: All that the ChePAN requires to train its neural network, φ , are the corresponding β -evaluation values of the training set, i.e. t x , β p x qu .
Revised: All that the ChePAN requires to train its neural network, φ , are the corresponding β -evaluation values of the training set, i.e. t x , β p x qu .

Original: Thus, the ChePAN calculates the conditioned response distribution to the input without assuming asymmetry or unimodality with respect to this distribution, as well as associating the desired statistic of this distribution to β p x q .
Revised: Thus, the ChePAN calculates the conditioned response distribution to the input without assuming asymmetry or unimodality with respect to this distribution, as well as associating the desired statistic of this distribution to β p x q .

Original: The formula used to calculate the constant of integration, C 0 p x q , will depend on which statistic we choose 1 .
Revised: The formula used to calculate the constant of integration, C 0 p x q , will depend on which statistic we choose 1 .

Original: If we impose the quantile τ “ 0 to be β (which we shall call ChePAN- β = q 0 ), then
Revised: If we impose the quantile τ “ 0 to be β (which we shall call ChePAN- β = q 0 ), then

Original: [Equation]
Revised: [Equation]

Original: However, if we force the quantile τ “ 1 to be the β (which we shall call ChePAN- β = q 1 ), then
Revised: However, if we force the quantile τ “ 1 to be the β (which we shall call ChePAN- β = q 1 ), then

Original: [Equation]
Revised: [Equation]

Original: For instance, the prediction of extreme weather events involves the forecasting system to predict the maximum or minimum values of p p y | x q .
Revised: For instance, the prediction of extreme weather events involves the forecasting system to predict the maximum or minimum values of p p y | x q .

Original: In these cases, this pre-trained system could be used as β in equation 9 or equation 10, respectively, to determine the overall quantile distribution of p p y | x q , taking β as a reference point.
Revised: In these cases, this pre-trained system could be used as β in Equation 9 or Equation 10, respectively, to determine the overall quantile distribution of p p y | x q , taking β as a reference point.

Original: If the median (equivalently, τ “ 0 . 5 ) is the β (which we shall call ChePAN- β =Med ), then
Revised: If the median (equivalently, τ “ 0 . 5 ) is the β (which we shall call ChePAN- β =Med ), then

Original: [Equation]
Revised: [Equation]

Original: Finally, the mean is forced to be the β (which we shall call ChePAN- β =Mean ), then
Revised: Finally, the mean is forced to be the β (which we shall call ChePAN- β =Mean ), then

Original: [Equation]
Revised: [Equation]

Original: Additionally, β p x q can be approximated by  another neural network when can be simultaneously optimised with φ p τ, x q .
Revised: Additionally, β p x q can be approximated by means of another neural network, which can be simultaneously optimised with φ p τ, x q .

Original: We will use this approach to compare  ChePAN and other baseline models in the results section regarding black-box modelling.
Revised: We will use this approach to compare the ChePAN and other baseline models in the results section regarding black-box modelling.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: The source code used to reproduce the results of the ChePAN in the following experiments can be found in the Github repository 2 .
Revised: The source code used to reproduce the results of the ChePAN in the following experiments can be found in the Github repository 2 .

Original: The DCT-II method referred to in Section 5 was used in the aforementioned source code.
Revised: The DCT-II method referred to in Section 5 was used in the aforementioned source code.

Original: [Table]
Revised: [Table]

Original: In this section, we describe the performance of the proposed models compared to other baselines.
Revised: In this section, we describe the performance of the proposed models compared to other baselines.

Original: The main goal is to show that, as with QR, the ChePAN, see Section 5.1, is an improvement on other black-box uncertainty modelling baselines because it avoids centrality or unimodality assumptions, while also allowing users to choose how to constrain the predicted quantiles with respect to the black-box prediction.
Revised: The main goal is to show that by using QR the ChePAN is an improvement on other black-box uncertainty modelling baselines because it avoids centrality or unimodality assumptions, while also allowing users to choose how to constrain the predicted quantiles with respect to the black-box prediction.

Original: 6. M ODELS U NDER E VALUATION
Revised: 6. M ODELS U NDER E VALUATION

Original: Exponential power distributions satisfy the condition that one of the parameters corresponds to the mode.
Revised: Exponential power distributions satisfy the condition that one of the parameters corresponds to the mode.

Original: Thus, those models that approximate such parametric distributions where the mode parameter is the black-box function and estimate the other parameter related to uncertainty can be used as baselines.
Revised: Thus, those models that approximate such parametric distributions where the mode parameter is the black-box function and estimate the other parameter related to uncertainty can be used as baselines.

Original: • The Heteroscedastic Normal distribution (N) Similarly to (Bishop (1994); Kendall & Gal (2017); Tagasovska & Lopez-Paz (2019); Brando et al.
Revised: • The Heteroscedastic Normal distribution (N) Similarly to (Bishop (1994); Kendall & Gal (2017); Tagasovska & Lopez-Paz (2019); Brando et al.

Original: , two neural networks, µ and σ , can be used to approximate the conditional normal distribution, N p µ p x q , σ p x qq , such that they maximise the likelihood.
Revised: , two neural networks, µ and σ , can be used to approximate the conditional normal distribution, N p µ p x q , σ p x qq , such that they maximise the likelihood.

Original: In the black-box scenario proposed here, µ is the black-box function and we only need to optimise the σ neural network.
Revised: In the black-box scenario proposed here, µ is the black-box function and we only need to optimise the σ neural network.

Original: Once optimised, the desired quantile τ can be obtained with 
Revised: Once optimised, the desired quantile τ can be obtained with F p τ, [Equation] [Equation]

Revised: p

Original: F p τ, x q “ µ p x q ` σ p x q? 2 ¨ erf ´ 1 p 2 τ ´ 1 q , τ P p 0 , 1 q , where erf ´ 1 is the inverse error function.
Revised: [Equation] τ [Equation] q , τ P p 0 , 1 q , where erf ´ 1 is the inverse error function.

Original: • The Heteroscedastic Laplace distribution (LP)
Revised: • The Heteroscedastic Laplace distribution (LP)

Original: As a more robust alternative to outlier values, a conditional Laplace distribution, LP
Revised: As a more robust alternative to outlier values, a conditional Laplace distribution, LP

Original: [Equation]
Revised: [Equation]

Original: , can be considered.
Revised: , can be considered.

Original: Here, the quantile function is F p τ, x 
Revised: Here, the quantile function is F p τ, x q “ µ p x

Original: [Equation]
Revised: [Equation]

Original: τ ą 12 s , τ P p 0 , 1 q . • The Chebyshev Polynomial Approximation Network (ChePAN)
Revised: p 0 , 1 q . • The Chebyshev Polynomial Approximation Network (ChePAN)

Original: In order to use the same black boxes as the other baselines, equation 12 is considered, given that these black boxes are optimising the mean square error.
Revised: In order to use the same black boxes as the other baselines, Equation 12 is considered, given that these black boxes are optimising the mean square error.

Original: Other alternative equations are considered in the pseudo code and in Figure 6 of the supplementary material.
Revised: Other alternative equations are considered in the pseudo code and in Figure 6 of the supplementary material.

Original: 6. D ATA S ETS AND E XPERIMENT S ETTINGS
Revised: 6. D ATA S ETS AND E XPERIMENT S ETTINGS

Original: All experiments were implemented in TensorFlow (Abadi et al.
Revised: All experiments were implemented in TensorFlow (Abadi et al.

Original: and Keras (Chollet et al.
Revised: and Keras (Chollet et al.

Original: (2019)), running in a workstation with Titan X (Pascal) GPU and GeForce RTX 2080 GPU.
Revised: (2019)), running in a workstation with Titan X (Pascal) GPU and GeForce RTX 2080 GPU.

Original: All the details of the data sets used and model hyper-parameters for the results section are described in the supplementary material.
Revised: All the details of the data sets used and model hyper-parameters for the results section are described in the supplementary material.

Original: 6. R ESULTS
Revised: 6. R ESULTS

Original: Table 1 shows a comparison of uncertainty modelled for two given black-box systems (a Random
Revised: Table 1 shows a comparison of uncertainty modelled for two given black-box systems (a Random

Original: Forest (RF) (Liaw et al.
Revised: Forest (RF) (Liaw et al.

Original: and a XGBoost (Chen & Guestrin (2016))) in four data sets.
Revised: and an XGBoost (Chen & Guestrin (2016))) in four data sets.

Original: The ﬁrst four columns correspond to each part of the synthetic distribution proposed by (Brando et al.
Revised: The ﬁrst four columns correspond to each part of the synthetic distribution proposed by (Brando et al.

Original: and shown in Figure 3, the ﬁfth column is the full Year Prediction MSD UCI dataset (Dua & Graff (2017a)), predicting the release year of a song from 90 audio features and, ﬁnally, the last two columns correspond to predicting the room price forecasting of Airbnb ﬂats (RPF) in Barcelona and Vancouver, extracted from (Brando et al.
Revised: and shown in Figure 3, the ﬁfth column is the full Year Prediction MSD UCI dataset (Dua & Graff (2017a)), predicting the release year of a song from 90 audio features and, ﬁnally, the last two columns correspond to predicting the room price forecasting of Airbnb ﬂats (RPF) in Barcelona and Vancouver, extracted from (Brando et al.

Original: The mean of the QR lossequation 1 value  is evaluated for ten thousand randomly selected quantiles for ten executions of each model t m k u 10 k “ 1 ,
Revised: The mean of the QR loss value (see Equation 1) is evaluated for ten thousand randomly selected quantiles for ten executions of each model t m k u 10 k “ 1 ,

Original: [Equation]
Revised: [Equation]

Original: where N test is the number of points in the test set, N τ “ 10 , 000 the number of Monte Carlo samplings and f m k any of the models considered in Table 1.
Revised: where N test is the number of points in the test set, N τ “ 10 , 000 the number of Monte Carlo samplings and f m k any of the models considered in Table 1.

Original: Considering how the QR loss is deﬁned in equation 1, its value not only informs us about each system’s performance but also how generically calibrated its predicted quantiles are.
Revised: Considering how the QR loss is deﬁned in Equation 1, its value not only informs us about each system’s performance but also how generically calibrated its predicted quantiles are.

Original: Furthermore, in Table 1 we observe that the ChePAN outperforms other methods in most cases due to it transferring the capacity to capture asymmetries and multimodalities of QR in p p
Revised: Furthermore, in Table 1 we observe that the ChePAN outperforms other methods in most cases due to it transferring the capacity to capture asymmetries and multimodalities of QR in p p

Original: y
Revised: y

Original: | x q to the black-box problem, where our uncertainty modelling needs to be restricted in order to maintain the corresponding statistic associated with the black box.
Revised: | x q to the black-box problem, where our uncertainty modelling needs to be restricted in order to maintain the corresponding statistic associated with the black box.

Original: This restriction of conserving the black box can be seen qualitatively in the upper part of Figure 3, where  the black box used is an inaccurate Random Forest predicting the mean.
Revised: This restriction of conserving the black box can be seen qualitatively in the upper part of Figure 3, where such a restriction must be met in any situation, i.e. even if performance worsens because the black box, β p x q , is not correctly ﬁtted (as described in Section 2). In this case, β p x q is an inaccurate Random Forest predicting the mean.

Original: As we can see, the quantiles predicted by the ChePAN respect the noise of the black box while capturing the heterogeneous patterns of the distribution pp y | x q .
Revised: Importantly, the ChePAN propagates the β p x q noise to the predicted quantiles (in blue) because the constraint is always forced.

Original: On the other hand, the ability  to model heterogeneous distributions using QR is better displayed in the lower part ofthe Figure 3.
Revised: On the other hand, the ability of ChePAN to model heterogeneous distributions using QR is better displayed in the lower part of Figure 3.

Original: In this case, the black box is a neural network that is learnt concurrently with the quantiles.
Revised: In this case, the black box is a neural network that is learnt concurrently with the quantiles.

Original: Since the black box is better approximated, the quantiles are better.
Revised: Since the black box is better approximated, the quantiles are better.

Original: Finally, since Table 1 shows that there is a similar performance order between the baselines when using the RF or XGBoost, we also want to show additional experiments that directly measure the calibration of the predicted quantiles and compare the predicted width of certain desired intervals.
Revised: Finally, since Table 1 shows that there is a similar performance order between the baselines when using the RF or XGBoost, we also want to show additional experiments that directly measure the calibration of the predicted quantiles and compare the predicted width of certain desired intervals.

Original: Following the UCI data sets used in (Hern ´ andez-Lobato & Adams (2015b); Gal & Ghahramani (2016); Lakshminarayanan et al.
Revised: Following the UCI data sets used in (Hern´andez-Lobato & Adams (2015b); Gal & Ghahramani (2016); Lakshminarayanan et al.

Original: (2017); Tagasovska & Lopez-Paz (2019))
Revised: (2017); Tagasovska & Lopez-Paz (2019))

Original: , we performed two empirical studies to assess this point in a black-box scenario where the black box is an MSE-XGBoost.
Revised: , we performed two empirical studies to assess this point in a black-box scenario where the black box is an MSE-XGBoost.

Original: Following the proposed hidden layers architecture in (Tagasovska & Lopez-Paz (2019)), the Prediction Interval Coverage Probability (PICP) and the Mean Prediction Interval Width (MPIW) are reported in Table 3 of the appendix considering the 0 .
Revised: Following the proposed hidden layers architecture in (Tagasovska & Lopez-Paz (2019)), the Prediction Interval Coverage Probability (PICP) and the Mean Prediction Interval Width (MPIW) are reported in Table 3 of the appendix considering the 0 .

Original: 025 and the 0 .
Revised: 025 and the 0 .

Original: 975 quantiles.
Revised: 975 quantiles.

Original: For the sake of completeness, in Figure 4 and its associated table we have also computed an additional metric XGBoost -N XGBoost -LP
Revised: For the sake of [Table]

Original: XGBoost -ChePAN
Revised: ChePAN

Original: Concrete
Revised: Concrete

Original: [Table]
Revised: [Table]

Original: Power
Revised: Power

Original: [Table]
Revised: [Table]

Original: Wine
Revised: Wine

Original: [Table]
Revised: [Table]

Original: Yacht
Revised: Yacht

Original: [Table]
Revised: [Table]

Original: Naval

Original: [Table]

Original: Energy

Original: [Table]

Original: Boston

Original: [Table] Kin8nm [Table] not only to verify the calibration of the 0 .
Revised: completeness, in Figure 4 and its associated table we have also computed an additional metric not only to verify the calibration of the 0 .

Original: 025 and 0 .
Revised: 025 and 0 .

Original: 975 quantiles, but also to obtain a measure of general calibration considering the entire quantile distribution.
Revised: 975 quantiles, but also to obtain a measure of general calibration considering the entire quantile distribution.

Original: Given  τ “ r 10 ´ 2 , . . . , 1 ´ 10 ´ 2 s N τ -equidistant set of quantiles to evaluate, the % of actual test data that falls into each predicted quantile can be compared to each real quantile value as follows,
Revised: Given N τ -equidistant set of quantiles to evaluate, τ “ r 10 ´ 2 , . . . , 1 ´ 10 ´ 2 s , the % of actual test data that falls into each predicted quantile can be compared to each real quantile value as follows,

Original: [Equation]
Revised: [Equation]

Original: In addition, two extra ﬁgures showing the disentangled visualisation of this calibration metric from each quantile can be found in Figure 5 of the Appendix.
Revised: In addition, two extra ﬁgures showing the disentangled visualisation of this calibration metric from each quantile can be found in Figure 5 of the Appendix.

Original: As all of the ﬁgures and tables show, in terms of calibration, the ChePAN generally displays a better performance in the black-box scenario than the other models.
Revised: As all of the ﬁgures and tables show, in terms of calibration, the ChePAN generally displays a better performance in the black-box scenario than the other models.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: The uncertainty modelling of a black-box predictive system requires the designing of wrapper solutions that avoid assumptions about the internal structure of the system.
Revised: The uncertainty modelling of a black-box predictive system requires the designing of wrapper solutions that avoid assumptions about the internal structure of the system.

Original: Speciﬁcally, this could be a non-deep learning model (such as the one presented in Table 1 and Figure 3) or even a nonparametric predictive system, as proposed in Figure 1.
Revised: Speciﬁcally, this could be a non-deep learning model (such as the one presented in Table 1 and Figure 3) or even a nonparametric predictive system, as proposed in Figure 1.

Original: Therefore, not all models or types of uncertainties can be considered using this framework.
Revised: Therefore, not all models or types of uncertainties can be considered using this framework.

Original: The present paper introduces the Chebyshev Polynomial Approximation Network (ChePAN) model, which is based on Chebyshev polynomials and deep learning models and has a dual purpose: ﬁrstly, it predicts the aleatoric uncertainty of any pointwise predictive system; and secondly, it respects the statistic predicted by the pointwise system.
Revised: The present paper introduces the Chebyshev Polynomial Approximation Network (ChePAN) model, which is based on Chebyshev polynomials and deep learning models and has a dual purpose: ﬁrstly, it predicts the aleatoric uncertainty of any pointwise predictive system; and secondly, it respects the statistic predicted by the pointwise system.

Original: To conclude, then, the ChePAN transfers the advantages of Quantile Regression (QR) to the problem of modelling aleatoric uncertainty estimation in another existing and ﬁxed pointwise predictive system (denoted as β and referred to as a black box).
Revised: To conclude, then, the ChePAN transfers the advantages of Quantile Regression (QR) to the problem of modelling aleatoric uncertainty estimation in another existing and ﬁxed pointwise predictive system (denoted as β and referred to as a black box).

Original: Experiments using different large-scale real data sets and a synthetic one that contains several heterogeneous distributions conﬁrm these novel features.
Revised: Experiments using different large-scale real data sets and a synthetic one that contains several heterogeneous distributions conﬁrm these novel features.