Evaluating Robustness to Dataset Shift via Parametric Robustness Sets
Authors: Nikolaj Thams
Michael Oberst
David Sontag
Venue: NeurIPS 2022 Accept (2022)
Keywords: Causality
Robustness
Distributional Robustness
Distribution Shift
Dataset Shift

Abstract: We give a method for proactively identifying small, plausible shifts in distribution which lead to large differences in model performance.  These shifts are defined via parametric changes in the causal mechanisms of observed variables, where constraints on parameters yield a "robustness set" of plausible distributions and a corresponding worst-case loss over the set. While the loss under an individual parametric shift can be estimated via reweighting techniques such as importance sampling, the resulting worst-case optimization problem is non-convex, and the estimate may suffer from large variance. For small shifts, however, we can construct a local second-order approximation to the loss under shift and cast the problem of finding a worst-case shift as a particular non-convex quadratic optimization problem, for which efficient algorithms are available.  We demonstrate that this second-order approximation can be estimated directly for shifts in conditional exponential family models, and we bound the approximation error. We apply our approach to a computer vision task (classifying gender from images), revealing sensitivity to shifts in non-causal attributes.

Evaluating Robustness to Dataset Shift via Parametric Robustness Sets
Nikolaj Thams* Dept. of Mathematical Sciences
Michael Oberst" CSAIL & IMES MIT Cambridge, MA
David Sontag CSAIL & IMES MIT
University of Copenhagen Copenhagen, Denmark thams@math. ku.
Cambridge, MA dsontag@csail mit. edu
moberst@mit.cm edu
Abstract
We give a method for proactively identifying small, plausible shifts in distribution which lead large differences in model performance. These shifts are defined via parametric changes in the causal mechanisms of observed variables, where constraints on parameters yield "robustness set" of plausible distributions and a corresponding worst-case loss over the set. While the loss under an individual parametric shift can be estimated via reweighting techniques such as importance sampling, the resulting worst-case optimization problem is non-convex, and the estimate may suffer from large variance. For small shifts, however, we can construct a local second-ore order approximation to the loss under shift and cast the problem finding a worst-case shift as a particular non-convex quadratic optimization problem, for which efficient algorithms are available. We demonstrate that this second-order approximation can be estimated directly for shifts conditional exponential family models, and we bound the approximation error. We apply our approach to a computer vision task (classifying gender from images), revealing sensitivity to shifts in non-causal attributes.
Introduction
Predictive models may perform poorly outside of the training distribution, problem broadly known as dataset shift Selihonner-Caanded et al., 2008]. In high-stakes applications, such as healthcare, it is important to understand the limitations of model in advance [Finlayson al., 2021]: given a model trained on data from one hospital, how will it perform under changes in the population of patients, in the incidence of disease, or the treatment policy?
In this paper, our goal is to proactively understand the sensitivity of predictive model to dataset shift, using only data from the training distribution. This requires domain knowledge, specify what type of distributional changes are plausible. Formally, for model f(X) trained on data from P(X.Y) with loss function {(f (X), Y), we seek understand the loss of the model under a set of plausible future distributions P. We seek to evaluate the worst-case loss over P.
sup E)({(f()),))) PEP
(1)
and provide an interpretable description of distribution which maximizes this objective. the value of the worst-case loss is low, this can build confidence prior to deployment, and otherwise. examining the worst-case distribution can help identify weaknesses use the following rumning example, inspired by Subbaswamy et al. [2021]]
model. To illustrate, we
Equal Contribution, order determined by coin flip. Code available at this link.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Age
Parametric shift intesting policy
2222 (0
order
Disease
Test Order
Peatures
Test Result
Figure 1: (Left) Causal graph for Example 1, with shift in conditional testing rates, parameterized by oordrr. (Right) We illustrate a shift using s(Y: dorder) = 61 YY + 6o (0(1-)), where óorder (do. 61). Here we plot the (non-concave) landscape of the expected cross-entropy loss of fixed model over distributions parameterized by (ōo. 61), with the training distribution given as the black star. Simulation details are given in Appendix
Example 1 (Changes in laboratory testing). We seek to classify disease (Y) based on the age (A) of patient, whether laboratory test has been ordered (O), and test results (L) if test was ordered. The performance of a predictive model may be sensitive to changes in testing policies, as the fact that test has been ordered itself is predictive of disease. Figure 1 (left) gives plausible causal relationship between variables. Let IP(O = 1|A, Y) = o (n),,, Y)), where o is the sigmoid function and mA,YY is the log-odds. In Figure (right),
show the loss under set of new distributions parameterized by 5 = (8o, 81), where we modify (O, = 1| Y) = (n,A, Y) + sY; for a shift function s(Y: 6) = 51 Y + 6p (1 Y). which modifies the log-odds of testing for both sick and healthy patients. If 60, 5, are unconstrained, the worst-case occurs when all healthy patients are tested, and no sick patients are tested.
The first challenge is to define a set of possible distributions P such that each distribution P E P satisfies two desiderata: First, they should be causally interpretable and simple specify, without placing unnecessary restrictions on the data-generating process. Second, they should be realistic, which often entails bounding the magnitude of the shift. We construct causally interpretable shifts by defining perturbed distributions P& using changes in causal mechanisms, parameterized by a miiee-dimensional parameter 8. Our main requirement is that the shifting mechanisms follow conditional exponential family distribution. For discrete variables, this places no restriction on PP: In Example 1, O is binary and the log-odds 1)( Y) can be any function of A, Y. We also demonstrate that constraining 8 can ensure that shifts are realistic: The unconstrained worst-case shift in Example implausible, where all healthy patients (and no sick patients) are tested. Equation (1) becomes
sup Es [e()) SEA
where Es is the expectation in the shifted distribution P3 and A is a bounded set of shifts.
The second challenge is evaluation of the expected loss under shift, as well as finding the worst-case shift. Under our definition of shifts, we show that the test distribution can always be seen as reweighting of the training distribution, allowing for reweighting approaches, such as importance sampling, to estimate the expected loss under shifts. While this practical for some distribution shifts, for others, importance sampling can lead to extreme variance in estimation. Further, finding the worst-case shift using a reweighted objective involves maximization over a non-concave objective (see Figure 1), problem that is generally NP-hard. We derive second-order approximation to the expected loss under shift, and show how can be estimated without the use of reweighting. When A is single quadratic constraint, in Equation (2) with a particular non-convex, quadratically constrained quadratic program (QCQP) for which efficient solvers exist [Conn et al., 2000, Section 7]. We bound the approximation error of this surrogate objective, and show in experiments that it tends to find impactful adversarial shifts.
approximate the general non-convex optimization problem
Our contributions are as follows:
1. We provide novel formulation of robustness sets which are defined using parametric shifts. This formulation only require that the shifting mechanisms (i.e., conditional distributions) can be modelled conditional exponential family (see Section 2).
2. We derive second-order approximation to the expected loss and provide bound on the ap- proximation error. We show that this translates the general non-convex problem into particular non-convex quadratic program, for which efficient solvers exist (see Section 3).

---

3. In computer vision task, we find that this approach finds more impactful shifts than a reweighting approach, while taking far less time to compute, and that the resulting estimates of accuracy are substantially more reliable (see Section 4).
1.1 Related Work
Distributionally robust optimization//valuation Distributionally robust optimization (DRO) seeks to learn models that minimize objectives like Equation (1) with respect to the model [Duchi and Namkoong, 2021, Duchi et al., 2020, Sagawa et al., 2020]. We focus on proactive worst-case evaluation of a fixed model, not optimization, similar to Subbaswamy et al. [2021]: Li et al. [2021], but we also differ in our definition of the set of plausible future distributions P. often called an "uncertainty set" in the optimization literature. Prior work often defines these sets using distributional distances (such as f-divergences): For instance, Joint DRO [Duchi and Namkoong, 2021] allows for shifts in the entire joint distribution (i.e., all distributions in an f f-divergence ball around P(X, Y)), which may be overly conservative. Marginal DRO [Duchi et al., 2020] considers shifts in a marginal distribution (e.g., P(X)), while assuming that the remaining conditionals (e.g., P(YY X)) are fixed. However, this assumption is not applicable in all scenarios: In Example 1, for instance, this assumption does not hold for shift in testing policy. Conditional shifts are considered in recent work that focuses on evaluation [Subbaswamy et al., 2021], using worst-case conditional subpopulations. However, choosing a plausible size of conditional subpopulation is often non-obvious. In Appendix D> we give a simple lab-testing example where taking worst-case 20% conditional subpopulations corresponds to an implausible shift: Healthy patients are always tested, and sick patients never tested.
In contrast, our approach uses explicit parametric perturbations to define shifts, as opposed to distributional distances or subpopulations. In addition, our approach allows for shifts in multiple marginal or conditional distributions simultaneously: In Example 1, for instance, we could model simultaneous change in both the marginal distribution of age P(4), well as the conditional distribution of lab testing P(O A, Y), leaving other factors unchanged.
Several approaches proactively specify shifting causal mechanismss/conditionl distributions, and then seek to learn predictors that have good performance under arbitrarily large changes in these mechanisms [Subbaswamy et al., 2019, Veitch et al., 2021, 1, Makar et al., 2022, Puli et al., 2022]. Other approaches use environments [Magliacane et al., 2018, Rojas-Carulla et al., 2018, Arjovsky et al., 2019] or identity indicators -eiiz and Meinshausen, 202.] I] to learn models that rely on invariant conditional distributions.
methods for learning robust models:
However, when shifts are not arbitrarily strong, notivaey predictors can be overly conser- vative. In Example 1, a model that ignores all test-related features (and only uses age as predictor) is a particularly simple example of causality- testing policy. Closer to our setting is a line of work that considers bounded mechanism changes in linear causal models [Rothenhäusler et al., 2021, Oberst et al., 2021], where estimation of the worst-case loss enables learning of worst-case optimal models. Our work can be seen as extending this idea to more general non-linear causal models, where we focus on evaluation rather than optimization.
predictor, with invariant risk over changes
Evaluating -isttr-iiitiitttion performance with unlabelled samples: A recent line of work has focused on predicting model performance in out- -isttriiitt settings, where unlabelled data is available from the target distribution [Garg et al., 2022, Jiang et al., 2022, Chen al.. 2021]. In contrast, our method operates using only samples from the original source distribution, and seeks to estimate the worst-case loss over a set of possible target distributions.
In Appendix we give a more detailed discussion of these approaches and others.
2 Defining parametric robustness sets
Notation: Let V denote all observed variables, where (X, Y') S V for features X and labels Y, and use F(V) to denote the probability density/mass function in the training distribution. We also refer to as simply "the training distribution". [5|-] and cov(..) refer to the mean and covariance in P. and for shifted distribution P. (Definition 1) we use E. cova(.,) -). For random variable Z, use 2 to denote the space of realizations, and dz for dimension e.g., 2 E 2 C Rdz, For set of random variables V- = (Vi,.. Va), we use V; to denote an individual element, and use (Vg(V)) to denote the set of parents in directed acyelic graph (DAG) G. omitting the subscript when otherwise clear.
3

---

We begin with general definition of parameterized robustness set of distributions P. Definition 1. A parameterized robustness set around I(V) is a family of distributions P with elements (o (V) indexed by å E A R with E A, where Po (V) = P(V).
We give examples shortly that satisfy this general definition. To construct such robustness set, we consider distributions P8 that differ from I in one or more conditional distributions (Assumption 1). We require that the relevant conditional distributions can be described by an exponential family.
Definition 2 (Conditional exponential family (CEF) distribution). P(W [ is conditional expo- nential family distribution if there exists function n(Z) - - gdr such that the conditional probability density (for continuous W) or probability mass function (for discrete W) is given by
P(WIZ) 2) = (W) exp (T(Z)TT(VV) T(W)
h(n(2)))
where T(W) is vector of sufficient statistics, T(W) Rdr g(-) specifies the density of a base measure and hz)( is the log-partition function.
Definition 2 does not restrict P(W) (2) for binary/categorical W, and captures wide range of distributions, including the conditional Gaussian (see Appendix B.1 for other examples). Definition 2 extends to marginal distributions where Z Ø and (7) is a constant function.
Example (Continued). Suppose the probability of ordering test (O) depends on age (A) and disease (Y), , such that IP(O = I|A, Y) = (A, Y)), where a is the sigmoid, and r is an arbitrary function. Here, Definition 2 satisfied with W O, (A, Y). and sufficient statistic T(O) O.
We now state our main assumption, where we distinguish between the terms in the joint distribution of P that shift, which we will need to model, and those that remain fixed, which we do not.
Assumption 1 (Factorization into CEF distributions). Let W (Wi...., Wm3 S V be a "inter- vention set" of variables and let
P(V)= II F(WIZ)) W&EW
II F(VJIU))
VEVIWW
Conditionals that shift
Conditional we do not model
be factorization, where Z,, J, Vj C are possibly overlapping (or empty) sets of variables, where P(V, 2) = PVVV)) For each W; we assume Z, is known and P(W, Z,) satisfies Definition 2.
If ((V) factorizes according to a DAG G, the factorization Assumption 1 is always satisfied by Z, = PAg (W.). While we assume data is generated according to Equation (4), we do not require knowledge of the full distribution, but only the conditionals that shift. In Appendix B.2 we show that we can also consider shifts that extend Z; to include additional variables, subject to an acyclicity constraint. We now define parametric perturbations and give the general form of the robustness sets that we consider in this work, involving simultaneous perturbations to multiple W.
Definition 3 (Parameterized shift functions and 6-perturbations). Let s(;;)) Rdz + gdr be parameterized shift function with parameters E C Rd which is twiee-diiferentiable with respect to and which satisfies s Z 0) 0 for all Z. For ()) satisfying Equation (3), refer
P& (W) = gW) exp (7o(Z)T((W)) - h(ns
as a 6-perturbation of F(WIZ) with shift function s s(Z 8), where ns(2) == n(() + s(Z;0). Note that this differs from Equation (3) in that 17( is replaced by (Z).
Example (Continued). model developer may be concerned about uniform change in testing rates across all types of patients. This can be modelled by choosing 6( 8, for 8 E R, an additive intervention on the log-odds scale. A separate change in testing rates for sick and healthy patients could instead be modeled using s( 02 = 5p(1 - Y) Y,Y, using 5 E R2. This reasoning extends readily more complex shifts (c.g., allowing for age-specific changes in testing rates, with non-linear dependence on age), as long as s s(Z; 5) remains a parametric function.
While the shift function s (2; is parametric, 17( unconstrained in Definitions and Note that this formulation includes multiplicative shifts nh8 (Z) = (1 + d))(() (Z) by letting 5(Z;)) 8 (2)).

---

Definition 4 (CEF parameterized robustness set), For distribution IP and intervention set W (W,.... ] S satisfying Assumption let each Fss, (W; be a 6. (Definition 3)
of Z... Then
(
PV(V)
II Po, (W. \WEW
II P(VJ)U)) VIEV
called d-perturbation of IP(V), and the robustness set P consists of all P8 for 5 E E x Am.
To estimate the expected loss under I,s. we will typically² need to estimate n (z)) for each W; E W. However, make distributional assumptions on the remaining variables V W. This is useful in applications such as computer vision, where we do not need to restrict the generative model of images given attributes (e.g., background, camera type, etc), but can still model the expected loss under changes in the joint distribution of those attributes.
Remark (Causal Interpretation of Shifts). If available, causal knowledge helps identify which factors in the joint distribution are subject shifts (e.g.. P(O A, in Example 1), and which remain stable. It is worth noting, however, that our methodology can be used to model any change in distribution that satisfies Assumption 1, including choices of "non-causal" factorizations and shifting factors. For example, in the context of Example 1, we could choose the factorization P(Y)P(O) Y)PP A Y.Y). and model a change only in the conditional P(O Y) while keeping other factors unchanged. This shift is not interpretable as change in causal mechanisms: The shifted distribution would imply change in the marginal distribution of age, which should be unaffected by real-world change in laboratory testing. Nonetheless, we can still estimate worst-case loss over such non-causal shifts in distribution. In short, our machinery can model shifts in non-causal conditionals (for example because the causal structure is unknown), though the resulting shifted distribution is not interpretable as a plausible shift in the ground ttrutthtthhhhh data generating mechanism.
Evaluation of the worst-case loss
For fixed predictor and loss function, we can use data from P(V) to estimate the expected loss [s[{] = E3 (X)),, Y)] for a fixed 8, and estimate the worst-case loss over 5 of bounded magnitude. In Section 3.1, we show that Ps shares support with P, suggesting the use of reweighting estimators. However, these estimators can exhibit high variance for shifts that produce large density ratios (see Appendix C.5 for example), and maximizing reweighted objective over d is generally a non-convex problem. Section 3.2 we derive an approximation to the expected loss under Iss. yielding tractable surrogate optimization problem under quadratic constraints such as ||ó1|| A. Remark 2. The methods here be used with an arbitrary predictor f and loss function L := ((f(X) Y). We do not even require access to the original predictor f. Both methods here simply treat & as random variable in P. for which we have samples from the training distribution.
3.1 Modelling shifted losses using reweighting
The shifts defined in Section 2 share common support, with the following density ratio. Proposition 1. For any P& (V),, P(V) that satisfy Definition 4, supp(F) = supp(Ps)) s) and the density ratio wi- = P3/P is given by
- M(n(Z)) Wg V) = exp Σ 8i (Z55,) (Z:56))T((W.))) exp Σ Mo(Z))
-
The proof can be found in Appendix G, along with all proofs for all other claims. Example (Continued). Suppose we perturb the probability of ordering a test O given age A and disease Y with shift function s(Y; :) = 6p(1 Y) + 61 Y. independently changing the conditional probability of testing for healthy and sick patients. Here, the density ratio is given by
+ exp((
I Wi (0,AY)) exp(=xxx(3X)))))) 0) 1+ exp(n(A, Y) + s(;;)))
2 As a special case, in Appendix C.2, we show the second-order approximation (Theorem 1) can be estimated in the case of scaled mean-shifts conditional Gaussian without estimation of all of (22).
5

---

To model the loss [5 using data from P, we can consider an importance sampling (IS) estimator [Horvitz and Thompson, 1952, Shimodaira, 2000], observing that Ex Bl (V) -f.. This requires estimation of the density ratio was wx(V), and (given sample (V) y==1 from IP) yields the estimator
1 Σ w V (((V).
Es = Es,1 =
Equation (6) can have high variance when density ratios are large, and maximizing this equation with respect to is a general non-convex optimization problem, which is generally NP-hard to solve.
3.2 Approximating the shifted loss for exponential family models
We now propose an alternative approach for approximating the loss Es[4]. Recalling that Ps=0 = P. we use a second-order Taylor expansion around the training distribution
E R E[(] 5 SG + SG² 5,
where [[[] denotes the loss in the training distribution and SG', SG² are defined as follows.
Definition 5 (Shift gradient and Hessian). For a parametric shift satisfying Definition where E. { wice-diifeeentiable, we denote the shift gradient SG and shift Hessian SG²
SG = [
and
SG [0
Equation (7) is local approximation of the loss, whose approximation error we bound in Theorem 2, with smaller approximation error for smaller shifts. 3 For Iss satisfying Definition 4, SG and SG² can be computed as expectations in the training distribution, without estimation of density ratios. Recall that the conditional covariance is given by cov(A, BJC)= == E[(A B B[A[CT) B[B[CT>[C).
Theorem 1 (Shift gradients and Hessians as covariances). Assume that Ps. IP satisfy Definition 4, with intervened variables W [Wi..... Wm } and shift functions s; (Z; 6,), where (61, 6m)) Then the shift gradient is given by SG
E Idda where
SG. = E [DT.. cov (c, (... (3 ]=
and the shift Hessian is a matrix of size (ds x ds). where the (i.5)th block of size ds, x ds, equals
(s E DT,, cov (e, eT, (.zz Dt,1 - -E [ DT2ET(z] i= $ j.
cov(l, DT, TTi2, D₁,1)
where Di. = V/, (7;; d:)l:o00= is the gradient of the shift function for k == 1, and the Hessian for k = 2. Here, T.(W;) is the sufficient statistic of ZZ: and CT Z, := T.WW,) (W) Zz))
Theorem 1 handles arbitrary parametric shift functions in multiple variables, but for simple shift functions in single variable, the notation simplifies substantially, as we show in Corollary 1. Corollary (Simple shift in single variable). Assume the setup of Theorem I, restricted to a shift in single variable W. and that s sZZ 8) 8. Then D
1, D2 = 0, and
SG = E cov (L.TWW)2) (2)
and SG² = E cov E, (4.r71zTTz/22)
where T(W) is the sufficient statistic of W and eTIZ == T(W) - B[T(W)
Example 1 (Continued). Suppose that age (A) follows a normal distribution with mean and variance o², and consider shift in the mean (without changing lab testing). We can parameterize ((A) as an exponential family with parameter vi = 1//a and sufficient statistic T(A) A)= = A/o. Here, s()) 8 implies a shift in the mean of 8 standard deviations 778 17 + s()) = (μ + σ8) /o and we can write that SG cov (e, A) /o and SG² cov (2, (A {[A))²)/o* /o²
"In Appendix C.3, we give example of a linear- Gaussian generative model where this second-order expansion is exact, corresponding the setting of Anchor Regression [Rothenhäusler al., 2021].

---

To estimate the shift gradient and Hessian from a sample from P., for each = I...., m we fit models and compute residuals on these predictions, which
Ae(Z,)) E[e] Z,] and Aw: (2,) ~ permits estimation of the gradient/Hessian as a sample average of residuals. A detailed treatment is given in Appendix C.I. Using estimates of the gradient and Hessian, we estimate the expected loss as
1 1 sG's. [4] RJ Bagraylor= == B|4] + "TS' + 20
Here, there are two sources of error: Finite-sample error, due to the estimates of SG SG², as well as approximation error. The latter is bounded by the norm of and a term that depends on the covariance between the loss and the deviations of the sufficient statistic from its shifted mean.
Theorem 2. Assume that Ps. P satisfy the conditions of Theorem 1, with shift in single variable W. where o) = 8. Let Esyoar be the population Taylor estimate (Equation (7)) and let o (M) denote the largest absolute value of the eigenvalues of matrix M. Then
Es [8]
Toylor
S
sup of covvoolf (T(TT) - co [1811
rE(O.1]
where T(W) is the sufficient statistic of W Z and 4,8,Z = T(W - E. [.
To build intuition, in Appendix C.8 we give a scenario where this bound can be simplified. In particular, we consider a "covariate shift" setting (juihonero-Candell et al., 2008] where X is standard Gaussian, Y = fo(X) + e with a noise term independent of X and we consider a shift in the mean of X. When evaluating a predictor f(X) with the loss l being the squared error, the bound in Theorem 2 depends on how the modelling error g(X) = fo(X) - f(X) behaves over the domain.
In particular, the bound scales as the supremum (over / E [0, 1) of Varlgg( + 6- (X))). As simple corollary, if our predictor is off by an additive constant factor, f= fo C. then the bound is zero, and the approximation is exact for any 5. On the other hand, if the squared modelling error g(X) x² at one point X tends be a poor predictor of the squared modelling error at another point X + - 5, then this variance will be large, and the approximation will be loose.
In exchange for considering second-order approximation of the loss, we gain two benefits: Variance reduction and tractable optimization. First, the variance of TayTa is ((1)a]i)) ) for large || while the variance of És,s can be much larger: We give a simple case in Appendix C.6 where var( (o, = O(**) while var( (É..s) O exp()?)). Second, maximizing Eay over the set || S À can be solved in polynomial time by exploiting the quadratic structure, while maximizing Es,s over the constraints is generally hard, and may be infeasible in high dimensions.
3.3 Identifying worst-case parametric shifts
For À > 0, we can locally approximate the worst-case loss over all distributions Ps where ||ó1|| S A by finding the worst-case loss in the Taylor approximation
B[A] |
sup
5T SG +30 SG² 8.
Since SG² is generally not negative definite, the maximization objective is non-concave. However, this partieular problem is an instance of the "trust region problem'* which well-studied in the optimization literature [Conn et al., 2000], and can be solved in polynomial time by specialized algorithms (see Pólik and Terlaky [2007, Section 8.1] for an example). This follows from the fact that strong duality holds, so that the optimal solution 5* can be characterized in terms of the Karush-Kulha Kuhn-Tucker conditions [Boyd and Vandenberghe, 2004, Section 5.2]. For this problem, we use the trsapp routine from NEWUOA [Powell, 2006], as implemented in the python package trustregion. Depending on the application and prior knowledge, one may choose constraint sets that differ from || < X. In particular, the strong duality of Equation (9) also holds when ||ō]|2 S A is replaced by any single quadratic constraint 8 A6 + 8 S X, allowing for e.g., larger shifts in some directions than in others.
Not to be confused with the "trust region method' which repeatedly solves the trust region problem.
7

---

0.85
Loss of
0.80
Est Loss
0.80-
Training Dist
training Dist.
bubustnes Set
0.75
Est. Worst-Case
Loss
0.70
0.70
0.85
0.60
0.60
0.35
0.50
0.50
0.0
0.2
0.4
0.6
0.8
1.0
Do
Po
Figure 3: The blue line gives the (unobserved) cross-entropy loss under parametric shifts, plotted with respect to the parameter dg (left) and the resulting change in the marginal laboratory testing rate (right). We also provide the quadratic approximation (orange line), estimated using validation data, and the predicted worst-case shift (red star) for [ōg 2 (region in grey).
4 Experiments
4.1 Illustrative example: Laboratory testing
To build intuition, we illustrate our method in a simple generative model, similar to Example 1, where lab tests more likely to be ordered (O) for sick patients (Y), and lab values (I.) are predictive of Y.
Y
O
Y ~ Ber(0.5) O|Y ~ Ber(a(a + BY)) LI(Y, = 1) ~N( 1)
L
where μ1 = 0.5, po -0.5, and we initialize with = 1, 3 = 2, so that P(O Y 0) 0.27 and IP(O 1 1) 0.73, and the marginal
Figure 2
probability of test ordering is POO 1) = 0.5. When O = O, we set L to dummy value of L O. The underlying causal graph given in Figure 2. The predictive model f(O, L) is trained on data from P to predict Y using all available features. lab tests are not available (0 = 0), this model predicts based on the observed likelihood of Y given 0, and otherwise uses logistic regression model trained on cases where O = 1 in the training data.
Defining a shift function: PP(OYY) is conditional exponential family with r)(Y)
BY, We consider the shift function 8); = 50 + Y, where 6n models an overall change in testing rate, and 61 models an additional change in the likelihood of testing sick (Y = 1) patients.
To start, we keep أ = 0 fixed and vary only do. which uniformly increases or decreases testing. In Figure 3, we show the ground-truth cross-entropy loss fOO L) under perturbed distributions Po,- We observe that the direction of the shift matters: In Figure 3, the model performance slightly increases under small increase in testing rates, but degrades if testing increases too much; moreover, the loss under shift is generally asymmetric,
Estimating the impact of shift using quadratic approximation:
decrease hurts more than increase in testing. In Figure (left), we demonstrate described in Section 3.2. For illustration, we consider a
the use of the quadratic approximation robustness set of dg E 2- 2), and see that the predicted worst-case shift coincides with the actual worst-case shift, and that the quadratic approximation accurate for smaller values of 8.
In Appendix D. we allow both do and é to vary, and compare approach to that of worst-case (1 - a) conditional subpopulation shifts [Subbaswamy al., 202 I]. In the context this example, demonstrate that for any
0.27, the worst-case conditional subpopulation loss is achieved by having all healthy patients get tested, and no sick patients get tested. We contrast this with an iterative approach to designing constraints that is made possible by considering parametric shifts, where end-users can restrict the degree to which the shift differs across sick and healthy populations.
4.2 Detecting sensitivity to non-causal correlations
predictive model may pick up on various problematic dependencies the data that may not remain stable under dataset shift. To understand the impact of these dependencies, a model user may wish to understand which changes in distribution pose the greatest threats to model performance, and to measure the impact of these changes. To illustrate this use-case, make use of the CelebA dataset
8

---

Age
Gender
Hyeglasses
Haid
Mustache
Smillng
Wearing Lipstick
Mouth Slightly Open
Narrow Eyes
Figure 4: Causal graph over attributes in the synthetic CelebA dataset, where lightning bolts indicate changes in mechanisms. All of these attributes are causal parents of the image X (not shown here), which is generated by a GAN conditioned on these attributes.
[Liu et al., 2015], which contains images of faces and binary attributes (e.g. glasses, beard, etc.) encoding several features whose correlations may be unstable (e.g., the relation between gender and being bald). We consider the task of predicting gender (Y) from images of faces (X), and assess sensitivity shift in the distributions of attributes (W).
Setup: To obtain ground trrtt shifts in distribution, we generate synthetic datasets of faces using CausalGAN [Kocaoglu et al., 2018], trained the CelebA data. We simulate attributes following the causal graph in Figure 4, and then simulate images from the GAN conditioned on those attributes. We draw training sample from this distribution P, and fit a gender classifier f(X) using the image data alone, by finetuning a pretrained ResNet50 classifier [Hu et al., 2018]. Each attribute W; is binary, so consider shifts in the log-odds n)( (Z,) of each attribute W. given parents Z, Here, we use maximally flexible shift function 8, (Z,; 6,) = Σrez, o,,= {2 z}. such that for Z, E 13 there are 2 parameters. Across all intervened variables, 8 E R.31, Due to the synthetic nature of our setup, we can simulate from (XX W. Y) to evaluate the ground-truth impact of this shift, simulating first from the shifted attribute distribution, and then simulating images from the GAN conditional on those attributes. We use the 0/1 loss & 1 (X) + }). and constrain ō by ||01|2 S A = 2.
Comparing importance sampling and Taylor across multiple simulations: We simulate K 100 validation sets from P. in each estimating the worst-case shifts naylor (via the approach in Section 3.3)
and ōis, where the latter corresponds minimizing És,s using a standard non-convex solver from the scipy library [Virtanen et al., 2020J. We simulate ground truth data from Ps.x and IP P to compare the two shifts. First, searching over the space of small, bounded shifts considered here. In Table I (right), we compare the average drop in accuracy using the Taylor shifts (3.8%) and the IS shifts (2.2%). In Figure (right) we plot the differences in test accuracy Eduaae [1 (S(X)= = Y} Esg (S(X) = YJJ, where the Taylor approach finds a more impactful shift in 96% of cases. Second, the Taylor approach has an average run-time of 0.01s, versus 2.14s for the IS approach. Third, when only used to evaluate the shift Taylor the IS estimator is comparable to the Taylor estimator, with a near-identical average bias (shown in Table (right)) and RMSE (0.0191 and 0.0192 respectively). Finally, however, in Table 1 (right) we observe that É is strongly biased in predicting Esg, yielding mean absolute prediction error (MAPE) of 0.069 (not shown in the table). This can be contrasted with MAPE of 0.015 when using E Taylor to predict Esuste This may suggest that optimizing the IS objective is prone to "overfittin"" choosing a sub-optimal 8 from region of the search space that has high variance. Here, where À = 2, the drop in accuracy is relatively mild for the shifts found by both approaches. In Appendix E.4 we show that larger values of À correspond to more substantial drops in accuracy (e.g., an average drop of 23% for À = 8 using the Taylor approach).
demonstrate that the Taylor approach finds more impactful shifts, when
Examining single shift: To illustrate the type of shift found by our approach, consider the Tayler (over the K runs) which yields the Pa with median test accuracy. We display the largest components of that 8 in Table (left). Among others, this shift entails 5% increase in the probability of an older woman being bald, and a 5% decrease in the probability of a young woman wearing lipstick. This suggests that the learned classifier f relies on these associations in the images for prediction. We validate that this shift leads to a decrease in accuracy of around 3.8%, using simulated data from PP.- To validate that this drop in accuracy is non-trivial occurrence, we simulate K 400 random shifts
'We do not endorse gender classification as an inherently worthwhile task. Nonetheless, gender classification is commonly studied in the context of understanding the implicit biases of machine learning models [Buolamwini and Gebru, 2018, Schwemmer al., 2020]. and consider the task with that context in mind.

---

Table (Left) Top 5 components (by magnitude) of the example shift vector d E R where IP and Po denote conditional probabilities. The full example shift vector can be found in Appendix E.2. (Right) Taylor and IS estimates vs. true accuracy for the oaalor found by the Taylor approach, and IS estimate vs. true accuracy for the ō1s found by the IS approach. Averages taken over 100 simulations.
Metric
Example 5
Avg.
Conditional
Ps
Original acc. ([
0.912
Bald
I Female, Old
0,899
010477
0.109
Acc. under Taylor (Bi..- [[((X) YJD
0.874
Bald
Male. Young
-0.000
0.378
0.214
IS est. of acc, under Taylor shift (E
0.829
0.863
Wearing Lipstik
Bald
Male.
Female, Young
-0.618
-0.680
0.924
0.622
0.368
0.455
Taylor est. of acc. under Taylor shift (Dota)
0.844
0.863
Wearing Lipstick
Fennll, Old
-0.543
0.953
0
Acc. under shift (E,,, (IXX) YJD IS est. of acc. under 18 shift (F1,,1)
0,889
0.821
at Predi Trikking
Toyter
Random shift acc. Higher tuan Seagus
-5.0%
-2.0%
0.0%%
Shift distrbutionn
Difference Shifted act. BRangter
Eaga)
Figure 5: (Left) Model accuracy at randomly drawn shifts. (Right) Difference in accuracy in the worst-case shifts identified by Taylor and importance sampling approaches. The Taylor method identifies a more adversarial shift than importance sampling in 96% of simulations (green).
5e where || and evaluate the model accuracy Pss (Figure 5, left). As expected, the chosen 5 yields a lower accuracy (red line) than all of the random shifts.
5 Conclusion
We argue for considering parametric shifts in distribution, to evaluate model performance under a set of changes that are interpretable and controllable. For parametric shifts in conditional exponential family distributions, we derive a local second-order approximation to the loss under shift. This approximation enables the use of efficient optimization algorithms (to find the worst-case shift), and empirically provides realistic estimates of the resulting loss. In a computer vision task, this approach finds more impactful shifts (in far less time) than optimizing reweighted objective, and the estimates of shifted accuracy under the chosen shift are substantially more reliable.
Of course, our method is not without limitations. Our definition of parametric shifts and resulting approximation relies on the relevant mechanisms P(W] Z) being conditional exponential family, and that the relevant variables model changes in the causal relationships between attributes of an image, but does not immediately extend to modelling changes in the distribution of images given a fixed set of attributes. As with any method that provides worst-case evaluation, there is potential for misuse and false confidence: If the specified shifts fail capture important real-world changes, the resulting worst-case loss may be overly optimistic and misleading. Even if used correctly, our approach examines a narrow measure of model performance, and small worst-case error should not be used claim that model is free of problematic behavior. For example, implicit dependence on certain attributes (e.g., race in medical imaging [Banerjee et al., 2021]) may problematic based on ethical grounds, even if does not lead to major issues with predictive performance under small shifts in distribution.
observed. As illustrated in experiments, this can be used to
Acknowledgements
We thank Jonas Peters, Tommi Jaakkola, Chandler Squires, and Stefan Hegselmann for helpful feedback and discussion, and Irene Chen and Christina Ji for providing comments on an earlier draft. MO and DS were supported in part by Office of Naval Research Award No. N00014-21-1-2877, NT was supported by research grant (18968) from VILLUM FONDEN.
10

---

References
A. Ahmadi-Javid. Entropic Value-at-Risk: new coherent risk measure. Journal of optimization theory and applications, 155(3):1105-1123) Dec. 2012.
M. Arjovsky, L. Bottou, I. Guirajani, and D. Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.0289; 2019.
I. Banerjee, A. R. Bhimireddy, J. L. Burns, L. A. Celi, L.-C. Chen, R. Correa, N. Dullerud, M. Ghas- semi, S.-C. Huang, P.-C. Kuo, M. P. Lungren, L. Palmer, B. J. Price, S. Purkayastha, A. Pyrros, L. Oayden-Rayyer, C. Okechukwu, L. Sryyed-Kllnntari, H. Trivedi, R. Wang, Z. Zaiman, H. Zhang, and J. W. Gichoya. Reading race: AI recognises patient's s racial identity in medical images. arXiv preprint drXiv:2107.10356 July 2021.
D. Berthelot, T. Schumm, and L. Metz. BEGAN: Boundary equilibrium generative adversarial networks. arXiv preprint arXiv: 703. 10717,
S. Bongers, P. Forré, Peters, and J. M. Mooij. Foundations of structural causal models with cycles and latent variables. The Annals of Statistics, 49(5):2885-299 2021.
S. Boyd and L. Vandenberghe. Convex optimization. Cambridge University Press, 2004.
J. Buolamwini and T. Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In S. A. Friedler and C. Wilson, editors, Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pages 77-91. PMLR, 2018
M. Chen, K. Goel, N. S. Sohoni, F. Poms, K. Fatahalian, and C. Re. Mandoline: Model evaluation under distribution shift. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 1617-1629. PMLR, 2021.
A. R. Conn, N. Gould, and L. Toint. Trust region methods. SIAM, 2000.
J. Duchi, Hashimoto, and H. Namkoong. Distributionally robust losses for latent covariate mixtures. arXiv preprint arXiv:2007. 13982, 2020.
J. C. Duchi and H. Namkoong. Learning models with uniform performance via distributionally robust optimization. The Annals of Statistics, 49(3):1378 1406, 2021.
S. G. Finlayson, A. Subbaswamy, K. Singh, Bowers, A. Kupke, J. Zittrain, I. S. Kohane, and S. Saria. The clinician and dataset shift in artificial intelligence. The New England journal of medicine, 385(3))283-286, July 2021.
S. Garg, S. Balakrishnan, 7. C. Lipton, B. Neyshabur, and H. Sedghi. Leveraging unlabeled data predict Out-of-Distribution performance. In ICLR, 2022.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
C. Heinze-Deml and N. Meinshausen. Conditional variance penalties and domain shift robustness. Machine learning. 110(2):303 348, Feb. 2021.
D. G. Horvitz and D. Thompson. A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47266))666666))633666555 1952.
W. Hu, G. Niu, I. Sato, and M. Sugiyama. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning, pages 2029-2037. PMLR, 2018.
Y. Jiang, V. Nagarajan, C. Back, and J. Zico Kolter. Assessing generalization of SGD via disagreement. In ICLR, 2022.
M. Kocaoglu, C. Snyder, A. G. Dimakis, and S. Vishwanath. CausalGAN: Learning causal im- plicit generative models with adversarial training. International Conference on Learning Representations, 2018. 2018.

---

L. Kook, B. Sick, and P. Bühlmann. Distributional anchor regression. Statistics and Computing, 32 (3)11-19, 2022.
H. Lam. Robust sensitivity analysis for stochastic systems. Mathematics of Operations Research, 41 (4):1248-1275, Nov. 2016.
M. Li, H. Namkoong, and S. Xia. Evaluating model performance under worst-case subpopulations. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
Liu, Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
S. Magliacane, T. Van Ommen, T. Claassen, S. Bongers, P. Versteeg, and J. M. Mooij. Domain adaptation by using causal inference to predict invariant conditional distributions. Advances in neural information processing systems, 31, 2018.
M. Makar, B. Packer, Moldovan, D. Blalock. Y. Halpern, and D' D'mmu... Causally motivated shortcut removal using auxiliary labels. In G. Camps- Valls, F. R. Ruiz, and I. Valera, editors, Proceedings The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 739-766. PMLR, 2022.
M. Oberst, N. Thams, J. Peters, and D. Sontag. Regularizing towards causal invariance: Linear models with proxies. In International Conference on Machine Learning, pages 8260-8270. PMIR, 2021.
Pearl. Causality, Cambridge University Press, 2009.
I. Pólik and Terlaky. A survey of the S-lemma. SIAM review, 49(3):371-418, 2007.
M. J. Powell. The NEWUOA software for unconstrained optimization without derivatives. In Large-scale nonlinear optimization, pages 255-297, Springer, 2006.
A. Puli, L. Zhang, E. K. Oermann, and Ranganath. Out-of-distribution generalization in the presence of Dibsance-Inducce spurious correlations. In International Conference on Learning Representations, 2022.
J. Mariooerr-Canndele M. Sugiyama, A. Schwaighofer, and N. D. Lawrence. Dataset Shift in Machine Learning. The MIT Press, 2008.
M. Dojas-Carulla, B. Schölkopf, R. Turner, and Peters. Invariant models for causal transfer learning. Journal of machine learning research: JMLR, 19(36):1-34, 2018.
D. Rothenhäusler, N. Meinshausen. P. Bühlmann, and J. Peters. Anchor regression: Heterogeneous data meet causality. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 83(2):215-246, 2021.
S. Sagawa, P. W. Koh, B. Hashimoto, and P. Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020.
C. Schwemmer, C. Knight, E. D. Bello-Pardo, S. Oklobdzija, M. Schoonvelde, and J. W. Lockhart. Diagnosing gender bias in image recognition systems. Socius sociological research for a dynamic world, 6, Jan. 2020.
H. Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelikel function.. Journal of Statistical Planning and Inference, 90(2):227-244, 2000. ISSN 0378-3758.
A. Subbaswamy, P. Schulam, and S. Saria. Preventing failures due to dataset shift: Learning predictive models that transport. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 31 3128-3127. PMLR, 2019.
A. Subbaswamy, R. Adams, and S. Saria. Evaluating model robustness and stability to dataset shift. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 261 1-2619. PMLR, 13-15 Apr 2021.
12

---

Veitch, D' D'Amour, Yadlowsky, and Eisenstein. Counterfactual invariance to spurious correlations: Why and how to pass stress tests. Advances neural information processing systems, 34, Dec. 2021.
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski,
P. Peterson, W. Weckesser, J. Bright, S. van der Walt, M. Brett. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, I. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, Perktold, R. Cimrman, I. Henriksen,
A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17:
261-272, 2020.
M. J. Wainwright, M. I. Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends@ in Machine Learning. 1(1-2):1-305, 2008.
13


Reviews:
[{"review_id": "d9Z3KvPg_O", "reviewer": "Reviewer_rEEg", "summary": "The authors work out a method that, given a structural equation model having exponential family conditional densities, returns/computes the worst-case loss (i.e., negative log-likelihood) of any predictive model fitted to data arising from the SEM over a class of distributional perturbations to the SEM.  Computing the worst-case loss \"naively\" requires solving a (constrained) non-convex optimization problem, so the authors propose a vanilla sequential convex programming (trust region) method instead that seems to work well. The method is useful from an interpretability standpoint b/c it can tell us which conditionals (e.g., which \"features\") had to be wiggled the most in order to attain the worst-case loss.", "questions": "In addition to the questions embedded in the \"Weaknesses\" section above, here are a few more Q's for the authors:\n* how would you do inference (e.g., get confidence intervals) for your estimation target (the worst-case loss)?\n* it seems taking the worst-case over alternate graph **structures** (i.e., not just **parameters**) would be super useful -- any thoughts as to how tractable that might be?", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Very cool/timely topic, w/ an approach that is apparently well-thought through / totally reasonable\n* The experiment in Sec 4.2 was neat\n* Thm 1 is interesting/useful, though not surprising\n* Clear / well-written paper\n\nWeaknesses:\n* I don't know that I totally get the approach here?  Why not define the worst-case loss to be taken over all distributions in a K-L (or any f-divergence, maybe) ball centered around the empirical measure?  I **think** the problem that falls out from that should be a (constrained) convex optimization problem.  Case closed.  Having said that, a difference in opinion on approach doesn't necessarily mean a paper is \"no good\" ...\n\n* The advantage of the K-L type approach I just mentioned is you don't have to make distributional assumptions, whereas in your case you do.  Actually you assume the user knows quite a bit about the problem at hand: the SEM, that there exist conditional densities that are exponential family, and the shift function(s) -- is all that realistic?  Moreover, the shift function has to be specified on the natural parameter scale (e.g., on the $\\mu/\\sigma$ scale, in the case of a Gaussian density) -- how interpretable is that / can people easily think in those terms?\n\n* Another potential approach: just draw data from several candidate SEM's (perhaps obtained by doing a very coarse grid search over your $\\delta$ parameter) + take the worst-case loss over this (finite) set of SEM's.  This is very roughly in line w/ what you do in Sec 4.2.  Clearly this Monte Carlo-type approach cannot be **too** good, but I'm wondering how much worse it actually is from your approach ...\n\n* There could probably be a few more / more thorough experiments in the paper ...", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "cca-I5-iPD6", "reviewer": "Reviewer_93gd", "summary": "The paper aims to tackle the problem of model evaluation by understanding the sensitivity of a predictive model to dataset shift using only data from the training distribution. The authors consider parametric shifts with Conditional Exponential Family (CEF) under a set of changes that are interpretable and controllable. For CEF distributions, authors derive a local second-order approximation to the loss (Theorem 1). This approximation enables the use of efficient optimization algorithms to find the worst-case shift. Authors also provide error bounds to their approximation (Theorem 2). On CelebA classification task, the authors show the efficacy of the proposed framework.  ", "questions": "While the motivation of the paper is very interesting and the proposed framework is novel, I have several suggestions/questions: \n\n- How to decide the bounds on $\\delta$, i.e., robustness set $\\mathcal{P}$ is unclear. Could the authors elaborate a bit more here?\n- To tackle the variance issue in the importance weights, in Eq. 6, did the authors try doubly robust estimators [1]? Maybe this can already improve the numbers in Table 2 (right) for IM.\n- As mentioned in the weaknesses above, could the authors explain how general their considered CEF framework is? \n- How does the framework considered here compare with Mandoline's framework [2]? It would be great to see the `pros and cons' of the CEF framework with the Mandoline framework, in terms of coverage of model evaluation settings, meta-information needed from humans, and underlying assumptions?\n- How does the accuracy estimation in CelebA experiment stand with other heuristic methods, e.g., GDE [3], ATC[4]. \n\n\n[1]  Reddi et al. 2014. Doubly Robust Covariate Shift Correction. https://www.cs.cmu.edu/~sjakkamr/papers/doublyrobust.pdf. \n\n[2] Chen et al. 2021. Mandoline: Model Evaluation under Distribution Shift https://arxiv.org/abs/2107.00643\n\n[3] Jiang et al. 2022. Assessing Generalization of SGD via Disagreement. https://arxiv.org/abs/2106.13799. \n\n[4] Garg et al. 2022. Leveraging Unlabeled Data to Predict Out-of-Distribution Performance. https://arxiv.org/abs/2201.04234", "limitations": "The authors provide a summary of the limitations of their work. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**:  \n\n- The problem considered in the paper is very interesting and timely. Understanding model performance beyond iid validation test is important prior to deployment. This can also enable examining the worst-case distribution P which can help identify weaknesses of the model.\n- Approximation of the loss with the CEF with second-order Taylor expansion is interesting. Theorem 1 derives an expression for SG$_i^1$ and Theorem 2 provides an error bound for the approximation error in population. \n- The running Example 1 is illustrative. Connecting back to it at many places helps in understanding the proposed framework and formulation \n\n\n**Weakness**: \n- The paper could do a better job in explaining what all kinds of distribution shifts are captured with Definition 3. Again the example in Section 4.1 is illustrative and interesting but overall, it is not clear how general the considered CEF framework is? \n- Experimental evaluation of the proposed framework is very slim and a major weakness of the paper. While I acknowledge the theoretical contribution and novelty of the paper, as mentioned before, the generality of the CEF framework is unclear and experimental validation is lacking. This is the main reason for my score. I am open to changing my score if the authors could comment more on this point or if I share any misunderstanding. \n  - (Minor) In the CelebA experiment, how can a user identify which variables to intervene upon and how to decide their bounds, i.e., the problematic dependencies (for which model performance changes considerably under dataset shift) in the data?  \n  - (Minor) Authors use Causal GAN to simulate distribution shifts, how reliable is even the ground truth evaluation in this framework? While I appreciate the details in Appendix D with some pictures of the generated example, a commentary on the generated dataset would be great. \n  - As mentioned in the limitations section, specified shifts can fail to capture important real-world changes and hence, the resulting worst-case loss may be overly optimistic and misleading. Can authors comment on this more in the context of CelebA experiment where the worst-case shift identified only has a drop of ~3 percentages in accuracy? \n- (Minor) While authors acknowledge the sources of error due to finite samples, complimenting Theorem 2 with errors due to finite-sample approximation would be insightful. \n- (Minor) Paper is a bit hard to understand at places. For example, \n  - Definition 3 gives the parametric family of shifts considered in the paper. The changes w.r.t. Definition 2 could be reminded to the reader which in particular is in the $\\eta_\\delta$ function. \n- (Minor) Writing of the paper could also be improved. For example, \n  -  A lot of references to \"these\" and \"them\" are used where the contextualization is a bit ambiguous and left for interpretation. E.g., abstract line 2-5. \n  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "REss6skjDdn", "reviewer": "Reviewer_mdu8", "summary": "The paper considers the problem of identification and evaluation of potential harmless data shifts. The paper proposes a parametric robustness set and proposes a local second-order approximation to efficiently solve the worst-case problem", "questions": "\nUnder which scenarios, the local second-order approximation performs well?", "limitations": "The definition of CEF parametrized robustness set seems restrictive.\n\n", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The problem of proactively understanding the policy's performance under distributional shifts is interesting to me.\n\nHowever, honestly, I feel this paper is hard to understand, perhaps I am not very familiar with the language used in this community.\n1. For Example 1, are A, and Y are allowed to change? There is a $\\delta_{age}$ in the caption of Figure 1, but it seems that the authors never use them.\n2. The author $\\mathbb{P}(V)$ to denote a distribution, which is not commonly used. I suggest the authors rigorous define everything using probability language, e.g. define the probability space {$ \\Omega,\\mathcal{F},P $}.\n3. I am confused with Equation (4) in Assumption 1. Why there is no $\\mathbb{P}(Z_i)$ and $\\mathbb{P}(U_j)$.\n4. I think the authors would better explain what shifts, intervention, and robust mean in this context.\n5. I did not quite get the intuition behind the CEF parametrized robustness set? Why is it a natural and good choice? The authors should better motivate.\n6. Are the variables $W$ the only variables that only to be intervened (shifted)? How about the variables Z,V,U?\n7. I think Proposition 1 implicitly assumes the distributions of Z do not change.\n\n\nOverall I find this paper needs to be further polished. And this paper at least in its current version will create difficulties for the general audience of NeurIPS to understand.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "-RvE9kQI0k", "reviewer": "Reviewer_Qr2b", "summary": "In this work, the present a method to evaluate the robustness of models. This is done by first parametrizing the space of distribution shifts and then maximizing an approximation of the model loss over the shift parameter space. The approximation is done via second-order Taylor expansion and some theory is provided about the approximation error. Finally, they show in experiments the ability of their method to find interpretable distribution shifts that negatively affect a model.", "questions": "1. The supremum factor in Theorem 2 is hard for me to reason about. Are there reasonable settings where this bound can be simplified (e.g. Lipschitz losses)?\n2. In Line 216, it is said that the variance of the Taylor approximation is $O(||\\delta||^4)$, but shouldn't the second term in (8) contribute $O(||\\delta||^2)$ variance? Should I be thinking of $\\delta$ as bounded away from 0?\n3. In section 3.3, the worst-case approximate loss is computed in a 2-norm ball. However, it seems to make more sense that the ball should be the sup-norm, with the idea being that each mechanism can be perturbed independently. Is it worth seeing what happens when you use that, or do you lose tractability?\n", "limitations": "1. I think the accuracy of the approximation needs to be better guaranteed in practice. It seems unclear based on just Thm 2 that the second-order Taylor will be accurate enough at moderately sized delta. One could perhaps verify the bound by sampling at the perturbed delta and checking the model's performance there, but that may not be always feasible.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Parameterizing shift space to probe biases is an original (to my knowledge) idea that could be a useful way to find interpretable biases.\n* Good experiments\n* Well written\n\nWeaknesses:\n* The approximation method could be too inaccurate in practice (see limitation 1)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept