D ISTRIBUTIONALLY R OBUST R ECOURSE A CTION
Anonymous authors
Paper under double-blind review
A BSTRACT
A recourse action aims to explain a particular algorithmic decision by showing one speciﬁc way in which the instance could be modiﬁed to receive an alternate outcome.
Existing recourse generation methods often assume that the machine learning model does not change over time.
However, this assumption does not always hold in practice because of data distribution shifts, and in this case, the recourse action may become invalid.
To redress this shortcoming, we propose the Distributionally Robust Recourse Action (DiRRAc) framework, which generates a recourse action that has a high probability of being valid under a mixture of model shifts.
We formulate the robustiﬁed recourse setup as a min-max optimization problem, where the max problem is speciﬁed by Gelbrich distance over an ambiguity set around the distribution of model parameters.
Then we suggest a projected gradient descent algorithm to ﬁnd a robust recourse according to the min-max objective.
We show that our DiRRAc framework can be extended to hedge against the misspeciﬁcation of the mixture weights.
Numerical experiments with both synthetic and three real-world datasets demonstrate the beneﬁts of our proposed framework over state-of-the-art recourse methods.
I NTRODUCTION
Post-hoc explanations of machine learning models are useful for understanding and making reliable predictions in consequential domains such as loan approvals, college admission and healthcare.
Recently, recourse is rising as an attractive tool to diagnose why the machine learning models have made a particular decision for a given instance.
A recourse action provides a possible modiﬁcation of the given instance to receive an alternate decision (Ustun et al., 2019).
Consider, for example, the case of loan approvals in which a credit application is rejected.
Recourse will offer the reasons for rejection by showing what the application package should have been to get approved.
A concrete example of a counterfactual in this case may be “the monthly salary should be higher by $500” or “20% of the current debt should be reduced”.
A recourse action has a positive, forward-looking meaning: they list out a directive modiﬁcation that a person should implement so that they can get a more favorable outcome in the future.
If a machine learning system can provide the negative outcomes with the corresponding recourse action, it can improve the user engagement and boost the interpretability at the same time (Ustun et al., 2019; Karimi et al., 2021b).
Explanations thus play a central role in the future development of human-computer interaction as well as human-centric machine learning.
Despite its attractiveness, providing recourse for the negative instances is not a trivial task.
For real-world implementation, designing a recourse needs to strike an intricate balance between conﬂicting criteria.
First and foremost, a recourse action should be feasible: if the prescribed action is taken, then the prediction of a machine learning model should be ﬂipped.
Further, to avoid making a drastic change to the characteristics of the input instance, a framework for generating recourse should minimize the cost to implement the recourse action.
An algorithm for ﬁnding recourse must make change to only features that are actionable, and should leave immutable features (relatively) unchanged.
For example, we must consider the date of birth as an immutable feature; in contrast, we can consider salary or debt amount as actionable features.
Various solutions have been proposed to provide recourses for a model prediction (Karimi et al., 2021b;
Stepin et al., 2021; Artelt & Hammer, 2019; Pawelczyk et al., 2021; 2020; Verma et al., 2020).
For instance, Ustun et al.
(2019) used an integer programming approach to obtain actionable recourses, and also provide a feasibility guarantee for linear models.
Karimi et al.
(2020) proposed a model-agnostic approach to generate nearest counterfactual explanations and focus on structured data.
Dandl et al.
(2020) proposed a method which ﬁnds counterfactual by solving a multi-objective optimization problem.
Recently, Russell (2019) and Mothilal et al.
(2020) focus on ﬁnding a set of multiple diverse recourse actions, where the diversity is imposed by a rule-based approach or by internalize a determinant point process cost in the objective function.
These aforementioned approaches make a fundamental assumption that the machine learning model does not change over time.
However, the dire reality suggests that this assumption rarely holds.
In fact, data shifts are so common nowadays in machine learning that they have sparkled the emerging ﬁeld of domain generalization and domain adaptation.
Organizations usually retrain models as a response to data shifts and this induces corresponding shifts in the machine learning models parameters, which in turns cause serious concerns for the feasibility of the recourse action in the future (Rawal et al., 2021).
In fact, all of the aforementioned approaches design the action which is feasible only with the current model parameters, and they provide no feasibility guarantee for the future parameters.
If a recourse action fails to generate a favorable outcome in the future, then the recourse action may become less beneﬁcial (Venkatasubramanian & Alfano, 2020), the pledge of a brighter outcome is shattered, and the trust on the machine learning system is lost (Rudin, 2019).
To tackle this challenge, Upadhyay et al.
(2021) proposed ROAR, a framework for generating instance level recourses that are robust to shifts in the underlying predictive model.
ROAR used a robust optimization approach that hedges against an uncertainty set containing plausible values of the future model parameters.
However, it is well-known that robust optimization solutions can be overly conservative because they may hedge against a pathological parameter in the uncertainty set (Ben-Tal et al., 2017; Roos & den Hertog, 2020).
A promising approach that can promote robustness, while at the same time prevent from over-conservatism is the distributionally robust optimization framework (El Ghaoui et al., 2003; Delage & Ye, 2010; Rahimian & Mehrotra, 2019;
Bertsimas et al., 2018).
This framework models the future model parameters as random variables whose underlying distribution is unknown, but is likely to be contained in an ambiguity set.
The solution is designed to counter the worst-case distribution in the ambiguity set in a min-max sense.
Distributionally robust optimization is also gaining popularity in many estimation and prediction tasks in machine learning (Namkoong & Duchi, 2017; Kuhn et al., 2019).
Contributions.
This paper combines ideas and techniques from two principal branches of explainable artiﬁcial intelligence: counterfactual explanations and robustness to resolve the recourse problem under uncertainty.
Concretely, our main contributions are the following:
We propose the framework of Distributionally Robust Recourse Action (DiRRAc) for designing a recourse action that is robust to mixture shifts of the model parameters.
Our DiRRAc maximizes the probability that the action is feasible with respect to a mixture shift of model parameters, while at the same time conﬁnes the action in the neighborhood of the input instance.
Moreover, the DiRRAc model also hedges against the misspeciﬁcation of the nominal distribution using a min-max form with a mixture ambiguity set prescribed by moment information.
We reformulate the DiRRAc problem into a ﬁnite-dimensional optimization problem with an explicit objective function.
We also provide a projected gradient descent to solve the problem.
We extend our DiRRAc framework along several axis to handle mixture weight uncertainty, to minimize the worst-case component probability of receiving unfavorable outcome, and also to incorporate the Gaussian parametric information.
We ﬁrst describe the recourse action problem with mixture shifts in Section 2.
In Section 3, we present our proposed DiRRAc framework, its reformulation and the numerical routine for solving it.
The extension to the parametric Gaussian setting will be discussed in Section 4.
Section 5 reports the numerical experiments showing the beneﬁts of the DiRRAc framework and its extensions.
Notations.
For each integer K , we have [ K ] = { 1 , . . .
, K } .
We use S d + ( S d ++ ) to denote the space of symmetric positive semideﬁnite (deﬁnite) matrices.
For any A ∈ R m × m , the trace operator is
Tr (cid:2) A (cid:3) =
(cid:80) di
=1 A ii .
If a distribution Q k has mean µ k and covariance matrix Σ k , we write Q k ∼ ( µ k , Σ k ) .
If additionally Q k is Gaussian, we write Q k ∼ N ( µ k , Σ k ) .
Writing Q ∼ ( Q k , p k )
k ∈
[ K ] means Q is a mixture of K components, the k -th component has weight p k and distribution Q k .
R ECOURSE A CTION UNDER M IXTURE S HIFTS
We consider a binary classiﬁcation setting with label Y = { 0 , 1 } , where 0 represents the unfavorable outcome while 1 denotes the favorable one.
The covariate space is R d , and any linear classiﬁer
C θ : R d → Y characterized by the d -dimensional parameter θ is of the form
[Equation]
Note that the bias term can be internalized into θ by adding an extra dimension, and thus it is omitted.
Suppose that at this moment ( t = 0 ), the current classiﬁer is parametrized by θ 0 , and we are given an input instance x 0 ∈ R d with un favorable outcome, that is, C θ 0 ( x 0 )
= 0 .
One period of time from now ( t = 1 ), the parameters of the predictive model will change stochastically, and are represented by a d -dimensional random vector ˜ θ .
This paper focuses on ﬁnding a recourse action x which is reasonably close to the instance x 0 , and at the same time, has a high probability of receiving a favorable outcome in the future.
Figure 1 gives a bird’s eye view of the setup.
To measure the closeness between the action x and the input x 0 , we assume that the covariate space is endowed with a non-negative, continuous cost function c .
In addition, suppose temporarily that ˜ θ follows a distribution (cid:98) P .
Because maximizing the probability of the favorable outcome is equivalent to minimizing the probability of the unfavorable outcome, the recourse can be found by solving
[Equation]
The parameter δ ≥ 0 in (1) governs how far a recourse action can be from the input instance x 0 .
Note that we constrain x in a set X which captures operational constraints, for example, the highest education of a credit applicant should not be decreasing over time.
In this paper, we model the random vector ˜ θ using a ﬁnite mixture of distributions with K components, the mixture weights are (cid:98) p satisfying (cid:80)
k ∈
[ K ] (cid:98) p k
= 1 .
Each component in the mixture represents one speciﬁc type of model shifts: the weights (cid:98) p reﬂect the proportion of the shift types while the component distribution (cid:98) P k representing the (conditional) distribution of the future model parameters in the k -th shift.
Further information on mixture distributions and their applications in machine learning can be found in (Murphy, 2012, §3.5).
Note that the mixture model is not a strong assumption.
It is well-known that the Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any speciﬁc nonzero amount of error by a Gaussian mixture model with enough components (Goodfellow et al., 2016;
McLachlan & Peel, 2000).
Thus, our mixture models are ﬂexible enough to hedge against distributional perturbations of the parameters under large values of K .
The design of the ambiguity set to handle ambiguous mixture weights and under the Gaussian assumption is extensively studied in the literature on distributionally robust optimization (Hanasusanto et al., 2015; Chen & Xie, 2021).
If each (cid:98) P k is a Gaussian distribution N ( (cid:98) θ k , (cid:98) Σ k ) , then (cid:98) P is a mixture of Gaussian distributions.
The objective of problem (1) can be expressed as
[Equation]
where the ﬁrst equality follows from the law of conditional probability, and Φ is the cumulative distribution function of a standard Gaussian distribution.
Under the Gaussian assumption, we can solve (1) using a projected gradient descent type of algorithm (Boyd & Vandenberghe, 2004).
Remark 2.1 (Nonlinear models) .
Our analysis focuses on linear classiﬁers, which is a common setup in the literature (Upadhyay et al., 2021; Ustun et al., 2019; Rawal et al., 2021; Karimi et al., 2020; Wachter et al., 2018; Ribeiro et al., 2016).
To extend to nonlinear classiﬁers, we can follow a similar approach as in Rawal & Lakkaraju (2020b) and Upadhyay et al.
(2021) by ﬁrst using
LIME Ribeiro et al.
(2016) to approximate the nonlinear classiﬁers locally with an interpretable linear model, then subsequently applying our framework.
D ISTRIBUTIONALLY R OBUST R ECOURSE A CTION F RAMEWORK
Our Distributionally Robust Recourse Action (DiRRAc) framework robustiﬁes formulation (1) by relaxing the parametric assumption and hedging against distribution misspeciﬁcation.
First, we assume that the mixture components (cid:98) P k are speciﬁed only through moment information, and no particular parametric form of the distribution is imposed.
In effect, (cid:98) P k is assumed to have mean vector (cid:98) θ k ∈ R d and positive deﬁnite covariance matrix (cid:98) Σ k
(cid:31) 0 .
Second, we leverage ideas from distributionally robust optimization to propose a min-max formulation of (1), in which we consider an ambiguity set which contains a family of probability distributions that are sufﬁciently close to the nominal distribution (cid:98) P .
We prescribe the ambiguity set using Gelbrich distance (Gelbrich, 1990).
Deﬁnition 3.1 (Gelbrich distance) .
The Gelbrich distance G between two tuples ( θ, Σ) ∈
R
d × S d + and ( (cid:98) θ, (cid:98) Σ)
∈ R
d × S d + amounts to G (( θ, Σ) , ( (cid:98) θ, (cid:98) Σ))
(cid:44) (cid:113) (cid:107) θ − (cid:98)
[Equation]
It is easy to verify that G is non-negative, symmetric and it vanishes to zero if and only if ( θ, Σ) =
( (cid:98) θ, (cid:98) Σ) .
Further, G is a distance on R d × S d + because it coincides with the type- 2 Wasserstein distance between two Gaussian distributions N ( µ, Σ) and N ( (cid:98) µ, (cid:98) Σ) (Givens & Shortt, 1984).
Distributionally robust formulations with moment information prescribed by the G distance are computationally tractable under mild conditions, deliver reasonable performance guarantees and also generate a conservative approximation of the Wasserstein distributionally robust optimization problem (Kuhn et al., 2019; Nguyen et al., 2021).
In this paper, we use the Gelbrich distance G to form a neighborhood around each (cid:98) P k as
[Equation]
Intuitively, one can view B k ( (cid:98) P k ) as a ball centered at the nominal component (cid:98) P k of radius ρ k ≥
prescribed using the distance G .
This component set B k ( (cid:98) P k ) is non-parametric, and the ﬁrst two moments of Q k are sufﬁcient to decide whether Q k belongs to B k ( (cid:98) P k ) .
Moreover, if Q k ∈
B k ( (cid:98) P k ) , then any distribution Q (cid:48) k with the same mean vector and covariance matrix as Q k also belongs to B k ( (cid:98) P k ) .
Notice that even when the radius ρ k is zero, the component set B k ( (cid:98) P k ) does not collapse into a singleton.
Instead, if ρ k = 0 then B k ( (cid:98) P k ) still contains all distributions of the same moment ( (cid:98) θ k , (cid:98) Σ k ) with the nominal component distribution (cid:98) P k , and consequentially it possesses the robustiﬁcation effects against the parametric assumption on (cid:98) P k .
The component sets are utilized to construct the ambiguity set for the mixture distribution as
[Equation]
[ K ] such that Q ∼ ( Q k , (cid:98) p k ) k ∈
[ K ] (cid:111) .
Any Q ∈ B ( (cid:98) P ) is also a mixture distribution with K components, with the same mixture weights (cid:98) p .
Thus, B ( (cid:98) P ) contains all perturbations of (cid:98) P induced separately on each component by B k ( (cid:98) P k ) .
We are now ready to introduce our DiRRAc model, which is a min-max problem of the form
[Equation]
The objective of (2) is to minimize the worst-case probability of unfavorable outcome of the recourse action.
Moreover, the last constraint imposes that for each component, the worst-case conditional probability of unfavorable outcome should be strictly less than one.
Put differently, this last constraint requires that the action should be able to lead to favorable outcome for any distribution in
B k ( (cid:98) P k ) .
By deﬁnition, each supremum subproblem in (2) is an inﬁnite-dimensional maximization problem over the space of probability distributions, and thus it is inherently difﬁcult.
Fortunately, because we use the Gelbrich distance to prescribe the set B k ( (cid:98) P k ) , we can solve these maximization problems analytically.
This consequentially leads to a closed-form reformulation of the DiRRAc model into a ﬁnite-dimensional problem.
Next, we will reformulate the DiRRAc problem (2), provide a sketch of the proof and propose a numerical solution routine.
3. R EFORMULATION OF D I RRA C
Each supremum in (2) is an inﬁnite-dimensional optimization problem on the space of probability distributions.
We now show that (2) can be reformulated as a ﬁnite-dimensional problem.
Towards this end, let X be the following d -dimensional set.
[Equation]
The next theorem asserts that the DiRRAc problem (2) can be reformulated as a d -dimensional optimization problem with an explicit, but complicated, objective function.
Theorem 3.2 (Equivalent form of DiRRAc) .
Problem (2) is equivalent to the ﬁnite-dimensional optimization problem
[Equation]
where the function f k admits the closed-form expression
[Equation]
Next, we sketch a proof of Theorem 3.2 and a solution procedure to solve problem (4).
3. P ROOF S KETCH
For any component k ∈ [ K ] , deﬁne the following worst-case probability of unfavorable outcome
[Equation]
To proceed, we rely on the following elementary result from (Nguyen, 2019, Lemma 3.31).
Lemma 3.3 (Worst-case Value-at-Risk) .
For any x ∈ R d and β ∈ (0 , 1) , we have
[Equation]
Note that the left-hand side of (6) is the worst-case Value-at-Risk with respect to the ambiguity set
B k ( (cid:98) P k ) .
Leveraging this result, the next proposition provides the analytical form of f k ( x ) .
Proposition 3.4 (Worst-case probability) .
For any k ∈ [ K ] and ( (cid:98) θ k , (cid:98) Σ k , ρ k ) ∈
R
d × S d + × R + , deﬁne the following constants A k (cid:44) − (cid:98) θ (cid:62) k x , B k (cid:44) (cid:113) x (cid:62) (cid:98)
Σ k x , and C k (cid:44) ρ k (cid:107) x (cid:107) 2 .
We have f k ( x ) (cid:44) sup
[Equation]
P
[Equation]
The proof of Theorem 3.2 follows by noticing that the DiRRAc problem (2) can be reformulated using the elementary functions
f k as
[Equation]
where the objective function follows from the deﬁnition of the set B ( (cid:98) P ) .
It sufﬁces now to combine with Proposition 3.4 to obtain the necessary result.
The detailed proof is relegated to the Appendix.
Next we propose a projected gradient descent algorithm to solve the problem (4).
3. P ROJECTED G RADIENT D ESCENT A LGORITHM
We consider in this section an iterative numerical routine to solve the DiRRAc problem in the equivalent form (4).
First, notice that the second constraint that deﬁnes X in (3) is a strict inequality, thus the set X is open.
We thus modify slightly this constraint by considering the following set
[Equation]
for some value ε > 0 sufﬁciently small.
Moreover, if the parameter δ is too small, it may happen that the set X ε becomes empty.
Deﬁne δ min ∈ R + as the optimal value of the following problem
[Equation]
Then it is easy to see that X ε is non-empty whenever δ ≥ δ min .
In addition, because c is continuous and X is closed, the set X ε is compact.
In this case, we can consider problem (4) with the feasible set being X ε , for which the optimal solution is guaranteed to exist.
Let us now deﬁne the projection operator Proj X ε as Proj X ε ( x (cid:48) )
(cid:44) arg min (cid:8) (cid:107) x
[Equation]
x
[Equation]
: x
[Equation]
X
[Equation]
(cid:9) .
If X is convex and c ( · , x 0 ) is a convex function, then X ε is also convex, and the projection operation can be efﬁciently computed using convex optimization.
In particular, suppose that c ( x, x 0 ) =
(cid:107) x − x 0 (cid:107) 2 is the Euclidean norm and X is second-order cone representable, then the projection is equivalent to a second-order cone program, and can be solved using off-the-shelf solvers such as GUROBI Gurobi Optimization, LLC (2021) or Mosek (MOSEK
ApS, 2019).
The projection operator Proj X ε now forms the building block of a projected gradient descent algorithm with a backtracking linesearch.
The details regarding the algorithm, along with the convergence guarantee, are presented in Appendix E.
To conclude this section, we visualize the geometrical intuition of our method in Figure 2.
The circular arc represents the proximity boundary c ( x, x 0 )
=
δ with c being an Euclidean distance.
Dashed lines represent the hyperplane − (cid:98) θ (cid:62)
k x = 0 for different k , while elliptic curves represent the robust margin − (cid:98) θ (cid:62) k x + ρ k (cid:107) x (cid:107) = 0 with matching color.
Increasing the ambiguity size ρ k brings the elliptic curves towards the top-right corner and farther away from the dash lines.
The set X taken as the intersection of elliptical and promixity constraints will move deeper into the interior of the favorable prediction region, resulting in more robust recourses.
G AUSSIAN D I RRA C F RAMEWORK
We here revisit the Gaussian assumption on the component distributions, and propose the parametric
Gaussian DiRRAc framework.
We make the temporary assumption that (cid:98) P k are Gaussian for all k ∈
[ K ] , and we will robustify against only the misspeciﬁcation of the nominal mean vector and covariance matrix ( (cid:98) θ k , (cid:98) Σ k ) .
To do this, we ﬁrst construct the Gaussian component ambiguity sets
[Equation]
where the superscript emphasizes that the ambiguity sets are neighborhoods in the space of Gaussian distributions.
The resulting ambiguity set for the mixture distribution is
[Equation]
k
[Equation]
∈
[ K
[Equation]
The Gaussian DiRRAc problem is formally deﬁned as
[Equation]
Similar to Section 3, we will provide the reformulation of the Gaussian DiRRAc formulation and a sketch of the proof in the sequence.
Note that the last constraint in (8) has margin 12 instead of 1 as in the DiRRAc problem (2).
The detailed reason will be revealed in the proof sketch in Section 4.2.
4. R EFORMULATION OF G AUSSIAN D I RRA C
Remind that the feasible set X is deﬁned as in (3).
The next theorem asserts the equivalent form of the Gaussian DiRRAc problem (8).
Theorem 4.1 (Gaussian DiRRAc reformulation) .
The Gaussian DiRRAc problem (8) is equivalent to the ﬁnite-dimensional optimization problem
[Equation]
[Equation]
where the function g k admits the closed-form expression g
[Equation]
=
[Equation]
Problem (9) can be solved using the projected gradient descent algorithm discussed in Section 3.3.
4. P ROOF S KETCH
The proof of Theorem 4.1 relies on the following analytical form of the worst-case Value-at-Risk (VaR) under parametric Gaussian ambiguity set (Nguyen, 2019, Lemma 3.31).
Lemma 4.2 (Worst-case Gaussian VaR) .
For any x ∈ R d and β ∈ (0 , 12 ] , let t =
Φ − 1 (1 − β ) .
Then inf
[Equation]
It is important to note that Lemma 4.2 is only valid for β ∈ (0 , 0 . 5] .
Indeed, for β > 12 , evaluating the inﬁmum problem in the left-hand side of (10) requires solving a non-convex optimization problem as t =
Φ − 1 (1 − β ) < 0 .
As a consequence, the last constraint of the Gaussian DiRRAc formulation (8) is capped at a probability value of 0 .
5 to ensure the convexity of the feasible set in the reformulation (9).
The proof of Theorem 4.1 follows a similar line of argument as for the
DiRRAc formulation, with g k being the worst-case Gaussian probability
[Equation]
To conclude this section, we provide a quick sanity check: by setting K = 1 and ρ 1 = 0 , we have a special case in which ˜ θ follows a Gaussian distribution N ( (cid:98) µ 1 , (cid:98) Σ 1 ) .
Thus, ˜ θ (cid:62) x ∼ N ( (cid:98) µ (cid:62) 1 x, x (cid:62) (cid:98) Σ 1 x ) and it is easy to verify from the formula of g 1 in the statement of Theorem 4.
that g 1 ( x )
= ( (cid:98) θ (cid:62) 1 x ) /
( x (cid:62) (cid:98) Σ 1 x ) 12 , which recovers the value of Pr(˜ θ (cid:62) x ≤ 0) under the Gaussian distribution.
N UMERICAL E XPERIMENTS
We compare extensively the performance of our DiRRAc model (2) and Gaussian DiRRAc model
(8) against four strong baselines: ROAR (Upadhyay et al., 2021), CEPM (Pawelczyk et al.,
, AR (Ustun et al., 2019) and Wachter (Wachter et al., 2018).
We conduct the experiments on three real datasets (German, SBA, Student).
Appendix A provides further comparisons with Karimi et al.
(2021a)
and ensemble variants of ROAR, along with the sensitivity analysis of hyperparameters.
Appendix A also contains the details about the datasets and the experimental setup.
Metrics.
For all experiments, we use the l 1 distance c ( x, x 0 )
= (cid:107) x − x 0 (cid:107) 1 as the cost function .
Each dataset contains two sets of data (the present and shifted data).
The present data is to train the current classiﬁer for which recourses are generated while the remaining data is used to measure the validity of the generated recourses under model shifts.
We choose 20% of the shifted data randomlytimes and train 100 classiﬁers respectively.
The validity of a recourse is computed as the fraction of the classiﬁers for which the recourse is valid.
We then report the average of the validity of all generated recourses and refer this value as M 2 validity .
We also report M 1 validity , which is the fraction of the instances for which the recourse is valid with respect to the original classiﬁer.
Results on real-world data.
We use three real-world datasets which capture different data distribution shifts (Dua & Graff, 2017): (i) the German credit dataset, which captures a correction shift.
(ii) the Small Business Administration (SBA) dataset, which captures a temporal shift.
(iii) the Student performance dataset, which captures a geospatial shift.
Each dataset contains original data and shifted data.
We normalize all continuous features to [0 , 1] .
Similar to Mothilal et al.
, we use one-hot encodings for categorial features, then consider them as continuous features in [0 , 1] .
To ease the comparison, we choose K = 1 .
The choices of K are discussed further in Appendix A.
[Figure]
cost 0.
M 2 v a li d i t y
German 1.
[Figure] [Figure]
SBA
[Figure]
[Figure] [Figure]
Student
DiRRAc
ROAR
[Table]
We split 80% of the original dataset and train a logistic classiﬁer.
This process is repeated independently 100 times to obtain 100 observations of the model parameters.
Then we compute the empirical mean and covariance matrix for ( (cid:98) θ 1 , (cid:98) Σ 1 ) .
To evaluate the trade-off between l 1 cost and
M 2 validity of DiRRAc and ROAR, we compute l 1 cost and the M 2 validity by running DiRRAc with varying values of δ add and ROAR with varying values of λ .
We deﬁne δ = δ min + δ add , δ min is speciﬁed in (7).
Figure 3 shows that the frontiers of DiRRAc dominate the frontiers of ROAR.
This indicates that DiRRAc achieves a far smaller l 1 cost for the robust recourses than ROAR.
Next, we evaluate the l 1 and l 2 cost, M 1 and M 2 validity of DiRRAc, ROAR and other baselines.
The results in Table 1 demonstrate that DiRRAc has high validity in all three datasets, while preserving low costs ( l 1 and l 2 cost) in comparison to ROAR.
Our DiRRAc framework consistently outperforms the AR, Wachter and CEPM in terms of M 2 validity.
Nonlinear models.
Following the previous work as in Rawal et al.
and Upadhyay et al.
(2021), we adapt our DiRRAc framework and other baselines (AR and ROAR) to non-linear models by ﬁrst generating local linear approximations using LIME (Ribeiro et al., 2016).
For each instance x 0 , we ﬁrst generate a local linear model for MLPs classiﬁer 10 times using LIME, each time using 1000 perturbed samples.
To estimate ( (cid:98) θ 1 , (cid:98) Σ 1 ) , we compute the mean and covariance matrix of parameters θ x 0 of 10 local linear models.
We choose randomly 10% of shifted dataset and concatenate with training data of original dataset 10 times, then train a shifted MLPs classiﬁer.
According to
Table 2.
On the German Credit and Student dataset, DiRRAc has a higher M 2 validity than other baselines, and a slightly lower M 2 validity on the SBA dataset than ROAR, while maintaining a low l 1 cost relative to ROAR and CEPM.
[Table]
Concluding Remarks.
In this work, we proposed the Distributionally Robust Recourse Action (DiRRAc) framework to address the problem of recourse robustness under shifts in the parameters of the classiﬁcation model.
We introduced a distributionally robust optimization approach for generating a robust recourse action using a projected gradient descent algorithm.
The experimental results demonstrated that our framework has the ability to generate the recourse action that has high probability of being valid under different types of data distribution shifts with a low cost.
We also showed that our framework can be adapted to different model types, linear and non-linear models, and allows for actionability constraints of the recourse action.
Remark 5.1 (Extensions) .
The DiRRAc framework can be extended to hedge against the misspeciﬁcation of the mixture weights (cid:98) p .
Alternatively, the objective function of DiRRAc can be modiﬁed to minimize the worst-case component probability.
These extensions are explored in Section C.
Corresponding extensions for the Gaussian DiRRAc framework are presented in Section D.
Remark 5.2 (Choice of ambiguity set) .
This paper’s results rely fundamentally on the design of ambiguity sets using a Gelbrich distance on the moment space.
This Gelbrich ambiguity set leads to the (cid:107)·(cid:107) 2 -regularizations of the worst-case Value-at-Risk in Lemmas 3.3 and 4.2.
If we consider other moment ambiguity sets, for example, the moment bounds in Delage & Ye (2010) or the KullbackLeibler-type sets in Taskesen et al.
(2021), then these regularization equivalence are not available, and there is no trivial way to extend the results to reformulate the (
Gaussian) DiRRAc framework.
A A DDITIONAL E XPERIMENT R ESULTS A. D ATASETS
Real-world datasets.
We use three real-world datasets which are popular in the settings of robust algorithmic recourse: German credit (Dua & Graff, 2017), SBA Li et al.
(2018), and Student performance Cortez & Silva (2008).
We select a subset of features from each dataset:
• For the German credit dataset from the UCI repository, we choose ﬁve features: Status, Duration, Credit amount, Personal Status, and Age.
We found in the descriptions of two datasets that feature Status in the data correction shift dataset corrects the coding errors in the original dataset (Dua & Graff, 2017).
• For the SBA dataset, we follow Li et al.
and Upadhyay et al.
and we choosefeatures: Selected, Term, NoEmp, CreateJob, RetainedJob, UrbanRural, ChgOffPrinGr, GrAppv, SBA Appv, New, RealEstate, Portion, Recession.
We use the instances during 1989-2006 as original data and the remaining instances as shifted data.
• For the Student Performance dataset, motivated by Cortez & Silva (2008), we choose G3 - ﬁnal grade for deciding the label pass or fail for each student.
The student who has G3 < 12 is labeled 0 (failed) and 1 (passed) otherwise.
For input features, we choose 9 features: Age, Study time, Famsup, Higher, Internet, Health, Absences, G1, G2.
We separate the dataset into the original and the geospatial shift data by 2 different schools.
We report the accuracy of the current classiﬁers and shifted classiﬁers for two types of models: logistics classiﬁers (LR) and MLPs classiﬁers (MLPs) on each dataset in Table 3.
[Table]
Synthetic data.
We synthesize two-dimensional data and simulate the shifted data by using K =different shifts similar to Upadhyay et al.
(2021): mean shift, covariance shift, mean and covariance shift.
First, we ﬁx the unshifted conditional distributions with X | Y =
y ∼ N ( µ y , Σ y ) ∀
y ∈ Y .
For mean shift, we replace µ 0 by µ shift= µ
+ [ α, 0] (cid:62) , where α is a mean shift magnitude.
For covariance shift, we replace Σ 0 by Σ shift= (1 + β )Σ 0 , where β is a covariance shift magnitude.
For mean and covariance shift, we replace ( µ 0 , Σ 0 ) by ( µ shift 0 , Σ shift 0 ) .
We generate 500 samples each class from the unshifted distribution with µ 0 =
[ − 3; − 3] , µ 1 = [3; 3] , and Σ
= Σ 1 = I .
To visualize the decision boundaries of the linear classiﬁers for synthetic data, we synthesize the shifted data in total 100 times including 33 mean shifts, 33 covariance shifts and 34 both shifts, then we visualize the 100 model’s parameters in a two-dimensional space in Figure 4 and Figure 5.
[Figure]
[Figure]
A. E XPERIMENTAL SETTINGS
Implementation details.
For all the baselines, we use the implementation of CARLA (Pawelczyk et al., 2021).
We use the hyperparameters of AR, Wachter and CEPM that are provided by CARLA.
For ROAR, we use the same parameters as in ROAR (Upadhyay et al., 2021).
Experimental settings.
The experimental settings for the experiments in the main text are as follows:
• In Figure 3, we ﬁx ρ 1 = 0 .
1 and vary δ add ∈
[0 , 2 . 0] for DiRRAc.
Then we ﬁx δ max = 0 .
1 and vary λ ∈ [0 . 01 , 0 . 2] for ROAR.
• In Table 1 and Table
we ﬁrst initialize ρ 1 = 0 .
and we choose the δ add that maximizes the M 1 validity.
We follow the same procedure as in original paper for ROAR (Upadhyay et al., 2021): choose δ max = 0 .
1 and ﬁnd the value of λ that maximizes the M 1 validity.
The detailed settings are provided in Table 4.
[Table] [Table]
(cid:98)
[Table]
[1] ρ [0 . 1]
[Table] [Table]
Choice of number of components K for real-world datasets.
To choose K for real-world datasets, we use the same procedure in Section 5 to obtain 100 observations of the model parameters.
Then we determine the number of components K on these observations by using K-means clustering and
Elbow method (Thorndike, 1953; Ketchen & Shook, 1996).
Then we train a Gaussian mixture model on these observations and obtain (cid:98) p k , (cid:98) θ k , (cid:98) Σ k for the optimal number of components K .
The Elbow method visualization for each dataset is shown in Figure 6.
[Figure]
[Table]
The results in Table 5 indicate that as we deploy our framework with the optimal number of components K , then DiRRAc deliver a smaller cost in all three datasets.
The M 2 validity of Gaussian
DiRRAc slightly increases in the Student Performance dataset.
Sensitivity analysis of hyperparameters δ add and ρ k .
Here we analyze the sensitivity of the hyperparameters δ add and ρ k to the l 1 cost of recourses and M 2 validity of DiRRAc.
From the results in Figure 3, we can observe that as δ add increases, both the cost and the robustness of the recourse increase.
We study the sensitivity of hyperparameters ρ k to M 2 validity by ﬁrst ﬁxing the δ add = 0 .
1 and vary ρ k ∈ [0 . 0 , 0 . 5] .
According to Figure 7, we can observe that as ρ k increases, the cost of recourses rises as well, yielding in more robust recourses.
[Figure]
A. R ESULTS ON REAL - WORLD DATA
Experiments with prior on (cid:98) Σ .
In some cases, we presume, we may not have access to the training data.
We set (cid:98) θ 1 = θ 0 , where θ 0 is the parameters of the original classiﬁer.
Then we choose (cid:98) Σ 1 = τI with τ = 0 . 1 .
We generate recourse for each input instance and compute the M 1 validity using the original classiﬁer and the M 2 validity using the shifted classiﬁers.
The results in Table 6 show that our methods produce the same performance while at the same time keeping the l 1 and l 2 cost lower than ROAR in all three datasets.
[Table]
Experiments with actionability constraints.
Using our two methods (DiRRAc and Gaussian
DiRRAc) and the AR method (Ustun et al., 2019)
[Table]
we analyze how the actionability constraints affect the cost and validity of the recourse.
We select a subset of features from each dataset and deﬁne each feature as immutable or non-decreasing as follows:
• In the German credit dataset, we select Personal status as an immutable attribute because it is challenging to impose changes in an individual‘s status and sex.
We view age as a non-decreasing feature.
In the SBA dataset, we select UrbanRural and Recession as two immutable attributes since it will be difﬁcult to change these features in the near future.
RetainedJob is another feature that we view as non-decreasing.
In the Student Performance dataset, we assume that a student’s
Higher education would not change, and select higher education as an immutable feature.
Age and Absences are considered as non-decreasing.
The above speciﬁcations are aligned with the existing numerical setup in algorithmic recourse (Ustun et al., 2019; Rawal & Lakkaraju, 2020a)
[Table]
For each dataset, we run the process of generating the recourse action by adding constraints to the projected gradient descent algorithm.
The experimental setup on three different real-world datasets is the same as in Section 5.
The results in Table 7 indicate that the M 2 validity of our 2 methods drops in the German Credit dataset.
The validity in shifted data of AR also decreases in this dataset.
In other datasets, the performance of our 2 methods remains the same.
The l 1 and l 2 cost of DiRRAc slightly increase in the Student Performance dataset.
Furthermore, there exists recourse for every input instance
[Table] [Table]
Comparison with RBR.
Here we compare our approach on the nonlinear model settings to a more recent approach on robust recourse (Nguyen et al., 2022)
[Table] [Table]
We provide the results in Table 8: we can observe that RBR has (nearly) perfect M 1 validity.
This result is natural because RBR is designed to handle the nonlinear predictive model directly.
Our methods do not have the perfect M 1 validity because we use the LIME approximation.
However, it is important to note that in the problem of robust recourse facing future model shifts, we regard the M 2 validity as the most crucial metric because it is the proportion of recourse instances that are valid with respect to the shifted (future) models.
In terms of l 1 cost and M 2 validity, the results demonstrate that our method has a competitive performance compared to the existing state-of-the-art methods.
In particular, LIME-DiRRAc outperforms
RBR in terms of M 2 validity for two datasets (German and Student).
In the SBA dataset, our approach has a lower M 2 validity, but the cost of recourses generated by our method is also lower.
This result is consistent with our discussion about the l 1 cost and M 2 validity trade-off in the Appendix.
Comparison with MINT on German Credit datasets.
We add a more recent baseline MINT proposed by Karimi et al.
(2021a)
for comparison purpose.
MINT requires a causal graph, thus we restrict the experiment to the German Credit dataset (the speciﬁcations of the causal graphs are not available for SBA and Student Performance).
We use the same set of features as in the MINT and
ROAR paper (Karimi et al., 2021a; Upadhyay et al., 2021) with 4 features: Sex, Age, Credit Amount and Duration.
The results in Table 9 demonstrate that the recourse generated by our framework is more robust to model shifts, but it has higher l 1 cost.
[Table]
Comparion with ensemble baselines.
Prior work suggested that model ensembles can be effective for out-of-distribution prediction (Ovadia et al., 2019; Fort et al., 2019).
Now we explore a model ensemble method to generate recourse based on ROAR as follows.
First we follow the procedure in Section 5 to obtain 100 model parameters θ i with i ∈ { 1 , . . .
Then we ﬁnd recourse by solving the following problem:
[Equation]
where (cid:96) is the cross-entropy loss function.
Second, we use the same 100 models and generate recourse for each model independently.
Then we average the ROAR recourses across those 100 models as follows.
[Equation]
[Table] [Table]
In Table 10, we provide results for the ROAR ensemble method as ROAR-Ensemble and the average
ROAR recourses as ROAR-Avg.
From this table, the M 1 and M 2 validity of ROAR-Ensemble and
ROAR-Avg remain the same for all datasets.
In almost every benchmark, the recourses generated by those two approaches are more costly than ROAR.
In comparison with our framework, our DiRRAc and Gaussian DiRRAc methods demonstrate advantages in terms of cost of recourses.
More discussions about cost-validity trade-off.
Previous work about robust recourses have suggested that recourses are more robust with the expense of higher costs (Rawal et al., 2021;
Upadhyay et al., 2021; Pawelczyk et al., 2020; Black et al., 2022).
Our results with DiRRAc and Gaussian
DiRRAc are consistent with this suggestion.
However, our framework can achieve robust and actionable recourses with a far smaller cost than ROAR (Upadhyay et al., 2021) and CEPM (Pawelczyk et al., 2020).
Comparison of run time.
Table 11 reports the average run time: we observe that Wachter has the smallest run time, and our (Gaussian) DiRRAc has a smaller run time than ROAR in all datasets.
[Table]
A. R ESULTS ON SYNTHETIC DATA
We deﬁne the adaptive mean and covariance shift magnitude as α = µ adapt × iter , β = Σ adapt × iter with µ adapt , Σ adapt are the factor of data shifts, iter is the index of iterative loop of synthesizing process.
[Figure]
ROAR
For data distribution shifts, we generate mean shifts and covariance shifts 50 times each type with adaptive mean and covariance shift magnitude, with the parameters µ adapt = Σ adapt = 0 . 1 .
To estimate (cid:98) θ k and (cid:98) Σ k , we deﬁne valid mixture weights (cid:98) p and generate data for each component for 100 times with the same ratio as the mixture weight.
We train 100 logistic classiﬁers to compute the empirical mean (cid:98) θ k and the empirical covariance matrix (cid:98) Σ
k for the k -th component.
We generate a recourse for each test instance that belongs to the negative class.
In Figure 8, we present the results of the cost-robustness analysis of DiRRAc and ROAR on synthetic data.
The results in Figure 9 show that recourses generated by our framework are robust to model shifts while the M 2 validity of recourses generated by other baselines starts to decline with a small shift magnitude.
We also provide additional results for the impact of magnitude of distribution shifts to the empirical validity with Gaussian DiRRAc model (8) in Figure 10.
B P ROOFS B. P ROOFS OF S ECTION
To prove Proposition 3.4, we are using the notion of Value-at-Risk which is deﬁned as follows.
Deﬁnition B.1 (Value-at-Risk) .
For any ﬁxed distribution Q k of ˜ θ , the Value-at-Risk at the risk tolerance level β ∈ (0 , 1) of the loss ˜ θ (cid:62) x is deﬁned as
[Equation]
[Figure] [Figure]
We are now ready to provide the proof of Proposition 3.4.
Proof of Proposition 3.4.
Using the deﬁnition of the Value-at-Risk in Deﬁnition B.1, we have
[Figure]
[Equation]
By Nguyen (2019, Lemma 3.31), we can reformulate the worst-case value-at-risk as
[Equation]
It is now easy to observe that in the ﬁrst case when − (cid:98) θ
[Equation]
2 ≥ 0 , then we should have
[Equation]
We now consider the second case when − (cid:98) θ (cid:62)
k x + [Equation] β (cid:107) x (cid:107) 2 < 0 .
It is easy to see, by the monotocity of the worst-case value-at-risk with respect to β , that the minimal value β (cid:63) should satisﬁes
[Equation]
Using the transformation t ← √ β (cid:63) , we have
[Equation]
By rearranging terms and then squaring up both sides, we have the equivalent quadratic equation
[Equation]
with A k (cid:44) − (cid:98) θ (cid:62) k x ≤ 0 , B k (cid:44) (cid:113)
x (cid:62) (cid:98) Σ k x ≥ 0 , and C k (cid:44) ρ k (cid:107) x (cid:107) 2 ≥ 0 as deﬁned in the statement of the proposition.
Note, moreover, that we also have A 2 k ≥ C 2 k .
This leads to the solution
[Equation]
Thus, we ﬁnd
[Equation]
This completes the proof.
We now provide the proof of Theorem 3.2.
Proof of Theorem 3.2.
We ﬁrst consider the objective function f of (2), which can be re-expressed as
[Equation]
where the equality in the second line follows from the non-negativity of (cid:98) p k , and the last equality follows from the deﬁnition of f k ( x ) in (5).
Applying Proposition 3.4, we obtain the objective function of problem (4).
Consider now the last constraint of (2).
Using the result of Proposition 3.4, this constraint is equivalent to
[Equation]
This leads to the feasible set X as is deﬁned in (3).
This completes the proof.
B. P ROOFS OF S ECTION
To prove Theorem 4.1, we ﬁrst deﬁne the following worst-case Gaussian component probability function
[Equation]
The next proposition provides the reformulation of f N k .
Proposition B.2 (Worst-case probability - Gaussian) .
For any x ∈ R d , any k ∈ [ K ] and any ( (cid:98) θ k , (cid:98) Σ k , ρ k ) ∈
R d × S d + × R + , deﬁne the following constants A k (cid:44) − (cid:98) θ (cid:62) k x , B k (cid:44) (cid:113) x (cid:62) (cid:98)
Σ k x , and
C k (cid:44) ρ k (cid:107) x (cid:107) 2 .
The following holds:
(i)
We have f N k ( x ) < 12 if and only if A k + C k < 0 .
(ii)
If x satisﬁes f N k ( x ) < 12 , then
[Equation]
Proof of Proposition B.2.
We ﬁrst prove Assertion (i).
Pick any Q k ∈ B N k
( (cid:98) P k ) , then Q k is a Gaussian distribution Q k ∼ N ( θ k , Σ k ) [Equation] and thus
[Equation]
Guaranteeing f N k
( x ) < 12 is equivalent to guaranteeing
[Equation]
Note that we also have
[Equation]
by the properties of the dual norm.
This leads to the equivalent condition that A k + C k < 0 .
We now prove Assertion (ii).
Using the deﬁnition of the Value-at-Risk in Deﬁnition B [Equation] 1, we have
[Equation]
Using the result from Nguyen (2019, Lemma 3.31), we have sup
[Equation]
with t =
Φ − 1 (1 − β ) .
Taking the inﬁmum over β is then equivalent to ﬁnding the root of the equation
[Equation]
Using a transformation τ = 1 /t , the above equation becomes
[Equation]
with solution
[Equation]
Notice that A k + C k < 0 , and we also have A 2 k > C 2 k , thus τ is well-deﬁned.
The result now follows by noticing that f N k ( x ) = 1 − Φ( t ) = 1 − Φ(1 /τ ) .
We are now ready to prove Theorem [Equation]
[Equation]
Proof of Theorem 4.1.
Problem (8) is equivalent to
[Equation]
Applying Proposition B.2, we obtain the necessary result.
C E XTENSIONS OF THE D I RRA C F RAMEWORK
Throughout this section, we explore two extensions of our DiRRAc framework.
In Section C.1, we study an additional layer of robustiﬁcation with respect to the mixture weights (cid:98) p .
Next, in
Section C.2, we consider an alternative formulation of the objective function to minimize the worstcase component probability.
C. R OBUSTIFICATION AGAINST M IXTURE W EIGHT U NCERTAINTY
The DiRRAc problem considered in Section 3 only robustiﬁes the component distributions (cid:98) P k .
We now discuss a plausible approach to robustify against the misspeciﬁcation of the mixture weights (cid:98) p .
Because the mixture weights should form a probability vector, it is convenient to model the perturbation in the mixture weights using the φ -divergence.
Deﬁnition C.1 ( φ -divergence) .
Let φ : R → R be a convex function on the domain R + , φ (1) = 0 , 0 × φ ( a/ 0) =
a × lim t ↑∞ φ ( t ) /t for a > 0 , and 0 × φ (0 / 0) = 0 . The φ -divergence D φ between two probability vectors p, (cid:98) p ∈ R K + amounts to D φ ( p (cid:107) (cid:98) p ) (cid:44) (cid:80) k ∈ [ K ] (cid:98) p k × φ ( p k / (cid:98) p k ) .
The family of φ -divergences contains many well-known statistical divergences such as the KullbackLeibler divergence, the Hellinger distance, etc.
Further discussion on this family can be found in Pardo (2018).
Distributionally robust optimization models with φ -divergence ambiguity set were originally studied in decision-making problems (Ben-Tal et al., 2013; Bayraksan & Love, 2015) and have recently gained attention thanks to their successes in machine learning tasks (Namkoong &
Duchi, 2017; Hashimoto et al., 2018; Duchi et al., 2021).
Let ε ≥ 0 be a parameter indicating the uncertainty level of the mixture weights.
The uncertainty set for the mixture weights is formally deﬁned as
[Equation]
which contains all K -dimensional probability vectors which are of φ -divergence at most ε from the nominal weights (cid:98) p .
The ambiguity set of the mixture distributions that hedge against the weight misspeciﬁcation is
[Equation]
that [Equation] where the component sets B k ( (cid:98) P k ) are deﬁned as in Section 3.
The DiRRAc problem with respect to the ambiguity set U ( (cid:98) P ) becomes
[Equation]
It is important to note at this point that the feasible set of (12) coincides with the feasible set of (2).
Thus, to resolve problem (12), it sufﬁces to analyze the objective function of (12).
Given the function φ , we deﬁne its conjugate function φ ∗ : R → R ∪ {∞} by
[Equation]
The next theorem asserts that the worst-case probability under U ( (cid:98) P ) can be computed by solving a convex program.
Theorem C.2 (Objective value) .
The feasible set of problem (12) coincides with X .
Further, for every x ∈ X , the objective value of (12) equals to the optimal value of a convex optimization problem
[Equation]
where f k ( x ) are computed using Proposition 3.4.
Proof of Theorem C.2.
From the deﬁnition of the set U ( (cid:98) P ) [Equation] we can rewrite F using a two-layer decomposition [Equation]
[Equation]
where the equality in the second line follows from the non-negativity of p k , and the last equality follows from the deﬁnition of f k ( x ) in (5).
By applying the result from Ben-Tal et al.
Corollary 4.2), we have
[Equation]
The proof is complete.
From the result of Theorem C.2, we can derive the gradient of the objective function of (12) using
Danskin’s theorem Shapiro et al.
(2009, Theorem 7.21), or simply using auto-differentiation.
Furthermore, φ ∗ is convex, and thus solving the minimization problem in Theorem C.2 can be done efﬁciently using convex optimization algorithms.
C. M INIMIZING THE W ORST -C ASE C OMPONENT P ROBABILITY
Instead of minimizing the (total) probability of unfavorable outcome, we can consider an alternative formulation where the recourse action minimizes the worst-case conditional probability of unfavorable outcome over all K components.
Mathematically, if we opt for the component ambiguity sets
B k ( (cid:98) P k ) constructed in Section 3, then we can solve
[Equation]
Interestingly, problem (13a) does not involve the mixture weighs (cid:98) p .
As a consequence, a trivial advantage of this model is that it hedges automatically against the misspeciﬁcation of (cid:98) p .
To complete, we provide its equivalent ﬁnite-dimensional form.
Corollary C.3 (Component Probability DiRRAc) .
Problem (13a) is equivalent to
[Equation]
D E XTENSIONS OF THE G AUSSIAN D I RRA C F RAMEWORK
In this section, we leverage the results in Section C to extend the Gaussian DiRRAc framework to (i) handle the uncertainty of the mixture weight and (ii) minimize the worst-case modal probability.
Remind that each individual mixture ambiguity set B N k ( (cid:98) P k ) is of the form
[Equation]
which is a ball in the space of Gaussian distributions.
D. H ANDLING M IXTURE W EIGHT U NCERTAINTY - G AUSSIAN D I RRA C
Following the notations in Section C.1, we deﬁne the set of possible mixture weights as
[Equation]
and the ambiguity set with Gaussian information is deﬁned as
U N ( (cid:98) P ) = (cid:110) Q : ∃ p ∈ ∆ , ∃ Q k ∈ B N k ( (cid:98) P k ) ∀ k ∈
[ K ] such that Q ∼ ( Q k , p k ) k ∈
[ K ] (cid:111) .
The distributionally robust problem with respect to the ambiguity set U ( (cid:98) P ) is
[Equation]
Following the results in Section 4, the feasible set of (14) coincides with the set X .
It sufﬁces now to provide the reformulation for the objective function of (14) [Equation]
Corollary D.1.
For any x ∈ X , we have
[Equation]
where the values f N k ( x ) are obtained in Proposition B.2.
Corollary D.2 follows from Theorem D.2 by replacing the quantities f k ( x ) by f N k ( x ) to take into account the Gaussian parametric information.
The proof of Corollary D.2 is omitted.
D. M INIMIZING W ORST -C ASE C OMPONENT P ROBABILITY
We now consider the Gaussian DiRRAc that minimizes the worst-case modal probability of infeasibility.
More concretely, we consider the recourse action obtained by solving
[Equation]
15a
[Equation]
The next corollary provides the equivalent form of the above optimization problem.
Corollary D.2.
Problem (15a) is equivalent to
[Equation]
E P ROJECTED G RADIENT D ESCENT A LGORITHM
The pseudocode of the algorithm is presented in Algorithm 1.
The convergence guarantee for Algorithm 1 follows from Beck (2017, Theorem 10.15), and is distilled in the next theorem.
Algorithm 1 Projected backtracking
Input: Input instance x 0 , feasible set X ε and objective function f
Line search parameters: λ ∈ (0 , 1) , ζ > 0 (Default values: λ = 0 .
7 , ζ = 1 )
Initialization: Set x 0 ← Proj X ε ( x 0 ) for t = 0 , . .
, T − 1 do
Find the smallest integer i ≥ 0 such that f
[Equation]
−
[Equation]
Set x t +1 =
Proj X ε
( x t − λ i ζ ∇ f ( x t )) .
end for
Output: x T
Theorem E.1 (Convergence guarantee) .
Let { x t } t =0 , 1 ,...,T be the sequence generated by Algorithm 1.
Then, all limit points of the sequence { x t } t =0 , 1 ,...,T are stationary points of problem (4) with the modiﬁed feasible set X ε .
Furthermore, there exists some constant C > 0 such that for any
T ≥ 1 , we have
[Equation]