Towards Explaining Distribution Shifts
Authors: Sean Kulinski
David I. Inouye
Venue: Submitted to ICLR 2023 (2023)
Keywords: Distribution Shifts
Explainable AI

Abstract: Distribution shift can have fundamental consequences such as signaling a change in the operating environment or significantly reducing the accuracy of downstream models. Thus, understanding such distribution shifts is critical for examining and hopefully mitigating the effect of such a shift. Most prior work has focused on merely detecting if a shift has occurred and assumes any detected shift can be understood and handled appropriately by a human operator. We hope to aid in these manual mitigation tasks by explaining the distribution shift using interpretable transportation maps from the original distribution to the shifted one. We derive our interpretable mappings from a relaxation of the optimal transport problem, where the candidate mappings are restricted to belong to a set of interpretable mappings. We then use quintessential examples of distribution shift in simulated and real-world cases to showcase how our explanatory mappings provide a better balance between detail and interpretability than the de facto standard mean shift explanation by both visual inspection and our PercentExplained metric.

Under review as a conference paper at ICLR 2023
TOWARDS EXPLAINING DISTRIBUTION SHIFT
Anonymous authors Paper under double-blind review
ABSTRACT
A distribution shift can have fundamental consequences such signaling change in the operating environment or significantly reducing the accuracy of downstream models. Thus, understanding distribution shifts is critical for examining and hope- fully mitigating the effect of such a shift. Most prior work has focused on merely detecting if a shift has occurred and assumes any detected shift can be understood and handled appropriately by human operator. We hope to aid in these man- ual mitigation tasks by explaining the distribution shift using interpretable trans- portation maps from the original distribution to the shifted one. We derive our interpretable mappings from a relaxation of the optimal transport problem, where the candidate mappings are restricted to a set of interpretable mappings. We then use quintessential examples of distribution shift in simulated and real-world cases to showcase how our explanatory mappings provide better balance between de- tail and interpretability than the de facto standard mean shift explanation by both visual inspection and our PercentExplained metric.
INTRODUCTION
Most real-world environments constantly changing, and in many situations, understanding how a specific operating environment has changed is crucial to making decisions respective to such a change. Such change might be new data distribution seen in deployment which causes machine learning model to begin to fail. Another example is a decrease in monthly sales data which could be due to temporary supply chain issue in distributing a product could mark shift in consumer preferences. When these changes are encountered, the burden is often placed on human operator to investigate the shift and determine the appropriate reaction, if any, that needs be taken, In this work, our goal is to aid these operators in providing an explanation of such change.
This ubiquitous phenomenon of having a difference between related distributions is known as dis- tribution shift. Much prior work focuses on detecting distribution shifts; however, there is little prior work that looks into understanding a detected distribution shift. As it is usually solely up to an operator investigating flagged distribution shift to decide what to do next, understanding the shift is critical for the operator to more efficiently mitigate any harmful effects of the distribution shift. Without defined approach to this task, the current de facto standard in analyzing a shift is looking at how the mean of the original, source, distribution shifted to the new, target, distribution. However, this simple explanation (e.g...Fi.22 2}. Further, in high-dimensional regimes, a shift in means could be uninterpretable due to its high dimensionality. Instead, if after flagging that shift has occurred, we could automatically provide more detailed information about the shift but still remain at level that is interpretable, we could reduce the manual load on the operator to understand the shift, and, ultimately, to take action if necessary.
miss crucial shift information due to being a coarse summary
Therefore, we propose novel framework for explaining distribution shifts, such as showing how features have changed or how groups within the distributions have shifted. Since a distribution shift can be seen as a movement from a source distribution æ Pare to a target distribution y ~ Ptgt, we define a distribution shift explanation as a transport map T(x) which maps point in our source distribution onto a point in our target distribution. For example, under this framework, the typical distribution shift explanation via mean shift can be written as T(œ) Intuitively, these transport maps can be thought of as a functional approximation of how the source distribution could have moved in distribution space to become without making assumptions on the type of shift, there exist many possible mappings that explain the shift (see subsection A.2 for examples). Thus, we leverage prior optimal transport work to define an
+ (Ny (lx).
target distribution. However,

---

Under review as a conference paper at ICLR 2023
Given
Baseline Mean Meyult, [ntrurp, wornnn delhipps 000, #ie day givemneer, 1o0, rile, +ick)
Expliin
Qussendde
O1 (weren, mma,, spoophe (kl which alge 7.35 of todi comments
Of (0r femure, tjst, rdon",, +haad, rmatel which aoceunts for it 65% ofe
Are sentiol
Tippraaze?
Oncic Soit from
Daseline idean Sppare:
Prosceed cckkeer Explass
Intepeentle sulls (bn)
e-duster maaps
Paxe
interpetatle (nd) imagrs)
Sobe Iey luoicn tranilation
T
Fondonn Set Explar:
Dunnbul and Traviation Explarc
maps
Methods
Figure 1: An overall look at our approach to explaining distribution shifts, where given a source Pare and shifted Pigt dataset pair, a user can choose to explain the distribution shift using k-sparse maps (which are best suited for high dimensional or feature-wist wise complex data), k-cluster maps (for tracking how heterogeneous groups change across the shift), or distribution translation maps (for data which has uninterpretable raw features such as images). For details on the results seen in the three boxes, please see experiments in Section 5) and (Section 6
www.
ideal distribution shift explanation and develop practical algorithms for finding and validating such maps. We summarize our contributions as follows:
In /ection 3
we define interpretable transport maps by constraining a relaxed form of the optimal transport problem to only search over set of interpretable mappings and suggest possible interpretable sets.
In Section 4
we develop practical methods for finding such interpretable mappings which allow to adjust the interpretability of an explanation to fit the needs of a situation.
In Section 5] we show empirical results on real-world tabular datasets demonstrating how our explanations and our PercentExplained metric can aid an operator in understanding how a distribution has shifted.
In Section 6 we use latent transport mappings and Image-to-Imane translation methods to extend this approach to explain image-basee shifts such as investigating how the staining of histopathological images varies across hospital network.
RELATED WORKS
The characterization of the problem of distribution shift has been extensively studied (Quiñonero-
Candela et al. 2009, Storkey, 2009) Moreno- Torres et al. 2012) via breaking down a joint distri- bution P(æ, M) of features and outputs y. into conditional factorizations such as P (z) P(x)) or P(x))) For covariate shift (Sugiyama et al. 2007) the P(x) marginal differs from source to target, but the output conditional P(y x the same, while label shift (also known prior probability shift) (Zhang et al. 2013; Lipton et al. 2018) is when the Pty) marginals differ from source to target, but the full- feature conditional P yy) remains the same. In this work, refer to general problem distribution shift, i.e. a shift in the joint distribution (with no distinction between and æ), and leave applications of explaining specific sub-genres of distribution shift to future work.
As far as we are aware, this is the first work specifically tackling explaining distribution shifts; however, there are distinct works that can be applied to explain distribution shifts. For exam- ple, one could use feature attribution methods SSarrela & Jauhiainen (2021) Molnar (2020) on
domain/distribttion classifier (e.g.. Shhnnhag et al. (2021) uses Shapley values (Shapley (1997) to
explain how changing input feature distributions affect a classifier's 's behavior), or once could find samples which are most illustrative of the differences between distributions Brockmeier al.) (202T).
Additionally, one could use counterfactual generation methods Karras et al. (2019); (auee & Geiger (202T:: Pawelczyk et al. (2020) and apply them for "distributional counterfactuals" which would show what sample from Pege would have looked like if instead came from Pare (e.g., cc.g..Pawelczyk ctt al. (2020) uses a classifier- guided VAE to generate class counterfactuals on tabular data). We explore this distributional counterfactual explanation approach in subsection 6.2
A sister field is that of detecting distribution shifts, This is commonly done using methods such as statistical hypothesis testing of the input features (Nelson 2003 Rabanser al. 2018 Quiñonero- Candela et al. 2209)), training a domain classifier to test between source and non-source domain samples Lipton et al. (2018), etc. In (Kulinski et al. 2020 Budhathoki et al. 2021). the authors
2

---

Under review as a conference paper at ICLR 2023
attempt to provide more information beyond the binary "has shift occurred?" via localizing shift to a subset of features or causal mechanisms. (Kulinski et al., 2020) does this by introducing the notion of Feature Shift, which first detects if shift has occurred and if so. localizes that shift to a specific subset of features that have shifted from source to target. In (Budhathoki et al.) (021). the authors take causal approach via individually factoring the source and target distributions into a product of their causal mechanisms (i.e. a variable conditioned on its parents) using a shared causal graph, which is assumed be known/discoverable. Then, the authors "replace" a subset of causal mechanisms from Pare with Pige, and measure divergence from Pere (i.e. measuring how much the subset change affects the source distribution). Both of these methods are still focused on detecting distribution shifts (via identifying shifted causal mechanisms or feature-level shifts), unlike explanatory mappings which help explain how the data has shifted.
EXPLAINING DISTRIBUTION SHIFTS VIA TRANSPORT MAPS
The underlying assumption of distribution shift is that there exists relationship between the source and target distributions. From distributional standpoint, we can view distribution shift as a move- ment, or transportation, of samples from the source distribution Pere to the target distribution Pigt- Thus, we can capture this relationship between the distributions via transport map T from the source distribution to the target, i.c., if x ~ Pare. then T( ~ Pigt. If this mapping is understand- able to operator investigating distribution shift, this can serve on what changed between the environments; thus allowing for more effective reactions to the shift. Therefore, we claim shift explanation to be: an interpretable transport map T which approximately maps a source distribution Pare onfo a target distribution Pegt such that T Psre R Ptgt- Similar to ML. model interpretabilityy |Monar (2020), interpretable map can either be that is intrinsically interpretable (subsection 3.1) or a mapping which is explained via post-hoc methods such as sets of input-output pairs (subsection 6.2).
explanation to the operator
3.1 INTRINSICALLY INTERPRETABLE TRANSPORIATION MAPS
In order to find such mapping between distributions, it is natural to look to Optimal Transport (OT) and its extensive prior work in this field Cuturi 2013); Arjovsky et al. 2017); Torres et al. (2021); Peyré & Cuturi 2019). An OT mapping is defined as a method of optimally moving points from
one distribution to another given a transport cost function c and is formally defined as:
Tor = arg min Ep... [c(æ, T(x))] s.t.TyPPre = Pigt
T
where T.P are is the push forward operator that can be viewed as applying the T mapping onto all points in Perc. and T.Pare are Pigt is the marginal constraint, meaning that Tor must perfectly align the source distribution and the target.
OT is natural starting point for shift explanations as it allows for rich geometric structure on the space of distributions, and by finding a mapping that minimizes a transport cost we are forcing our mapping to retain much information about the original æ samples when aligning Pare and Pigt- For more details about OT, please see Villani (2009); Peyré & Cuturi (2019).
However, since OT considers all possible mappings which satisfy the marginal constraint, this means the resulting Tor can be arbitrarily complex and thus possibly uninterpretable as a shift explanation. We can alleviate this by restricting the candidate transport maps to belong to a set of user-defined interpretable mappings Ω. However, this problem can be infeasible if Ω does not contain mapping that satisfies the marginal constraint. Therefore, we can use Lagrangian relaxation to relax the marginal constraint, giving us an Interpretable Transport mapping Trr:
Trr := arg arg min Ep [c(2, T T())] + X ((Pr(m)) Pegt)
(I)
TEQ
where is assumeed to be the squared Euclidean cost and +(. ).. the divergence function, is assumed to be the squared Wasserstein-2 metric, unless stated otherwise. Due to the heavily complex and context-specific nature of distribution shift, it is likely that Q would be initialized based on context. However, we suggest two general methods in the next section as a starting point and hope that future work can build upon this framework for specific contexts.
3.2 INTERPRETABLE TRANSPORT SETS
The current common practice for explaining distribution shifts is comparing the means of the source and the target distributions. The mean shift explanation be generalized as Qyecuer (T T(x)
3

---

Under review as a conference paper at ICLR 2023
x +++ where ó is constant vector and mean shift being the specific case where ò is the difference of the source and target means. By letting 6 be a function of æ, which further generalizes the notion of mean shift by allowing each point to move variable amount per dimension, we arrive at transport set that includes any possible mapping T : IR. + TRd.. However, even a simple transport set like Avector can yield uninterpretable mappings in high dimensional regimes (e.g., a shift vector of over 100 dimensions). To combat this, we can regulate the complexity of mapping by forcing it only move points along a specified number of dimensions. We define this k-Sparse Transport:
k-Sparse Transport: For a given class of transport maps, S2 and a given l E {I,..., d}, we can find subset Dxpares which is the set of transport maps from & which only transport points along k dimensions or less. Formally, we define an active set A to be the set of dimensions along which a given T moves points: A(T) - {i E (l....,d d} kæ, Tw)) Ij + 0}. Then, we define 1.60 Opprrse {T Ω [A(T)) S k}.
k-sparse transport is most useful in situations where a distribution shift has happened along a subset of dimensions, such as explaining a shift where some sensors in a network are picking up a change in an environment. However, in situations where points shift in different directions based on their original value, e.g., when investigating how a heterogeneous population responded to an advertising campaign, s-sparse transport is not ideal. Thus, we provide shift explanation that breaks the source and target distributions into & sub-populations and provides a vector-based shift explanation per cot-population. We define this as cluuste transport:
k-Cluster Transport Given & E (1...., d) we define k-cluster transport to be a mapping which moves each point æ by constant vector which is specific to æ's cluster. More formally, we de- fine labeling function oa;; M) & arg min, ||m, æl12, which returns the index of the col- umn in M (i.e. the label of the cluster) which x is closest to. With this, we define (A) (T:
cluster
+ (o( M E IRdxk, [Rdd }. where 5, is the jth column of A.
Since measuring the exact interpretability of mapping heavily context-dependent, we can instead use k in the above transport maps to define partial ordering of interpretability of mappings within a class of transport maps. Let ki and k2 be the size of the active sets for k-sparse maps (or the number of clusters for k-cluster maps) of Ti and T2 respectively. If k: I k2, then Inter(Ti) > (nter(T2), where Inter(T) is the interpretability of shift explanation T. For example, we claim the interpretability of Ti E D.ppr1s is greater than (or possibly equal to) the interpretability of Te E Qsperse since a shift explanation in Q which moves points along only 10 dimensions is more interpretable than similar mapping which moves points along 100 dimensions. A similar result can be shown for k-cluster transport since an explanation of how 5 clusters moved under a shift is less complicated than an explanation of how 10 clusters moved. The above method allows to have partial ordering on interpretability without having to determine the absolute value of interpretability of an individual explanation T, as this requires expensive context-specific human evaluations, which is out of scope for this paper.
4 PRACTICAL METHODS FOR FINDING AND VALIDATING SHIFT EXPLANATIONS
In this section, we discuss practical methods for shift explanations. We first discuss using k- sparse and k-cluster maps to allow a user to automatically change the level of interpretability of a shift explanation as desired. Coupled with PercentExplained metric, this gives an operator various levels/complexities of explanation and a way to validate them. Next, we propose a practical approx- imation to Eqn. I the Interpretable Transport equation, and in Sections 4.3 and 4.4 we cover how to find the optimal explanation from D.porse and (R) 2 for this equation.
4.1 INTERPRETABILITY AS A HYPERPARAMETER
By optimizing Eqn. we can find the best shift explanation for a given set of interpretable transport maps Ω. However, directly defining a Ω which contains candidate mappings that are guaranteed to be both interpretable and expressive enough to explain a shift can be a difficult task. Thus, we can instead set Ω to be a super-class, such as Quector given in |subsection 3.2] and then adjust until (R) is found which matches the needs of the situation. This allows human operator to
4

---

Under review as a conference paper at ICLR 2023
request mapping with better alignment by increasing k which correspondingly will decrease the mapping's ss interpretability, or request a more interpretable mapping by decreasing the complexity (i.e. decreasing k) which will decrease the alignment.
To assist an operator in determining if the interpretability hyperparameter should be adjusted, we introduce PercentExplained metric, which we define to be:
PercentExplained (Paccs Pigt: T) = ( Puge) W Pige)
(2)
W2 Pige)
where (2.+)) is the squared W distance between two distributions. By rearranging terms (and ignoring the percentage scaling factor) we get WT which shows this metric's correspondence to the statistics coefficient of determination R², where (2, Pigt) is analogous to the residual sum of squares and W (Parc: P(ge) similar to the total sum of squares. This gives approximation of how much a current shift explanation T accurately maps onto target distribution. This can be seen as a normalization of a mapping's 's fidelity with the extremes being TyPare Pigt. which fully captures shift, and T = Id, which does not move the points at all. When provided this metric along with a shift explanation, an operator can decide whether to accept the explanation (e.g., the PercentExplained is sufficient and T is still interpretable) or reject the explanation and adjust k:
W.(P (P.(PrrePPep)
4.2 EMPIRICAL INTERPRETABLE TRANSPORT
Since the divergence term in Eqn. can be computationalll--eeaa expennsivv to optimize in practice, suggest an empirical approximation to the interpretable transport solution:
N
e (e())(()))) Ad (To(())TTrr((()))
min TER
N
where d is distance function such as the 12 distance or squared euclidean distance. Most notably, the divergence value in Eqn. 1 is replaced with the sum over distances between T(x) and the op- timal transport mapping for æ. This computationally attractive as the optimal transport solution only needs to be calculated once, rather than calculating the Wasserstein distance once per iter- ation like in the Interpretable Transport solution (which even if the W disiittece is approximated, can be expensive over many iterations). For optimization purposes, this is also reasonable since (o)((())) (z) upper-bounds ((Pr(x)) Pige),, when d Eg. Φ W3 and N ap- proaches the population size of Parc (proof shown in appendix).
EN, (x)() T Tor
4.3 FINDING k- SPARSE MAPS
Let be desired level of interpretability; our goal is to find the optimal k features include in our active feature set AA and then find the best transport along those features for a given transport class Ω. simple (and often ideal) approach the feature selection problem is select the & features which have the largest average shift from the source distribution to the target distribution; this approach used throughout this paper. Although the chosen T will depend on the optimization over Ω we provide two closed-form solutions that give optimal alignment for given 1 under cases where Qvector and when is all possible mappings. The mapping which gives the best alignment in (k) is k-sparse mean shift, i.e. T(x) x + ft where ğ is a vector where the jth coordinate is [Nege= = if j E A. else, it is O. When (2) is all k-sparse functions, the shift explanation which minimizes the distance term in Egq. 3ss the k--saars optimal transport solution which sets each feature in A to match that of the OT push forward for that feature, i.e. [T))], = (To((x)] if j E A, else [æ];, thus allowing for arbitrary conditional transports for features A (see proof ??. The proofs for the two previous claims can be seen in the Appendix.
4.4 FINDING C-CLUSTER MAPS
Instead of shifting respective to features, we can define & vector shifts for groups in our source domain, with the goal of explaining how each group changed from source target. To do this, we perform paired clustering in the source and target domains, so that we can relate a given cluster in Pare to its most similar counterpart in Pigt (as opposed to pushing the i clusters in Pere onto the entire target domain). With this, we construct More and Megs where the columns of M represent the k cluster means for the source and target distributions, respectively. Then, we define
5

---

Under review as a conference paper at ICLR 2023
A MMgt Mrr so that each vector shift oj is the difference in means between the 7th source and the target clusters. In practice, the set of paired clusters can be found by performing clustering in a joint Z space of Psre and (To where the resultant & cluster centroids in this space are of the form [Mares Megi]. Formally, this is done using the algorithm seen in Alg.
5 EXPERIMENTS
While in Appendix C we have experiments on simulated and simpler shifts which can be used to gain intuition on how the different shift explanation techniques work, in this section, we study the performance of our methods when applied to real-world data. We first present results using k-sparse transport our method of explaining shifts between toxic and non-toxic comments across splits from the Stanford WILDS distribution shift benchmark Koh et al. (202I) version of the "CivilCom- ments" Dataset Borkan al. (2019). We then, use k-cluster transport to explain the difference between different groups of the male population and groups of the female population in the U.S. Census "Adult Income" dataset Kohavi & Becker (1996). Finally, in the next section, we provide a general framework for explaining image-bascd shifts in high dimensional regimes (c.g., images).
Table I: A baseline vanilla mean shift explanation, k-sparse mean shift explanation, (k-p--Ex), and k-sparse OT explanations (k-OT-Ex) for the three splits from CivilComments (to save space the baseline is only used for F F-+M). Each cell represents adding/subtracting a unigram from Pare to align it with the comment distribution of Pegt and the respective PercentExplained (excluding the baseline method). For example, in k-ju-Ex(Fo-1T))) FI), adding "stupid" aligns the non-toxic female comments to the toxic female comments and cumulatively explains 0.2% of the shift.
Uasselines p(* M]
keyelbatR M)
k-UVT-EXEE M)
Anye Ex.FP. )
k-UTT-ExtFF F.)
R
R-n-Ex(M,, M.)
k-OT-Ex(M2 M2)
white
trump
2.6%
+rrmp
wumen
2,0%
+ men
men
2,8%
+ men
9,25%
2
wamen
0.1%%
+ pepople
3.7%
2
-women
0.3%)
men
4.1%
woman
wornan
2.6%
woman
10.8%
3
like
0.%
4.7%
3
black 0.3%
5.3%
white
white
2.7%
white
12.0%
4
stupid
0.2%
like
5.7%
4
tburch
0.3%)
+ people
6.3%
129K wore ennteg....
male
2.8%
people
13.0%
5
church
0.2%
meen
6.6%
5
+ stupid
0.3%)
+ Like
7.3%
29,553
martina
black
like
13.9%
6
hillary
+just
gay
t.rrmpp
29,554
dielm/uwwse
7
god
2.8%
Just
10.7%
7
black
0.2%
+ don't
8.3%
7
radsst
0.05%
+just
9,2%
29.555
femall
2.0%
male
15.4%
8
0.2%
wwitte
9.2%
8
god
0.455
1 don't
10.9%
29.556
bodyhuilder
abortton
2.8%
don't 16.3%
female
0.2%
+ man
9.9%
-jesus
29.557
+ philbiblers
10 males
2.8%
gud 16.0%
10
-borttion
0.3%)
think
10.5%
1.0
4 man
0.4%6
male
12.3%
Civil Comments Dataset Here we present results using k-sparse shifts to explain the difference between three splits of the CivilComments dataset [Brrkan et al. (2019) from the WILDS datasets Koh et al. (2021). This dataset consists of comments scraped from the internet where each comment is paired with a binary toxicity label and demographic information pertaining to the content of the comment. If we were an operator trying to see how the comments and their toxicity change across targeted demographics, we could create three splits: {F. M). [Fo. Fr}. and {Mo. M, where F represents all female comments, M are all male comments, and Fo. Fj are nontoxic, toxic female comments, respectively (and likewise for males). We can explain these three splits using vanilla mean shift, k-sparse mean shift (k-ju), and k-sparse OT (k-OT) shift explanations, as seen in Ta- ble I which shows results for the unigrams which the maximize the alignment between the unigram distributions created for each split. The baseline vanilla mean-shift explanation yields all 30K fea- tures at once (with no guide for truncating), while the s-sparse shifts provide explanations up to a k words as well as a corresponding PercentExplained aid in determining if additional words should be added to the explanation. Note that for k-p explanations, when transporting word, that word is added equally to all comments in Pares while since k-OT allows for each comment to be shifted op- timally (via conditioning on the other words in each comment), thus k-OT can explain significantly more of the shift by transporting the same word.
It is clear that performing vanilla full mean shift explanation the unigram data between splits unwise due to the high dimensionality of the data and, without further information, it is unclear when to truncate such an explanation. However, in approach, by iteratively reporting the shifted unigrams along with the cumulative PercentExplained, a practitioner can better understand the im- pact each additional word has the shift explanation. For example, it makes sense that adding "man", "men", and subtracting "woman" " were the three unigrams that best aligned the female and male comment distributions and could account for much 10% of the shift. By moving from nontoxic to toxic comments we can see a decrease in plural words such as "women" and "men" suggesting individuals are being targeted, and when narrowing to male nontoxic toxic mappings we see an increase in words such as "gay" and "racist" as opposed to Fo-+1 which shows an increase in words such as "sex" and "white".
6

---

Under review as a conference paper at ICLR 2023
Baseline Mean Shift Explanation:
Baseline Feature
High
Feature Value
Importance via Shapley
Income
Values Explanation:
Age
Per [Age: 39.7, Edu: 10.2, Inc: 0.29]
Education
[Age: 36.6, Edu: 10.1, Inc: 0.10]
Shap Value (impact model output)
-100%
4- Cluster Explanation [ours]:
1500-
Transsort Coot
Percent Expla
Mojk
[Age: 35.6, 12.9, 0.00] Meg
: [Age: 29.7, Edu: 8.9, Inc: 0.03]
2.000
50%
Hijh +
(Age: 32.1, Edu: 12.5, Inc: 0.01]
Moj-c3 IAge: 26.3, Edu: 9.0, Inc: 0.00)
40%
500
20%
Heg : [Age: 56.3, Edu: 8.4, Inc: 0.13] Meg
: [[Ag:: 43.5, Edu: 12.1, Inc: 1.00]
Meg-r]
[Age: 53.7, Edu: 8.7, Inc: 0.01]
Heg-ct [Age: 40.2, Edu: 11.9, Inc: 0.38)
Nuumbe Clsitesss (8)
10
Figure 2: Using k-cluster transport (bottom) to explain the shift from the male population to the fe- male population of the Adult Income dataset allows us to capture how heterogeneous groups within the dataset moved. For example, our bluster-based explanation (where k 4) highlights the larger income disparity between males and females with a bachelor's 's degree (edu=12) seen in C*, which is hidden in the mean shift explanation. On the other hand, comparing the means of the two distribu- tions (top left) or the Shapley values calculated from a Random Forest classifier trained to classify between the Male/Female distributions (top right) only allows us to see that the income gap predominate difference, but provide no additional actionable detail.
Adult Income Dataset This dataset originally comes from the United States 1994 Census database and is commonly used to predict whether person's 's annual income exceeds $50k using 14 demo- graphic features. Similar to BBddhathoki et al. (202I), I). we consider a subset of non-redundant fea- tures: age, years of education (where 12+ is beyond high school), and income (which is encoded as 1 if the person's 's anmual income is greater than S50k and 0 if is below). We then split this dataset along the sex dimension, and define our source distribution as the male population and the target as the female population. In order to find the set of paired clusters, we first standardize a copy of the data to have zero mean and unit variance across all features, where the μ and used for the stan- dardization are found via the feature-wiss mean and standard deviation of the source distribution and perform clustering in the standardized joint space using the method described in Section 4.4 The k clustering labels are then used to label points to clusters in the original (unstandardized) data space.
Let's assume the role of a researcher looking to implement a social program targeting gender in- equalities. We could compare the means of the male/female distributions which shows on average a 20% lower chance of having annual income above $50k when moving from the male population to the female population. Additionally, we could train a classifier to predict between male/female data points and use feature importance measurement tool like Shapley values Lundberg & Lee (2117) to determine that income is a main differing feature. However, let's say we want to dig deeper. We could instead use k-cluster transport a range of clusters, as seen in Fig. 2] If we accept the explanation at k = 4 (since beyond this, the marginal advantage of adding an additional cluster is minimal in terms of both transport cost and PercenlExplained), with this more detailed explanation we could see in Mcg, the more significant drop in high-income likelihood between middle-age adults with a bachelors degree (from males having nearly a 100% likelihood of having an income 250k only 38% chance when pushed onto the female cluster). This sub-shift, which was hidden in both the mean-shift and distribution classifier explanations, gives us a significantly narrower scope as a starting point to look deeper at the discrepancies between given population of men and women.
how heterogeneous subgroups shifted across
EXTENDING TO EXPLAINING SHIFTS IN IMAGES
Thus far, our focus has been on building foundation for explaining distribution shifts via intrinsi- cally interpretable transportation maps and showing the baseline efficacy of this approach. As seen in the Stanford Wilds dataset Koh et al. (2021), which contains benchmark examples of real-world image-based distribution shifts, image-based shifts can be immensely complex and context-speciffic even when the oracle shift is known. In order to provide an adequate intrinsically interpretable map- ping explanation of a distribution shift in image data, multiple new advancements must first be met (e.g., finding disentangled latent space with semantically meaningful dimensions, approximating high dimensional empirical optimal transport maps, etc.), which are out of scope of this paper. Thus, first present initial framework for finding intrinsically interpretable transport maps to explain
7

---

Under review as a conference paper at ICLR 2023
distribution shifts in images, which we hope future work can build upon. Then, in subsection 6.2 we use visualizations of complex distributional mappings as a post-hoc transport mäp interpretabil- ity method which alleviates the intrinsic interpretability requirement (allowing for more expressive mappings), but leaves more up to operator interpretation.
6.1 HIGH DIMENSIONAL. INTERPRETABLE TRANSPORT MAPS
In order to find interpretable transport mappings for high dimensional spaces like images, we can project Pare and Pagt onto interpretable latent space (e.g., space which has disentangled and semantically meaningful dimensions) which is learned by some (pseudo-)inverting 9 Rd + Be where k d (e.g., an autoencoder). Then, we can solve for interpretable mapping such that it aligns the distributions in the latent space, Pr(g(=)) R Piyy) For counter- factual purposes, we can use g- to project T (((a)) back to R. to display the transported image to an operator. With this, we can define our set of high dimensional interpretable transport maps:
function
Onigh dim == - TT:T=g-1 (f (o(x)))) 7 (e.g., k-sparse mappings) and I is the set of (pseudo- Jinvertiie functions with an interpretable (i.e. semantically meaningful) latent space. Finally, given an interpretable g E I, this gives us High- dimensional Interpretable Transport, THIT:
22 E I where li the of interpretable mappings
arg min Ep..e Te (ola))((((az))) + (P(((()))))) Po(w)) FgQU0
(4)
which results in an interpretable map T which approximately shows how images from Pare shifted to Pigt in a semantically meaningful way (e.g., how the H&E staining in histopathology images changes across hospitals). For further details about THIT, its variants, and results for an experiment on explaining Colorized-MNIST, please see see|Appendix
6.2 EXPLAINING IMAGE. BASEED SHIFTS VIA COUNTERFACTUAL EXAMPLES
In some cases, solving for an interpretable latent space can be too difficult/costly, and thus a shift cannot be expressed by an interpretable mapping function. However, if the samples themselves are easy to interpret (c.g., images), we can still explain a transport mapping by visualizing trans- lated samples. Specifically, we can remove the interpretable constraint on the mapping and lever- age methods from the unpaired mmage-to-Image translation (12I) literature to translate between the source and target domain while preserving the content. For a comprehensive summary of the re- cent I2I works and methods, please see Pang et al.) (2021). Once a distributional mapping is found, to serve as an explanation, we can provide an operator with a set of counterfactual pairs {(xx, T(x)) x ~ Psre,
Ptge). ). Then, by determining what commonly stays invariant and what commonly changes across the set of counterfactual pairs, this can serve as an explanation of how the source distribution shifted to the target distribution. While more broadly applicable, this ap- proach could put higher load on the operator than an intrinsically interpretable mapping approach.
Explaining Shifts in H&E Images Across Hospitals We apply this distribution counterfactual approach to the Camelyon17 17 dataset Bandi et al. (2018) which is a real-world distribution shift dataset that consists of whole-slide histopathology images from five different hospitals. We use the Stanford WILDS Koh et al. (202T) variant of the dataset which converts the whole-slide images into over 400 thousand patches. Since each hospital has varying hematoxylin and eosin (H&E) staining characteristics, this, among other batch effects, leads to heterogeneous image distributions across hospitals as can be seen in Fig. 3
To generate the counterfactual examples, we treat each hospital as a domain and train a StarGAN model Choi et al. (2018) to translate between each domain. For training, we followed the original training approach seen in Choi et al. (2018), with the exception that we perform no center cropping. After training, we can generate distribution counterfactual examples by inputting a source image and the label of the target hospital domain to the model. Counterfactual generation was done for all five hospitals and can be seen on the right-hand side of Fig. 3 It can be seen that the StarGAN model captures the different staining characteristics across the hospitals. For example, hospital 1 (Pi) consists of mostly light staining, and thus transporting to this domain usually involves lightening of the image while P3 seems to have more hematoxylin stain thus leading deeper purple images when pushing onto this domain. We can also see that the model tends to respect the content of the image where patches that contain tumor cells (e.g., the P3 sample the right-hand side) still
8

---

Under review as a conference paper at ICLR 2023
Baseline:
Visual Inspection of Samples
Original
Counterfactual Examples (ours)
Pt
P2
P3
P4
P5
Px
P
1x-1xx-2
Px-3
Px
xx-4
P
P1
P2
Baseline: Saliency Maps for Domain Classifier
P3
P1
Pz
P.
P4
Ps
P4
Ps
Figure 3: Our explanation approach (right) of showing paired counterfactual images translated between the hospital domains (represented as P1, P2. ...) quickly makes it clear how the stain- ing/coloring differs across the hospital domains (where the (i,j) row, column pair represents the pushforward of the jt domain onto the j"h domain). The baseline method of unpaired samples (top- left) requires many more samples to begin to understand the differences across the hospital domains and using Grad-CAM [elvaraaju et al. 2016) to explain a ResNett-50fe He et al. (2016) domain classi- fier (bottom-left) does not capture the appropriate staining/coloring details.
contain tumor cells in the counterfactual cases and likewise for lymphocyte cells (e.g.. the Pa sample on the right-hand side). We further explore content- -attent-based changes (as opposed to style changes) of an image using the CelebA dataset in /sussection C.4
LIMITATIONS
primary challenge in developing distribution shift explanations is determining how to evaluate the efficacy of given explanation in a given context. Evaluating explanations is active area of research RRonnikvSikoojaa & Bohanec (2018): Molnar (2020); Doshi Velez & Kim (2017) with explanation should be contrastive, suceinet, should highlight abnormal-
commonalities such ities, and should have high fidelity Molnar 2020). For the case of distribution shift explanations, as this is highly context-dependent problem (dependent on the data setting, task setting, and op- erator knowledge) and our approach is designed to tackle this problem in general, we do not have general automated way of measuring whether given explanation is indeed interpretable. Instead, provide a general contrastive method that supplies the PercentExplained (approximation of fi- delity) and the adjustable k-level of sparse/cluster mappings (which trades off between succinetness and fidelity) but ultimately leaves the task of validating the explanation up the operator. We be- lieve developing new shift explanation maps and criteria for specific applications (e.g., explaining the results of experiments ity measurement bounds, is a rich area for future work. An additional challenge is that while the PercentExplained metric shows the fidelity of explanation (i.e. how aligned Ty (P.re) and Pige are), we do not have a method of knowing specifically what is missing from the explanation which explanation can be considered "known unknown". For further discussions of challenges with explaining dis- tribution shifts (c.g., finding an interpretable latent space, approximations of Wasserstein distances in high dimensional regimes, etc.) we point the reader to Apppeddix
with different initial conditions). thus offering tighter interpretabil-
DISCUSSION AND CONCLUSION
In this paper, we introduced a framework for explaining distribution shifts using a transport map T between a source and target distribution. We constrained a relaxed form of optimal transport the- oretically define interpretable mapping Tyr and introduced two interpretable transport methods: k-sparse and k-cluster transport. provided practical approaches to calculating a shift explana- tion, which allows to treat interpretability as a hyperparameter that be adjusted based on a user's need and showed how our methods can help an operator investigate a distribution shift on real-world examples.
We hope our work suggests multiple natural extensions such as using trees as a feature-axie -ww-aaiiii--iieeeee form of clustering even other forms of interpretable sets. Given results and potential ways forward, we ultimately hope our framework lays the groundwork for providing more information to aid in investigations of distribution shift.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214 223. PMLR, 2017.
Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Transactions on Medical Imaging, 2018.
Dimitris Bertsimas, Agni Orfanoudaki, and Holly Wiberg. Interpretable clustering: an optimization approach. Machine Learning, 1 110(1):89-133 138. 2021.
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion Pro- ceedings of The 2019 World Wide Web Conference, 2019.
Austin Brockmeier, Claudio Cesar Claros-Olivares, Matthew Emigh, and Luis Gonzalo Sanchez Giraldo. Identifying the instances associated with distribution shifts using the max-sliced bures divergence. In NeurlPS 2021 Workshop on Distribution Shifts: Connecting Methods and Appli- cations, 2021.
Kailash Budhathoki, Dominík Janzing, Patrick Blocbaum, and Hoiyi Ng. Why did the distribution change? In Arindam Banerjee and Kenji Fukumizu (eds.), Proceedings of The 24th Interna- tional Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pp. 1666-1674. PMLR, 13-15 Apr 2021. URL URL.ttppp/proeeedio
mlr. httess/sw11///uucthoootttt
html
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha. Sunghun Kim, and Jacgul Choo. Star- gan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 2018
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26:2292-2033 2300, 2013.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 2916):111-14,, 2012.
Finale Doshi Velez and Been Kim. Towards rigorous science of interpretable machine learning. arXiv preprint arXiv: 100.::100.0000 082008808, 2017.
Ricardo Fraiman, Badih Ghattas, and Marcela Svarc. Interpretable clustering using unsupervised binary trees. Advances in Data Analysis and Classification, 7(2):125-155, 2013.
Aude Genevay, Lénaic Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample complex- ity of sinkhorn divergences. In The 22nd international conference on artificial intelligence and statistics, pp. 1574-1583. PMLR, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Maximilian Ilse, Jakub M Tomezak, Christos Louizos, and Max Welling. Diva: Domain invariant variational autoencoders. In Medical Imaging with Deep Learning, pp. 322-348. PMLR, 2020.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative the IEEE/CVF conference on computer vision and pattern
adversarial networks. In Proceedings the recognition, pp. 4401-4410, 2019.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of tn-the-wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664. PMLR, 2021.
10

---

Under review as a conference paper at ICLR 2023
R Kohavi and B Becker. Uci machine learning repository: Adult data set, 1996. URL https: //archive. ics uci. http://////mmmccieeeeeeaaaaaaa-a
Alexander Korotin, Vage Egiazarian, Arip Asadulaev, Alexander Safin, and Evgeny Burnaev.
Wasserstein-2 generative networks. arXiv preprint arXiv: 13082 2019.
Sean Kulinski, Saurabh Bagchi, and David I Inouye. Feature shift detection: Localizing which fea- tures have shifted via conditional distribution tests. Advances in Neural Information Processing Systems, 33, 2020.
Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In International conference machine learning, pp. 3122-3130. PMLR, 2018.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Ashok Makkuva, Amirhossein Taghvaei, Sewoong Oh, and Jason Lee. Optimal transport mapping via input convex neural networks. In International Conference on Machine Learning, pp. 6672- 6681. PMLR, 2020.
Olvi L Mangasarian and William H Wolberg, Cancer diagnosis via linear programming. Technical report, University of Wisconsin- Madiison Department of Computer Sciences, 1990.
Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.
Jose G Moreno-Torres, Troy Raeder, Rocío Alaiz-Rodríguez, Nitesh Chawla, and Francisco Her-
rera. A unifying view dataset shift in classification. Pattern recognition, 45(1):521-553 2012.
Wayne B Nelson. Applied life data analysis, volume 521. John Wiley & Sons, 2003.
Yingxue Pang, Jianxin Lin, Tao Qin, and Zhibo Chen. Image-to-image translation: Methods and applications. IEEE Transactions on Multimedia, 2021.
Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. Learning model-agnostic counterfactual explanations for tabular data. In Proceedings of The Web Conference 2020, pp. 3126-3132 2020.
Gabriel Peyré and Mareo Cuturi. Computational optimal transport. Foundations and Trends in Machine Learning. 11(5-6):355-6)) 007,5-60,2200000 2019.
Joaquin Quiñonero-
Masashi Sugiyama, Neil D Lawrence, and Anton Schwaighofer. Dataset shift in machine learning. Mit Press, 2009.
Stephan Rabanser, Stephan Günnemann, and Zachary Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. arXiv preprint 11X11:1810.11953, 2018.
Marko Robnik-vSikonja and Marko Bohanec. Berturbation-basee explanations of prediction models. In Human and machine learning, pp. 159-175. Springer, 2018.
Mirka Saarela and Susanne Jauhiainen. Comparison of feature importance measures as explanations for classification models. SN Applied Sciences, 3(2):1-12, 2021.
Axel Sauer and Andreas Geiger. Counterfactual generative networks. International Confer-
ence on Learning Representations, 2021. URL https:://pperevieee net. forttmffo
BXewEAYMmJW
RR Selvaraju, Cogswell, Das, Vedantam, D Parikh, and Batra, Grad-cam: visual explana- tions from deep networks via gradient-based localization. 2016. arXiv preprint arXiv:1610.02391,
2016.
Aalok Shanbhag. Avijit Ghosh, and Josh Rubin. Unified shapley framework to explain prediction drift. arXiv preprint arXiv:2 2021.
11

---

Under review as a conference paper at ICLR 2023
Lloyd S Shapley. A value for n-person games. Classics in game theory, 69, 1997.
N. Siddharth, Brooks Paige, Jan- Willem van de Meent, Alban Desmaison, Noah D. Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. Learning disentangled representations with semi- supervised deep generative models. In I. Guyon, U. V. Luxburg, S. Bengio. H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 5927-5937. Curran Associates, Inc., 2017.
Amos Storkey. When training and test sets are different: characterizing learning transfer. Dataset shift in machine learning, 30:3-28, 2009.
Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007.
Luis Caicedo Torres, Luiz Manella Pereira, and M. Hadi Amini. A survey on optimal transport for machine learning: Theory and applications, 2021.
Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009.
Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under
target and conditional shift. In International Conference on Machine Learning, pp. 819-827.
PMLR, 2013.
12

---

Under review as a conference paper at ICLR 2023
A ADDITIONAL TRANSPORTATION MAPPINGS DETAILS AND PROOFS
A.1 FINDING K-PAIRED CLUSTERS
Algorithm Finding & Paired Clusters
Input: X. X.Y.k
4 X.ndim
Tor & OpinmaTrrnspootNlg((X Y) /le.g. Sinkhorn
2 + [X,ToT(X))
Zetuster controids ClusteringAlg(2 Z, k) //e.g., k-means
Mare* +- Zccuster-centroieell //slccing column-wise
Migt- Zdduttrr-cnntroids d:2d
Output: Mares Magt
A.2 INFINITE NUMBER OF POSSIBLE MAPPINGS BETWEEN DISTRIBUTIONS
As stated the introduction, given two distributions, there exist many possible mappings such that TyPore Ptgt (it should be noted that here the optimal transport problem which can be shown via Brenier's s theorem Peyré & Cuturi (2019) to have a unique matching for some cases). For instance, given isometric Gaussian distributions æ ~ (i I). ~ N22 I), where I is the Identity matrix, there exist an infinite number of T's such that T(x) .N2- Specifically, any T of the form: T(x:) #22 R(x 11), where is R is an arbitrary rotation matrix, will shift TW to have a mean of μ22 and perfectly align the two distributions (since any rotation an isometric Gaussian will still be isometric Gaussian).
speaking of the general mapping problem, not
A.3 DISTANCE EMPIRICAL INTERPRETABLE TRANSPORT UPPER-BOUNDS THE WASSERSTRIN DISTANCE
First, let's remember our empirical method for finding
arg min TER
Σ " ()) Ad(T(z Tor((a))
(5)
where Tor is the optimal transport solution between source and target domains with the given cost function. The distance term d on the right-hand side this equation is assumed to he the &2 cost squared euclidean cost and is empirical approximation of the divergence term oi (r(()) Py)) in Eqn. I where & is assumed to be the Wasserstein distance, W. We claim this is a reasonable approximation since as N approaches the size of the dataset (or for densities, lim the dis- tance term becomes the expectation: Egw.P... (TT (x) (+)( which is an upper-bound the ((Pr(=); Pv) distance. To show this, we start with the expanded W distance:
W(Pr(a)) Pv) = min Euw (T(x), R(T(x))) REW
T = (R: R₂T(x) Py}
I E d (T R(T())))
WR T
If we let lor 1 and since Q E we can say
< Ex-P, d (T(x), Q(T(=))T)))))))))))))))))))(())) = Ex- d (T(x), (or(z)))
((Pr(m)).P Py) S Ex (T(x), Tor (x)))
A.4 PROVING THE K-SPARSE OPTIMAL TRANSPORT IS THE K-SPASSE TRANSPORT THAT MINIMIZES OUR DISTANCE FROM OT LOSS
When performing unrestricted s--saare transport, i.e. where (66) 2aparse is any transport which only moves points along & dimensions, the k-sparse optimal transport solution is the exact mapping that minimizes the distance function in the right-hand side of (Egn 5) is the lg distance or squared Fuclidean distance. As reminder, k-sparse optimal transport is: [T(=)];
(w)) if j E A.
13

---

Under review as a conference paper at ICLR 2023
else |z) where is the active set of lt dimensions which our s-spas transport T can points.
Let A be A's complement (i.e. the dimensions which are unchanged under T). Let = T(x)),
2oT (o)( and a TRaxd is the squared Fuclidean distance:
d(2, = E
E
(zr. ZOT..)) )
jE(d sefni
ΣΣ Σ
(zi.) ZOT,,) )²
Σ
[ii
)
JEAiE(n]
jEA+E|M
WEET
ΣE (Zi.) ZOT..)) ++
JEAiE[n] now if T is the truncated optimal transport solution, [z]; |ZOT]i Vj EA = +
Since a is the minimum of d(= ZOT) for a given A. the truncated optimal transport problem minimizes the d(T(x(()), (z) active set for this case is the one that minimizes thus the active set should be the & dimensions which have the largest squared difference between æ and Zor.
distance. This can easily be extended to show that the optimal
A.5 PROOF THAT K-MEAN SHIFT THE K- V-CCRO SHIFT THAT GIVES US THE BEST ALIGNMENT
When performing k-sparse vector transport, i.. where 2 = {T' T(x) = x + where = j j & A else [ō]; 0 and E Rd, |A| < l;, the k-sparse mean shift solution is the exact mapping that minimizes the distance function in the right-hand side of Eq.. 5 when is the & distance.
d(z, 2OT) = Σ
Σ
(Zi.j - )
SE(d iE(n)
Σ Σ
(Zi. ZOT..)
[ii.j ZOT.,)
JEAiE(M
jEAiE|M
=0, since constant witt
ΣΣ
(zi ZOT..) +
JEAiETM
Σ Σ
(miS + +5 -2OT;s) +2
JEAiEIM
Σ Σ JEAiEM
( + 63 + 2 ZOT., + +28,(Z1.5 ZOT..) 22OT;,5 -2x,,) +
Similar the k-sparse optimal transport solution, we can see that A should be selected as the dimensions which have the largest shift, thus minimizing a. The coordinate-wise gradient of the above equation is:
V6, d(z, ZOT) Eiejol (26, + 2x,,; 22OT.,) j jEA A
jEA
Now with this say:
14

---

Under review as a conference paper at ICLR 2023
Vosea 2OT)
Σ
(26g + 2x;s 2ZOT..))
iE(n]
2ndj +
Σ (2xr,) 220T.,))
iE(m]
now let 8j 67
0- 2nd + Σ (2x1.. )2oT;,)
kE(m]
no;
Σ
(ZOT; - Ii.j)
EE ml
1
8] n iE(N]
(ZOT:, Fi.j)
5) = MIOTy
Mm,
Thus showing the optimal delta vector minimize k-vector transport is exactly the k-sparse mean shift solution.
B CHALLENGES OF EXPLAINING DISTRIBUTION SHIFTS AND LIMITATIONS OF OUR METHOD
Distribution shift is a ubiquitous and quite challenging problem. Thus, we believe discussing the challenges of this problem and the limitations of our solution should aid in advancements in this area of explaining distribution shifts.
As mentioned in the main body, distribution shifts can take many forms, trying explain distri- bution shift is a highly context-dependent problem (i.e., dependent on the data setting, task setting, and operator knowledge). Thus, a primary challenge in developing distribution shift explanations is determining how to evaluate whether a given explanation is valid for a given context. In this work, we hope to introduce the problem of explaining distribution shifts general (i.e. not with a specific task nor setting in mind), therefore we do not have an automated way of measuring whether a given explanation indeed interpretable. Evaluating explanations is an active area of research Robnik- Molnar (2020); Doshi- Velez & Kim (207)) with commonalities such
vSikonja & Bohanec | as an explanation should be contrastive, suceinct, should highlight abnormalities, and should have high fidelity. Instead, we introduce a proxy method that supplies the operator with the PercentEx- plained and the adjustable k-level of sparse/cluster mappings but leaves the task of validating explanation up to the operator. We believe developing new shift explanation maps and criteria for specific applications (c.g., explaining the results of experiments run with different initial conditions) rich area for future work.
Explaining distribution shifts becomes more difficult when the original data is not interpretable. This typically can take two forms: 1) the raw data features uninterpretable but the samples are interpretable (e.g., a sample from the CelebA dataset Liu et al. (2015) is interpretable but the pixel-level features are not) 2) when both the raw data features and samples are uninterpretable (e.g., raw experimental outputs from material science simulations). In the first case, one can use the of counterfactual pairs method outined in subsection 6.2 (see Fig. 8 for examples with CelebA), however, as mentioned in the main paper, this is less sample efficient than an interpretable transport map. For the second case, the original features are not interpretable, one must first find an interpretable latent feature space which is challenging problem by itself. As seen in |Fi.. 10, is possible to solve for a semantic latent space and solve interpretable transport maps within the latent space, in this case, the latent space of a VAE model. However, if the meaningful latent features are not extracted, then any transport map within this latent space will be meaningless. In the case of Fig. 10 the 3- 3-cluster explanation is likely only interpretable because know the ground truth and thus know what to look for. As such, this is still an open problem and one we hope future work can improve on.
15

---

Under review as a conference paper at ICLR 2023
Additionally, while the PercentExplained metric shows the fidelity of explanation (i.e. how aligned (P(Pe))) and Pege are), we do not have a method of knowing specifically what is missing from the explanation. This missing part of the explanation can be considered "known unknown". For example, a given T has a PercentExplained of 85%, we know how much is missing, but we do not know what information is contained in the missing 15%. Similarly, when trying explain an image-based distribution shift with large differences in content (e.g., a dataset with blonde humans and dataset with bald humans), leveraging existing style transfer architectures (where one wishes to only change the style of an image while retaining as much of the original content as possible) to generate distributional counterfactuals can lead to misleading explanations. This is because explain- ing image-based distribution shifts might require large changes in content (such as removing head hair from an image), which most style-transfer models are biased against doing. As an example, Fig. 8 shows an experiment that translates between five CelebA domains (blond hair, brown hair, wearing hat, bangs, bald). can be seen that stylistic differences such "blond hair" "brown hair" but is unable to make significant content changes such as "bangs" + "bald".
the StarGAN model can successfully translate between
The above issues mainly problems that affect distribution shift explanations in general, but below are issues specific to our shift explanation method (or any method which similarly uses empirical OT). Since we rely on the empirical OT solution for the sparse and cluster transport (and the percent explained metric), the weaknesses of empirical OT are also inherited. For example, empirical OT, even using the Sinkhorn algorithm with entropic regularization, scales at least quadratically in the number of samples Cuturi 2013). Thus, this is only practical for thousands of samples. Further- more, empirical OT is known poorly approximate the true population-leve OT in high dimensions although entropic regularization can reduce this problem Genevay et al. (2019). Finally, empirical OT does not provide maps for new test points. Some of these problems could be alleviated by using recent Wasserstein-2 approximations to optimal maps via gradients of input-convex neural networks based on the dual form of Wasserstein-2 distance Korotin et al.] (2019); Makkuva et al.) (2020). Additionally. when using k-cluster maps, the clusters are not guaranteed to be significant (i.e. might be indiscernible what makes this cluster different than another cluster), and thus if using k- cluster maps datasets that do not have natural significant clusters (e.g., Pare ~Uniform(0, 1), Ptgt ~Uniform(1, 2)) an operator might waste time looking for significance where there is none. While this cannot be avoided in general, using clustering method that is either specifically de- signed for finding interpretable clusters Fraiman et al. )2013); Bertsimas et al. (2021) or one which directly optimizes the objective in interpretable transport equation Eqn. might lead to easier to explanations which are easier to interpret or validate.
C EXPERIMENTS ON KNOWN SHIFTS
Here we present additional results on simulated experiments as well as an experiment on UCI "Breast Cancer Wisconsin (Original)" dataset Mangasarian & Wolberg (1990). Our goal is to il- luminate when to use the different sets of interpretable transport, and how the explanations can be interpreted, where in this case, ground truth explanation known.
C.I SIMULATED EXPERIMENTS
In this section we study three toy shift problems: a mean shift between two, otherwise identical, Gaussian distributions, Gaussian mixture model where each mixture component has different mean shift, and a flipped and shifted half-moon dataset, as seen in figures (a), (d), and (g) in FFg. 4
The first case is of mean shift between two, otherwise identical, Gaussian distributions can be easily explained using k-sparse mean shift (as well as vanilla mean shift). We first calculate the OT mapping Tor between the two Gaussian distributions, which has closed form solution of Tor(x) = Aegt + A(x - (rre), where A is a matrix that can be seen as a conversion between the source and target covariance matrices, and because the covariance matrices are identical, A is the identity.
!Code to recreate all experiments seen here and in the main body of the paper will be released upon accep- tance.
16

---

Under review as a conference paper at ICLR 2023
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
Figure 4: Three toy dataset shift examples showing the advantages of the different shift explanation methods, where a mean shift between Gaussians (top row) can be easily explained using k-sparse vector shifts, varying mean shift across mixture components of Gaussian mixture model (middle row) is best explained using k-sparse transport maps, while complex shift (bottom row) requires a complex feature-wise mapping, such as k-sparse optimal transport, which maximally aligns the distributions as it can perform conditional transport mappings for each sample (as seen by the dif- fering vertical shifts in (h) depending on where the blue sample lies on the horizontal axis), at the expense of interpretability. Each example shows three levels of decreasing interpretability, where the leftmost column shows the original shift (which has maximal interpretability since 0) from source (blue diamonds) target (red down arrows), and the rightmost column shows shift with near-pere near-perfect fidelity.
The second toy example of distribution shift is shifted Gaussian mixture model which represents case where groups within a distribution shift in different ways. An example of this type of shift could be explaining the change in immune response data across patients given different forms of treatment for disease. Looking (d) in Fig. 4 is clear that sparse feature transport will not easily explain this shift. Instead, we turn to bluster-based explanations, where we first find & paired clusters and attempt to show how these shift from Pare to Ptgt- Following the mean-shift transport of paired clusters approach outlined in ssussection 4.4, the l = 3 case as seen in the Appendix shows that three clusters sufficiently approximate the shift by averaging the shift between similar groups. more faithful explanation is required, (f) Fig. 4 shows that increasing to 6 clusters can recover the full shift, i.e. PercenfExplained-1000 at the expense of being less interpretable (which is especially true in a real-world case where the number of dimensions might be large).
The half-moon example, figure (g) in Fig. 4 shows case where complex feature-wise dependency change has occurred. This example is likely best explained via feature-wise movement, so will use k-sparse transport. we follow the approach in subbecctii 4.3 with our interpretable set as the (4) and let 1, get mapping that is interpretable, but has poor alignment (see Figure (h) in
17

---

Under review as a conference paper at ICLR 2023
Fig. 4).
For this example, we can possibly reject this explanation due poor PercentExplained. With the understanding that this shift is not explainable via just one feature, we can instead use a k 2-sparse OT solution. The 2 case can be seen in (i) of 4 4wwich shows that this approach aligns the distributions perfectly, at the expense of interpretability.
C.2 EXPLAINING SHIFT IN WISCONSIN BREAST CANCER DATASET
Features: "Bare Nuclei and Cell Size shifted 6.28 and 5.27"
Features: "Bare Nuclei, Cell Size,
Shape, Nuclcoli, and Marginal Adhesion were shifted by 6.28, 5.27, 5.15, 4.6, 4.24"
60%
40%
All Features: "Bare Nuclei, Cell Size, Cell Shape, Normal
20%
Nucleoli, Marginel Adhesicn, clump Thickness, Bland Chromatin, Single spithelial Cell Size, and Nitoses were shifted by 6.28, 5.27, 5.15, 4.6, 4.24, 4.22,
0%
Number of Features Changed
3.22, 1.54
Figure 5: comparison of the performance h-sparse mean shift explanations (solid line) and k- sparse optimal transport explanations (dashed line) when explaining the shift from the benign tumor samples to malignant tumor samples for the UCI Wisconsin Breast Cancer dataset. On the right are example explanations human operator would see as they change the level of interpretability during k-sparse mean shift explanations (where "All Features" is the baseline full mean shift explanation).
This tabular dataset consists of tumor samples collected Mangasarian & Wolberg (1990) where each sample is described using nine features which are normalized to integers from [0, 10]. We split the dataset along the class dimension and set Pa be the 443 benign tumors and Page the 239 malignant samples. To explain the shift, we used two forms of k-sparse transport, the first being k- sparse mean transport and the second being k-sparse optimal transport. The left of Fig. (1Fig.5]shar Shoows that the k-sparse mean shift explanation is sufficient for capturing the 50% of the shift between P and Ptge using only four features, and nearly 80% of the shift with all 9 features. However, if an analyst requires more faithful mapping, they can use the k-sparse OT explanation which can recover the full shift, at the expense the interpretability. The right of Fig. shows example explanations that an analyst can use along with their context-specific expertise for determining the main differences between the different tumors they are studying.
C.3
COUNTERFACTUAL EXAMPLE EXPERIMENT TO EXPLAIN A MULTI-MNIST SHIFT
S
7
3
3
3
3
8
0
4
2
6
5
/
5
1
3
7
3
5
6
O
3
3
4
8
2
9
5
6
2
2
6
8
5
2
8
6
8
9
3
6
3
#
4
5
2
8
6
g
g
6
g
0
Y
0
2
3
4
5
7
7
8
2
6
3
9
9
9
9
3
8
7
6
2
1
5
9
8
8
2
4
4
6
7
)
3
3
2
Le
2
6
1
3
5
3
9
2
2
|
8
3
6
7
&
2
8
2
I
9
6
9
2
3
9
6
9
Figure 6: A grid of 25 raw samples from each domain (left is Parc and right is Pigt). Even for the relatively simple shift seen the Shifted Multi-MNIST dataset, it may be hard to tell what is different between the two distributions by just looking at samples (without knowing the oracle shift).
18

---

Under review as a conference paper at ICLR 2023
As mentioned in subsecction 6.2 image-based shifts be explained by supplying an operator with a set of distributional counterfactual images with the notion that the operator would resolve which semantic features distribution-specifte Here provide toy experiment (as opposed to the real-world experiment seen (susecttonn 6.2) to illustrate the power of distributional counterfactual examples. To do this, apply the distributional counterfactual example approach to Multi- MNIST dataset where each sample consists of a row of three randomly selected MNIST digits Deng (2112) and is split such that Psre consists of all samples where the middle digit even and zero and Pigt is all samples where the middle digit is odd, seen in Fig. 6
P Prcc
Ptgt
Psrc
Psrc
Ptgt
Ptgt Psrc
Ptgt
S
3
4
4
4
4
/
1
6
5
0
7
3
4
4
O
O
2
6
2
3
9
9
3
3
3
5
9
Ø
/
8
b
8
7
7
2
/
2
2
8
8
8
3
1
1
6
9
4
8
a
a
9
9
3
O
O
9
9
2
3
5
0
5
0
0
4
7
7
Figure 7: A comparison of the baseline grid of unpaired source and target samples (left) and counter- factual pairs (right) which show how counterfactual examples can highlight the difference between the two distributions. For each image, the top left digit represents the class label, the middle digit represents distribution label (where Pare only contains even digits and zero and Pegt has odd digits), and the bottom right digit is noise information and is randomly chosen. The second, third columns show the counterfactuals from Pare + Pige and Pigt > Parc- respectively. Hence we can under the push forward of each image the "evenness" of the domain digit changes while the class and noise digits remain unchanged.
Algorithm Generating distributional counterfactuals using DIVA
Input: 31 D1, Z2 D2. model
Zys 2d, Presidual, <- model..encode(11.
Eya 2dg Fresiduats +- model.encode 2r2)
41 model.decode((,, Znı Edg Zresidualy $2-+1 f- model.dccode((z., 2d, 2reeiduol
Output: #1-22 $2--1
To generate the counterfactual examples, we use Domain Invariant Variational Autoencoder (DIVA) [lse et al. (2020), which is designed to have three independent latent spaces: one for class information, one for domain-specifee information (or in this case, distribution-specifie information), and one for any residual information. We trained DIVA on the Shifted Multi-MNIS' dataset for 600 epochs with KL-3 value of 10 and latent dimension of 64 for each of the three sub-spaces. Then, for each image counterfactual, we sampled one image from the source and one image from the target and encoded each image into three latent vectors: Zy. 2d, and Zresiduat- The latent encoding za was
19

---

Under review as a conference paper at ICLR 2023
then "swapped" between the two encoded images, and the resulting latent vector set decoded to produce the counterfactual for each image. This process is detailed in Algorithm below. The resulting counterfactuals odd digits) to the target (i.e., even digits) and vice versa while keeping the other content unchanged (i.e., the top and bottom digits).
be seen in Fig. where the middle digit maps from the source (i.e.,
C.4 USING STARGAN TO EXPLAIN DISTRIBUTION SHIFTS CELEBA
Here we apply the distributional counterfactual approach seen in subsection 6.2 to the CelebA dataset Liu al. (2015), which contains over 200K images of celebrities, each with 40 attribute annotations. We split the original dataset into 5 related sets, P=="blonde hair", Pa="brunette hair", Pe="wea hat". Pi="bangs", Ps="bald". These five sets were chosen as they are related con- cepts (all related to hair) yet mostly visually distinct. Although there are images with overlapping attributes, such as a blonde/brunette person with bangs, these are rare and naturally occurring, thus they were not excluded.
P1
P2
P3
P4
P5
Original
Counterfactual Examples
i P Px-11
OVV
Px
Px-2
Px-3 Px-44 Px-5
P1
P2
P3
P4
P5
Figure 8: StarGAN is able to adequately translate between distributions with similar content but different style (e.g., P1 - P2), however, when transporting between distributions with different content (e.g.. "no hat" - P3) the 121 model is unable to properly capture the shift. This is likely due the model being biased to only change the style of the image, while maintaining as much content as possible. The figure breakdown is similar to Fig. 3 with the baseline method of unpaired samples on the left and paired counterfactual images on the right, where here P Pbonbe hair". "runbrunette hair", P hat", Pa="bangs", "s="bald".
We trained a StarGAN model Choi et al. 2018) to generate distributional counterfactuals following same approach seen in subsectioo 6.2 The result of this process can be seen in Fig. where we can see the model successfully translating "stylistic" parts of the image such as hair color. However, model unable translate between distributions with larger differences in "content" such as removing hair when translating to "bald". This highlights difference between I2I tasks such as style transfer (where one wishes to bias a model to only change the style of an image while retain- ing much of the original content possible) the mappings required for explaining image-based based distribution shifts, which might require large changes in content (such as adding a hat to an image).
D EXPLAINING SHIFTS IN IMAGES VIA HIGH-DIMENSIONAL INTERPRETABLE TRANSPORTATION MAPS
& is an image with domain 1 then any non-trivial transportation map in this space is likely to be hard to optimize for well as uninterpretable. However, if P Parey Pagt can be expressed some
20

---

Under review as a conference paper at ICLR 2023
interpretable lower dimensional manifold which is learned by some manifold-invertine function IR + R where h < d, we can project Perc: Ptgt onto this latent space and solve for an interpretable mapping such that it aligns the distributions in the latent space, Pr(g(=)) Note, in practice, an encoder-decoder with an interpretable latent space can be used for g. however, requiring g to be exactly invertible allows for mathematical simplifications, which will see later. For explainability purposes, we can use g- to re-project T' (g(x)) back to IR in order to display the transported image to an operator. With this, we can define our set of high dimensional interpretable
Po(y)-
transport maps: Qhigh-dim :- { T g-1 (f(()) T 22(h),
9 I where 2(4) is the set of k-interpretable mappings (e.g., k-sparse or k-cluster maps) and I is the set of invertible functions with interpretable (i.e. semantically meaningful) latent space.
Looking at our interpretable transport problem:
arg min E [c(æ, T(x)) + (riro(Pr(a)) Pa) Tigga
(6)
Although our transport is now happening in a semantically meaningful space, our transportation cost is still happening in the original raw pixel space. This is undesirable since we want transport cost that penalizes large semantic movements, even if the true change in the pixel space is small (e.g., change from "dachshund" to "hot dog" would be a large semantic movement). We can take a similar approach as before and instead calculate our transportation cost in the space. This logic can similarly be applied to our divergence function & (especially if ø is the Wasserstein distance, since this term can be seen as the residual shift not explained by T). Thus, calculating our cost and alignment functions within the latent space gives us:
arg min Epace [c(2x)) gEZ,TEN(1)
(((x)))] (()o))),(((=)))) + Ao(P (( Pa(m)
This formulation has critical problem however. Since we are jointly learning our representation g and our transport map T, trivial solution for the above minimization is for to map each point to an arbitrarily small space such that the distance between any two points e(g(x)), gg))) z 0, thus giving us a near zero cost regardless of how "far" we move points. To avoid this, we can use pre- defined image representation function h, e.g., the latter layers in Inception V3, and calculate pseudo- distances between transported images in this space. Because h expects an image an input, we can utilize the invertibility of g and perform our cost calculation as: c (h(æ)), h (g- (T(())))) or
more simply, en (x, (x)),, where T = = (f (F (o(2))) Similar to the previous equation, we also apply this i pseudo-distance to our divergence function to get Φh. With this approach, we can still use g jointly learn semantic representation which is specific to our source and target domains (unlike which is trained on images in general, c.g., ImageNet) and an interpretable transport map T within g's latent space. This gives us:
arg min Ep... [ch (z, ((x)) + Xon Pa) gEI,TER
While the above equation is an ideal approach, it can be difficult to use standard gradient approaches to optimize over in practice due it being a joint optimization problem and any gradient information having to first pass through h which could be a large neural network. To simplify this, we can op- timize T and g separately. With this, we can first find g which properly encodes our source and target distributions into a semantically meaningful latent space, and then find the best interpretable transport to align the distributions in the fixed latent space. The problem can be even further simpli- fied by setting the pre-trained image representation function / to be equal to the pretrained since the disjoint learning of and T removes the shrinking cost problem. By setting h := g. we can see that e (h(a)) h o g 1 f o g(xx) = c (g(x)) T y(a))) = c,(x,, f(x)), which simplifies Eqn. back to our interpretable transport problem, Eqn. 6 where g is treated pre-processing step on the input images:
arg min EP... [c(g(z), (T()))) + (Pr(()))))))))
(9)
Ten
21

---

Under review as a conference paper at ICLR 2023
Another way to simplify Egn. 8 is relax the constraint that g maanfold-invertiin and instead use a pseudo-inven invertible function such as an encoder g and decoder g* structure where g is pseudo- inverse such that gtt (g(x)) x This gives us:
arg min Er... (cn (T.g((((((x)))) + AFia O (((le)(())))))) Fell.g.t
(10)
+ ARccon E Esportt [+(
where -(x,,) is some reconstructive-loss function.
D.1 EXPLAINING A COLORIZED-MNE -NNIS SHIFT VIA HIGH- NIGH-DIMENSIONAL INTERPRETABLE TRANSPORT
In this section we present preliminary experiment showing the validity of our framework for ex- plaining digh-dimensional shifts. The experiment consists of using k-cluster maps to explain shift in golorzed--versinn of MNIST, where the source environment is yellow/light red digits with light grayscale background color (i.e. light gray) and the target environment consists of darker red digits and/or darker grayscale background colors. Like the lower dimensional experiments before, our goal is test our method on a shift where the ground truth is known and thus the explanation can validated against. We follow the framework presented in Eqn. where the fixed y is semi- supervised VAE Siddharth et al. (2017) which trained on a concatenation of Pere and Pigt Our results show that k-cluster transport can capture the shift and explain the shift, however, pect the given explanation interpretable because the ground truth is already known. Our hope is that future work will improve upon this framework by better finding latent space which is inter- pretable and disentangled, leading to better latent mappings, and thus better high-dimensional shift explanations.
sus-
4
6
6
4
7
1
4
3
5
7
3
8
E
3
7
7
2
6
3
5
3
2
/
7
O
7
S
8
7
&
7
5
?
8
T
6
9
4
4
5
7
O
8
7
9
2
8
2
2
0
/
8
7
S
9
7
2
8
Figure 9: The left figure shows samples from the source environment which has lighter digits and backgrounds while the right figure shows the target environment which has darker digits and/or darker backgrounds
Data Generation The base data the 60,000 grayscale handwritten digits from the MNIST dataset Dengl 2012). We first colored each digit by copying itself along the red and green chan- nel axes with an empty blue channel, yielding an initial dataset of yellow digits. We then randomly sampled 60,000 points from wooddimensional Beta distribution with shape parameters, The first dimension of our Beta distribution represented how much of the green channel would be visible per sample meaning low values would result in red image, while high values would result
= 5.
22

---

Under review as a conference paper at ICLR 2023
in yellow image. The second dimension of our Beta distribution represented how white black the background of the image would be, where 0 == black background and 1 = white background.
Specifically, the data was generated as follows. With Traw representing grayscale digit from the unprocessed MNIST dataset, a mask of representing the background was calculated m = Traw S 0.1, where any pixel value below 0.1 is deemed to be the background (where each pixel value = -[1- - m) -2rras by- (11 m) Braw: 0J, where 0 is zero-valued matrix matching the size of Traw and bj Beta(a, B). The background color was calculated via Thack = Ib2 -m -nraam b2 -m m-Eramm b2 m -taaa Then Fcolored I = + cback-colors which results colorized MNIST digit with a stochastic foreground and background coloring.
E [0, 1). Then, the foreground (i.e. digit) color was created Edigit-
The environments were created by setting the source environment to be any images where b1 N 0.4 and b2 2 0.4, i.e. any colorized digits that had over 40% of the green channel visible and background least 40% white, and the target environment is all other images. Informally, this split can be thought of as three sub-shifts: a shift which is only reddens the digit, second shift which only a darkens the background, and a third shift which is both a digit reddening and background darkening. The environments can be seen in Fig. 9
Model To encode and decode the colored images, we used a semi-supervised VAE (SSVAE) Sid- dharth et al. (2017). The SS encoder consisted of an initial linear layer with input size of 3. 28 28 and a latent size of 1024. This was then multi-headed into a classification linear layer of size 1024 10, and for each sample with label, digit label classification was performed on the output of this layer. The second head of the input layer was sent to style linear layer of size 1024 to 50 which is to represent the style of the digit (and not used in classification). The decoder followed a typical VAE decoder approach on a concatenation of the classification and style latent dimensions. The SSVAE was trained for 200 epochs on a concatenation of both Psre and Ptge with 80% of the labels available per environment, and batch size of 128 (for training details please see (Siddharth ct al. 2017)). The transport mapping was then found on the saved lower-dimensional embeddings.
2G
7:50
2
9
2
7
2
8
8
8
0
0
0
0
8
8
8
8
/
1
/
D
D
2
2
2
2
2
2
2
2
2
2
6
6
6
6
6
4
3
3
3
3
3
7
7
1
3
3
3
3
3
4
4
4
4
4
6
6
6
6
The linear interpolation explanations for the three clusters where the left cluster seems to explain the darkening digit shift, the right-most figure explains the shift which darkens the back- ground, and the middle cluster explains the case where both digit and background darkens. For each cluster, the left-most digit æ is the reconstruction of original encoding from the source distribution, the right-most digit is the clsste-bbsed push-forwat of that digit T(x), and the three middle images are reconstructions of a linear interpolations À æ + (1 - A) (x) with X E (0.25, 0.5, 0.75J.
Figure 10:
Shift Explanation Results Given the shift is divided into three main sub-shifts, we used i = 3 cluster maps to explain the shift. We followed the approach given in Eqn. 9. where the three clus- ter labels and transport were found in the 60 dimensional latent space using the algorithm given in Algorithm 1. Since our current approach is not able to find latent space with disentangled and
23

---

Under review as a conference paper at ICLR 2023
semantically meaningful axes, cannot the mean shift information per cluster the explana- tion itself (as it is meaningless the space is uninterpretable). Instead, we provide an operator with samples from our source environment and the linear interpolation to the samples' puswaforward versions under the target environment, for each cluster. The goal is for the operator to discern the meaning of each cluster's mean shift by finding the invariances across the linear interpolations. The explanations can be seen in Fig. 10
The linear interpolations from the first cluster (the left of Fig. 10) seem to show darkening of the source digit, while keeping the background relatively constant. The third cluster (right-most side of the figure) represents the situation where only the background is darkened but the digit is not. Finally, the third cluster seems to explain the sub-shift where both the background and the digit are darkened. However, the changes made in the figures are quite faint, and without priori knowledge of the shift it is possible that this could be an insufficient explanation. As mentioned in [Section 6 this could improved by finding disentangled latent space with semantically meaningful dimen- sions, better approximating high dimensional empirical optimal transport maps, jointl finding representation space and transport map like in Eqn. 4 and more; however, these advancements are out of scope for this work. We hope that this current preliminary experiment showcases the validity of using transportation maps to explain distribution shifts in images and inspires future work to build upon this foundation.
24


Reviews:
[{"review_id": "tFTUtqD7Ik", "reviewer": "Reviewer_Lnku", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors focus on providing an approach to interpret distributions\nshifts. To this end, they build on Optimal Transport by considering a\nrelaxation of the problem. They propose two general ways to approach\ninterpretability of distribution shifts that trade off complexity for\ninterpretability. Results on real-world data show the approach is\npromising in providing explanation of distribution shifts.", "summary_of_the_review": "The idea of approaching the interpretability of distribution shifts with\ntransport maps is interesting; the relaxation from optimal to\ninterpretable transport helps narrow OT to a set of interpretable OT\n\\Omega. One concern revolves around the need of relying excessively on\nhuman knowledge to define \\Omega; I do understand the problem is\nchallenging and the authors propose two general methods to kickstart the\nprocess. It would be great to integrate this reasoning in the\nexperimental section too.\n\nA key point of achieving IT, however, is to rely on k-Sparse Transports.\nhow can we identify what k dimensions to transport points from P_src to\nP_tgt? Does this also need to be user-defined? Similar reasoning applies\nto k-Cluster Transport. It seems the approach requires domain knowledge\nand this is clearly applicable in specific contexts, where dimensions\nand clusters are still manageable, for instance, as the experiments show.\n\nThe evaluation is interesting although I am not sure the benefit of the\nauthors' approach stands out. Looking at the k-mean explanation only on\nthe specific evaluation shown in Table 1, it is possible to understand\nthe shift quite well without resorting to the authors' k-OT approach.\nThe authors provide an explanation in Section 5 but it still seems\nblurred.  Is it because of the higher percentage? That comes with a\nhigher computation induced by the authors' approach. What is missing\nhere is a qualitative evaluation that would help humans to understand\nwhat approach helps the most. (Also, how would this compare against\nShapley values?)", "strength_and_weaknesses": "+  Important research problem\n+  Interesting idea of building on OT and relaxing the problem to make it feasible\n+  Realistic evaluation\n\n-  Excessive reliance on domain expertise\n-  Lack of qualitative evaluation", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "hpdhl_cLle8", "reviewer": "Reviewer_6o1f", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors explore explaining distribution shifts using interpretable transport mappings. The authors first propose a method which identifies $k$ features which maximally account for the optimal transport solution. They evaluate this method on the CivilComments dataset, finding that it outperforms a mean shift baseline. Then, the authors propose a method for explaining shifts by selecting $k$ clusters, which they evaluate on the Adult Income dataset. Finally, the authors demonstrate their ability to explain shifts in images by applying interpretable transport mappings to the latent space of a generative model to generate counterfactual images. ", "summary_of_the_review": "The paper presents an interesting and novel method for explaining distribution shifts using transport mappings. However, I believe that several aspects of the method are not well explained or are not well explored, and the empirical results are not particularly convincing. I am leaning towards rejection pending the rebuttal.", "strength_and_weaknesses": "Strengths:\n- The paper tackles an important and practical problem in distribution shifts.\n- The authors test their method on a variety of datasets from different modalities, including images.\n\nWeaknesses:\n1. There are several components of the method which are not explained very clearly in the paper:\n- It is unclear how the argmin problem in Equation 3 is actually solved. Is it always possible to decompose this function into a sum over individual features? What is the computational complexity of this problem?\n- I cannot find where the authors state the form of the $c$ function.\n- It is not clear what types of distribution shifts the proposed methods can identify. It seems that the OT method can identify arbitrary conditional shifts (i.e. $P(X_i | X_{\\neg i})$), whereas the mean method can only pick up marginal shifts ($P(X_i)$). The authors should clarify this, and perhaps contrast the difference with a new synthetic dataset in the appendix.\n- Is the output of the method dependent on the relative scale of the features? It seems that the authors normalize the Adult dataset, but do not state this fact explicitly.\n- The experiment in Section 6 is missing some details. For example, what is $\\Omega$ defined in $\\Omega_{high-dim}$ at the top of page 8? If it is a set of k-sparse mappings, then what is $k$ in Figure 3?\n\n2. The authors propose the PercentExplained metric, which is a measure of the effectiveness of the _mapping_. However, in practice, we only care about the effectiveness of the _explanation_. As an example, in the CivilComments result from Table 1, it seems that the output of the mean method (in terms of the features selected) is very similar to that of the OT method. For example, the top 4 features are identical for the F->M case. It seems that the reason why the OT method has much higher PercentExplained is because its underlying mapping is better. To compare the explanations of the two methods fairly, the authors should consider showing the PercentExplained for the mean method using the underlying mappings from OT.\n\n3. The output of the method for the Adult Income experiment in Figure 2 does not seem very meaningful to me. For example, the first cluster $C_M^1$ (presumably accounting for 80% of the shift), when mapped to the target (female) distribution, lowers the average age and the average education level, with a small increase in the average income. Does this indicate that middle-aged men have similar income to women who are slightly less educated and younger? It is not clear to me how this result would be actionable to a practitioner. \n\n4. The authors only compare against very simple baselines. For the Section 6 experiment, it seems that many I2I methods (which the authors reference) such as CycleGAN would be reasonable baselines. For the Section 5 experiment, I think Section 3 from Budhathoki et al., 2021 could work, using the attribution magnitude as a ranking.\n\n5. There are several spelling mistakes in the paper. For example, \"yeild\" at the top of page 4, a missing bracket in Equation 3, $P_y$ which should probably say $P_{tgt}$ at the end of Section 4.2, and a left over comment in Appendix D.\n\n6. Overall, the proposed methods do not seem very cohesive, and instead seems like a patchwork of approaches for different settings.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "exwKZfZkWdx", "reviewer": "Reviewer_xS74", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an interpretable transport map based method for explaining the distribution shift. It further proposes methods to calculate and obtain k-spare and k-cluster transport, based on pre-defined hyperparameters. Extensive evaluation has been performed to demonstrate the promising of the proposed methods.", "summary_of_the_review": "Overall, I think this paper works on an important problem with a feasible solution. \n\nAlthough the proposed transport map based method seems to be simple, less well-grounded, and somehow heuristic, I think it could be a useful step and feel positive about this paper. I also appreciate the authors’ effort in extensive evaluation and studies. Even though, I hope the authors could address the concerns, posted in the weakness part, during the response.\n\nAfter rebuttal:\nI appreciate the authors' feedback on the posted questions. However, at the current stage, it would be very important to make the proposed method actionable, since I would say it is not at a very early stage. Even though, I do appreciate the authors' effort in working in this direction and it is important, and hope authors could address this critical issue and enhance the paper in the future version.", "strength_and_weaknesses": "Strength:\n\n- Important problem\n- Sound and feasible solution\n- Extensive evaluation and showcase to demonstrate the potential usefulness\n\nWeakness:\n\n- The selection of hyperparameter is somehow empirical, lacks constructive ways to obtain a most optimized k.\n- Not very clear how to deal with cases that do not have balanced samples from source and target data distribution.\n- Not clear whether the obtained k-sparse and k-sparse cluster always have real and interpretable meanings.\n- Lack of detailed discussion on how the obtained explanation of the shifts could be useful in practice, in an actionable way for an operator.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FJN9qWmSWR", "reviewer": "Reviewer_Sirc", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors introduced a framework for explaining distribution shift using a transport map T between a source and target distribution. They constrained a relaxed form of optimal transport to theoretically define an interpretable mapping TIT and introduced two interpretable transport methods: k-sparse and k-cluster transport. The authors also provided practical approaches to calculating a shift explanation, which allows us to use treat interpretability as a hyperparameter that can be adjusted based on a user’s need and showed how our methods can help an operator investigate a distribution shift on real-world examples.", "summary_of_the_review": "The motivations in this paper are not clear. In addition, it is hard to see the potential application of the results when there are no extensive discussions regarding the distribution shifts considered in the machine learning community.", "strength_and_weaknesses": "Pros:\n\n1.  This paper contributes an interesting idea to see where the distribution changes.\n\n2. This paper is clear and easy to follow.\n\n\nCons:\n\n1. The major problem contained in this paper is that the criterion design is not well motivated. Eq. (2) does not have a clear motivation.\n\n2. Discussions regarding the down-stream tasks are not enough. It is hard to believe that understanding the distribution shift can be directly applied in some scenarios.\n\n3. The motivations behind the K-S and K-C are not clear as well. \n\n4. How to apply the results to improve the general machine learning algorithm? Distribution shifts happen everywhere, what we care about is how to adapt to the distribution shift. After the understanding part, it is very important to say how to use this kind of results.\n\n5. Adversarial data-based distribution shift might be a good example to analyze.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject