Explaining Low Dimensional Representation, a reproduction
Anonymous Author(s)
AfﬁliationAddress email
Scope of Reproducibility
This report covers our reproduction of the paper ’Explaining Low dimensional Representation’ [8] by Plumb et al.
In this paper, a method (Transitive Global Translations, TGT) is proposed for explaining different clusters in lowdimensional representations of high dimensional data.
They show their method outperforms the Difference Between the Means (DBM) method, is consistent in explaining differences with few features and matches real patterns in data.
Weverify these claims by reproducing their experiments and testing their method on new data.
We also investigate the useof more complex transformations to explain differences between clusters.
Methodology
We reproduce the original experiments using their source code.
We also replicate their ﬁndings by re-implementing theauthors’ method in PyTorch [7] and evaluating on two of the dataset used in the paper and two new ones.
Furthermore,we compare TGT with our own extension of TGT, which uses a larger class of transformations.
Results
We were able to reproduce their results using their code, yielding mostly similar results.
TGT generally outperforms
DBM, especially when explanations use few features.
TGT is consistent in terms of the features to which it attributescluster differences, across different sparsity levels.
TGT matches real patterns in data.
When extending the types offunctions used for explanations, performance did not improve signiﬁcantly, suggesting translations make for adequateexplanations.
However, the scaling extension shows promising performance on the modiﬁed synthetic data to recoverthe original signal.
What was easy
The easiest part was running the existing code with the pre-trained model ﬁles.
The original authors had set up theircode base in an organized manner with clear instructions.
What was difﬁcult
The ﬁrst difﬁculty that we encounter was ﬁnding the right environment.
The source code depends on deprecatedfunctionality.
The clustering method they used, had to be re-implemented for us to use it in our replication.
Anotherdifﬁculty was the selection of clusters.
The authors did not prove a consistent method for selecting clusters in a latentspace representation.
When retraining the provided models, we get a latent space representation different to the originalexperiments.
The clusters have to be manually selected.
The metrics that they used to evaluate their explanations arealso depend on the clustering.
This means that there is some variability in the exact veriﬁcation of reproducibility.
Communication with original authors
We asked the original authors for clariﬁcation on how to choose the (cid:15) hyper-parameter.
However, it became apparentthat we had misread, and the procedure is indeed adequately reported in the paper.
Introduction
The curse of dimensionality [3] is a long-standing problem in Machine Learning.
Data in many domains and applications(e.g.
Bioinformatics) has high-dimensional representations.
Finding patterns in such high-dimensional data is achallenging task.
To this end, dimensionality reduction [12] techniques have greatly helped in data-analysis, informationextraction, building computational models, and in doing inference.
Given an input x ∈ R d , dimensionality reductionlearns a function r : xr ( x ) , r ( x ) ∈ R m , where m << d .
Such a dimensionality reduction function r naturally arisesin deep learning due to the expressivity and representational power of neural networks.
The goal of r is to encode usefulknowledge about the input space, thus providing distinctive information in the transformed output r ( x ) .
This results in“clusters" or “groups of points" in the transformation space.
The downside of this exercise, however, is that the outputspace is usually non-interpretable.
There is usually no easy way to know what information is present in the transformedpoints r ( x ) and what sort of distinctive knowledge they contain.
In this work, we reproduce the paper ‘Explaining Groups of Points in Low-Dimensional Representations’ by Plumbet al.
This paper proposes a method for explaining different clusters in latent space representation.
They look atthe problem of explaining the points in the latent space representation through the lens of Interpretability in Machine
Learning.
We reproduce their ﬁndings and expand upon their work with our an extension.
We extend their research byapplying their method to a larger class of explanation functions and testing their method on new dataset.
We furtherinvestigate the efﬁcacy of the explanations using a probing classiﬁer [2].
Methodology
Counterfactual Explanations [11] have emerged as an active research area in the ﬁeld of Interpretable Machine Learning.
A counterfactual explanation is deﬁned as the smallest perturbation to the input that would change the output of amachine learning model.
As such, these explanations are promising as they can provide suggestive recourse to thebeneﬁciary in a machine learning based decision system.
As an interpretable machine learning problem, Plumb et al.[8] aim to ﬁnd such counterfactual explanations in order to explain the differences between the groups in latent space.
To this end, they employ the function r itself to ﬁnd what perturbation δ needs to be made to the input x ∈ R d so thatr ( x + δ ) belongs to the different target group.
The goal is to ﬁnd the global explanations that apply to the whole groupas opposed to the local explanations which explain only individual examples [4].
Furthermore, the explanations needto be sparse for them to be interpretable by practitioners.
Finally, these explanations should be be both symmetricand transitive .
To obtain these Global Counterfactual Explanations (GCE), the authors propose the algorithm called, Transitive Global Translations (TGT), explained hereafter.
Following the previous notation, let r : R d → R m denote our dimensionality reduction function, where d is thedimensionality of the input space and m is the latent space’s dimensionality.
Suppose X i , X j ⊂ R d get mappedto the clusters R i , R j ⊂ R m respectively.
The goal is to deﬁne the transformation t i
→ j : R d → R d on x ∈ X i asx (cid:48)
=
t i → j ( x ) , so
that r ( x (cid:48) ) ∈ R j , or equivalently x (cid:48)
∈
X j .
The proposed algorithm TGT considers the transformations of the form t i
→ j ( x ) =
x
+ δ i → j .
To ﬁnd the optimalparameters of the transformation function, authors imply a compressed-sensing based objective function as below:
[Equation]
(1) where λ (cid:107) δ i → j (cid:107) 1 is a regularization term to incentivize sparser explanations, and
¯ X i ∈ R d and ¯ R j ∈ R m denote themeans of the clusters in the input space and latent space respectively.
Given clusters 0 , 1 , . . .
, n , we get a total of12 n ( n + 1) transformations.
To further increase sparsity, we can truncate δ i
→ j to only the k features with the largestabsolute value, for some k .
An issue with this is that the translation using the truncated δ i
→ j might no longer correctlytransform inputs that get mapped to R i into inputs that get mapped to R j .
Furthermore, the transformations t i → j have to adhere to several mathematical properties.
Namely, for any clustersi, j, k these transformations should be : a) Symmetric, i.e. t i → j = t j → i − 1 and b) Transitive, i.e. t j → k ◦ t i → j = t i → k .
From these properties it follows that t i → i is the identity function I as
[Equation]
We deﬁne this condition as self-similarity .
Furthermore, the group of translations is uniquely deﬁned by t 0 → 1 , . . .
, t 0 → n ,because for any i, j :
[Equation]
Plumb et al.
[8] compare their method against the naive baseline of Difference Between the Means (DBM).
With DBM,each transformation is still a translation: t i → j ( x ) = x
+ δ i → j .
However, now δ i → j = ( ¯ X j − ¯ X i ) .
We also use this as abaseline for comparison in this report.
Since translations are a very narrow class of functions, we expanded upon the research by investigating other transformations that still satisfy the GCE requirements.
We investigate the transformations of the form t
→ i ( x ) =exp( γ 0 → i ) (cid:12) x
+ δ
→ i .
These always have a well deﬁned inverse, given by t 0 → i − 1 ( x ) = exp( − γ
→ i ) (cid:12) ( x − δ
→ i )and only have O ( d ) parameters.
The inclusion of scaling could enhance performance, while the necessary componentsof GCE are maintained.
2.1 Metrics to evaluate Global Counterfactual Explanations
To measure the efﬁcacy of the transformation function t i
→ j , the authors propose two metrics, Coverage and Correctness.1.
The Coverage ( cv ( t i → j )) is the fraction of points a ∈ R j for which there is a point b ∈
X i such that(cid:107) r ( t i → j ( b )) − a (cid:107) 2 < (cid:15) , i.e.
[Equation]
The Correctness ( cr ( t i → j )) is the fraction of points b ∈
X i for which there is some a ∈ R j such that(cid:107)
r ( t i → j ( b )) − a (cid:107) 2 < (cid:15) , i.e.
[Equation]
Note that both these metrics have the hyperparameter (cid:15) which is to be chosen carefully.
When i =
j we do not countthe point itself, there must be some other point within distance (cid:15) .
Furthermore, the Similarity metric measures the consistency of the explanations at different sparsity levels.
Given twoexplanations e 1 , e 2 where e 1 is more sparse than e 2 , the similarity of e 1 and e 2 is deﬁned as
[Equation]
This is equal to 1 if e 1 uses a subset of the features that e 2 uses.
By deﬁnition, DBM always has similarity 1 .
Scope of Reproducibility
We investigate the following claims from the original paper:
In terms of the average correctness and coverage, TGT performs equally well or better than the DBM method.
This remains true, especially for sparser explanations.
TGT explanations have similarity close to 1 .
It is consistent in which features it uses for explanations acrossdifferent sparsities.3.
TGT correctly identiﬁes known causal structure in data.
Furthermore, TGT explanations are consistent.
When altering the dataset by adding a copy of a cluster with aspeciﬁc feature altered, TGT recovers the modiﬁcation with little change to the other explanations.
Methodology of Reproducibility
We make use of the code made available by the original authors 2 for our pilot investigative study.
We ﬁrst verify thatthe provided models and explanations stay true to the claims made in the paper.
We further retrain their models on theprovided dataset.
We also made our own PyTorch [7] implementation to to further verify the claims, and to performexperiments with the proposed extension.
4.1 Model description
We identify that the scope of the original paper is to explain clusters in the low-dimensional representations.
However,obtaining meaningful and discernible low-dimensional representations is an active area of research.
The original authorsemploy a t-SNE [10] objective based Variational Autoencoder (called, henceforth, as scVIS) [5] as the r function.
Theymake use of library 3 by the original scVIS authors in their implementation.
We also implement this model in Pytorchfor our experiments.
However, we deliberately decide not to match the model implementation exactly.
This is doneto study the model-agnosticism of the TGT algorithm.
By design, TGT should be able to explain the clusters for anydifferentiable r function.
However, we maintain that r should give discernible latent representations with preservedglobal structure in the data.
In our implementation of the scVIS library, we therefore do not employ the hyperparametersand the training settings from the original library.
4.2 Dataset Description
We reproduce the ﬁndings of the authors on four datasets that they used.
We use two of these datasets as well as twonew ones to test our PyTorch implementation.
Single cell RNA [9]: This dataset has 13166 features.
We use the same number of clusters at the originalauthors, 18 in this case.2.
UCI Boston housing
This dataset has 506 entries with 13 features.
We use 6 clusters for both reproductionand replication.3.
UCI Heart disease This dataset has 303 entries with 13 features and 1 binary label.
We used 8 clusters in thereproduction and 4 in the replication.
The data was normalized to be in the range [0, 1].
UCI Iris This dataset has 150 entries with 4 features and 1 ternary label.
Ran in the reproduction with 3clusters.
N =
Breast Cancer Wisconsin (Diagnostic)
This dataset has 569 entries with 30 features and 1 binary label.
Weuse 3 clusters in the replication.
Pima Indians Diabetes Database
This dataset has 768 entries with 8 features and 1 binary label.
We used3 clusters in the replication.
The data was normalized to be in the range [0, 1]. Note that the number of clusters depend on the latent-space representation, and thus, are user dependent.
4.3 Hyperparameters
Tensorﬂow [1] Experiments For the reproduction of the original experiments, we use the same hyperparameters asthe original authors.
Pytorch For our implementation of the scVIS model, we use l2 regularization of 0 .
001 , learning rate 0 .
01 , andperplexity of 10 .
Furthermore, the degree-of-freedom for the studentT distribution is set to 2.0.
Perplexity and thedegree-of-freedom is used same as the original scVIS implementation.
We use validation set to monitor the trainingprocess of the scVIS model, and stop training when the ELBO(Evidence Lower BOund)[6] stops improving.
Fortraining the TGT explanations, we closely follow the settings from Plumb et al.
We initialize the deltas s aszero vectors.
We tune the regularization parameter λ by grid search over a ﬁxed range [0 . 0 , 5 . 0] incremented by 0.5.
Deﬁning the metrics for TGT requires careful setting of the (cid:15) hyper-parameter.
We follow the self-similarity condition(transformations of clusters to themselves should, theoretically, have correctness and coverage to be 1.0), and increasethe (cid:15) in the range [0 . 0 , 2 . 0] with increments of 0.02 until the correctness and coverage metrics are greater than 0.95.
Furthermore, we use the truncation values(TV)(refer Table 1) to evaluate on the sparsity of the explanations.
For the Pima Indians Diabetes Database and Breast Cancer Wisconsin(Diagnostic) dataset, we use the same truncation valuesas for UCI Boston Housing dataset.
4.4 Experimental setup and code
We closely follow the experimental setup in the original paper for our experiments.
We make our Pytorch code available 153to further support the reproducible research.
We reran the code of the original authors with new clustering models
[Table]
[Table]
and new explanations.
We optimize the compressed-sensing based objective function for the TGT algorithm using thegradient descent algorithm.
Our scaling extension is easily integrated in the source code, and can be optimized in asimilar way.
We train the scVIS models on the Lisa computing cluster 7 .
We use approximately 30 hours of GPU time.
We train the TGT explanations on CPU (Intel i5).
Results
For the reproduction of the authors’ experiments, we achieve approximately similar results to the original paper.
The
TGT method does seem to outperform DBM method.
The TGT explanations also have high similarity across sparsitylevels.
However, the TGT algorithm is unable to identify known causal structure in synthetic data with as good precisionas reported in the original paper.
We are also unable to match the results on the modiﬁed and corrupted data to a goodprecision.
We describe the results in the following sections:
5.1 Results reproducing original paper
5.1.1 Coverage, Correctness and similarity
In ﬁgure 1, we can see a comparison between the correctness, coverage and similarity of the TGT and DBM methods.
Note that the DBM always has similarity 1 .
The similarity of TGT stays between 1 and 0 . 9 , which supports claim 2.
We see that the coverage and correctness are similar for the UCI Heart disease dataset.
On the UCI Iris dataset, thecoverage is comparable but the correctness is better for TGT.
In both housing and RNA, the coverage and correctnessare better at less features and similar for more features.
Overall, these results support claim 1, especially for a smallamount of features.
5.2 Explaining Causal Structure in the Synthetic Data
We verify the claim that TGT identiﬁes the causal structure in the data (claim 3).
The synthetic dataset is generatedsame as the original paper, i.e. x 1 , x 2 ∼ N (0 , 0 . 2) + Bern(0 . 5) , x 2 ∼ N (0 , 0 . 05) , x 4 ∼ x 1 + N (0 , 0 . 05) .
Note thatthis dataset has four different clusters, caused by the ﬁrst two dimensions x 1 and x 2 , x 3 is noise, and x 4 is correlatedwith x 1 and x 2 .
The authors claim that for this synthetic data, TGT is able to ﬁnd that x 4 is not the cause for any group.
However, the said claim cannot be re-veriﬁed.
Interestingly, the re-run of their code doesn’t provide the justiﬁcationeither to the degree as mentioned in the paper.
We observe that both TGT and DBM are able to identify x 3 is notcausing any groups.
Thus, in this scenario, both TGT and DBM are comparable.
Refer table 2 for the explanationsobtained.
We, hereby note, that the explanations vary across multiple runs and we use the experimental setup same asthe original authors.
However, the values across the third dimension are consistently approximately 0.
5.2.1 Feature modiﬁcations
For each of the UCI datasets, the original authors add a ‘corrupted’ version where an extra cluster is added with artiﬁcialfeature modiﬁcation.
With the exception of the modiﬁed features, the corrupted class is a copy of a chosen target class.
They train TGT explanations using both the original scVIS model for the respective dataset and a model retrained on thecorrupted dataset.
We reproduce these experiments to see if TGT correctly attributes the difference between the targetand corrupted class to the right features.
Refer to Appendix A.1 for the illustrated ﬁgures and description.
Overall, weobserve that TGT is unable to identify the modiﬁcations to as good a precision as reported in the original paper.
TGT isable to identify the modiﬁcation for the UCI Iris dataset.
For UCI Heart Disease Dataset (ﬁgure 7), it does not identifythe features modiﬁed and on the UCI Boston Housing Dataset (ﬁgure 6), it identiﬁes noisy modiﬁcations.
However,with the retrained scVIS model and new representations, TGT is consistent in identifying the modiﬁcations across allthe datasets.
5.3 Results beyond original paper5.3.1 PyTorch replication
We also replicate the TGT algorithm in PyTorch.
Our Pytorch implementation includes the entire method along withthe scVIS clustering method.
In our implementation, we use the Scikit learn 8 kmeans module for our cluster selectionas opposed to the manual clustering in the Tensorﬂow implementation. However, our number of clusters argument tothe kmeans algorithm was informed by the learned low-dimensional representations for each dataset.
Due to differencesin the clustering model and cluster selection, we cannot directly compare the coverage and correctness metrics betweenour Pytorch replication and the TensorFlow reproduction.
We additionally experiment with our scaling extension to the
TGT algorithm. In the scaling extension of the TGT algorithm, along with the δ ( δ ) parameters, each cluster now has aγ ( γ ) parameters. The transformation from cluster 0 to i is now given by: t 0 → i = e γ i (cid:12) x + δ i
The gammas( γ s ) aretruncated just like the deltas and their L 1 norm is added to the regularization term.
Note that these transformations arestrictly more expressive.
If γ is the zero vector, these transformations reduce to regular TGT.
5.3.2 UCI Heart Disease and UCI Boston Housing Dataset
In ﬁgure 2 we see the results of our replication on the UCI Boston Housing and UCI Heart Disease dataset. For the UCI
Boston Housing data, the TGT method seems to slightly outperform DBM both with and without scaling.
This supportsclaim 1.
The deltas( δ s ) and gammas( γ s ) show high similarity, supporting claim 2.
For the UCI Heart Disease dataset,we do not see a difference in performance without scaling while TGT with scaling performs slightly worse. Figure [Table]
5.3.3 Breast Cancer Wisconsin (Diagnostic) and Pima Indians Diabetes Database
In ﬁgure 3 we see the results for the Pima Indians Diabetes and Breast Cancer Wisconsin (Diagnostic) dataset.
For thediabetes dataset, TGT with and without scaling outperforms DBM when more than one feature is used.
This supportsclaim 1.
Since the delta ( δ ) similarity is close to 1 , claim 2 is also supported.
For the Breast Cancer dataset, we seesimilar performance for DBM and TGT and slightly worse performance with scaling.
The deltas ( δ s ) still have highsimilarity, supporting claim 2.=
5.3.4 Scaling extension
In ﬁgure 2, we can see the difference in performance on two dataset included in the original experiments.
Scaling doesnot seem to improve performance on the UCI Boston Housing dataset and slightly decreases performance on the UCI Heart Disease dataset.
The similarity of the gammas ( γ s ) is mostly above 0 .
In ﬁgure 3, we see the same metrics for the Breast Cancer and Pima Indians Diabetes Dataset.
For the Diabetesdataset, the performance improves slightly and the gammas( γ s ) show high similarity.
For the Breast Cancer dataset, theperformance is about the same but the gammas( γ s ) show relatively low similarity.
Altogether, these results suggest that the addition of scaling does not signiﬁcantly improve the accuracy and correctnesswhile making the transformations more complex.
Based on our experiments, we do not recommend the addition ofscaling in the explanation functions, and conclude that the original TGT is expressive enough.
5.3.5 Experiment with Modiﬁed Synthetic Data
In order to study the efﬁcacy of the proposed scaling function, we perform experiments on the synthetic dataset.
Wemodify one of the groups of points G by performing the operation ax ki + b , where i corresponds to the group numberand k denotes the feature dimension which we modify.
We deﬁne a ∼ U (1 . 0 , 2 . 0) and b ∼ U ( − 0 .
We addmodiﬁed group G (cid:48) into the original data D to get the new data D (cid:48) .
We follow the experiment setup from the originalpaper as: a) r ( G (cid:48) ) should form a different group of its own.
b) G (cid:48) should be within the distribution of the original D .
In this study, we want to investigate whether the TGT with scaling is able to recover the modiﬁcations, and if in doingso it affects the explanations between other groups.
The sampling procedure gave a=2.0, b=0.60 and we keep k=0.
Weobserve that the explanations with scaling are able to recover the modiﬁcation to an approximate degree(scaling factore γ ≈ 2 . 38 , actual a=2.0), and give better correctness as compared to the regular TGT (refer ﬁgure 3).
Interestingly, thetranslations explanations of the scaling extension are approximately equal to the deltas of the regular TGT.
The exactresults can be found in Table 3.
Figures 8 and 9 in the appendix show the data spread and resulting translations.
5.3.6 Experiments with Probing Classiﬁer
To further investigate the efﬁcacy of TGT explanations, we use a probing-classiﬁer [2] as a proxy to study the qualitativedifferences of the features selected by TGT and DBM.
For each cluster, we train a binary classiﬁer with features rankedhighest by TGT and DBM at different sparsity levels K. We compute the overall accuracy at each sparsity level usingthe ensemble of these binary classiﬁers.
As can be seen in Figure 4, the results demonstrate that for sparser explanations,
TGT selects features that lead to higher accuracy of the ensemble classiﬁer than those selected by DBM.
This furthervalidates the paper’s claim that TGT leads to better sparse explanations as compared to DBM.
Furthermore, we alsouse the probing classiﬁer to understand the differences between the groups.
For each pair of group, we train a Binary
Linear Classiﬁer to predict the group of a test point.
We, then, investigate the feature importances of the classiﬁertowards decision making.
We ascertain that the features classiﬁer give more importance to while decision making arethe deﬁning property of the class.
Interestingly, we ﬁnd that the more important features according to the classiﬁercorrespond to the explanations provided by the TGT algorithm.
Refer to ﬁgure 11.
This provides further evidence that TGT is able to ﬁnd real distinctive signals as explanations.
Discussion
Based on the reproduction of the original experiments, claims 1 and 2 seem to hold, the experiments for claim 4 do notall support it, but the claim does seem to hold.
Claim 1 and 2 seem to hold in particular for sparse explanations.
Theevidence for claim 3 is inconclusive.
The coverage and correctness in our reproduction were not always the same as inthe original paper.
It is difﬁcult to compare these metrics for different clustering outcomes, as they depend on the (cid:15)parameter which depends on the clustering.
A major difﬁculty in reproduction is the cluster selection.
When retraining the scVIS model, the latent space representation structure changes.
The authors provide no method as to determine the different clusters other than visualinspection.
Cluster selection could be an explanation for the differences in results between the original experiments andour reproduction.
To verify the results with more conﬁdence, a robust method for cluster selection might be required
A.2 Synthetic data
[Figure]
A.3 Probing Classiﬁer and Feature Importance