DiSC: Differential Spectral Clustering of Features
Abstract
Selecting subsets of features that differentiate between two conditions is a key taskin a broad range of scientific domains.
In many applications, the features of interestform clusters with similar effects on the data at hand.
To recover such clusterswe develop DiSC, a data-driven approach for detecting groups of features thatdifferentiate between conditions.
For each condition, we construct a graph whosenodes correspond to the features and whose weights are functions of the similaritybetween them for that condition.
We then apply a spectral approach to computesubsets of nodes whose connectivity differs significantly between the conditionspecific feature graphs.
On the theoretical front, we analyze our approach with atoy example based on the stochastic block model.
We evaluate DiSC on a varietyof datasets, including MNIST, hyperspectral imaging, simulated scRNA-seq andtask fMRI, and demonstrate that DiSC uncovers features that better differentiatebetween conditions compared to competing methods.
Introduction
Detecting variables or features that separate between two or more conditions is a critical task in manyscientific domains.
Often, the separation between conditions is caused by a large number of stronglydependent features that form one or more differentiating clusters.
Recovering those clusters canprovide insights into the data and its underlying mechanisms.
Strongly dependent differentiatingfeatures are common, for example, in computational biology.
In transcriptome analysis, the dataconsists of the gene expression of multiple cells.
Often, the cells correspond to two or more biologicalconditions, such as various cell types, the existence of a particular disease, response to medicaltreatment, or two-time points in an evolutionary process [45].
Uncovering groups of differentiatinggenes (also known as pathways ) may significantly contribute to our understanding of biologicalprocesses happening in one condition and not in the other.
Differential feature grouping has variousother interesting applications in neuroscience [40], computer vision and machine learning [6].
Inall applications, detecting groups that are significant for differentiating between conditions addsinterpretability to the otherwise black box models.
Identifying differential features has been addressed in various contexts.
Here we briefly review twomain settings: 1) grouping of features with a similar contribution to a prediction task, and 2) datafusion - identifying shared features in datasets from different sources or views.
In the context of featureselection, regularization based methods are a popular approach.
Most of these methods assume a linearregression model over the features to predict a function over the data points, e.g., class label.
To selectonly meaningful features, a regularizer is added to the fidelity term.
For imposing a sparse solution,classical Lasso relaxes the l 0 norm of the coefficient vector with its convex surrogate l 1 [12, 34].
Alternatively, a recent approach aims to approximate the l 0 regularizer via stochastic gates [41].
The sparsity induced by these methods makes them problematic for detecting groups of dependentfeatures since they typically yield only a small number of features out of a large dependent group.
To address the case of correlated features, various extensions of Lasso were developed.
Clustered
Lasso [30] promotes sparsity among the coefficients and equal significance to non-zero coefficientsin the same group.
Ordered Weighted L1 regularization (OWL) [4] penalizes the coefficients in theorder of their magnitude, i.e., the higher the absolute value of the coefficient, the higher the penaltythat will be imposed.
This induces equal coefficients to the correlated features [13].
Elastic Net[46] uses a combination of l 1 and l 2 regularizers which creates a feature grouping effect.
Cluster
Elastic Net [39] assumes that each feature belongs to one of k-distinct clusters and uses a clusteringpenalty, along with Lasso, which minimizes the sum of pairwise distances between the associationsof features with the prediction within each cluster.
A method that combines feature selection andgrouping was also developed in [5].
A few methods explicitly use the correlation between the featuresin the regularization terms.
[24] estimates the covariance between features from which they form agraph.
The graph Laplacian matrix is then used as a quadratic regularizer function.
This line of work typically addresses a linear regression model.
However, in many settings thedependency on the important features is highly non-linear.
This is typically the case, for example, inmost discrete settings such as classification and clustering.
Furthermore, most of these methods donot separate different groups of correlated features that are significant in a classification or regressionproblem.
In many settings, such a separation is important in order to gain insight to different sourcesof the variability in the data.
A different approach related to this work is feature selection via discriminant subspace analysismethods such as linear discriminant analysis [33, 29] and the Fukunaga-Koontz transform [16, 28].
Here, the primary goal is to identify the subspace that best discriminates the classes accordingto a given criterion.
For example, the classic Fischer criterion [15] maximizes the ratio betweeninter-class variance and intra-class variance.
Recent works applied similar methods in the contextof non-negative and Boolean matrix factorization [19, 20] .
Some metric learning approaches [27],which take into account triplet relationships between points, learn a linear transform on the features sothat the distance in the projected space better separates the classes.
However, these approaches do notexplicitly perform feature selection to identify group of correlated features that separate the classes.
In this paper we develop a data-driven method to detect and group relevant features that does notrely on a specific model.
We consider the classification setting, where our aim is to identify classspecific information in the feature space that distinguishes between different conditions.
To thatend, we develop DiSC, a data-driven method to reveal groups of differential features.
Our approachconsists of two main steps: first, for each class, we compute a class-specific graph, whose nodescorrespond to the features of that class separately.
Next, we apply a spectral approach to obtain anembedding of the features that identifies groups of features whose connectivity differs between thegraphs.
Our contributions in this paper are: 1) develop a nonlinear spectral approach on the featurespace to identify groups of correlated features that distinguish between datasets, 2) our solution isnon-symmetric such that we identify class-specific differential features, 3) we provide theoreticalanalysis in the setting of a stochastic block model underlying the features.
Our spectral approach is related to recent papers that address the challenges of multiview and datafusion.
Here, data samples are observed by multiple sets of sensors, and the goal is to identify latentrepresentations of the samples that are shared for all sets.
The shared information can be recoveredvia the shared latent space between multiple sources.
Alternating Diffusion Maps (ADM) [23] is anonlinear manifold learning approach to reveal shared latent variables.
Shnitzer et al.
[32] extendthis to identifying both common structures and the differences between the manifolds underlyingthe different modalities.
In an alternative solution, [25] proposes a kernel and distance metric fordiffusion maps on multiview datasets.
There are numerous other data fusion techniques [26], [9] andapplications [21].
Most of these techniques require one-to-one correspondence between data samplesfrom different views, e.g., simultaneous recordings from different sources, and they mainly focusextracting a shared subspace or a shared hidden variable of the samples.
On the other hand, in oursetting the correspondence is between features in two datasets, and not between samples.
Our goal isto uncover connectivity patterns that are condition specific .
Problem formulation
We begin with a formal description of our problem, followed by relevant notation.
We consider twodatasets X A ∈ R n A × p and X B ∈ R n B × p where the rows in each matrix are the high-dimensional samples and the columns are the features, see illustration in Fig.
1a.
Both datasets have the same pfeatures, such that the feature column in A , denoted X A · i ∈ R n A corresponds to the feature columnin B , denoted X B · i ∈ R n B .
Our goal in this work is to detect one or more subsets of features thattogether differentiate between the two states A and B .
These subsets can be, for example, groups ofgenes that participate in some biological pathways, or a subset of brain parcels in fMRI data withsimilar blood oxygen level dependent (BOLD) activity.
In our approach, we compute two graphs, denoted G A , G B with p nodes that correspond to thefeatures of the given data (Fig. 1b).
We denote by W A , W B the weight matrices of the two graphs,whose elements are functions of the similarity between features, as computed by two kernel functions,
K A ( X A · i , X A · j ) :
R n A × R n A → R and
K
[Equation] [Equation]
×
[Equation]
These for example can be the RBF kernel K ( x, x ′ )
=
exp {−∥ x − x ′ ∥
2 /ϵ 2 } .
Our underlyingassumption is that differences between the two states A, B are expressed as differences in connectivitybetween the two graphs G A and G B .
For example, a pair of features i, j may be insignificant in state
A and highly significant and dependent in B .
This will imply,
K
[Equation]
( X
[Equation]
i , X
[Equation]
≈and
K B ( X
[Equation]
X
[Equation]
> 0 .
If there is a subset of l correlated features i 1 , . .
, i l that are strongly dependent in B but not in A , theresultant nodes in G B will form an independent component with dense connections among the nodes.
However, this independent component will not exist in G A , see illustration in Figure 1.
Thus, thetask of obtaining sets of significant features boils down to identifying the independent componentsthat appear in one graph but not the other.
DiSC
[Figure]
3.1 A graph cut perspective
Given a graph G with n nodes and its associated weight matrix W ∈ R n × n , the minimum-cut of G isthe minimum, over all possible partitions α and β , of the sum of the edge weights (cid:80) i ∈ α,j ∈ β W ( i, j ) .
This task is strongly related to the spectral clustering algorithm.
For completeness, we begin with abrief description of this relation.
For a more thorough review see [38].
A variation of the minimum-cut, designed to avoid highly imbalanced partitions, is the ratio-cut
[Equation]
where | α | , | β | denote the subset size.
This can be formulated as
[Equation]
and L u is the unnormalized graph Laplacian matrix given by D − W , where D is a diagonal matrix containing in the diagonal the degrees of each node.
Note that f is a non-binary indicator vector and if α, βare non-empty, then f is orthogonal to the constant vector with ∥ f ∥ 22 = n .
Minimizing Eq.
(3) over allpartitions is a discrete optimization problem.
To avoid it, one can relax the discrete requirement overthe elements of f in Eq.
(3), while still maintaining the constraints f T 1 = 0 and ∥ f ∥ 22 = n .
This relaxation yields a simple spectral solution, where the nodes are partitioned according to the sign of thegraph Laplacian eigenvector corresponding to the second smallest eigenvalue (a.k.a Fiedler vector).
The mincut criterion is designed to attain densely connected components that have low connectivitywith the rest of the graph.
In our work, however, densely connected components that appears in bothgraphs are of no interest, as they do not differentiate between the two states.
Rather, we would liketo detect a partition ( α, β ) , say on G B , that has two properties: 1) Rcut ( α, β ) , on G B is minimized,which indicates that α and β are independent connected components in G B ; 2) To avoid detection ofcomponents that exist both in G A and G B , Rcut ( α, β ) on G A is lower bounded by some constant.
This double objective can be formulated by
[Equation]
G B on the feature space for each dataset, we aim to find which nodes have different connectivity pattern between the two datasets.
c)
The weight matrices each follow a stochastic block model, where in B a subset of of the nodes form a separate block.
d) Subspaces spanned by the eigenvectors of P A and P B , and the differential feature between B and A .
The constraint ∥ f ∥ = 1 is added for uniqueness, since L A , L B are rank deficient.
Relaxing thediscrete requirement in Eq.
(3), the solution to Eq.
(4) is given by the generalized eigenvalueproblem L B f = µL
A f .
If L A is invertible, the solution to this is given by the eigenvalue problem
L −
A L B f = µf .
But if L A is not invertible, one numerical trick is to strengthen the diagonal of L Ato make it a full rank matrix.
Then the solution is given by the eigenvalue problem,
[Equation]
Motivated by the above derivation, in the next section we present our spectral approach for groupfeature discovery.
In the appendix we present a similar derivation that is based on the normalized cut,whose relaxation relates to the eigenvectors of the random walk graph Laplacian.
3.2 Spectral approach for group differential feature extraction
In practice, our approach is based on the spectral decomposition of the two random-walk Laplacianmatrices P A
= D − 1 A W A and P B = D − 1 B W B , where D A and D B represent degree matrix of Aand B respectively, rather than the unnormalized graph Laplacian.
Let U B ∈ R p ×
d B be a matrixcontaining the d B leading right eigenvectors of P B .
These vectors, scaled by their correspondingeigenvalues, are the diffusion vectors of the two graphs [8].
A vector containing the i -th elementof the columns of U A is defined as the diffusion map representation of the i -th node.
Here, we definea second operator Q B , that forms a projection matrix onto the complementary subspace of U B :
[Equation]
We define the differential vectors of G A , denoted ( V A ) i and their significance level ( σ A ) i via [Equation] i
[Equation]
and
[Equation]
where E is a subspace in R p and S ( E ) denotes unit Euclidean sphere in E .
The relation betweenthe diffusion vectors U A and the differentiating vectors V A is similar in nature to the relation betweenthe outcome of the min-cut criterion in Eq.
(3) and the double objective in Eq.
The diffusionvectors in U B highlight significant processes that underlie state B .
However, we are only interested inprocesses that differentiate between state A and B .
These are highlighted in the differential vectors V B .
Thus, our approach for detecting differences between graphs is similar to the solution of the doubleobjective criterion given in Eq.
However, to increase the stability of our approach we make twochanges: 1) replace the inverse matrix in Eq.
(5) with the projection matrix Q B to the complementarysubspace, thus avoiding potential complications and additional constraints required for the inversecomputation.
2) Instead of the right eigenvectors, we compute the leading singular vectors.
Note that the operator P A Q B whose singular vectors are used to attain differential features is notsymmetric in A and B .
The differential vectors in V A highlight components that are significant in Abut not in B .
The differential vectors V B are attained from the singular vectors of P B Q A .
Differential vectors for downstream analysis The differential vectors V A , V B and their scoresσ A , σ B can be used in several ways.
Here, and in the experimental section, we apply the vectors in oneof two applications: (i) Detecting of differential subsets of features.
This can be done by performingk -means clustering over the rows of the matrix V A ; (ii) Computing differential meta-features via
V TA X A and V TB X B .
In section 4 and supplementary material we provide several examples forusing both in downstream analysis (e.g., clustering, classification).
In the next section we derivea theoretical justification for application (i) for our approach that is motivated by the stochasticblock model.
Algorithm 1 summarizes the steps of the DiSC algorithm.
Further details about thecomputation of the graph and discussion on choice of hyperparameters is given in App. A.
3.3 Two stochastic block models
The stochastic block model (SBM) has received a lot of attention due to its significant role in obtainingtheoretical guarantees for community detection.
In this setting, the individual community membersare modeled as nodes in a random graph G .
The edge weights W ij are sampled according to a
Bernoulli distribution, with
Pr( W ij = 1) = (cid:26) p if i, j belong to the same community
[Equation]
Usually, one assumes p > q such that the connectivity within a block is stronger than the connectivitybetween blocks, See [1] for further details.
In our setting, we model the features in both states by two random graphs G A and G B .
We considera toy problem in which the features that differentiate two states A and B have different graphconnectivity in G A and G B .
In graph G A the nodes are partitioned into two communities of sizesl and l + s , with s < l .
In contrast, the graph G B has three communities, denoted α, β and γ .
Community α is equal to the first community in G A , while β, γ are a partition of the second into twogroups of size l and s .
Our goal is to detect the elements in γ via a spectral approach whose steps are similar to algorithm 1.
For simplicity of exposition, instead of using the graph Laplacian matrix as in step 2 of the algorithm,we use the symmetric weight matrices W A , W B .
Our goal in this analysis is to provide insight intoour ability to recover differentiating groups such as γ via algorithm 1.
Our main parameters ofinterest are the size of the differentiating group γ , and the ratio between the size of γ and the sizeof the original block.
Elements that depend on other parameters of the model (e.g. p and q ) arereferred to as constants.
Our derivation consists of three main steps.
The proofs of our results aregiven in Appendix B.
We note that all matrix norms (i.e. ∥ X ∥ ) in the following section and relevantappendices are the spectral norm.
Step 1: Here we address the non-random matrix E [ W B ] where E [ · ] is the expectation operator.
Let v α , v β , v γ be the leading three eigenvectors of E [ W B ] and by e γ a binary indicator vector withelements ( e γ )
i = 1 if i ∈ γ .
Lemma 1.
The distance between v γ and 1 √ s e γ is bounded by
[Equation]
Lemma 1 shows that if the ratio s/l is small enough, one can recover the elements in γ by applying athreshold to v γ .
However, in practice we only have access to W B and W A .
In the next step we boundthe difference between v γ and the eigenvector we compute by our spectral approach.
Step 2: Let u α , u β be the two leading eigenvectors of the random weight matrix W A , and let
[Equation]
In this step we address the leading eigenvector of Q W A W B Q W A .
Lemma 2.
Let ˜ v γ denote the leading eigenvector of Q W A W B Q W A .
Then, with high probability
[Equation]
Step 3:
Observing the two lemmas, we see that there is a tradeoff concerning the size s of thedifferentiating group γ .
On the one hand, Lemma 1 shows that having a small value for s makes theelement more distinguishable in v γ and thus easier to detect.
On the other hand, if s ≪ √ l then thecomputed vector ˜ v γ is too noisy to actually detect the relevant features.
Combining the two lemmas,we conclude with the following theorem.
Theorem 1.
We assume that s, l are large such that s, l ≫ max i ( C i ) , and s = b √ l for some constantb .
Applying a threshold to ˜ v γ yields, with high probability, the elements of γ , with a number of errorsproportional to √ s .
Proof of Theorem 1.
Combining Lemmas 1, 2 with the assumption s = b √ l yields the inequality
[Equation]
for some constant C 6 ( p, q, b ) .
We recover the elements in γ by setting a threshold of 12 √ s to ˜ v γ .
Eachmisclassified element contributes at least 1 / (2 √ s ) to the l 2 distance between 1 √ s e γ and ˜ v γ .
Thus, thenumber of errors is bounded with high probability by C 6 ( p, q, b ) √ s .
Theorem 1 provides insight as to our ability to detect small differentiating groups of features. Theanalysis is based on a simple toy model where the differentiating subset is densely connected to alarger block in one state, but only sparsely connected with the block in the second state, see illustrationin Figure 1. In Sec. 4, we present results on additional models of feature connectivity.
Experiments
4.1 Toy problems
We demonstrate the usefulness of our methodology in three toy datasets which present different scenarios of feature correlation patterns: 1) a subset of features that is uncorrelated in one dataset becomescorrelated in the second 2) a subset of features that are correlated in one dataset are divided into twocorrelated groups in the second.
The third problem demonstrates generalizing DiSC to more thantwo datasets.
In all experiments the kernels K A , K B are the RBF kernel with an adaptive bandwidth.
The choice of the all the hyperparameters (bandwidth, d A and d B are discussed in Appendix A.
Identifying newly connected features We generated two datasets X A and X B with p = 250features and n A = n B = 10 , 000 samples from a Gaussian mixture model (Fig. 2(a)), whose featurecorrelation is shown in Fig. 2(b).
In the two datasets, the first 100 features are i.i.d samples from anormal distribution and the next 50 features are sampled from two Gaussian distributions with lowrank covariance matrices.
Features 151-200 in X A are sampled from two other Gaussian distributionswith low rank covariance matrices, whereas in X B , they are independent noise.
Features 201-250 in
X A are independent noise, whereas in X B , they are sampled from two other Gaussian distributionswith low rank covariance matrices.
Our goal is to identify the features 151-200 as the features that distinguish X A from X B , and viceversa for features 201-250.
We apply DiSC with d A = d B = 20 .
The first two significance values of the differential features are high and then significance drop drastically (Fig. 2(c)), indicating that thefirst two differential vectors are important in each dataset.
These top two differential features for X Aand X B are shown in Figure 2(d).
Clearly, ( V A ) 1 and ( V A ) 2 highlight the features between 151-200and more precisely, the two Gaussian mixtures are separately represented in each of these vectors.
Similarly, ( V B ) 1 and ( V B ) 2 highlight the features between 201-250 and these represent the other two
Gaussian mixtures.
On the other hand, the diffusion maps eigenvectors, U A and U B , captures all theconnected components and not just the differential features as unique groups, as shown in Fig. 2(e).
Identifying subsets of connected features We generated two datasets X A and X B with p =
200features and n A = n B = 10 , 000 with correlation between the features as shown in Fig. 3(a).
In boththe datasets, the first 100 features are correlated.
In X A , all the remaining features form a secondcorrelated group, yielding two connected components in the feature space.
In X B the remainingfeatures are composed of two groups of correlated features, namely, feature indices 101-125 and126-200.
Here the goal is to identify these smaller subsets of correlated features in X B , as thissignifies an increase in the dimensionality of correlation structure compared to X A .
This also meansthat X B has richer feature information than X A , therefore we have to identify that there are nodifferential features in X A .
We use the significance level associated with the differential vectors todetermine if the differential vectors are meaningful.
We apply DiSC with d A = d B = 20 .
Fig.
3(c)shows the differential vectors of X A and X B and the corresponding significance levels are shownin Fig. 3(b).
The significance level associated with the differential vectors of X A is negligiblecompared to that of X B , indicating that feature information in X A is contained in X B .
In addition,the difference between the significance of the first two differential vectors of X B is large.
Therefore,the first differential vector of X B captures the major difference, and it groups feature indices 101-125and 126-200 with positive and negative values respectively (Fig. 3(c)).
In contrast, diffusion mapscaptures the two connected components in X A and three connected components in X B , and not justthe differential components (Fig. 3(d)).
4.2 MNIST
The MNIST dataset [22] consists of images of hand written digits from 0 to 9, each of dimension28 × 28 pixels.
It has 60k training samples and 10k testing samples.
We used Algorithm 2, DiSCon multiple datasets, on the MNIST dataset to extract differential features for each of the digits:features present in a single digit but are not present in each of the other digits.
To have a quantitativemetric to measure the significance of these differential features, we develop a classifier for a 10-classclassification problem.
For this, we use K-means clustering on the differential features for each digit(K=10) and compute a meta-feature by averaging pixel values for each cluster.
Logistic regression isperformed on these meta features to distinguish between the 10 classes.
We compare our approach with Diffusion maps, Elastic Net (EN) and Elastic Net - logistic.
Fordiffusion maps, we replaced the differential features with diffusion maps and followed the sameprocedure.
For EN and EN-logistic, we obtained the feature importance vectors by training a classifierto distinguish between the 10-classes.
K-means clustering is performed on these feature importancevectors.
This cluster information is further used to compute meta-feature and finally for classification.
Additionally, the entire data (784 features) is used to obtain the clusters using K-means clusteringwhich are further used to compute classification accuracy following the procedure mentioned above.
We consider this as a baseline.
The results are tabulated below in Table 1.
We can see that our proposed method has the best performance, as it can capture the importantdifferential information needed for classification.
Since diffusion maps capture both shared anddifferential features, and entire data has all the information, their performance is poor compared to
DiSC.
Elastic Net has the least accuracy because it is designed for a regression setup but this is aclassification problem.
Elastic Net-logistic has better performance over Elastic Net as we use a crossentropy loss function – the one used for classification setup.
Further, DiSC has better performancethan Entire Data as it is designed to capture the differential features.
However, DiSC outperformed
EN-logistic as it is designed to extract groups of differential features unlike EN-logistic which justextracts all the differential features as a single group, especially when those features have similareffect on the classification problem.
Note in App.
E we include an additional experiment on pairwiseclassification of MNIST digits which also includes more details about the choice of hyper-parametersfor each of these methods.
Entire Data
[Table]
291 4.3 Hyperspectral imagery
We apply DiSC to a hyperspectral imagery change detection dataset [11], consisting of hyperspectralimages of a particular scene that were captured during different weather conditions, lighting conditions, and across four different months of the year (August, September, October and November).
Each hyperspectral image is captured at a resolution of 100 × 100 pixels and 124 different bands.
We denote these by X aug , X sep , X oct and X nov .
These images consist of a metal frame, grass andtrees in the background, as shown in Fig.
An additional hyperspectral image, referred to as‘October-change’ and denoted X oct-c , was captured in October, in which an added object, a tarp , wasincluded in the scene, see Fig.
The goal is to detect the tarp as an added object, albeit various otheraforementioned different conditions during which the image is captured.
We consider X A ∈ { X aug , X sep , X oct , X nov } and X B = X oct-c and compute the differential vectorsfor these four pairs of datasets.
Note that this is similar to the toy problem presented in Sec.
4.1 andthe theoretical analysis in Sec.
3.3, where a group of correlated features in X A (the pixels belongingto the grass) are divided into two groups of correlated features in X B (grass and tarp).
With theaddition of the tarp, the pixels belonging to the tarp remain correlated but their connectivity with theother pixels in the grass is lost.
The differential features of X B identify the tarp, see App.
E Fig.
We pick the top four significant differential vectors of X B and perform k-means clustering onthese with k=3.
Figure 4(b) shows these three feature clusters for the four pairs of datasets.
Thetarp is revealed as a dominant cluster in all months.
We compare our results with Diffusion mapsfor changing data (DM-changing data) [7], which introduces a distance metric that measures thedistance between diffusion maps calculated on a dataset that changes over time.
We compute thispixel-wise distance between X A and X B and cluster the pixels into 3 groups based on this distance.
These clusters are illustrated in Figure.4(c).
This approach is much more affected by the acquisitionconditions, For example: weather and lighting, than our approach, as it groups additional objects thathave not changed along with the tarp.
Finally, DiSC performs better than DM-changing data in thepresence of added noise (see Appendix E).
4.4 fMRI
We assess the performance of DiSC in identifying groups of brain parcels with correlated
BOLD activity in a working memory task fromthe Human Connectome Project 1 [36], wheresubjects executed interleaved blocks of 0-backand 2-back working memory tasks.
In thesetasks, subjects are instructed to monitor a sequence of visual items and to respond whenevera presented item is the same as the one previously presented 2 items ago (2-back) or a predetermined item (0-back).
This fMRI datasetconsists of 515 subjects (subjects with high motion or incomplete data were removed), and awhole-brain, functional atlas [31] was used toextract time-courses from p
= 268 brain parcels.
Dataset X A is composed of all blocks from the0-back task and X B is composed of all blocksfrom the 2-back task.
For each subject we calculated the top 2 diffusion maps eigenvectors as well as the top 2differential features for each dataset, and average these across all subjects.
Fig.
5 displays the correlation of each of the averaged vectors with indicator vectors for 10 canonical brain networks [14].
Results show that diffusion maps is mainly correlated with visual networks, while the differentialvector for the 0-back task being most correlated with the visual II network.
On the other hand, thedifferential vector for the 2-back task is most correlated with the frontal-parietal network, which hasbeen shown to be predictive of working memory performance [2].
Thus, as opposed to diffusion maps,
DiSC reveals that the 2-back task incorporates more high-level cognitive regions (e.g., prefrontal)compared to the 0-back task which has lower cognitive load [17, 18].
Discussion and future work
In this paper we introduced DiSC, a spectral approach for finding differential features between two ormore datasets.
We demonstrate the results of our model on various synthetic and real-world datasetsand show that DiSC extracts better differential features as compared to the competing techniques.
We also show the experimental results on more than two datasets.
One limitation of our method isthat it addresses only differences in “connectivity", or correlation, between features, not the featurevalues themselves.
Another limitation is the choice of the hyperparameters, d A and d B .
High valueswould result in extracting noise or nuisance features as the differential features, and low values mightnot detect the essential differential features.
Finally, the problem of “redundant" eigenvectors [10]arising in spectral clustering and manifold learning may further complicate choosing the correctdimensionality.
This can be mitigated by using non-redundant eigenvectors [3] which we will explorein future work.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 2 , We assume that difference between the two datasets are represented in the differences in their
connectivity.
(c) Did you discuss any potential negative societal impacts of your work?
[No] Our work does not have any direct potential negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix B3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] We will include
the code in the supplemental material
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No] Our algorithm is not computationexpensive.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] We used open source data
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No] We are using publicly available and vastly used datasets.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A](b)
Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[-Thesecond (and smallest non-zero) eigenvalue of E [ W A ] is larger than ( p − q ) l 
-]
[N/A]