Factorized linear discriminant analysis for phenotype-guided representation learning of neuronal gene expression data

A central goal in neurobiology is to relate the expression of genes to the structural and functional properties of neuronal types , collectively called their phenotypes . Single-cell RNA sequencing can measure the expression of thousands of genes in thousands of neurons . How to interpret the data in the context of neuronal phenotypes ? We propose a supervised learning approach that factorizes the gene expression data into components corresponding to individual phenotypic characteristics and their interactions . This new method , which we call factorized linear discriminant analysis ( FLDA ) , seeks a linear transformation of gene expressions that varies highly with only one phenotypic factor and minimally with the others . We further leverage our approach with a sparsity-based regularization algorithm , which selects a few genes important to a specific phenotypic feature or feature combination . We applied this approach to a single-cell RNA-Seq dataset of Drosophila T4/T5 neurons , focusing on their dendritic and axonal phenotypes . The analysis confirms results obtained by conventional methods but also points to new genes related to the phenotypes and an intriguing hierarchy in the genetic organization of these cells . 1 INTRODUCTION . The complexity of neural circuits is a result of many different types of neurons that specifically connect to each other . Each neuronal type has its own phenotypic traits , which together determine the role of the neuronal type in a neural circuit . Typical phenotypic descriptions of neurons include features such as dendritic and axonal laminations , electrophysiological properties , and connectivity ( Sanes & Masland , 2015 ; Zeng & Sanes , 2017 ; Gouwens et al. , 2019 ) . However , the genetic programs behind these phenotypic characteristics are still poorly understood . Recent progress in characterizing neuronal cell types and investigating their gene expression , especially with advances in high-throughput single-cell RNA-Seq ( Zeng & Sanes , 2017 ) , provides an opportunity to address this challenge . With massive data generated from single-cell RNA-Seq , we now face a computational problem : how to factorize the high-dimensional data into gene expression modules that are meaningful to neuronal phenotypes ? Specifically , given phenotypic descriptions of neuronal types , such as their dendritic stratification and axonal termination , can one project the original data into a low-dimensional space corresponding to these phenotypic features and their interactions , and further extract genes critical to each of these components ? Here we propose a new analysis method named factorized linear discriminant analysis ( FLDA ) . Inspired by multi-way analysis of variance ( ANOVA ) ( Fisher , 1918 ) , this method factorizes data into components corresponding to phenotypic features and their interactions , and seeks a linear transformation that varies highly with one specific factor but not with the others . The linear nature of this approach makes it easy to interpret , as the weight coefficients directly inform the relative importance of each gene to each factor . We further introduce a sparse variant of the method , which constrains the number of genes contributing to each linear projection . We illustrate this approach by applying FLDA to a single-cell transcriptome dataset of T4/T5 neurons in Drosophila ( Kurmangaliyev et al. , 2019 ) , focusing on two phenotypes : dendritic location and axonal lamination . 2 FACTORIZED LINEAR DISCRIMINANT ANALYSIS ( FLDA ) . Suppose that we are given gene expression data of single neurons which are typically very highdimensional . These cells are classified into cell types , as a result of clustering in the highdimensional space and annotations based on prior knowledge or verification outcome ( Macosko et al. , 2015 ; Tasic et al. , 2016 ; Shekhar et al. , 2016 ; Tasic et al. , 2018 ; Peng et al. , 2019 ) . We know the phenotypic traits of each neuronal type , therefore each type can also be jointed defined by the phenotypic features . We want to find an interpretable low-dimensional embedding in which certain dimensions represent factors of phenotypic features or their interactions . This requires that variation along one of the axes in the embedding space causes the variation of only one factor . In reality , this is hard to satisfy due to noise in the data , and we relax the constraint by letting data projected along one axis vary largely with one factor while minimally with the others . In addition , we ask that cells classified as the same type are still close to each other in the embedding space , while cells of different types are far apart . As a start , let us consider only two phenotypic features of neurons , dendritic stratification , and axonal termination , both of which can be described with discrete categories , such as different regions or layers in the brain ( Oh et al. , 2014 ; Euler et al. , 2014 ; Sanes & Masland , 2015 ; Kurmangaliyev et al. , 2019 ) . Suppose that each cell type can be jointly represented by its dendritic location indexed as i and axonal lamination indexed as j , with the number of cells within each cell type nij . This representation can be described using a contingency table ( Figure 1A , B ) . Note here that we allow the table to be partially filled . Let xijk ( k ∈ 1 , 2 , ... nij ) represent the expression values of g genes in each cell ( xijk ∈ Rg ) ) . How to find linear projections yijk = uTxijk and zijk = vTxijk that are aligned with features i and j respectively ( Figure 1C ) ? We first asked whether we could factorize , for example , yijk , with respect to components depending on features i and j . Indeed , motivated by the linear factor models used in multi-way ANOVA and the idea of partitioning variance , we constructed an objective function as the following , and found u∗ that maximizes the objective ( see detailed analysis in Appendix A ) : u∗ = arg max u∈Rg uTNAu uTMeu ( 1 ) When we have a complete table , and there are a levels for the feature i and b levels for the feature j , we have NA = MA − λ1MB − λ2MAB ( 2 ) where MA , MB , and MAB are the covariance matrices explained by the feature i , the feature j , and the interaction of them . λ1 and λ2 are hyper-parameters controlling the relative weights of MB and MAB with respect to MA . Me is the residual covariance matrix representing noise in gene expressions . Formal definitions of these terms are the following : MA = 1 a− 1 a∑ i=1 b∑ j=1 ( mi . −m .. ) ( mi . −m .. ) T ( 3 ) MB = 1 b− 1 a∑ i=1 b∑ j=1 ( m.j −m .. ) ( m.j −m .. ) T ( 4 ) MAB = 1 ( a− 1 ) ( b− 1 ) a∑ i=1 b∑ j=1 ( mij −mi . −m.j + m .. ) ( mij −mi . −m.j + m .. ) T ( 5 ) Me = 1 N − ab a∑ i=1 b∑ j=1 [ 1 nij nij∑ k=1 ( xijk −mij ) ( xijk −mij ) T ] ( 6 ) where m .. = 1 ab a∑ i=1 b∑ j=1 mij ( 7 ) mi . = 1 b b∑ j=1 mij ( 8 ) m.j = 1 a a∑ i=1 mij ( 9 ) in which mij = 1 nij nij∑ k=1 xijk ( 10 ) An analogous expression provides the linear projection v∗ for the feature j , and w∗ for the interaction of both features i and j . Similar arguments can be applied to the scenario of a partial table to find u∗ or v∗ as the linear projection for the feature i or j ( see Appendix B for mathematical details ) . Note that NA is symmetric and Me is positive definite . Therefore the optimization problem is a generalized eigenvalue problem ( Ghojogh et al. , 2019 ) . When Me is invertible , u∗ is the eigenvector associated with the largest eigenvalue of M−1e NA . In general , if we want to embed xijk into a ddimensional subspace aligned with the feature i ( d < a ) , we can take the eigenvectors with the d largest eigenvalues of M−1e NA , which we call the top d factorized linear discriminant components ( FLDs ) . Since multi-way ANOVA can handle contingency tables with more than two dimensions , our analysis can be easily generalized to more than two features . 3 SPARSITY-BASED REGULARIZATION OF FLDA . For this domain-specific application in neurobiology , there is particular interest in finding a small group of genes that best determine one of the phenotypic features . This leads to finding axes that have only a few non-zero components . To identify such a sparse solution , we solved the following optimization problem : u∗ = arg max u∈Rg uTNAu uTMeu subject to ||u||0 ≤ l ( 11 ) from which the number of non-zero elements of u∗ is less or equal to l. This is known as a sparse generalized eigenvalue problem , which has three unique challenges , as listed in Tan et al . ( 2018 ) : first , when the data are very high-dimensional , Me can be singular and non-invertible ; second , because of the normalization term uTMeu , many solutions for sparse eigenvalue problems can not be applied directly ; finally , this problem involves maximizing a convex objective over a nonconvex set , which is NP-hard . To solve it , we used truncated Rayleigh flow ( Rifle ) , a method specifically developed to solve sparse generalized eigenvalue problems . The algorithm of Rifle is composed of two steps ( Tan et al. , 2018 ) : first , to obtain an initial vector u0 that is close to u∗ . We used the solution from the non-sparse FLDA as an initial estimate of u0 ; second , iteratively , to perform a gradient ascent step on the objective function , and then execute a truncation step that preserves the l entries of u with the largest values and sets the remaining entries to 0 . Pseudo-code for this algorithm can be found in Appendix C. As proved in Tan et al . ( 2018 ) , if there is a unique sparse leading generalized eigenvector , Rifle will converge linearly to it with the optimal statistical rate of convergence . The computational complexity of the second step is O ( lg + g ) for each iteration , therefore Rifle scales linearly with g , the dimensionality of the original data . Based on the theoretical proof , to guarantee convergence , the hyperparameter η was selected to be sufficiently small such that ηλmax ( Me ) < 1 , where λmax ( Me ) is the largest eigenvalue of Me . In our case , the other hyperparameter l , indicating how many genes to be preserved , was empirically selected based on the design of a follow-up experiment . As mentioned later in Results , we chose l to be 20 , a reasonable number of candidate genes to be tested in a biological study . 4 RELATED WORK . 4.1 LINEAR DIMENSIONALITY REDUCTION . FLDA is a method for linear dimensionality reduction ( Cunningham & Ghahramani , 2015 ) . Formally , linear dimensionality reduction is defined as the following : given n data points each of g dimensions , X = [ x1 , x2 , ... , xn ] ∈ Rg×n , and a choice of reduced dimensionality r < g , optimize an objective function f ( . ) to produce a linear projection P ∈ Rr×g , and Y = PX ∈ Rr×n is the low-dimensional transformed data . State-of-the-art methods for linear dimensionality reduction include principal component analysis ( PCA ) , factor analysis ( FA ) , linear multidimensional scaling ( MDS ) , linear discriminant analysis ( LDA ) , canonical correlations analysis ( CCA ) , maximum autocorrelation factors ( MAF ) , slow feature analysis ( SFA ) , sufficient dimensionality reduction ( SDR ) , locality preserving projections ( LPP ) , and independent component analysis ( ICA ) ( Cunningham & Ghahramani , 2015 ) . Some of them are obviously unsuitable to the problem we want to solve , for example , MAF and SFA are developed for data with temporal structures ( Larsen , 2002 ; Wiskott & Sejnowski , 2002 ) , and LPP focuses on the local structures of the data instead of their global organization ( He & Niyogi ) . The remaining approaches can be roughly grouped for either unsupervised or supervised linear dimensionality reduction , and we discuss them separately .

Summary: This manuscript describes a generalization of ANOVA that is intended to be used in the interpretation of single-cell RNA-seq data. The method requires specification of an orthogonal, discrete categorization of cells, nominally by phenotype.  The method then linearly factorizes the observed gene expression values into features and their interactions, relative to the phenotypic categories.  The factors can then be used to help interpret the categories, especially in conjunction with a regularizer to reduce the number of genes involved in the factors.