Removing Inter-Experimental Variability from Functional Data in Systems Neuroscience

Integrating data from multiple experiments is common practice in systems neuroscience but it requires inter-experimental variability to be negligible compared to the biological signal of interest . This requirement is rarely fulfilled ; systematic changes between experiments can drastically affect the outcome of complex analysis pipelines . Modern machine learning approaches designed to adapt models across multiple data domains offer flexible ways of removing inter-experimental variability where classical statistical methods often fail . While applications of these methods have been mostly limited to single-cell genomics , in this work , we develop a theoretical framework for domain adaptation in systems neuroscience . We implement this in an adversarial optimization scheme that removes inter-experimental variability while preserving the biological signal . We compare our method to previous approaches on a large-scale dataset of two-photon imaging recordings of retinal bipolar cell responses to visual stimuli . This dataset provides a unique benchmark as it contains biological signal from well-defined cell types that is obscured by large inter-experimental variability . In a supervised setting , we compare the generalization performance of cell type classifiers across experiments , which we validate with anatomical cell type distributions from electron microscopy data . In an unsupervised setting , we remove inter-experimental variability from data which can then be fed into arbitrary downstream analyses . In both settings , we find that our method achieves the best trade-off between removing inter-experimental variability and preserving biological signal . Thus , we offer a flexible approach to remove inter-experimental variability and integrate datasets across experiments in systems neuroscience . Code available at https : //github.com/eulerlab/rave . ˚ ; Equal contributions , 1 University of Tübingen , 2 Norwegian University of Science and Technology . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1 Introduction . Systems neuroscientists are often concerned with identifying and characterizing how properties of neurons vary along certain dimensions of interest . Differences in these properties between neurons form the basis for sorting them into discrete categories . Both the advance of large-scale data acquisition techniques in experimental neuroscience as well as the development of more efficient and powerful data analysis methods allow collecting and analyzing datasets of increasing size ; and hence the discovery of more subtle variations in neural function between cell types [ e.g . 1–4 ] . However , as data acquisition is often an incremental process , it has become common practice to pool data from multiple experiments . This practice ignores variability in the data stemming from external factors , which include non-biological ones ( e.g . sample handling resulting in small differences in tissue quality , or temperature fluctuations affecting the rates of biochemical processes ) but potentially also unforeseen biological ones ( e.g . subtle genetic variations ) [ 3 , 5 , 6 ] . Such variability due to external factors , here referred to as inter-experimental variability , can confound and obscure the biological signals of interest . In some cases , the source of inter-experimental variability is known and can be modeled [ 5 ] , but if this is not the case , a method for removing it from the data is required . The issue of inter-experimental variability in systems neuroscience is analogous to the problem of domain shift in machine learning , where the data distribution changes between training ( ‘ source ’ ) and test ( ‘ target ’ ) data , causing an algorithm to fail when deployed on data from an unseen target domain [ 7–10 ] . Methods that address this issue have to perform some form of domain adaptation , i.e . adapting the algorithm to work both on the training as well as some ( usually unseen ) test domain [ 11 ] . In single-cell genomics , a number of different studies have proposed methods for removing inter-experimental variability ( see Section 2 ) , but related works in systems neuroscience are lacking , despite the recognized need for such approaches [ 3 , 5 ] . Here , we contribute to closing this gap as follows : • We cast the removal of inter-experimental variability from functional data in systems neuroscience in the theoretical framework of domain adaptation ( Figure 1 and Section 3 ) . • We adapt and evaluate different approaches and demonstrate improved performance of cell type assignment , while preserving the biological signals of interest ( Table 1 and Figure 4 ) . • We demonstrate that our method produces cell type predictions on a new dataset that are best aligned with anatomical data ( Table 2 and Figure 5 ) . • Finally , we showcase in a downstream analysis that the corrected data ( Figure 3 ) clearly exhibits biological effects that were obscured by inter-experimental variability ( Figure 6 ) . 2 Related Work . As mentioned before , few studies have proposed specialized solutions to the issue of interexperimental variability in systems neuroscience . Two studies have approached the problem of temporal alignment of neural responses across experiments . Zhao et al . [ 5 ] proposed a solution to deal with the specific effects of temperature fluctuations on the response kinetics of retinal neurons by modeling them explicitly . Williams et al . [ 12 ] proposed a more general method for the temporal alignment of data across trials or recording sessions . Other studies have suggested models of neural function that integrate data across experiments . Shah et al . [ 3 ] build encoding models to predict the responses of retinal ganglion cells across different experiments [ see also 13 ] and compare it to covariates such as the gender of an animal . Sorochynsky et al . [ 14 ] propose a way to measure noise correlations in each recording and integrate those into models of neural populations of a specific cell type . This latter approach is complementary to our method because it allows the study of the structure and function [ see also 15 ] of noise correlations , which we discard as nuisance variability . Somewhat related to the example application in our paper , Jouty et al . [ 16 ] suggested a method to perform non-parametric physiological classification of retinal ganglion cells in the mouse retina while trying to find matching clusters of cell types across experiments . Crucially , all of these approaches offer specialized solutions that do not represent general purpose correction methods . In single-cell genomics , a number of approaches for removing inter-experimental variability from data have been developed [ 17–24 ] . Two such methods are Harmony [ 25 ] and scGen [ 26 ] . Harmony performs iterative clustering using a variant of soft k-means until convergence to align cells from different datasets in a joint embedding . scGen , on the other hand , combines a variational autoencoder adapted for scRNA-seq data with latent space arithmetics to predict gene expression , while removing inter-experimental variability between datasets . In this paper , we compare our approach against these methods as they have been found to perform particularly well in two benchmarking studies [ 27 , 28 ] . 3 Theoretical Framework The generative process of data denoted by a random variableX with image X is depicted in Figure 1 . The biological signal shared across experiments ( e.g . variation due to cell types ) is represented by a random variable S ( ‘ signal ’ ) with image S. We define D ( ‘ domain ’ ) as a random variable with image D that represents inter-experimental variability . Now , our objective is to learn a function f that transforms the data into a new random variable Z : “ fpXq with image Z . Importantly , we distinguish two settings : ( i ) unsupervised — where S is unknown and we simply try to retain in Z as much information about the data as possible while removing inter-experimental variability ; ( ii ) supervised — where S is known and we additionally try to retain in Z as much information about S as possible . These objectives can be formulated in terms of mutual information , giving the unsupervised loss function L “ IpZ ; Dq ´ IpZ ; Xq ( 1 ) and , provided knowledge about S , we obtain the supervised loss function L ` “ L´ IpZ ; Sq . ( 2 ) Now , IpZ ; Dq attains its minimum for IpfpXq ; Dq “ 0 because of the non-negativity of mutual information . And IpZ ; Sq attains its maximum for IpfpXq ; Sq “ IpX ; Sq because of the data processing inequality . If f were a bijection , it would follow that IpfpXq ; Sq “ IpX ; Sq , but also IpfpXq ; Dq “ IpX ; Dq . But by assumption , IpX ; Dq ą 0 ( otherwise there is no inter-experimental variability and we are done ) and so we would have IpfpXq ; Dq ą 0 . Thus , at the minimum of IpZ ; Dq , f can not be a bijection . Generally , if there is an interaction between recording , signal and domain i.e . IpX ; S ; Dq ‰ 0 , then there will be a trade-off between maximizing IpfpXq ; Sq and minimizing IpfpXq ; Dq . This trade-off becomes even more apparent in the unsupervised setting where IpfpXq ; Xq and IpfpXq ; Dq are clearly competing . Mutual information quantifies the dependence between two variables but it is difficult to estimate [ 29–32 ] . Instead , we measure dependency through nonlinear regression with an appropriate distance metric d.2 UsuallyD is a discrete random variable indicating the experiment of a recording , and so , to estimate IpZ ; Dq , we can perform classification with a classifier function h : Z Ñ D , minimizing the standard cross-entropy dCEphpZq , Dq ( Figure 1 ) . Lemma 10 in [ 34 ] shows that this gives a variational lower bound to IpZ ; Dq [ see also 35 ] . In some cases S may also be discrete ( e.g . cell types ) and we can do the same , in other cases it might be a ( high-dimensional ) continuous random variable and so , to approximate IpZ ; Sq , we can perform regression , minimizing the mean squared error dMSEpgpZq , Sq . Similarly , in the unsupervised setting , to approximate IpZ ; Xq , we minimize dMSEpgpZq , Xq . To keep notation simple , in the unsupervised setting we define the mapping g : Z Ñ X , and in the supervised setting g : Z Ñ X ˆ S . Putting this together , in the unsupervised setting our objective is minL ÝÑ min h max g , f λ dphpfpXqq , Dq ´ dpgpfpXqq , Xq ( 3 ) where we have introduced a hyperparameter λ that mitigates the trade-off discussed above . In the supervised setting our objective becomes minL ` ÝÑ min h max g , f λ dphpfpXqq , Dq ´ dpgpfpXqq , pX , Sqq . ( 4 ) 2If the joint probability density of two random variables is a bivariate normal distribution , then the mutual information is proportional to their linear correlation [ 33 ] . In both equations we find a min´max optimization , where h is trying to predictD fromZ , tightening the lower bound on IpZ ; Dq [ see 34 ] , while f is trying to prevent that by removing information about D from Z , effectively lowering IpZ ; Dq . Practically , this optimization scheme has become a standard adversarial setting in machine learning , for instance , in the training of generative adversarial networks [ 36 ] or for discriminative adversarial domain adaptation [ 37 ] .

Summary: This paper introduces a new framework to remove non-biological variability to facilitate data aggregation across studies. The problem is viewed as a transformation of the input data, such that the transformed output is not informative of the “domain” (e.g., experimental condition or study) while simultaneously capturing relevant information from the data. This high-level procedure is implemented via an adversarial training strategy, where the mutual information terms of the objective function are approximated via a standard cross-entropy or MMSE loss. The authors validate their method using two murine datasets, in which the neural recording procedures are slightly different; the proposed framework outperforms the standard Harmony and scGen in the supervised and unsupervised settings.