MoReL: Multi-omics Relational Learning

1 INTRODUCTION . Multi-view learning tries to fully leverage the information from multiple sources ( i.e . different types of omics data in molecular biology ) and represents them in a shared embedding space , which is beneficial for many downstream tasks with a limited number of training samples . In biomedical applications , the shared embedding space also enables better understanding of the underlying biological mechanisms by discovering interactions between different types of molecules , which is our focus in this paper . Existing multi-omics data integration methods are limited in their applicability . First , most of them attempt to derive low-dimensional embeddings of the input samples and are not designed to infer a multi-partite graph that encodes the interactions across views . In unsupervised setting , matrix factorization based methods , such as Bayesian Canonical Correlation Analysis ( BCCA ) ( Klami et al. , 2013 ) and Multi-Omics Factor Analysis ( MOFA ) ( Argelaguet et al. , 2018 ) , can achieve the similar goal of cross-view relational learning but often through two-step procedures , in which the factor loading parameters are used for downstream interaction analyses across views . Second , a very recent relational inference for multi-view data integration , BayRel ( Hajiramezanali et al. , 2020 ) , is built on three strict assumptions , which may limit its practical application , including in multi-omics data integration : 1 ) A graph of dependency between features of each view is available ; 2 ) Input dataset is complete on all views with no missing samples ; 3 ) The samples in different views are well-paired . While the first limitation might be solved by learning a graph using an ad-hoc technique , the last two issues are common in many multi-omics data integration problems . Integrated samples commonly have one or more view with various missing patterns . This is mostly due to limitations of experimental designs or compositions from different data platforms . In addition , data might be collected in different laboratories or the sample IDs are not available due to patient identification or security concerns , leading to unpaired datasets . Apart from these , we might not have access to a priori graph structure data in some view ( s ) as the nature of data might not be structured , or we only have incomplete or very noisy prior knowledge . For such multi-omics data , leaving out such a view may lose some complementary information while enforcing graph structures may cause degraded performance . In this work , we propose a new Multi-omics Relational Learning method , MoReL , based on the fused Gromov-Wasserstein ( FGW ) regularization , mitigating the dependency of multi-view learning on the aforementioned two assumptions . The proposed method contains four major contributions : 1 ) MoReL provides a new Bayesian multi-omics relational learning framework with efficient variational inference and is able to exploit non-linear transformations of data by leveraging deep learning models for either unstructured or graph-structured data ; 2 ) MoReL learns a multi-partite graph across different features from multiple views using a FGW-based decoder , facilitating meaningful biological knowledge discovery from integrative multi-omics data analysis while accounting for arbitrarily permutation and/or transformation caused by processing features with different deep functions across the views ; 3 ) MoReL can flexibly integrate both structured and unstructured heterogeneous views in one framework , in which only confident constraints need to be imposed to improve the model performance ; 4 ) MoReL is able to integrate multiple views with unpaired samples and/or arbitrary sample-missing patterns . 2 RELATED WORKS . Optimal transport . There have been extensive efforts to utilize Gromov-Wasserstein ( GW ) discrepancy to solve the alignment problems in shape and object matching ( Mémoli , 2009 ; 2011 ) . A similar attempt has been made recently to investigate its potential for more diverse applications , such as aligning vocabulary sets between different languages ( Alvarez-Melis & Jaakkola , 2018 ) , and graph matching ( Chowdhury & Mémoli , 2019 ; Vayer et al. , 2018b ; Xu et al. , 2019b ) . Peyré et al . ( 2016 ) have proposed a fast Sinkhorn projection-based algorithm ( Cuturi , 2013 ) to compute the entropy-regularized GW distance . Following this direction , Xu et al . ( 2019b ) have replaced the entropy regularizer with a Bregman proximal term . To further reduce the computational complexity , the recursive GW distance ( Xu et al. , 2019a ) and the sliced GW distance ( Vayer et al. , 2019 ) have been proposed . In Bunne et al . ( 2019 ) , a pair of generative models are learned for incomparable spaces by defining an adversarial objective function based on the GW discrepancy . It imposes an orthogonal assumption on the transformation between the sample and its latent space . However , it can not incorporate the graph structured data . Similar to our model in this paper , Vayer et al . ( 2018a ) and Xu et al . ( 2020 ) have proposed to impose the fused GW regularization in their objective functions by combining GW and Wasserstein discrepancies . Graph CCA ( gCCA ) . In order to utilize a priori known information about geometry of the samples , gCCA methods ( Chen et al. , 2019 ; 2018 ) have been proposed to construct a dependency graph between samples and directly impose it into a regularizer . Similar to classical CCA , gCCA learns an unstructured shared latent representation . Unlike our MoReL , though , they can neither take advantage of the dependency graph between features , nor explicitly model relational dependency between features across views . Therefore , they rely on ad-hoc post-processing procedures as a second step to infer inter-relations . Graph representation learning . Graph neural network architectures have been shown to be effective for link prediction ( Hamilton et al. , 2017 ; Kipf & Welling , 2016 ; Hasanzadeh et al. , 2019 ) as well as matrix completion for recommender systems ( Berg et al. , 2017 ; Monti et al. , 2017 ; Kalofolias et al. , 2014 ; Ma et al. , 2011 ) . The first group of models is dealing with a single graph and is not able to deal with heterogeneous graphs , with multiple types of nodes and edges , and node attributes ( Zhang et al. , 2019 ) . The second group utilizes the known item-item and user-user relationships and their attributes to complete the user-item rating matrix . However , they rely on two strict assumptions : 1 ) the inter-relation matrix is partially observed ; and 2 ) both views have structured information . The proposed MoReL achieves robust multi-view learning without these assumptions , making it more practical in multi-omics data integration . 3 PRELIMINARIES . 3.1 WASSERSTEIN DISTANCE . Wasserstein distance ( WD ) is a measure for comparing probability distributions on the same or different spaces where a meaningful distance across domains can be computed ( Vayer et al. , 2019 ) . Given two distributions Λ and ∆ defined on the corresponding space X and Y , and a transportation cost c : X× Y→ R+ , WD is the solution to the following optimization problem : inf π∈Π E ( x , y ) ∼π [ c ( x , y ) ] = inf π∈Π ∫ c ( x , y ) dπ ( x , y ) , where π ∈ Π ( X×Y ) is the transport map such that its marginals are Λ and ∆ , respectively . Assuming that the probability distributions are discrete , i.e . Λ = ∑n i=1 aiδxi and ∆ = ∑m j=1 bjδyj with δ as the Dirac delta function , WD optimization could be simplified as follows : DW ( Λ , ∆ ) = min T∈Π ( a , b ) n∑ i=1 m∑ j=1 Ti , j c ( xi , yj ) , where Ti , j is an element of the transport matrix T whose row-wise and column-wise sums equal to [ ai ] n i=1 and [ bj ] m j=1 , respectively . 3.2 GROMOV-WASSERSTEIN DISTANCE . Gromov-Wasserstein distance ( GWD ) has been proposed as a natural extension of WD when a meaningful transportation cost between the distributions can not be defined for WD . For example when two distributions are defined in Euclidean spaces with different dimensions ( Vayer et al. , 2019 ) . Instead of measuring inter-domain distances , GWD measures the distance between pairs of samples in one distribution and compares it to those in the other domain . More specifically , given two distributions Λ and ∆ defined on spaces X and Y , as well as two domain-specific transportation costs c ( X ) and c ( Y ) , GWD is the solution to the following optimization problem : inf π∈Π E ( x , y ) ∼π , ( x′ , y′ ) ∼π [ L ( x , x ′ , y , y′ ) ] = inf π∈Π ∫ ∫ L ( x , x′ , y , y′ ) dπ ( x , y ) dπ ( x′ , y′ ) , where L ( x , x′ , y , y′ ) =‖ c ( X ) ( x , x′ ) − c ( Y ) ( y , y′ ) ‖ and π ∈ Π ( X × Y ) is the transport map . Likewise , this can be derived for discrete distributions Λ = ∑n i=1 aiδxi and ∆ = ∑m j=1 bjδyj , as follows : DGW ( Λ , ∆ ) = min T∈Π ( a , b ) n∑ i , i′=1 m∑ j , j′=1 Ti , j Ti′ , j′ L ( xi , xi′ , yj , yj′ ) , ( 1 ) where Ti , j is an element of transport matrix T whose row-wise and column-wise sums equal to [ ai ] n i=1 and [ bj ] m j=1 , respectively . 4 METHOD . 4.1 PROBLEM FORMULATION AND NOTATIONS . We propose a novel hierarchical generative model for multi-omics data integration that incorporates view-specific structure information when it is available . Given observations from structured and unstructured views , our model , Multi-omics Relational Learning ( MoReL ) , aims to infer the interrelations among entities , i.e . features , across all of the views . More specifically , assume that multiple views , V , of data are given . Without loss of generality , we assume that the structure information , provided as a graph , is available for some of the views Vs ⊂ V , and the remaining views Vu = V \Vs are unstructured . We note that every structure could be represented as a graph . For example , image and sequential data could be represented over grid and directed path graphs , respectively . We represent the set of graphs for structured views by Gs = { G ( v ) } v∈Vs and their adjacency matrices by As = { A ( v ) } v∈Vs . We also define Xs = { X ( v ) } v∈Vs as the set of node attributes for structured views , and Xu = { X ( v ) } v∈Vu as the set of data for unstructured views . Moreover , Nv denotes the number of nodes in structured views and number of features for unstructured views . MoReL infers the interactions among the nodes in Gs and features in Xu . We represent these inter-relations by a multi-partite graph with ∑ v∈V Nv nodes and a multi-adjacency tensor A = { A ( vv ′ ) } v , v′∈V , v 6=v′ , where A ( vv ′ ) is the Nv ×Nv′ bi-adjacency matrix between views v and v′ .

Summary: The paper proposed a deep Bayesian model for heterogeneous multi-omics data integration. The Gromov-Wasserstein (FGW) regularization between latent representations of heterogeneous views is used to align nodes/features in every pair of views. The experimental results have demonstrated improvement in inferring meaningful relations.