Global graph curvature

1 INTRODUCTION . Representation learning is an important tool for learning from structured data such as graphs or texts ( Grover & Leskovec , 2016 ; Perozzi et al. , 2014 ; Mikolov et al. , 2013 ) . State-of-the-art algorithms typically use Euclidean space for embedding . Recently , however , it was found that hyperbolic spaces demonstrate superior performance for various tasks ( Nickel & Kiela , 2018 ; Sala et al. , 2018 ) , while in some cases spherical spaces can be useful ( Liu et al. , 2017 ) . A key characteristic classifying the above-mentioned spaces is curvature , which is negative for hyperbolic spaces , zero for Euclidean and positive for spherical spaces . These findings , therefore , show that certain graphs are better represented in spaces with non-zero curvature . While some methods simply fix curvature ( e.g. , -1 for hyperbolic space ) and then find the optimal embedding of the graph in the corresponding space ( Nickel & Kiela , 2018 ) , others try to learn the right curvature and embedding simultaneously ( Gu et al. , 2019 ) . In this paper , we analyze the problem of determining a graph curvature suitable for embedding without actual embedding the graph itself . The aim is to use some simple graph characteristics to understand whether it is reasonable to embed a graph into some non-Euclidean space and if yes , then which curvature to choose . Having such an estimator would save computational resources , since learning an embedding is computationally expensive and should be done for each space separately . More importantly , it can also save human resources by showing whether it is worth investing resources in the implementation of a more complex embedding algorithm . We make an important first step in this direction by introducing a concept of global graph curvature , which depends on both the dimension and loss function used for the embedding . We consider two loss functions : distortion , which is widely used in embedding literature , and threshold-based loss ( see Section 2 ) , which is more suitable for some practical applications . We demonstrate , both theoretically and empirically , that these loss functions lead to fundamentally different graph curvatures . We also compare several estimators of the global curvature , in particular , the ones based on well-known Ollivier-Ricci and Forman-Ricci local graph curvatures . Our analysis shows that all these notions give curvatures that are far from the optimal curvature for embedding the graph ( in particular , because they are dimension-independent ) . This raises the important question of how to properly estimate the best curvature for embedding graphs . We approach this problem by introduc- ing a new curvature estimator which , in contrast to all existing ones , is dimension-specific . This estimator agrees with our theoretical results and is also able to distinguish between “ hyperbolic ” and “ Euclidean ” networks . One important insight , which we gained based on both theoretical and empirical analysis , is the fact that the optimal curvature is usually smaller for smaller dimensions . In other words , a network which seems to be negatively curved ( for some small dimension ) usually becomes more neutral as dimensionality grows . Finally , while in the current research we assume that the global graph curvature is constant , this concept can be extended in the future to cover , e.g. , the product spaces from ( Gu et al. , 2019 ) . We focused on the constant curvature since , first , such spaces are widely used in various applications ( Tifrea et al. , 2018 ) and , second , even for constant curvature we observed many non-trivial theoretical and empirical results . These results and insights can eventually be extrapolated to analyze curvature in other , more complicated spaces . To the best of our knowledge , the problem of analyzing different notions of curvature for embedding graphs ( even in simple spaces ) was not considered before , so we hope that more results will follow . 2 BACKGROUND AND RELATED WORK . Graph embeddings For an unweighted connected graph G = ( V , E ) equipped with the shortest path distance metric dG , a graph embedding f is a map f : V → U , where U is a metric space . We refer to Goyal & Ferrara ( 2018 ) for a survey of several graph embedding methods and their applications . The goal of an embedding is to preserve some structural properties of a graph . Depending on an application , different loss/quality functions are used to measure the quality of a graph embedding . The most well-known is distortion:1 D ( f ) = 1 ( n 2 ) ∑ u 6=v |d ( f ( u ) , f ( v ) ) − dG ( u , v ) | dG ( u , v ) . Distortion is a global metric , it takes into account all graph distances . However , in some practical applications , it may not be the best choice . For example , in recommendation tasks , we usually deal with a partially observed graph , so a huge graph distance between nodes in the observed part does not necessarily mean that the nodes are not connected in the full graph . Additionally , as discussed in Section 4.3.1 , graph distances are hard to preserve : there are simple graphs on just 4 nodes that can be perfectly embedded only in a space of curvature −∞ for any dimension . Another measure , often used for embeddings , is Mean Average Precision ( MAP ) , which , for a given node v , compares the distance-based ranking of other embedded nodes with the graphneighborhood-based ranking . In contrast to distortion , MAP is scale-invariant , as it cares only about the order . Since changing curvature is equivalent to changing scale , MAP is insensitive to curvature.2 However , in practical applications like recommendation systems , ranking scores are usually complemented by minimum score thresholds to have personalized sizes of top recommended items . This motivated us to consider another loss function , which is at the same time local and scale-dependent . Let us define the following class of threshold-based loss functions . Given an embedding f of a graph G , we ( re ) construct a graphG′ in the following way : v and u are connected inG′ iff d ( f ( v ) , f ( u ) ) ≤ 1 . Then , any loss function which is based on the comparison of G and G′ is called thresholdbased . While all our theoretical results hold for any threshold-based loss function ( see Section 4 ) , in our experiments we measure and compare several functions of this type including , e.g. , zeroone loss for the edge classification problem ( whether a given node pair is connected or not ) , see Appendix E.2 for definitions and discussion . Such loss functions are natural in many applications ( graph reconstruction , link prediction , recommendations ) . 1There are other definitions of distortion in the literature , see , e.g. , ( Sala et al. , 2018 ) . 2In other words , it is sufficient to consider only the curvatures -1 , 0 , 1 , corresponding to hyperbolic , Euclidean and spherical spaces . Moreover , by considering a small enough region in hyperbolic or spherical space we get geometry similar to the Euclidean one , so for MAP it is important to distinguish only between -1 and 1 . Hyperbolic and Spherical Spaces For many years , Euclidean space was the primary choice for data embeddings ( Goyal & Ferrara , 2018 ) . However , it turned out that many observed datasets are well fitted into hyperbolic space ( Krioukov et al. , 2010 ) . Nickel & Kiela ( 2017 ) have shown that such hyperbolic embeddings can improve state-of-the-art quality in several practical tasks , e.g. , lexical entailment and link prediction . On the other hand , spherical spaces are also used for embeddings ( Liu et al. , 2017 ) . Gu et al . ( 2019 ) goes even further by suggesting mixed spaces : product manifolds combining multiple copies of spherical , hyperbolic , and Euclidean spaces . The main advantage of hyperbolic space is that it is “ larger ” : the volume of a ball grows exponentially with its radius . Hence , such spaces are well suited for tree-like structures . On the other hand , spherical spaces are suitable for embedding cycles ( Gu et al. , 2019 ) . Spherical and hyperbolic spaces are parametrized by curvature c , which is positive for spherical space and negative for hyperbolic space . As c → 0 , geometry of both these spaces becomes similar to the Euclidean one . We discuss some geometrical properties of these different spaces in Appendix A . 3 LOCAL GRAPH CURVATURES . While in this paper we analyze the global graph curvature , there are several local ones proposed in the literature . Many of them are based on the notion of sectional curvature and Ricci curvature defined for Riemannian manifolds . Intuitively , Ricci curvature controls whether the distance between small spheres is smaller or larger than the distance between the centers of the spheres . Ricci curvature is positive if small spheres are closer than their centers are . We refer to ( Jost , 2009 ; O ’ neill , 1983 ) for more details on Ricci curvature . Ollivier curvature Ollivier curvature translates the definition of Ricci curvature to graphs . Again , the idea is to compare the distance between two small balls with the distance between their centers . The distance between balls is defined by the well-known optimal transport distance ( a.k.a . Wasserstein distance or earth-mover distance ) . Formally , for a graphG we consider the shortest path metric on G , denoted by dG , and let WG1 denote the Wasserstein metric with respect to the metric space ( G , dG ) . Furthermore , for each node v we let mv denote the uniform probability measure on the neighbors of v , i.e. , mv ( u ) = 1u∼vdeg ( v ) , where deg ( v ) denotes the degree of v. Then , the classic definition3 of Ollivier curvature between two neighboring nodes v ∼ u in G is defined as κG ( u , v ) = 1−WG1 ( mv , mu ) . ( 1 ) We note that Ollivier curvature always belongs to the interval [ −2 , 1 ] ( Jost & Liu , 2014 ) . Forman curvature Forman curvature ( Sreejith et al. , 2016 ) is based on the discretization of Ricci curvature proposed by Forman ( 2003 ) . It is defined for a general weighted graph G , with both node and edge weights . When the graph G is not weighted , the definition becomes : FG ( u , v ) = 4− ( deg ( v ) + deg ( u ) ) . ( 2 ) Forman was interested in a general discretization of curvature for Riemannian manifolds and his formula includes faces of any dimension . Although this can be translated to graphs ( Weber et al. , 2017 ) , the formula becomes quite cumbersome . Therefore , in Equation 3 only 1-dimensional faces ( edges ) are included . One can extend this expression by including higher dimensional faces . This was considered in ( Samal et al. , 2018 ) , where 2-dimensional faces on three nodes ( triangles ) were included . In the case of an unweighted graph , we then obtain F̂G ( u , v ) = F ( u , v ) + 3∆uv = 4− deg ( v ) − deg ( u ) + 3∆uv , ( 3 ) where ∆uv is the number of triangles that contain the edge ( u , v ) . Based on the definitions , both Forman curvatures , especially FG ( u , v ) , are expected to often be highly negative . Indeed , this is what we observe in Sections 4.3 and 5 . 3Note that Ollivier curvature is defined in much more generality in terms of metrics and random walks , see ( Ollivier , 2009 ) . Thus , different version on graphs can be considered . Equation 1 corresponds to the classical choices of graph distance and random walk on the graph . Heuristic sectional curvature A different notion of curvature used by Gu et al . ( 2019 ) is based on the following geometric fact . Let abc be a geodesic triangle and let m be the ( geodesic ) midpoint of bc . Then the value d ( a , m ) 2 + d ( b , c ) 2 4 − d ( a , b ) 2+d ( a , c ) 2 2 is equal to zero in euclidean space , is positive in spherical space and negative in hyperbolic space . For graphs , let v be a node in G , b , c neighbors of v and a any other node . Then , we define ξG ( v ; b , c ; a ) = 1 2dG ( a , v ) ( dG ( a , v ) 2 + dG ( b , c ) 2 4 − dG ( a , b ) 2 + dG ( a , c ) 2 2 ) . ( 4 ) This resembles the formula above with m = v and the normalization constant 2dG ( v , a ) is included to yield the right scalings for trees and cycles . To define the graph sectional curvature of a node v and its neighbors b , c , we average ξG ( v ; b , c ; a ) over all possible a : ξG ( v ; b , c ) = 1 |V |−3 ∑ a∈G\ { v , b , c } ξG ( v ; b , c ; a ) . 4

Summary: This paper is about curvature as a general concept for embeddings and ML applications. The origin of this idea is that various researchers have studied embedding graphs into non-Euclidean spaces. Euclidean space is flat (it has zero curvature), while non-Euclidean spaces have different curvature; e.g., hyperbolic space has a constant negative curvature. It was noted that trees don't embed well into flat space, while they embed arbitrarily well into hyperbolic space.