Non-Euclidean Mixture Model for Social Network Embedding

Authors: roshni g iyer
yewen  wang
wei  wang
yizhou  sun
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-30 20:49:58

Abstract: It is largely agreed that social network links are formed due to either homophily or social influence. Inspired by this, we aim at understanding the generation of links via providing a novel embedding-based graph formation model. Different from existing graph representation learning, where link generation probabilities are defined as a simple function of the corresponding node embeddings, we model the link generation as a mixture model of the two factors. In addition, we model the homophily factor in spherical space and the influence factor in hyperbolic space to accommodate the fact that (1) homophily results in cycles and (2) influence results in hierarchies in networks. We also design a special projection to align these two spaces. We call this model Non-Euclidean Mixture Model, i.e., NMM. We further integrate NMM with our non-Euclidean graph variational autoencoder (VAE) framework, NMM-GNN. NMM-GNN learns embeddings through a unified framework which uses non-Euclidean GNN encoders, non-Euclidean Gaussian priors, a non-Euclidean decoder, and a novel space unification loss component to unify distinct non-Euclidean geometric spaces. Experiments on public datasets show NMM-GNN significantly outperforms state-of-the-art baselines on social network generation and classification tasks, demonstrating its ability to better explain how the social network is formed.

Main paper: 
 introduction: 
 Social networks are omnipresent because they are used for modeling interactions among users on social platforms. Social network analysis plays a key role in several applications, including detecting underlying communities among users [1] , classifying people into meaningful social classes [2] , and predicting user connectivity [3] . Most existing embedding models are designed based on the homophily aspect of social networks [4, 5] . They utilize the intuition that associated nodes in a social network imply feature similarity, and an edge is usually generated between similar nodes. Prior works have used shallow embedding models to represent homophily, like matrix factorization and random-walk (Section 2), which are parameter intensive and do not employ message passing. As an improvement, graph neural network (GNN) models (Section 2) have been proposed to more effectively capture homophily by representing a node through its local neighborhood context. However, research of RaRE [6] and work of [7] show homophily is insufficient, and social influence is also critical in forming connections. This is due to popular nodes having direct influence in forming links [8] . For example, in Twitter network, users tend to follow celebrities in addition to users who share similar interests [9] . Though RaRE jointly models both factors, it has limitations in modeling capabilities. Specifically, it is parameter intensive as each node embedding is fully parameterized through a Bayesian framework. Further, RaRE assumes graphs are transductive, limiting its performance in the practical inductive setting where new links not seen during training must be predicted. Moreover, nearly all works embedding social networks utilize a single zero-curvature Euclidean space, when in reality, network factors may create different topologies. Specifically, edges generated by homophily tend to form cycles [10] , while edges generated by social influence tend to form tree structures [11, 12] . From Riemannian geometry, people have found that networks with cycles are best represented by spherical space embeddings [13] , while tree structured networks are best represented by hyperbolic space embeddings [14] . Thus, an end-to-end model to bridge social network embeddings of distinct non-Euclidean geometric spaces is a promising direction.Our motivation is two-fold: (1) We aim to understand how the social network is generated e.g., which factors affect node connectivity and what topological patterns emerge in the network as a result. (2) Using our learning from (1) , we aim to design a more realistic deep learning model to explain how the network is generated (inferring new connections). We summarize our contributions as follows:• We propose Graph-based Non-Euclidean Mixture Model (NMM) to explain social network generation. NMM represents nodes via joint influence by homophily (modeled in spherical space) and social influence (modeled in hyperbolic space), while seamlessly unifying embeddings via our space unification loss. • To our knowledge, we are also the first to couple NMM with a graph-based VAE learning framework, NMM-GNN. Specifically, we introduce a novel non-Euclidean VAE framework where node embeddings are learned with a powerful encoder of GNNs using spherical and hyperbolic spaces, non-Euclidean Gaussian priors, and unified non-Euclidean optimization. • Extensive experiments on several real-world datasets on large-scale social networks, Wikipedia networks, and attributed graphs demonstrate effectiveness of NMM-GNN in social network generation and classification, which outperforms state-of-the-art (SOTA) network embedding models.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The abstract summarizes the main paper claims and we also provide a contribution summary with bullet points in the Introduction. Further, all these main claims of the paper are supported through Section 3 (detailing our model architecture), Section 4 on Experiments, as well as our (four) ablation studies in the Appendix. Our model novelly represents both homophily and social influence factors when modeling the social network, an advancement over the recent network embedding methods. Further, the novel utilization of non-Euclidean geometric spaces to model the resulting topologies due to network factors through appropriate positive and negative curvatures naturually exibiting properties of cycles and hierarchy (the observed topological heterogeneity), tremendously improves the representation capability of network embedding modes. This is evidenced by the empirical results as ablation studies for our work. Moreover, NMM-GNN not only improves against SOTA models in the performance metrics (Jaccard Index, Hamming Loss, , F1 score, AUC Score), but is also applicable to the Inductive Setting, other large-scale information networks like Wikipedia networks, and attributed networks (Facebook and Google+), while consistently achieving the best performance indicating model generalizability. 
preliminary and related work: 
 We provide an overview of social network embedding models and discuss advancements in non-Euclidean graph learning.Several works that embed social networks merely model homophily [15] , capturing node-node similarity, without also considering a node's social influence or node popularity e.g., a celebrity. Homophily-based models include shallow embedding models and GNN embedding models. Most shallow embedding models are either based on matrix factorization [16] or random-walk [17] . Though GNN models [18] effectively learn on large networks and in inductive settings, they still fail to model the social influence factor in the network. Further, even the model that captures both homophily and social influence, RaRE [6] , has limitations in that its fully-parameterized node embeddings require large parameter size to be learned, and it models all nodes in Euclidean space, an approach not effective in capturing different topologies (e.g., cycles and hierarchy) in the social network.Non-Euclidean geometric spaces, commonly used to model surfaces in mathematics and physics, are curved geometries that include spherical spaces with positive curvature, and hyperbolic spaces with negative curvature [19] . Works have recently found Euclidean space modeling to be insufficient for non-Euclidean graph-structured data [20] . Namely, spherical spaces have been shown to effectively embed graphs with cyclic structure due to their positive curvature [21, 13] , while hyperbolic spaces have been shown to effectively embed graphs with hierarchical structure due to their negative curvature and exponential or "tree-like" growth of the space [22, 23] . HGCN [24] and [25] are critical works exploring GNNs in non-Euclidean spaces. Both these hyperbolic GNN methods have shown significant improvement on benchmark datasets by preserving hierarchical structure of graphs. In knowledge graphs, [26] has achieved notable performance by modeling graphs in non-Euclidean spaces of various curvatures, using both hyperbolic and spherical space. . z H i from the hyperbolic space is projected to its corresponding point in the spherical space, z S ′ i , such that its geodesic distance is ensured to be close to vi's existing spherical space representation, z S i .A social network G = (V, A) consists of a set of vertices V = {v i } N i=1 , and associated adjacency matrix e ij ∈ A, where e ij is an edge from v i to v j . We aim to design a model to jointly learn both node homophily and social influence representation, denoted z S i and z H i respectively, that can best explain the social network in terms of link reconstruction. In this section, we describe our architecture. First, we introduce a novel non-Euclidean mixture model, called NMM, to model the probability of a new link. Second, we add a non-Euclidean GNN encoder to enrich NMM, called NMM-GNN, which is connected to the variational autoencoder framework. Source code is in the Appendix. For details on future directions, the reader is also referred to the Limitations section in the Appendix.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: We have an entire section on Limitations in the Appendix (Section A) that addresses all the guidelines below. Our model also makes the assumption the homophily regulated nodes lie on the surface of the spherical ball and the social influence regulated nodes lie on the open Poincare ball, which are both natural representations for modeling nodes. People have observed that more popular celebrity nodes tend to have smaller norm space and are embedded towards the center of the ball (similar to higher order concepts in the knowledge graph space), while less popular nodes (containing less social influence) are embedded towards the boundary of the ball (similar to entities in the knowledge graph space). To ensure that we can compute an intersection space (for the space unification component to unify the homophily and social influence representation for the same node), we enforce that the spherical surface norm is in the Poincare ball e.g., any learned soft value norm space between 0 and 1. In compute, while we evaluate on a cluster of 8 GPUs, at least one GPU is necessary for running our experiments (which may been seen as compute limitation).Regarding ethical considerations and fairness, privacy and fairness are often topics of focus regarding social networks. Our model uses information like node popularity based on graph structure (in degree vs. out degree ratio), as well as attributed features if available. Further it also infers links through neighborhood context by looking at connected nodes. In general, a network embedding model uses personal information from the user which may impinge on their privacy. However, we make every attempt in our model to allow the user to select their granular choice of model personalization (e.g., we provide the option to choose from whether or not users want to enable their attributed information like profile interest and history being learned). Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
overview: 
 To model both factors of homophily and social influence that affect graph connectivity, we model each node v i with homophily regulated representation, z S i , and social influence regulated representation, z H i1 . A unified framework is designed such that z S i and z H i influence each other bi-directionally and are seamlessly merged through NMM. NMM utilizes non-Euclidean geometric spaces to better represent homophily and social influence components which produce curved structures like cycles and trees. To improve NMM, we enrich its encoder by GNNs, q ψ (z H i |G) and q ϕ (z S i |G). To ensure generated embeddings are in spherical and hyperbolic spaces respectively, non-Euclidean GNNs are adopted. The enhanced model is NMM-GNN, which has a clear connection to the VAE framework.Figure 1 (a) illustrates the architecture overview of NMM-GNN. Specifically, the encoder component maps nodes into homophily based embedding, z S , in spherical space (for homophily generated cycles) and social influence based embedding, z H , in hyperbolic space (for social influence generated trees), which follow non-Euclidean prior distributions. The embeddings are passed into our mixture model decoder, which models the probability of a link as a mixture of a homophily based distribution component and a social influence based distribution component. The objective is to maximize the likelihood to observe the links, or equivalently to minimize the link reconstruction loss. In addition, the two geometric spaces are ensured to be aligned together via a space unification regularization term, to make sure the two embeddings of the same node are corresponding to each other.Geometrically, the space of NMM is a spherical surface inside a unit Poincaré ball. Each node v i corresponds to (1) a point in the Poincaré ball, z H i , and (2) a point on the spherical surface, z S i , where both spaces are embedded inside a Euclidean space. These two points are aligned by enforcing projection of z H i onto the spherical surface to be close to z S i , as projecting a star onto earth's atmosphere, shown in Fig. 1(b ). We note that the space alignment does not mean embeddings are enforced to be the same in two spaces 2 . Rather, we require the projection of z H i is close to i's embedding in spherical space z S i . In this case, the two distances of spherical and hyperbolic spaces are also different from each other (norm space difference vs. spherical geodesic distance). Without the space alignment, z H i has too much degree of freedom, which can move freely as long its norm is kept the same. Lastly, the probability of generating a link is a mixture of the probability of generating the link in each non-Euclidean space.Representation of homophily regulated nodes. We embed homophily based representation of node v i , or z S i , on the surface of the spherical ball in spherical space, S d . Formally,S d = {z S i ∈ R d ∥z S i ∥ = w S } is the d-dimensional w S -norm ball,where ∥•∥ is the Euclidean norm and w S ∈ [0, 1) is a constant to ensure the spherical surface is inside the unit Poincaré ball. To better capture homophily distribution, p(z S i ), we use spherical Gaussian distribution G S (•) as spherical prior, described in Table 1 . β is the axis or direction of the lobe controlling where the lobe is located on the sphere, and points towards the center of the lobe; λ is the sharpness of the lobe such that as this value increases, the lobe will become narrower in width; and a is the amplitude or intensity of the lobe, corresponding to the height of the lobe at its peak.Link prediction using homophily based distribution. The probability of link e ij = 1 between nodes v i and v j , also described in Table 1 , is determined by the geodesic distance between z S i and z S j . dist s (z S i , z S j ) = arccos(⟨z S i , z S j ⟩) is the geodesic distance between z S i and z S j ; J > 0 and B ≥ 0 are model parameters; and ⟨•, •⟩ denotes the inner product. The probabilistic model is designed so nodes greatly dissimilar (exhibiting low homophily), or dist s (z S i , z S j ) → +∞, have low probability of a link being generated, or p hom (e ij = 1) → 0. In the case of nodes exhibiting high homophily, they either (1) may be connected due to highly similar characteristics, or (2) may not be connected simply because the individuals do not know each other. Our distribution models both scenarios. Note when dist s (z S i , z S j ) → 0, p hom (e ij = 1) → 1 1+e B , which can be interpreted as a factor to control sparsity of the network. p(z S i ) = GS(z S i ; β, λ, a) = ae λ(β•z S i -1) (b) Link Generation Probability p hom (eij = 1) = p(eij = 1|dists(z S i , z S j )) = 11 + e J×dists(z S i ,z S j )+BRepresentation of social influence regulated nodes. We embed social influence based representations of node v i , or z H i , on the Poincaré ball from hyperbolic space, H d+1 , to better capture resulting hierarchical structures. Social influence regulated nodes are represented as points,z H i , belonging in- side the Poincaré (open) ball in H d+1 . Formally, H d+1 = {z H i ∈ R d+1 ∥z H i ∥ = w H zi }, w H zi ∈ [0, 1), is (d+1)-dimensional w Hzi -norm ball, where ∥•∥ is Euclidean norm. We assume center of the Poincaré ball is aligned with center of the sphere, and is one dimension larger than the sphere to ensure the spherical surface is inside the Poincaré ball. To better capture a social influence regulated distribution, p(z H i ), we use Hyperbolic Gaussian distribution G H (•) as non-Euclidean Gaussian prior, described in Table 2 . z H i is the origin of (r, ω) for radius r and angle ω in polar coordinates. z H i is the center of mass and ζ > 0 is the dispersion parameter, where the dispersion dependent normalization constant Z(ζ) accounts for the underlying non-Euclidean geometry. Z(ζ) is partitioned into angular ω and radial r components. Γ(•) is Euler's gamma function, and dist H (•) is the hyperbolic distance between two hyperbolic space node embeddings, x and y, where ∥•∥ denotes Euclidean norm.Link prediction using social influence based distribution. We model existence of an edge, e ij = 1 between nodes v i and v j , also described in Table 2 , as a function of norm space difference, dist h (•), between two hyperbolic space node embeddings, z H i and z H j . We utilize norm space difference as opposed to hyperbolic distance, since nodes of similar social influence status may have large distance in the Poincaré (open) ball due to nodes possibly being placed towards the ball's boundary. This is because, at the boundary of the ball, nodes become infinitely distanced apart. Thus, to allow for numerical stability and to capture social influence difference (in which the higher the social influence of nodes indicated by large in-degree and smaller out-degree, the closer they are embedded towards the center of the ball), we utilize norm space as indicator. Consistent with the notation of [6] , a node with higher social influence is associated with a smaller norm value. C and D are learned model parameters and norm space of a vector is norm(x) = ∥x∥. dist h (z H i , z H j ) = |norm(z H i ) -norm(z H j )| is the norm difference between z Hi and z H j ; C > 0 and D ≥ 0 are model parameters; and norm(•) denotes the L1 normalization function. The probabilistic model is designed such that nodes largely different in popularity (exhibiting high social influence), or dist h (z H i , z H j ) → +∞, have high probability of a link being generated, or p rank (e ij = 1) → 1. In the case both nodes exhibit low social influence (such as having similar social rank), they either (1) may be connected due to highly similar characteristics, or (2) may not be connected simply because individuals do not know each other. Our distribution models both scenarios. When dist h (z H i , z H j ) → 0, p rank (e ij = 1) → e D 1+e D , which can be interpreted as another factor to control sparsity of the network. EQUATIONEQUATIONEQUATION(b) Link Generation Probabilityp rank (eij = 1) = p(eij = 1|dist h (z H i , z H j )) = e C×dist h (z H i ,z H j )+D 1 + e C×dist h (z H i ,z H j )+DLink Prediction Using Mixed Space Distribution. Since both homophily and social influence affect the connectivity structure of social networks, we model existence of a new link between nodes, p θ (e ij = 1), as a weighted combination distribution of these factors. Specifically, our non-Euclidean mixture model is a weighted combination of homophily based distribution, p hom (e ij = 1), and social influence based distribution, p rank (e ij = 1), with learned weight γ:EQUATIONwhere p hom (e ij = 1) is modeled in positively curved spherical space since homophily based links may form cycles due to similarity connections between node clusters. p rank (e ij = 1) is modeled in negatively curved hyperbolic space since social influence based links may form tree-like structures due to popularity-based social hierarchy between node clusters. We would like to highlight that the link between nodes i and j is a mixture model because each link is a weighted combination of influence from both spherical and hyperbolic spaces (not one or the other) as evidenced in Equation 6.As shown in Figure 1b , the same node has two representations -one in the spherical space and one in the hyperbolic space, and because they represent the same underlying node, they need to be aligned. Therefore, these two network factors do not contradict each other, but rather work together to explain how links are formed between users.We enrich the encoder of NMM to generate better embeddings. To do so, we explore GNN methods which have been shown to be more effective than shallow embedding methods (see Experiments and Ablation Studies). We refer to this enriched NMM model as NMM-GNN. NMM-GNN uses non-Euclidean VAE as its learning framework. The framework integrates a mixture of different non-Euclidean geometric spaces e.g., hyperbolic and spherical spaces, for learning of encoder, decoder, and node prior distributions, and ensures geometric spaces are unified during training.Encoder model. The encoder learns two corresponding embedding representations per node v i to produce spherical embedding z S i and hyperbolic embedding z H i . For homophily regulated nodes in spherical space, S d , any spherical space GNN (SGNN) can be applied, and for social influence regulated nodes in hyperbolic space, H d+1 , any hyperbolic space GNN (HGNN) can be applied. The general framework for SGNN and HGNN are shown in Tables 3 and 4 . z S (l) i ∈ R d (l) and z H (l) i ∈ R d+1 (l)are spherical and hyperbolic feature representations of node v i at layer l, with dimensionality d and (d + 1) respectively. f is a message-specific neural network function of incoming messages to v i from neighborhood context N i , and activation function σ(•), typically ReLU(•) for all layers but the last one being softmax(•). z S (l+1) i = qSGNN(z S i |A, vi) = q ϕ (z S i |A, vi) = σ j∈N i f (z S (l) i , z S (l) j ) (b) Example: Spherical GCN z S (l+1) i = q ϕ (z S i |A, vi) = exp S 0 σ W T l j∈N i ∪{i} ej,i √ mjmi log S 0 (z S (l) j )H(l+1) i = qHGNN(z H i |A, vi) = q ψ (z H i |A, vi) = σ j∈N i f (z H (l) i , z H (l) j ) (b) Example: Hyperbolic GCN z H(l+1) i = q ψ (z H i |A, vi) = exp H 0 σ W T l j∈N i ∪{i} ej,i √ mjmi log H 0 (z H (l) j )Example Non-Euclidean Encoder models. We illustrate Non-Euclidean Graph Convolutional Neural Network (Non-Euclidean GCN) as an example GNN. Tables 3 and 4 describe the model architectures for both Spherical GCN and Hyperbolic GCN respectively. Node features are initialized by sampling each embedding dimension from a distribution uniformly at random for all nodes where z S (0EQUATION)) d+1 respectively. The retraction operator, R(•), involves mapping between spaces. For non-Euclidean spaces, retraction is performed between non-Euclidean space and approximate tangent Euclidean space using logarithmic and exponential map functions. Specifically,log H 0 (z H i ) = tanh -1 (i • ∥z H i ∥) z H i i•∥z Hi ∥ is a logarithmic map at center 0 from hyperbolic space to Euclidean tangent space, andexp H 0 (z H i ) = tanh(i•∥z H i ∥) z H i i•∥z H i ∥is an exponential map at center 0 from Euclidean tangent space to hyperbolic space.log S 0 (z S i ) = tanh -1 (∥z S i ∥) z S i ∥z Si ∥ is a logarithmic map at center 0 from spherical space to Euclidean tangent space and exp S 0 (z S i ) = tanh(∥z S i ∥)z S i ∥z Si ∥ is an exponential map at center 0 from Euclidean tangent space to spherical space. where z S (l) i , z H (l) i are embeddings of node v i at layer l ∈ [0, L), L = 2; W l is a layer-specific learnable weight matrix; N i is the set of nodes in the neighborhood context of v i ; e j,i is the edge-weight between nodes v j → v i , with default edge weight being 1.0 if an edge exists. m i ,m j are entries of the degree matrix, with m i = 1 + j∈Ni e j,i .Decoder model. NMM can be considered as a probabilistic decoder for link generation, which maps embeddings of two nodes into the probablity to generate a link between them.The training loss involves components of reconstruction loss (to ensure the generated graph is consistent with the original graph), KL divergence loss (to ensure predicted embeddings z S i and z H i closely match their non-Euclidean Gaussian distributions), and space unification loss (to ensure z S i and z H i map to the same node v i ). Reconstruction Loss. Table 5 shows reconstruction loss, which minimizes the upper bound on the negative log-likelihood. λ A is a hyperparameter; A ′ = XAX T , given X ∈ 0, 1 k×d where X a,i = 1 only if node a ∈ G is assigned to i ∈ G and X a,i = 0 otherwise, where G is the predicted graph. EQUATIONEQUATIONKL Divergence Loss. The KL divergence loss is formed by minimizing the equations described in Table 6 . Minimizing the KL divergence loss ensures that the homophily regulated nodes and social influence regulated nodes closely align to their underlying non-Euclidean distribution priors. As described in Section 3, these distributions are designed to appropriately capture the distinct topologies that emerge as a result of the respective social network factors.Space Unification Loss. The space unification loss is formed by minimizing the equations described in Table 6 . Minimizing the space unification loss ensures that the hyperbolic space representation of node v i in spherical space, proj S (z H i ), is close to the corresponding learned representation of node v i in the spherical space, z S i . Note that using a normalized hyperbolic disk is not a substitute for the projection operator from the social influence hyperbolic space onto the homophily spherical space. The projection operation solely projects out-of-sphere nodes onto the w S norm space, or the norm at the surface of the spherical ball. A normalization operator would instead change the embedding values of all nodes. More importantly, the space unification loss component ensures minimal spherical geodesic distance between z H i 's representation on the spherical ball and z S i , illustrated in Figure 1 (b). EQUATION)EQUATIONTotal Loss. The overall loss function for homophily regulated and social influence regulated nodes respectively is then a summation of the above loss components given by:EQUATIONEQUATIONThis section details NMM-GNN's training framework, using non-Euclidean VAE, for representing social networks. We describe the optimization method for variables from Sections 3.2 and 3.3.Embedding Initialization. We randomly initialize all embeddings of z S i and z H i . For homophily regulated nodes, we choose a value for norm w S , that is sampled from uniform distribution: w S : w S ∈ [0, 1) → Unif([0, 1)) for all nodes, and for social influence regulated nodes, we choose a value for norm w H zi , assigned uniformly at random per node. We set curvature values of spherical and hyperbolic spaces as K S = 1 and K H = -1 respectively. We leave the non-trivial problem of learning optimal curvatures as future work.Training procedure for homophily regulated nodes. Parameter optimization for learning node embeddings is performed using Riemannian stochastic gradient descent (RSGD) for the spherical space as shown in Table 7 . To ensure the updated node embeddings remain in norm-w S space, we perform a rescaling operation, proj S , to project out-of-boundary embeddings back to the surface of EQUATIONEQUATION(b) Social Influence Regulated Nodes the w S -ball. We further update scalar parameters J and B (from the homophily regulated distribution) and β, λ, a (from the spherical gaussian prior) through stochastic gradient descent (SGD) as defined below via SGD S (J), SGD S (B), SGD S (β), SGD S (λ), SGD S (a).EQUATIONEQUATIONTraining procedure for social influence regulated nodes. Parameter optimization for learning node embeddings is performed using RSGD for the hyperbolic space as shown in Table 7 . The corresponding norm space, w H zi , is also learned through RSGD by updating embeddings of z H i . We further update scalar parameters C and D (from the social influence regulated distribution) and ζ (from the hyperbolic gaussian prior) through SGD as defined below viaSGD H (C), SGD H (D), SGD H (ζ).Training procedure for NMM-GNN weights. Parameter optimization for γ uses SGD as follows, whereEQUATION4 ExperimentsWe comprehensively evaluate NMM-GNN on social network generation for popular large-scale social networks through multi-label classification and link prediction tasks against competitive SOTA baselines in various categories. The Appendix section further details Ablation Studies where we test quality of using (1) a mixture model, (2) distinct non-Euclidean geometric spaces, (3) non-Euclidean GNN-based encoders and non-Euclidean GraphVAE framework (through the inductive setting), and (4) space unification loss component. All experiments and each of the ablation studies consistently show NMM-GNN outperforms baseline network embedding models on all metrics for all datasets.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: The assumptions about the social network graph as well as our model are comprehensively listed in Section 3 which contains the "Methodology" and "Overview" assumptions as well as subsections that each list the associated assumptions for that particular model component. Moreover, Section 4 (Experiments) and Appendix (Additional Experiments + Ablation Studies) detail the entire experiment evaluation procedure and design as well as all the corresponding graph assumptions. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
datasets: 
 For comprehensive evaluation, we assess our models on real-world datasets from well-known social media venues: BlogCatalog (BC) [27] , LiveJournal (LJ) [28] , and Friendster (F) [29] which are friendship networks among bloggers. Table 8 provides statistics of the datasets. In the Appendix, we also include experiments for Wikipedia datasets, to show that our model can also benefit other networks. Our research goal is to design an embedding model to better explain how the social network is formed. Thus, to separate model contribution from the learned representation, we focus on the setting of featureless graphs since quality of node features can be a confounding factor in determining quality of our model. However, since our model can handle feature graphs, we also provide these experiments in the Appendix, with our model outperforming all baselines on attributed networks.Baselines. We compare NMM-GNN to SOTA network embedding models, in Table 9 , and report results in Table 10 . We omit comparison to prior models e.g., LINE [30] due to lower performance. We also highlight that unlike prior works like κ-GCN, our model overcomes limitations of the product space, e.g., where the entire model belongs to a Cartesian product of non-Euclidean geometric spaces by default. Our work is in a category called mixed space model that uses a multi-geometric space framework where different portions of the graph may possibly belong to different spaces (based on the amount of impact each of homophily and social influence has for that personalized pair of nodes). In the extreme case (Case 1) where only social influence is at play, e.g., weight of homophily representation is learned close to 0, the hyperbolic space will be used. On the other hand if only homophily is at play (Case 2), e.g., weight of homophily representation is learned close to 0, the spherical space will be used. In the normal case of both factors at play (Case 3), then both spaces will be used and can be jointly aligned with our space alignment mechanism. When using product space, Cases 1, 2, and 3 will all not be distinguished from each other as all cases will be modeled by one complex non-Euclidean geometric space as a Cartesian product of spherical and hyperbolic spaces.Time Complexity of NMM-GNN. Regarding our model, NMM is highly efficient, with time complexity O(ed + nd), where n is number of nodes, e is number of edges, and d is dimension size.In comparison, the time complexity analyses for the remaining baseline models are as follows: the mixture model of RaRE is O(ed + nd) which is comparable to the NMM mixture model, and the GNN embedding models of GCN, GAT (with one-head attention), and κ-GCN (for κ = 0) are O(ed + nd 2 ). κ-GCN (for κ ̸ = 0) and HGCN's time complexities are O(ed + a • nd 2 ), where a is the filter length, and NMM-GNN is O(ed + nd 2 ) which is comparable to GNN embedding models.As our work focuses on improving accuracy of learned embeddings, we further use GraphVAE training with NMM to achieve SOTA performance. We would like to point out that GraphVAE (of NMM-GNN) training is also designed to be highly parallelizable, which allows for scalability. Moreover, our model is capable of learning on real-world, highly large-scale graphs on the order of millions of nodes and billion of edges, e.g., Friendster, while achieving the best performance, which attests to its practical value to the network science community.We detail our evaluation procedure for multi-label classification and link prediction. For all experiments, for fairness of comparison to baselines, we utilize the experiment procedure of [6] . Specifically, 90% of links are randomly sampled as training data. We do not perform cross-validation, since it may cause overfitting to occur as our framework uses learnable parameters e.g., z S , z H , J, B, C, D, γ, β, λ, α, W l , and ζ which is a function of z H equivalently interpreted as mean square error. Per dataset, we choose hyperparameter values for λ A in reconstruction loss: {0, 1, 2, 4, 8, 16, 32, 64}, step sizes η t : {0.005, 0.001, 0.01, 0.05, 0.1}, and experiments are performed on AWS cluster (8 Nvidia GPUs).Evaluation results are in Table 10 . We observe that mixture models (homophily and social influence), achieve better performance on all datasets for all metrics. Specifically, it improves over structural embedding models (GraphWave), GNNs (GAT, HGCN), and homophily-based models (GELTOR, NRP). We also see learning embeddings in non-Euclidean geometric spaces helps better represent structures in social networks (HGCN vs. GAT, RaRE vs. NMM). We further observe using GNNbased encoders with GraphVAE learning yields additional improvement (NMM vs. NMM-GNN).As from [6] , we measure quality of link prediction by sorting probability scores of every pair of nodes per model and evaluating them using area under the ROC curve (AUC) score. Specifically, 10% of existing edges and non-existing edges are hidden from training set, and probabilities are examined by show more realistic heterogeneity in network structure compared to BlogCatalog dataset e.g., cyclic structures produced by homophily based nodes and tree-like structures produced by social influence based nodes. Thus, modeling these structures in non-Euclidean spaces (RaRE vs. NMM) also shows more improvement. Moreover, in our NMM variant models, every node is influenced by both factors of homophily and social influence through our non-Euclidean mixture model. It is a weighted combination personalized per node of these factors that influence the links formed, rather than being generated solely through homophily vs. social influence.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Justification: We have an entire training procedure section and loss function information inside Section 3 (Methodology) which comprehensively describes our model architecture.All optimizer and loss function details are presented here. Our Section 4 (Experiments) and Appendix (which contains our four Ablation Studies) detail the experiment evaluation for the corresponding experiments and detail all the training and test details as well as any additional information needed to understand the results. We provide detailed information about hyperparameter settings (Section 4), which were selected based on performance on the validation set (train/validation/test split details also in this section). Further, the use of all evaluation metrics from the experiments are clearly defined (paper resource references pointed) and justified.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusions: 
 We are among the first to explore a Graph-based non-Euclidean mixture model for social networks.As social networks are influenced by homophily and social influence, we design a model to represent both factors jointly for nodes. Further, we model resulting unique network topologies (cycles and trees) using distinct non-Euclidean geometric spaces and introduce a GNN-based non-Euclidean variational autoencoder framework for our model, to effectively learn embeddings. The resulting model, NMM-GNN, significantly outperforms various state-of-the-art models for social networks.As future work, we hope to explore alternatives to graph-based VAE methods for improved learning. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA] Justification: We provide all details about distribution priors, random embedding initialization etc., and evaluate our model on standard experiment metrics important to the social network community for determining model quality. Error bars are not applicable in this scenario but our paper details all experiment conditions.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: The paper has a section devoted to analyzing the memory complexity of the model (Section 4: Experiments), as well as also provide compute and hyperparameter details in this section and Section 3 (model architecture). Additionally, we release all of our source code and datasets for maximal reproducibility (Supplement Material).Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: All paper guidelines have been carefully followed regarding page limit, formatting, content, research topic and relevance, writing, figures etc. All privacy and data security concerns are also addressed as we utilize public benchmark datasets that do not contain any sensitive user information. Our Limitations section further provides details regarding ethical considerations.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: The paper provides clear motivation of the importance of the research problem of investigation and further identifies key challenges in the network science community in state-of-the-art embedding models, which thus motivates our model architecture. Further, our paper provides clear intuition and justification for each design choice decision in our model components. Additionally, impact of our model is discussed with respect to the social network community and more broadly for other online information networks e.g., Wikipedia datasets and attributed networks (in the Appendix section on Ablation studeies). Lastly, we also discuss Limitations of our model in Appendix Section A. This model effectively also learns without needed data attributes, which could beneficially influence privacy considerations in social networks (that users do not need to have their profile information and history being stored). Of course, any network science model could have potential for misuse in areas like surveillance or manipulation of online communities, so litigation and action should be taken to safeguard against misuse.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: We have been extremely careful on the type of data and model we use and to consider all ethical aspects respectively. No human subjects were used in our research study (N/A), and all of our datasets for evaluation are large-scale public benchmark popular social media datasets evaluated on numerous papers in the recent years. As such, these datasets are highly suitable for direct comparisons against various SOTA models as they have been widely tested on.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: All source codes are attributed to the respective authors and we even identify in our paper the training procedures if used from prior work for fairness of comparison e.g., RaRE [6] that we indicate in Section 4 (Experiments). We also release all our source code and datasets (in addition to referencing each of them in our main paper for the source information) for reproducibility under MIT and CC-by licences.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. Justification: As mentioned in previous questions this is not applicable for this paper and all datasets evaluated are popular large-scale networks from social media that have been widely evaluated on in recent years in the network science community.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: NMM is highly efficient, with time complexity O(ed + nd), where n is number of nodes, e is number of edges, and d is dimension size. As our work focuses on improving accuracy of learned embeddings, we further use GraphVAE training with NMM to achieve SOTA performance, called NMM-GNN. Using the GraphVAE training pipeline may be seen as a limitation as it is less efficient compared to GNNs for training time. However, in the worst case scenario, NMM-GNN still achieves efficiency comparable to popular heterogeneous graph models including GraphRNN [40] , and training is designed to be highly parallelizable, which allows for scalability. Moreover, our model is capable of learning on real-world, highly large-scale graphs on the order of millions of nodes and billion of edges, e.g., Friendster, while achieving the SOTA performance, which attests to its practical value to the network science community.Our model also makes the assumption the homophily regulated nodes lie on the surface of the spherical ball and the social influence regulated nodes lie on the open Poincare ball, which are both natural representations for modeling nodes. People have observed that more popular celebrity nodes tend to have smaller norm space and are embedded towards the center of the ball (similar to higher order concepts in the knowledge graph space), while less popular nodes (containing less social influence) are embedded towards the boundary of the ball (similar to entities in the knowledge graph space). To ensure that we can compute an intersection space (for the space unification component to unify the homophily and social influence representation for the same node), we enforce that the spherical surface norm is in the Poincare ball e.g., any learned soft value norm space between 0 and 1.In compute, while we evaluate on a cluster of 8 GPUs, at least one GPU is necessary for running our experiments (which may been seen as compute limitation).Regarding ethical considerations and fairness, privacy and fairness are often topics of focus regarding social networks. Our model uses information like node popularity based on graph structure (in degree vs. out degree ratio), as well as attributed features if available. Further it also infers links through neighborhood context by looking at connected nodes. In general, a network embedding model uses personal information from the user which may impinge on their privacy. However, we make every attempt in our model to allow the user to select their granular choice of model personalization (e.g., we provide the option to choose from whether or not users want to enable their attributed information like profile interest and history being learned).Our work can be generalized to the graph-level because our method learns to represent social science network factors based on topologies in the graph on clusters of nodes and edges. Thus, if the cluster of nodes and edges comprised of the entire graph and we subsequently applied graph pooling per node embedding (that we currently learn), we can reduce the embedding from node level to graph level. In this way, homophily and social influence can be modeled at the graph level. That said, it is unclear whether graph-level modeling would be specifically useful or interpretable for social network embedding models as compared to node-level modeling. This is due to the social network setting requiring links to be generated that are per node and not at the graph level, because a user (modeled as a node) is recommended to another specific user in the practical social network setting. This is different from other network domains like molecular classification where the entire graph represents one molecule e.g., atoms form individual nodes and chemical bonds form the edges. For this reason, node-level learning is in fact consistent with the recent state-of-the-art NN methods in the network science community though NMM-GNN can still be generalized to learning at the graph-level.In this section, we provide experiment results on Wikipedia networks and on attributed graphs. As in the case for social networks, we also evaluate on multi-label classification and link prediction for Wikipedia networks and attributed graphs. For all experiments, for fairness of comparison to baselines, we utilize experiment procedure of [6] . Specifically, 90% of links are randomly sampled as training data. We do not perform cross-validation, since it may cause overfitting to occur as our framework uses learnable parameters e.g., z S , z H , J, B, C, D, γ, β, λ, α, W l , and ζ which is a function of z H equivalently interpreted as mean square error. Per dataset, we choose hyperparameter values for λ A in reconstruction loss: {0, 1, 2, 4, 8, 16, 32, 64}, and step sizes η t : {0.005, 0.001, 0.01, 0.05, 0.1}.Evaluation on Wikipedia networks. In Table 12 , we additionally present multi-label classification and link prediction experiments on Wikipedia datasets to show the benefits of our model on other networks. These include Wikipedia Clickstream [6] , which contains counts of (referrer, resource) pairs extracted from the request logs of Wikipedia, and Wikipedia Hyperlink [6] , which contains edges as hyperlinks from one page to another. Dataset statistics for the Wikipedia datasets are summarized in Table 11 . Our NMM model variants consistently achieves the best performance over all the competitive baseline models belonging to categories of (1) structural embedding models, (2) GNN embedding models (Euclidean space), (3) homophily-based embedding models, (4) GNN embedding models (non-Euclidean space), and (5) mixture models. This shows that NMM's model is generalizable and widely applicable because it can learn effective representations on online information networks that go beyond social networks. Evaluation on attributed graphs. As our model can also handle attributed graphs, we provide experiments against SOTA attributed network embedding models that are summarized below. Since these models require the presence of network attributes, we do not include them in our experiments on featureless graphs for fairness of comparison. We conduct evaluation on well-known largescale attributed graph datasets which include Facebook [9] and Google+ [9] social networks where consistent with [27] , we treat each ego-network as a label and extract attributes from their user profiles. Dataset statistics are in Table 13 , and results are reported in Table 14 . It can be seen that our NMM-GNN model consistently achieves the best performance in both the large-scale graph datasets, indicating that our model's inherent learning ability is effective to learn graph structure and topology, which goes beyond simply exploiting information from node and edge attributes. Below is a summary of the baseline attributed network embedding models: • PANE [41] , random-walk based attirubted network embedding (ANE) model• NRP [27] , described in Table 9 of the main paper• CONN [42] , GNN for attributed networks via collaborative aggregation on bipartite graphs This section details ablation studies where we test quality of using (1) a mixture model, (2) distinct non-Euclidean geometric spaces, (3) non-Euclidean GNN-based encoders and non-Euclidean GraphVAE framework (through the inductive setting), and (4) space unification loss component. Each of the ablation studies provides insight to motivate our architecture design choices, as well as consistently shows NMM-GNN outperforms baseline network embedding models.Quality of using a mixture model architecture. We deconstruct our mixture model of NMM, to observe the effect different network factors have on learning embeddings, where α and β are learnable. We report results on embedding dimension 64, evaluated on AUC score, with the models summarized below:• NMM hom , deconstructed homophily component: p θ (e ij = 1) = α • p hom (e ij = 1)• NMM rank , deconstructed social influence component: p θ (e ij = 1) = β • p rank (e ij = 1)• NMM, which is our mixture model defined by Eqn. 6As shown in Figure 2 (a), the mixture model of NMM outperforms that of its subcomponents NMM hom and NMM rank on all datasets for link prediction. This validates the effectiveness of using a mixture model architecture for modeling both homophily and social influence factors jointly.Quality of using distinct non-Euclidean geometric spaces. We study combinations of geometric spaces to model NMM (homophily/social influence) to observe the effect it has on learning topological structure, denoted with NMM(•) † in Table 10 (main paper). As shown, the choice of modeling homophily based nodes in spherical space and modeling social influence based nodes in hyperbolic space leads to the best performance. Further, NMM outperforms its Euclidean space counterpart showing that the social network exhibits structures (cycles and hierarchy) that need to be appropriately represented in curved geometric spaces. There is also evidence of non-Euclidean topologies in the datasets. For example, the average node on LiveJournal has in-degree of 17 but outdegree of 25, showing several hierarchical structures present, and 17.7% of LiveJournal data contains cycles [43] .Link prediction on unseen nodes (inductive task). We study ability of NMM-GNN to learn on the inductive setting (in addition to the standard transductive setting of Table 10 of the main paper). To do so, we randomly sample % of nodes being {10, 30, 50, 70, 90} and their corresponding links as our training set. Test nodes are ensured to have no overlap with training nodes, to allow for link prediction on unseen graphs. Figure 2 (b) reports results on NMM-GNN and RaRE for LiveJournal on embedding dimension of 64 for AUC score. NMM-GNN outperforms RaRE on all settings of training nodes. This shows the effectiveness of NMM-GNN in using non-Euclidean GNNbased encoders and a non-Euclidean GraphVAE training framework during the learning process, two components that RaRE lacks. Further, as less training nodes are observed, NMM-GNN outperforms RaRE by larger margins (e.g., 10% vs. 70% training nodes), showing NMM-GNN better generalizes to unseen graphs.Quality of using space unification loss. We test quality of using our proposed space unification loss (SUL) in our NMM-GNN, by conducting experiments for with the loss component (NMM-GNN) and without it (NMM-GNN-no-SUL). We report results on embedding dimension 64, evaluated on AUC score in Figure 2 (c), for datasets Wikipedia Clickstream (WC), BlogCatalog (BC), LiveJournal (LJ), Wikipedia Hyperlink (WH), and Friendster (F). As shown, using SUL improves performance on all datasets, indicating importance of bridging together representations of distinct non-Euclidean spaces (spherical and hyperbolic space) at node level.• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.The geometric space community conventionally uses S to represent spherical space and H to represent hyperbolic space.Note that it is impossible to equate these two embeddings directly as they are in different geometric spaces. 


Decision: The paper proposes a graph embedding methodology in which link generation is modeled via a mixture model that leverages non-Euclidean geometric spaces. The proposed approach has merits that the reviewers have highlighted. These mostly include novel aspects of the methodology, such as modeling both homophily and social influence, as well as well-executed experiments.  The response of the authors to the different questions raised further clarified several aspects of the paper. Therefore, I recommend accepting the paper.

Reviews: review: paper_summary: paper_summary: This work addresses the embedding of social networks with downstream tasks such as link prediction in mind. In this manuscript the authors propose to model the link as the mixture of two factors of node embedding, Spherical and Hyperbolic. Concretely, two kinds of node embedding are unified into a single loss by projecting from hyperbolic space into spherical space, and jointly trained in the framework of VAE on Graaph. The authors also conduct experiments against a wide range of baseline methods to highlight the capacity of the proposed method.
strength_weakness: strength_weakness: The major issue is on limited datasets in experiments. I would suggest the authors consider other datasets used in previous works, such as synthetic datasets, citation networks (PubMed, wikipedia citation,  DBLP, Microsoft Academic Graph ) and other social networks (e.g. Twitter). 

Also another factor limiting the impact of the proposed method is that the proposed method does not work on the whole-graph level, and thus only applies to node classification and link prediction tasks. If by any extension the embeddings could be aggregated to the whole-graph leve, there could be more significance (e.g. tasks like molecular classification, protein-to-protein interactions)
questions: questions: N/A
limitations: limitations: Well addressed.
paper_summary: paper_summary: This paper proposes a new Graph-based non-Euclidean mixture model for social networks.
Under the assumptions that social network links are formed due to either homophily or social influence, 
the homophily factor is modeled in spherical space and the social influence factor is in hyperbolic space.
The homophily regulated nodes lie on the surface of the spherical ball and the social influence-regulated
nodes lie on the open Poincare ball. 
The projection to align these two spaces is also proposed.
The non-Euclidean GraphVAE is also integrated into the model.
Experiments show the proposed model outperforms other baselines.
strength_weakness: strength_weakness: - The motivation of the space unification is unclear (See the questions below).
questions: questions: - Why is the space unification necessary? Each node has two coordinates, in the spherical and the hyperbolic space. The link between nodes i and j is generated either from the spherical or hyperbolic proximity (it is the "explaining away" situation as explained in the RaRE paper [6] ). The homophily and social influence relationships may be unrelated or may even contradict to each other (you may hate your boss). Therefore the two coordinates may not necessarily be aligned. It would be appreciated if the authors would elaborate on this. 

- The $\kappa$-GCN proposed in [39] also deals with spherical and hyperbolic spaces. I would like to know the similarities and/or differences between the two approaches as the two coordinates in the proposed model can also be regarded as a coordinate in the product space. The authors do not discuss them. They just compare the experimental performance.

- What is $i$ in the definition of $\log_0^H$ and $\exp_o^H$ ?
limitations: limitations: As for the computational efficiencies, more formal and quantitative comparisons would be necessary in the main texts.
paper_summary: paper_summary: The authors understand why links are generated through node-embedded representations of social networks. Specifically, spherical space is utilized to represent the homogeneity of nodes and hyperbolic space is utilized to represent the hierarchy and influence of nodes. By mixing these two spaces together, the corresponding link representation is finally obtained.
strength_weakness: strength_weakness: 1.The idea of the article is not novel. Both hyperbolic and spherical spaces are geometric models that are very commonly used in the field of social networking.[1]
2.Although the authors are trying to explain linking relationships in terms of homogenization and hierarchy, however the learning of the model embedding is still unknown, which does not explain the emergence of social networks.
3.There has been much better work on embedding methods in hybrid spaces. [2]The authors don't contribute as much to this as their article suggests.
[1]Network geometry. Nature physics
[2]Motif-aware Riemannian Graph Neural Network with Generative-Contrastive Learning AAAI 2024
[3]Product Manifold Learning. AISTATS 2021
questions: questions: 1.How to analyze the reasonable causes of social network link generation from complex feature representations?
2.Can this model explain small-world networks in complex networks?
limitations: limitations: 1.The model does not really give an adequate explanation of links in complex networks, but only from the perspectives of homogeneity and hierarchy.
paper_summary: paper_summary: This paper proposes NMM-GNN, a non-Euclidean mixture model that captures both homophily and hierarchies in social networks for embedding.
strength_weakness: strength_weakness: 1. The related work section does not cover most existing social embedding works. For example, many works, in addition to RaRE, also consider both the similarities and the social impact of nodes.

2. There is no clear proof or at least empirical experiments to demonstrate that it is more reasonable to embed both node similarity and social impact in spherical space instead of Euclidean space.
questions: questions: 1. My main concern is that social network embedding algorithms considering social impact are not novel. For example, when performing edge prediction, one can directly assume the probability of the edge is proportional to the degree of both nodes. I hope the author could further explain why the proposed method is novel enough in terms of encoding both similarity and impact.

2. Despite the experimental results and ablation study in the main article showing the good performance of the proposed method, I want to know how the speed of the method compares with existing baselines.
limitations: limitations: The author discussed the potential limitations of the paper in detail in the appendix.

Comments: title: Response to Follow-Up Question from Reviewer 3fkq
Response to Rebottal
comment: We sincerely appreciate your positive feedback and the increase in the score. We are glad that we were able to address your questions.
We sincerely appreciate your positive feedback and the increase in the score. We are glad that the discussion on the contribution and novelty of our work helped to address your questions.
We sincerely appreciate your positive feedback and the increase in the score. We are glad that the additional clarifications we provided have helped to address some misunderstandings of the paper and demonstrate the importance of the space alignment. We will rewrite some of our text to include these insights in the next revision, in case other readers also also have this confusion.
Thanks for the clarification. 
The geodesic relationship between zHi and zHj in the hyperbolic metric space does not directly govern the link probability, but only the norm difference matters. 
I will change my rating from 4 to 6.
Thanks for your follow-up question. Please find our response below:

We would like to address a misunderstanding in your interpretation of the paper: It seems that you thought the alignment is to enforce the embeddings the same in two spaces, which is untrue. We require the projection of Z_i^H is close to i's embedding in spherical space Z_i^S. Note, it's impossible to equate these two directly as they are in different geometric spaces.

In this case, the two distances (of spherical and hyperbolic spaces) are also different from each other. Note in our paper dist_h(zHi , zHj ) = |norm(zHi ) − norm(zHj )| is not geodesic distance in hyperbolic metric space. It is defined based on their norm to reflect their rank difference.

Without the alignment, Z_i^H has too much degree of freedom, which can move freely as long their norm kept the same. 

We also have an ablation study on this part. Please refer to Appendix section's Figure 2 (c), where we show the quality of using the space unification component (both with and without), and we observe that the performance improves consistently on across all datasets with the space unification component.
Thanks for the detailed explanation. 

As for Q1,  I am aware that the proposed model is a mixture model. 

What I wanted to know is the reason why the space unification is necessary (the motivation of the space unification loss). The space unification regularization term ensures that two geometric spaces are aligned together to make sure the two embeddings of the same node correspond to each other. Why do the two embeddings of the same node have to be close to each other? 

If $\boldsymbol{Z}_i^H \approx \boldsymbol{Z}_i^S$ and  $\boldsymbol{Z}_j^H \approx \boldsymbol{Z}_j^S$, 

I wonder $\mathrm{dist}_h(\boldsymbol{z}_i^H,\boldsymbol{z}_j^H)
\approx$ $\mathrm{dist}_s(\boldsymbol{z}_i^S,\boldsymbol{z}_j^S)$
(I know this is oversimplified, but you may still say that the former distance is small then the latter distance is also small)

then I wonder

$p(e_{ij}=1\vert \mathrm{dist}_h(\boldsymbol{z}_i^H,\boldsymbol{z}_j^H))$ 

may tend to be equivalent to 

$p(e_{ij}=1\vert \mathrm{dist}_s(\boldsymbol{z}_i^S,\boldsymbol{z}_j^S))$
Thanks a lot for your response. I appreciate and agree with your response, and I will change my rating from 6 to 7.
Thanks for your response! I have known the contribution and novelty of your work. At last, I have raised my score.