Title: Transfer Learning for Latent Variable Network Models
Venue: NeurIPS 2024
Accepted: yes
Keywords: Transfer learning

network estimation

latent variable models
Abstract: We study transfer learning for estimation in latent variable network models. In our setting, the conditional edge probability matrices given the latent variables are represented by $P$ for the source and $Q$ for the target. We wish to estimate $Q$ given two kinds of data: (1) edge data from a subgraph induced by an $o(1)$ fraction of the nodes of $Q$, and (2) edge data from all of $P$. If the source $P$ has no relation to the target $Q$, the estimation error must be $\Omega(1)$. However, we show that if the latent variables are shared, then vanishing error is possible. We give an efficient algorithm that utilizes the ordering of a suitably defined graph distance. Our algorithm achieves $o(1)$ error and does not assume a parametric form on the source or target networks. Next, for the specific case of Stochastic Block Models we prove a minimax lower bound and show that a simple algorithm achieves this rate. Finally, we empirically demonstrate our algorithm's use on real-world and simulated graph transfer problems.

Transfer Learning for Latent Variable Network Models

Abstract

We study transfer learning for estimation in latent variable network models. In our setting, the conditional edge probability matrices given the latent variables are represented by P for the source and Q for the target. We wish to estimate Q given two kinds of data: (1) edge data from a subgraph induced by an o (1) fraction of the nodes of Q , and (2) edge data from all of P . If the source P has no relation to the target Q , the estimation error must be Ω(1) . However, we show that if the latent variables are shared, then vanishing error is possible. We give an efficient algorithm that utilizes the ordering of a suitably defined graph distance. Our algorithm achieves o (1) error and does not assume a parametric form on the source or target networks. Next, for the specific case of Stochastic Block Models we prove a minimax lower bound and show that a simple algorithm achieves this rate. Finally, we empirically demonstrate our algorithm's use on real-world and simulated network estimation problems.

1 Introduction

Within machine learning and statistics, the paradigm of transfer learning describes a setup where data from a source distribution P is exploited to improve estimation of a target distribution Q for which a small amount of data is available. Transfer learning is quite well-studied in learning theory, starting with works such as Ben-David et al. (2006); Cortes et al. (2008); Crammer et al. (2008), and at the same time has found applications in areas such as computer vision (Tzeng et al., 2017a) and speech recognition (Huang et al., 2013). A fairly large body of work in transfer learning considers different types of relations that may exist between P and Q , for example, Mansour et al. (2009); Hanneke and Kpotufe (2019, 2022), with emphasis on model selection, multitask learning and domain adaptation. On the other hand, optimal nonparametric rates for transfer learning have very recently been studied, both for regression and classification problems (Cai and Wei, 2021; Cai and Pu, 2024).

In this paper, we study transfer learning in the context of random network/graph models . In our setting, we observe Bernoulli samples from the full n × n edge probability matrix for the source P and only a n Q × n Q submatrix of Q for n Q ≪ n . We would like to estimate the full n × n probability matrix Q , using the full source data and limited target data, i.e., we are interested in the task of estimating Q in the partially observed target network, utilizing information from the fully observed source network. This is a natural extension of the transfer learning problem in classification/regression

to a network context. However, it is to be noted that network transfer is a genuinely different problem owing to the presence of edge correlations.

While transfer learning in graphs seems to be a fundamental enough problem to warrant attention by itself, we are also motivated by potential applications. For example, metabolic networks model the chemical interactions related to the release and utilization of energy within an organism (Christensen and Nielsen, 2000). Existing algorithms for metabolic network estimation (Sen et al., 2018; Baranwal et al., 2020) and biological network estimation more broadly (Fan et al., 2019; Li et al., 2022) typically assume that some edges are observed for every node in the target network. One exception is Kshirsagar et al. (2013), who leverage side information for host-pathogen protein interaction networks. For the case of metabolic networks, determining interactions in vivo 1 requires metabolite balancing and labeling experiments, so only the edges whose endpoints are both incident to the experimentally chosen metabolites are observed (Christensen and Nielsen, 2000). For a non-model organism, the experimentally tested metabolites may be a small fraction of all metabolites believed to affect metabolism. However, data for a larger set of metabolites might be available for a model organism.

To study transfer learning on networks, one needs to fix a general enough class of networks that is appropriate for the applications (such as the biological networks mentioned above) and also suitable to capture the transfer phenomenon. The latent variable models defined below appear to be a natural candidate for that.

Latent Variable Models. Latent variable network models consist of a large class of models whose edge probabilities are governed by the latent positions of nodes. This includes latent distance models, stochastic block models, random dot product graphs and mixed membership block models (Hoff et al., 2002; Hoff, 2007; Handcock et al., 2007; Holland et al., 1983; Rubin-Delanchy et al., 2022; Airoldi et al., 2008). They can also be unified under graph limits or graphons (Lovász, 2012; Bickel and Chen, 2009), which provide a natural representation of vertex exchangeable graphs (Aldous, 1981; Hoover, 1979). In addition to their theoretical breadth and usefulness, latent variable models are relevant and applicable to real-world settings such as neuroscience Ren et al. (2023), ecology Trifonova et al. (2015), international relations Cao and Ward (2014), political pscyhology Barberá et al. (2015), and education research Sweet et al. (2013).

For unseen latent variables x 1 , . . . , x n ∈ X ⊂ R d and unknown function f Q : X × X → [0 , 1] where X is a compact set and d an arbitrary fixed dimension, the edge probabilities are:

Typically, in network estimation, one observes adjacency matrix { A ij } distributed as { Bernoulli ( Q ij ) } , and either has to learn x i or directly estimate f Q . There has been much work in the statistics community on estimating x i for specific models (usually up to rotation). For stochastic block models, see the excellent survey in Abbe (2017).

Estimating f Q can be done with some additional assumptions (Chatterjee, 2015). When f Q has appropriate smoothness properties, one can estimate it by a histogram approximation (Olhede and Wolfe, 2014; Chan and Airoldi, 2014). This setting has also been compared to nonparametric regression with an unknown design (Gao et al., 2015). Methods for network estimation include Universal Singular Value Thresholding (Chatterjee, 2015; Xu, 2018), combinatorial optimization (Gao et al., 2015; Klopp et al., 2017), and neighborhood smoothing (Zhang et al., 2017; Mukherjee and Chakrabarti, 2019).

Transfer Learning on Networks. We wish to estimate the target network Q . However, we only observe f Q on ( n Q 2 ) pairs of nodes, for a uniformly random subset of variables S ⊂ { 1 , 2 , . . . , n } . We assume S is vanishingly small, so n Q := | S | = o ( n ) .

Absent additional information, we cannot hope to achieve o (1) mean-squared error. To see this, suppose f Q is a stochastic block model with 2 communities of equal size. For a node i ̸∈ S , no edges incident to i are observed, so its community cannot be learned. Since n Q ≪ n , we will attain Ω(1) error overall. To attain error o (1) , we hope to leverage transfer learning from a source P if available. In fact, we give an efficient algorithm to achieve o (1) error, formally stated in Section 2.

Theorem 1.1 (Theorem 2.3, Informal) . There exists an efficient algorithm such that, if given source data A P ∈ { 0 , 1 } n × n and target data A Q ∈ { 0 , 1 } n Q × n Q coming from an appropriate pair ( f P , f Q )

of latent variable models, outputs ̂ Q ∈ R n × n such that

There must be a relationship between P and Q for them to be an appropriate pair for transfer learning. We formalize this relationship below.

Relationship Between Source and Target. It is natural to consider pairs ( f P , f Q ) such that for all x , y ∈ X , the difference ( f P ( x , y ) -f Q ( x , y )) is small. For example, Cai and Pu (2024) study transfer learning for nonparametric regression when f P -f Q is close to a polynomial in x , y . But, requiring f P -f Q to be pointwise small does not capture a broad class of pairs in the network setting. For example, if f P = αf Q . Then f P -f Q = ( α -1) f Q can be far from all polynomials if f Q is, e.g. a Hölder-smooth graphon. 2 However, under the network model, this means A P and A Q are stochastically identical modulo one being α times denser than the other.

We will therefore consider pairs ( f P , f Q ) that are close in some measure of local graph structure. With this in mind, we use a graph distance introduced in Mao et al. (2021) for a different inference problem.

̸

Definition 1.2 (Graph Distance) . Let P ∈ [0 , 1] n × n be the probability matrix of a graph. For i, j ∈ [ n ] , i = j , we define the graph distance between them as follows:

where e i , e j ∈ R n are standard basis vectors.

Intuitively, this first computes the matrix P 2 of common neighbors, and then computes the distance between two rows of the same (ignoring the diagonal elements). We will require that f P , f Q satisfy a local similarity condition on the relative rankings of nodes with respect to this graph distance. Since we only estimate the probability matrix of Q , the condition is on the latent variables x 1 , . . . , x n of interest. The hope is that the proximity in graph distance reflects the proximity in latent positions.

̸

Definition 1.3 (Rankings Assumption at Quantile h n ) . Let ( P, Q ) be a pair of graphs evaluated on n latent positions. We say ( P, Q ) satisfy the rankings assumption at quantile h n ≤ 1 if there exists constant C > 0 such that for all i ∈ [ n ] and all j = i , if j belongs to the bottom h n -quantile of d P ( i, · ) , then j belongs to the bottom Ch n -quantile of d Q ( i, · ) .

To further motivate Definition 1.3, recall our motivating example of biological network estimation. Previous works require some form of similarity between networks to enable transfer Sen et al. (2018); Fan et al. (2019); Baranwal et al. (2020). For example, Kshirsagar et al. (2013) require a commonality hypothesis : if pathogens A, B target the same neighborhoods in a protein interaction network, one can transfer from A to B. Our rankings assumption similarly posits that to transfer knowledge from A to B, A and B have similar 2-hop neighborhood structures.

Note that Definition 1.3 involves quantiles of graph distances; therefore it is a relative condition, because it depends on a rank-ordering within both graphs P, Q before comparison. On the other hand, an absolute condition would require that for nodes i, j ∈ [ n ] , if e.g. d P ( i, j ) < 100 then d Q ( i, j ) < C · 100 . Our condition is more flexible and will hold for a larger set of graph pairs ( P, Q ) , such as pairs where one graph is much more dense than the other.

Finally, to illustrate Definition 1.3, consider stochastic block models f P , f Q with k P ≥ k Q communities respectively. If nodes i, j are in the same communities then P e i = P e j , so d P ( i, j ) = 0 . We require that j minimizes d Q ( i, · ) . This occurs if and only if d Q ( i, j ) = 0 . Hence if i, j belong to the same community in P , they are in the same community in Q . Note that the converse is not necessary; we could have Q with 1 community and P with arbitrarily many communities.

With the relationship between the source and target defined by the rankings assumption, our contributions are as follows.

(1) Algorithm for Latent Variable Models. We provide an efficient Algorithm 1 for latent variable models with Hölder-smooth f P , f Q . The benefit of this algorithm is that it does not assume a parametric form of f P and f Q . We prove a guarantee on its error in Theorem 2.3.

(2) Minimax Rates. We prove a minimax lower bound for Stochastic Block Models (SBMs) in Theorem 3.2. Moreover, we provide a simple Algorithm 2 that attains the minimax rate for this class (Proposition 3.4).

(3) Experimental Results on Real-World Data. We test both of our algorithms on real-world metabolic networks and dynamic email networks, as well as synthetic data (Section 4).

All proofs are deferred to the Appendix.

1.1 Other Related work

Transfer learning has recently drawn a lot of interest both in applied and theoretical communities. The notion of transferring knowledge from one domain with a lot of data to another with less available data has seen applications in epidemiology Apostolopoulos and Bessiana (2020), computer vision Long et al. (2015); Tzeng et al. (2017b); Huh et al. (2016); Donahue et al. (2014); Neyshabur et al. (2020), natural language processing Daumé (2007), etc. For a comprehensive survey see Zhuang et al. (2019); Weiss et al. (2016); Kim et al. (2022). Recently, there have also been advances in the theory of transfer learning Yang et al. (2013); Tripuraneni et al. (2020); Agarwal et al. (2023); Cai and Wei (2021); Cai and Pu (2024); Cody and Beling (2023).

In the context of networks, transfer learning is particularly useful since labeled data is typically hard to obtain. Tang et al. (2016) develop an algorithmic framework to transfer knowledge obtained using available labeled connections from a source network to do link prediction in a target network. Lee et al. (2017) proposes a deep learning framework for graph-structured data that incorporates transfer learning. They transfer geometric information from the source domain to enhance performance on related tasks in a target domain without the need for extensive new data or model training. The SGDA method Qiao et al. (2023) introduce adaptive shift parameters to mitigate domain shifts and propose pseudo-labeling of unlabeled nodes to alleviate label scarcity. Zou et al. (2021) proposes to transfer features from the previous network to the next one in the dynamic community detection problem. Simchowitz et al. (2023) work on combinatorial distribution shift for matrix completion, where only some rows and columns are given. A similar setting is used for link prediction in egocentrically sampled networks in Wu et al. (2018). Zhu et al. (2021) train a graph neural network for transfer based on an ego-graph-based loss function. Learning from observations of the full network and additional information from a game played on the network Leng et al. (2020); Rossi et al. (2022). Wu et al. (2024) study graph transfer learning for node regression in the Gaussian process setting, where the source and target networks are fully observed.

Levin et al. (2022) proposes an inference method from multiple networks all with the same mean but different variances. While our work is related, we do not assume E [ P ij ] = E [ Q ij ] . Cao et al. (2010) do joint link prediction on a collection of networks with the same link function but different parameters.

Another line of related but different work deals with multiplex networks (Lee et al., 2014, 2015; Iacovacci and Bianconi, 2016; Cozzo et al., 2018) and dynamic networks Sarkar and Moore (2005); Kimet al. (2018); Sewell and Chen (2015); Sarkar et al. (2012); Chang et al. (2024); Wang et al. (2023). One can think of transfer learning in clustering as clustering with side information. Prior works consider stochastic block models with noisy label information (Mossel and Xu, 2016; Mazumdar and Saha, 2017a) or oracle access to the latent structure (Mazumdar and Saha, 2017b).

Notation. We use lowercase letters a, b, c to denote (real) scalars, boldface x , y , z to denote vectors, and uppercase A,B,C to denote matrices. Let a ∨ b := max { a, b } and a ∧ b := min { a, b } . For integer n > 0 , let [ n ] := { 1 , 2 , . . . , n } . For a subset S ⊂ [ n ] and A ∈ R n × n , let A [ S, S ] ∈ R | S |×| S | be the principal submatrix with row and column indices in S . We denote the ℓ 2 vector norm as ∥ x ∥ = ∥ x ∥ 2 , dot product as ⟨ x , y ⟩ , and Frobenius norm as ∥ A ∥ = ∥ A ∥ F . For functions f, g : N → R we let f ≲ g denote f = O ( g ) and f ≳ g denote f = Ω( g ) . All asymptotics O ( · ) , o ( · ) , Ω( · ) , ω ( · ) are with respect to n Q unless specified otherwise.

2 Estimating Latent Variable Models with Rankings

In this section, we present a computationally efficient transfer learning algorithm for latent variable models. Algorithm 1 learns the local structure of P based on graph distances (Definition 1.2). For

Algorithm 1 ̂ Q -Estimation for Latent Variable Models

1: Input: A P ∈ { 0 , 1 } n × n , A Q ∈ { 0 , 1 } n Q × n Q , S ⊂ [ n ] s.t. | S | = n Q .

2: Initialize ̂ Q ∈ R n × n to be all zeroes.

̸

3: For all i , all j = i , compute graph distances:

4: Fix a bandwidth h ∈ (0 , 1) based on n, n Q .

5: for i = 1 to n do

7: if i ∈ S then

6: Let T A P i ( h ) ⊂ S be bottom h -quantile of S with respect to d A P ( i, · ) .

8: Update T A P i ( h ) ← T A P i ( h ) ∪ { i } .

9: end if

10: end for

11: for i = 2 to n do

12: for 1 ≤ j < i do

13: Compute ̂ Q ij = ̂ Q ji by averaging:

14: end for

15: end for

16: Return ̂ Q .

each node i of P , it ranks the nodes in S with respect to the graph distance d P ( i, · ) . For most nodes i, j ∈ [ n ] , none of the edges incident to i or j are observed in Q . Therefore, we estimate ̂ Q ij by using the edge information about nodes r, s ∈ S such that d P ( i, r ) and d P ( j, s ) are small.

Formally, we consider a model as in Eq. (1) with a compact latent space X ⊂ R d and latent variables sampled i.i.d. from the normalized Lebesgue measure on X . We set X = [0 , 1] d without loss of generality and assume that functions f : X × X → [0 , 1] are α -Hölder-smooth.

Definition 2.1. Let f : X ×X → R and α > 0 . We say f is α -Hölder-smooth if there exists C α > 0 such that for all x , x ′ , y ∈ X ,

To exclude degenerate cases where a node may not have enough neighbors in latent space, we require the following assumption.

Assumption 2.2 (Assumption 3.2 of Mao et al. (2021)) . Let G be a graph on x 1 , . . . , x n . There exist c 2 > c 1 > 0 and ∆ n = o (1) such that for all x i , x j ,

The second inequality follows directly from Hölder-smoothness, and the first is shown to hold for e.g. Generalized Random Dot Product Graphs, among others (Mao et al., 2021).

We establish the rate of estimation for Algorithm 1 below.

Theorem 2.3. Let ̂ Q be as in Algorithm 1. Let f P be α -Hölder-smooth and f Q be β -Höldersmooth for β ≥ α > 0 , and let c be an absolute constant. Suppose ( P, Q ) satisfy Definition 1.3 at h n = c √ log n Q n Q and P satisfies Assumption 2.2 with ∆ n = O (( log n n Q ) 1 2 ∨ α ∧ 1 d ) . Then there exists an absolute constant C > 0 such that

To parse Theorem 2.3, consider the effect of various parameter choices. First, observe that our upper bound scales quite slowly with n . Even if n is superpolynomial in n Q , e.g. n = n log n Q Q , then log n = O ((log n Q ) 2 ) = n o (1) Q , so the overall effect on the error is dominated by the n Q term.

Second, the bound is worse in large dimensions, and scales exponentially in 1 d . This kind of scaling also occurs in minimax lower bounds for nonparametric regression (Tsybakov, 2009), and upper bounds for smooth graphon estimation (Xu, 2018). However, we caution that nonparametric regression can be quite different from network estimation; it would be very interesting to know the dependence of dimension on minimax lower bounds for network estimation, but to the best of our knowledge this is an open problem. Finally notice that a greater smoothness β results in a smaller error, up to β = 1 , exactly as in (Gao et al., 2015; Klopp et al., 2017; Xu, 2018).

3 Minimax Rates for Stochastic Block Models

In this section, we will show matching lower and upper bounds for a very structured class of latent variable models, namely, Stochastic Block Models (SBMs).

Definition 3.1 (SBM) . Let P ∈ [0 , 1] n × n . We say P is an ( n, k ) -SBM if there exist B ∈ [0 , 1] k × k and z : [ n ] → [ k ] such that for all i, j , P ij = B z ( i ) z ( j ) . We refer to z -1 ( { j } ) as community j ∈ [ k ] .

We first state a minimax lower bound, proved via Fano's method.

Theorem 3.2 (Minimax Lower Bound for SBMs) . Let k P ≥ k Q ≥ 1 with k Q dividing k P . Let F be the family of pairs ( P, Q ) where P is an ( n, k P ) -SBM, Q is an ( n, k Q ) -SBM, and ( P, Q ) satisfy Definition 1.3 at h n = 1 /k P . Moreover, suppose S ⊂ [ n ] is restricted to contain an equal number of nodes from communities 1 , 2 , . . . , k P of P . Then the minimax rate of estimation is:

Note that Definition 1.3 at h n = 1 /k P implies that the true community structure of Q coarsens that of P . The condition that k Q divides k P is merely a technical one that we assume for simplicity.

We remark that minimax lower bounds for smooth graphon estimation are established by first showing lower bounds for SBMs, and then constructing a graphon with the same block structure using smooth mollifiers (Gao et al., 2015). Therefore, we expect that Theorem 3.2 can also be extended to the graphon setting, using the same techniques. However, sharp lower bounds for other classes such as Random Dot Product Graphs will likely require different techniques (Xie and Xu, 2020; Yan and Levin, 2023).

Remark 3.3 (Clustering Regime) . In Appendix A.4 we also prove a minimax lower bound of log k Q n Q in the regime where the error of recovering the true clustering z dominates. This matches the rate of Gao et al. (2015), but for estimating all n 2 entries of Q , rather than just the n 2 Q observed entries.

Theorem 3.2 suggests that a very simple algorithm might achieve the minimax rate. Namely, use both A P , A Q to learn communities, and then use only A Q to learn inter-community edge probabilities. If ( P, Q ) are in the nonparametric regime where regression error dominates clustering error (called the weak consistency or almost exact recovery regime), then the overall error will hopefully match the minimax rate.

We formalize this approach in Algorithm 2, and prove that it does achieve the minimax error rate in the weak consistency regime. To this end, we define the signal-to-noise ratio of an SBM with parameter B ∈ [0 , 1] k × k as follows:

̸

where p = min i B ii , q = max i = j B ij .

Proposition 3.4 (Error Rate of Algorithm 2) . Suppose P, Q ∈ [0 , 1] n × n are ( n, k P ) , ( n, k Q ) -SBMs with minimum community sizes n ( P ) min , n ( Q ) min respectively. Suppose also that ( P, Q ) satisfy

Algorithm 2 ̂ Q -Estimation for Stochastic Block Models

1: Input: A P ∈ { 0 , 1 } n × n , A Q ∈ { 0 , 1 } n Q × n Q , S ⊂ [ n ] s.t. | S | = n Q .

2: Estimate clusterings ̂ Z P ∈ { 0 , 1 } n × k P , ̂ Z Q ∈ { 0 , 1 } n Q × k Q using Chen et al. (2014) on A P , A Q respectively.

3: Let ̂ W Q ∈ R k Q × k Q be diagonal with

4: Initialize ̂ Π ∈ { 0 , 1 } k P × k Q to be all zeroes.

5: for i ∈ S do

6: Let j P ∈ [ k P ] , j Q ∈ [ k Q ] be the unique column indices at which row i of ̂ Z P , ̂ Z Q respectively are nonzero.

7: Let ̂ Π j P ,j Q = 1 .

8: end for

9: Let ̂ B Q ∈ [0 , 1] k Q × k Q be the block-average:

10: return ̂ Q := ̂ Z P ̂ Π ̂ B Q ̂ Π T ̂ Z T P .

Definition 1.3 at h n = n ( P ) min /n . Then if the signal-to-noise ratios are such that: s P ≥ C ( √ n n ( P ) min ∨ log 2 ( n ) √ n ( P ) min ) and s Q ≥ C ( √ n Q n min ( Q ) ∨ log 2 ( n Q ) √ n ( Q ) min ) for large enough constant C > 0 , Algorithm 2 returns ̂ Q such that

4 Experiments

In this section, we test Algorithm 1 against several classes of simulated and real-world networks. We use quantile cutoff of h n = √ log n Q n Q for Algorithm 1 in all experiments.

Baselines. To the best of our knowledge, our exact transfer formulation has not been considered before in the literature. Therefore, we implement two algorithms as alternatives to Algorithm 1.

Algorithm 2. Given A P ∈ { 0 , 1 } n × n , A Q ∈ { 0 , 1 } n Q × n Q , let k P = ⌈ √ n ⌉ , k Q = ⌈ √ n Q ⌉ . Compute spectral clusterings ̂ Z P , ̂ Z Q with k P , k Q clusters respectively. Let J S ∈ { 0 , 1 } n Q × n is such that J S ; ij = 1 if and only if i = j and i ∈ S . The projection ̂ Π ∈ R k P × k Q solves the leastsquares problem min Π ∈ R k P × k Q ∥ J S ̂ Z P Π -̂ Z Q ∥ 2 F . We compute the ̂ Π differently from steps 4-7 in Algorithm 2 to account for cases where Q is not a true coarsening of P . When Q is a true coarsening of P , this reduces to the procedure in steps 4-7. Given ̂ Z P , ̂ Π we return ̂ Q as in Algorithm 2.

(2) Oracle. Suppose that an oracle can access data for Q on all n ≫ n Q nodes as follows. Fix an error probability p ∈ (0 , 1) . The oracle is given symmetric A ′ Q ∈ { 0 , 1 } n × n with independent entries following a mixture distribution. For all i, j ∈ [ n ] with i < j let X ij ∼ Bernoulli ( p ) and Y ij ∼ Bernoulli ( Q ( x i , x j )) . Then:

Given A ′ Q , the oracle returns the estimate from Universal Singular Value Thresholding on A ′ Q Chatterjee (2015). As p → 0 , the error will approach O ( n -2 β 2 β + d ) for a β -smooth network on on d -dimensional latent variables (Xu, 2018), so the oracle will outperform any transfer algorithm.

Table 1: Comparison of different algorithms on simulated networks. Each cell reports ̂ µ ± 2 ̂ σ of the mean-squared error over 50 independent trials. Error numbers are all scaled by 1 e 2 for ease of reading. Bold: Best algorithm. Emphasis: Second-best algorithm.

Simulations. We first test on several classes of simulated networks. For n Q = 50 , n = 200 , we run 50 independent trials for each setting. We report results for each setting in Table 1, and visualize estimates for stylized examples in Figure 1.

At a glance, Figure 1 shows that Algorithms 1 and 2 both work well on Stochastic Block Models (first row), that only Algorithm 1 works well on graphons (second and third rows), and that the Oracle performs well in all cases.

Smooth Graphons. The latent space is X = [0 , 1] . We consider graphons of the form f γ ( x, y ) = x γ + y γ 2 where P, Q have different γ . We denote this the γ -Smooth Graphon.

Mixed-Membership Stochastic Block Model. Set k P = ⌊ √ n ⌋ , k Q = ⌊ √ n Q ⌋ . The latent space X is the probability simplex X = ∆ k P := { x ∈ [0 , 1] k P : ∑ i x i = 1 } ⊂ R k P . The latent variables x 1 , . . . , x n are iid-Dirichlet distributed with equal weights 1 k P , . . . , 1 k P . Then P ij = x T i B P x j and Q ij = Π( x i ) T B Q Π( x j ) , for connectivity matrices B P ∈ [0 , 1] k P × k P , B Q ∈ [0 , 1] k Q × k Q , and projection Π : ∆ k P → ∆ k Q for a fixed subset of [ k P ] . For parameters a, b, ϵ ∈ [0 , 1] we generate B ∈ [0 , 1] k × k by sampling E ∈ Uniform ( -ϵ, ϵ ) k × k and set B = clip (( a -b ) I + b 11 T + E, 0 , 1) . We call this Noisy-MMSB ( a, b, ϵ ) .

Latent Distance Model. The latent space is the unit sphere X = S d -1 ⊂ R d . For scale parameter s > 0 , we call f s ( x , y ) = exp( -s ∥ x -y ∥ 2 ) the R d -Latent ( s ) model.

Discussion. When the latent dimension is larger than 1 (the Noisy MMSB and Latent Variable Models), our Algorithm 1 is better than both Algorithm 2 and the Oracle with p = 0 . 1 . Note that

2

Algorithms 1 and 2 use n Q n 2 ≈ 0 . 06 unbiased edge observations from Q , while the Oracle with p = 0 . 1 observes (1 -p ) n 2 -n 2 Q n 2 ≈ 0 . 9 unbiased edge observations in expectation.

Real-World Data. Next, we test on two classes of real-world networks. We summarize our dataset characteristics in Table 2. See Appendix C for further details.

Transfer Across Species in Metabolic Networks. For a fixed organism, a metabolic network has a node for each metabolite, and an edge exists if and only if two metabolites co-occur in a metabolic

Source

Algorithm 1

Algorithm 2

Oracle (p = 0.1)

1.0

0.5

0.0

1.0

0.5

0.0

1.0

0.5

0.0

reaction in that organism. We obtain the unweighted metabolic networks for multiple gram-negative bacteria from the BiGG genome-scale metabolic model dataset (King et al., 2016; Norsigian et al., 2020). In the left half of Figure 2, we compare two choices of source organism in estimating the network for BiGG model iJN1463 ( Pseudomonas putida ). For a good choice of source, Algorithm 1 is competitive with the Oracle at p = 0 . 1 .

Transfer Across Time in the Email Interaction Networks. We use the EMAIL-EU interaction network between n = 1005 members of a European research institution across 803 days Leskovec and Krevl (2014); Paranjape et al. (2017). The source graph A P is the network from day 1 to ≈ 80 ( [1 , 80] ). In Figure 2 we simulate transfer with targets [81 , 160] (left) and [561 , 640] (right). We visualize results for arbitrary target periods; similar results hold for other targets. Unlike metabolic networks, Algorithm 2 has comparable performance to both our Algorithm 1 and the oracle algorithm with p ∈ { 0 . 01 , 0 . 05 } . Compared to the metabolic networks, this indicates that the email interaction networks are relatively well-approximated by SBMs, although Algorithm 1 is still the best.

Additional Experiments and Baseline. In Appendix B.1, we present additional ablation experiments that test the dependence of Algorithms 1 and 2 on all relevant parameters. We compare their performance to the Oracle baseline with p = 0 . 0 (the non-transfer setting), and an additional baseline adapted from Levin et al. (2022). We find that our Algorithms outperform this new baseline but are worse than the Oracle with p = 0 . 0 , as expected. Further, in Appendix B.2, we test our Algorithms and original baselines on a link prediction task in the setting of Figure 2. We find that the relative accuracy of the methods for link prediction is qualitatively similar to that of Figure 2, and the Oracle performs even better with sparsity tuning.

50

100

150

200

nQ

, with

n

=251

10

1

4×10

2

6×10

2

2×10

1

MSE (Log Scale)

50

100

150

200

nQ

, with

n

=251

10

1

4×10

2

6×10

2

2×10

1

20

40

60

80

100

nQ

, with

n

=1005

7×10

3

8×10

3

20

40

60

80

100

nQ

, with

n

=1005

7×10

3

8×10

3

Metabolic Networks

Alg. 1

Alg. 2

Oracle,

p

=0.1

Oracle,

p

=0.3

Oracle,

p

=0.5

Oracle,

p

=0.01

Oracle,

p

=0.05

Left half : Estimating metabolic network of iJN1463 ( Pseudomonas putida ) with source iWFL1372 ( Escherichia coli W ) leftmost, and source iPC815 ( Yersinia pestis ) second-left.

Right half : Using source data from days 1 -80 of EMAIL-EU to estimate target days 81 -160 (third-left) and target days 561 -640 (rightmost). Note that we use smaller values of p for the Oracle in EMAIL-EU.

5 Conclusion

In this paper, we study transfer learning for network estimation in latent variable models. We show that there exists an efficient Algorithm 1 that achieves vanishing error even when n ≥ n ω (1) Q , and a simpler Algorithm 2 for SBMs that achieves the minimax rate.

There are several interesting directions for future work.

First, we believe that Algorithm 1 works for moderately sparse networks with population edge density Ω( 1 √ n ) . This is because the concentration of empirical graph distance (Algorithm 1 line 3) requires expected edge density ˜ Ω( n -1 / 2 ) Mao et al. (2021). It would be interesting to see if a similar approach can work for edge density Ω( log n n ) . For example, in the aforementioned paper it is shown that a variation of the graph distance of Definition 1.2 concentrates at expected edge density ˜ Ω( n -2 / 3 ) . While is this still far from the Ω( log n n ) regime, it suggests that variations on the graph distance might ensure our Algorithm 1 works for sparser graphs.

Second, the case of multiple sources is also interesting. We have focused on the case of one source distribution, as in Cai and Wei (2021); Cai and Pu (2024), but expect that our algorithms can be extended to multiple sources as long as they satisfy Definition 1.3.

Acknowledgments and Disclosure of Funding

We thank the anonymous reviewers for their valuable feedback.

AJ and PS gratefully acknowledge NSF grants 2217069, 2019844, and DMS 2109155.

AMwas supported by NSF awards 2217058 and 2133484.

SSM was partially supported by an INSPIRE research grant (DST/INSPIRE/04/2018/002193) from the Dept. of Science and Technology, Govt. of India and a Start-Up Grant from Indian Statistical Institute.

References

Abbe, E. (2017). Community detection and stochastic block models. arXiv preprint arXiv:1703.10146 .

Agarwal, A., Song, Y., Sun, W., Wang, K., Wang, M., and Zhang, X. (2023). Provable benefits of representational transfer in reinforcement learning. In Neu, G. and Rosasco, L., editors, Proceedings of Thirty Sixth Conference on Learning Theory , volume 195 of Proceedings of Machine Learning Research , pages 2114-2187. PMLR.

Airoldi, E. M., Blei, D. M., Fienberg, S. E., and Xing, E. P. (2008). Mixed membership stochastic blockmodels. Journal of Machine Learning Research , 9:1981-2014.

Aldous, D. J. (1981). Representations for partially exchangeable arrays of random variables. Journal of Multivariate Analysis , 11:581-598.

Apostolopoulos, I. D. and Bessiana, T. (2020). Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine , 43:635 - 640.

Baranwal, M., Magner, A., Elvati, P., Saldinger, J., Violi, A., and Hero, A. O. (2020). A deep learning architecture for metabolic pathway prediction. Bioinformatics , 36(8):2547-2553.

Barberá, P., Jost, J. T., Nagler, J., Tucker, J. A., and Bonneau, R. (2015). Tweeting from left to right: Is online political communication more than an echo chamber? Psychological science , 26(10):1531-1542.

Ben-David, S., Blitzer, J., Crammer, K., and Pereira, F. (2006). Analysis of representations for domain adaptation. Advances in neural information processing systems , 19.

Bickel, P. J. and Chen, A. (2009). A nonparametric view of network models and newman-girvan and other modularities. Proceedings of the National Academy of Sciences , 106(50):21068-21073.

Cai, T. T. and Pu, H. (2024). Transfer learning for nonparametric regression: Non-asymptotic minimax analysis and adaptive procedure. arXiv preprint arXiv:2401.12272 .

Cai, T. T. and Wei, H. (2021). Transfer learning for nonparametric classification: Minimax rate and adaptive classifier. The Annals of Statistics , 49(1).

Cao, B., Liu, N. N., and Yang, Q. (2010). Transfer learning for collective link prediction in multiple heterogenous domains. In Proceedings of the 27th international conference on machine learning (ICML-10) , pages 159-166. Citeseer.

Cao, X. and Ward, M. D. (2014). Do democracies attract portfolio investment? transnational portfolio investments modeled as dynamic network. International Interactions , 40(2):216-245.

Chan, S. and Airoldi, E. (2014). A consistent histogram estimator for exchangeable graph models. In International Conference on Machine Learning , pages 208-216. PMLR.

Chang, S., Koehler, F., Qu, Z., Leskovec, J., and Ugander, J. (2024). Inferring dynamic networks from marginals with iterative proportional fitting. arXiv preprint arXiv:2402.18697 .

Chatterjee, S. (2015). Matrix estimation by universal singular value thresholding. The Annals of Statistics , pages 177-214.

Chen, Y., Sanghavi, S., and Xu, H. (2014). Improved graph clustering. IEEE Transactions on Information Theory , 60(10):6440-6455.

Christensen, B. and Nielsen, J. (2000). Metabolic Network Analysis , pages 209-231. Springer Berlin Heidelberg, Berlin, Heidelberg.

Cody, T. and Beling, P. A. (2023). A systems theory of transfer learning. IEEE Systems Journal , 17(1):26-37.

Cortes, C., Mohri, M., Riley, M., and Rostamizadeh, A. (2008). Sample selection bias correction theory. In International conference on algorithmic learning theory , pages 38-53. Springer.

Cozzo, E., De Arruda, G. F., Rodrigues, F. A., and Moreno, Y. (2018). Multiplex networks: basic formalism and structural properties , volume 10. Springer.

Crammer, K., Kearns, M., and Wortman, J. (2008). Learning from multiple sources. Journal of Machine Learning Research , 9(8).

Daumé, H. (2007). Frustratingly easy domain adaptation. ArXiv , abs/0907.1815.

Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. (2014). Decaf: A deep convolutional activation feature for generic visual recognition. In Xing, E. P. and Jebara, T., editors, Proceedings of the 31st International Conference on Machine Learning , volume 32 of Proceedings of Machine Learning Research , pages 647-655, Bejing, China. PMLR.

Fan, J., Cannistra, A., Fried, I., Lim, T., Schaffner, T., Crovella, M., Hescott, B., and Leiserson, M. D. (2019). Functional protein representations from biological networks enable diverse cross-species inference. Nucleic acids research , 47(9):e51-e51.

Gao, C., Lu, Y., and Zhou, H. H. (2015). Rate-optimal graphon estimation. The Annals of Statistics , pages 2624-2652.

Guruswami, V., Rudra, A., and Sudan, M. (2019). Essential coding theory.

Handcock, M. S., Raftery, A. E., and Tantrum, J. M. (2007). Model-based clustering for social networks. Journal of the Royal Statistical Society: Series A (Statistics in Society) , 170(2):301-354.

Hanneke, S. and Kpotufe, S. (2019). On the value of target data in transfer learning. Advances in Neural Information Processing Systems , 32.

Hanneke, S. and Kpotufe, S. (2022). A no-free-lunch theorem for multitask learning. The Annals of Statistics , 50(6):3119-3143.

Hoeffding, W. (1994). Probability inequalities for sums of bounded random variables. The collected works of Wassily Hoeffding , pages 409-426.

Hoff, P. (2007). Modeling homophily and stochastic equivalence in symmetric relational data. Advances in neural information processing systems , 20.

Hoff, P. D., Raftery, A. E., and Handcock, M. S. (2002). Latent Space Approaches to Social Network Analysis. Journal of the American Statistical Association , 97(460):1090-1098.

Holland, P. W., Laskey, K. B., and Leinhardt, S. (1983). Stochastic blockmodels: First steps. Social Networks , 5(2):109-137.

Hoover, D. N. (1979). Relations on probability spaces arrays of random variables . Institute for Advanced Study,RI.

Huang, J.-T., Li, J., Yu, D., Deng, L., and Gong, Y. (2013). Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. In 2013 IEEE international conference on acoustics, speech and signal processing , pages 7304-7308. IEEE.

Huh, M., Agrawal, P., and Efros, A. A. (2016). What makes imagenet good for transfer learning? arXiv preprint arXiv:1608.08614 .

Iacovacci, J. and Bianconi, G. (2016). Extracting information from multiplex networks. Chaos: An Interdisciplinary Journal of Nonlinear Science , 26(6).

Kim, B., Lee, K. H., Xue, L., and Niu, X. (2018). A review of dynamic network models with latent variables. Statistics Surveys , 12(none):105 - 135.

Kim, H., Cosa-Linan, A., Santhanam, N., and et al. (2022). Transfer learning for medical image classification: a literature review. BMC Medical Imaging , 22(1):69.

King, Z. A., Lu, J., Dräger, A., Miller, P., Federowicz, S., Lerman, J. A., Ebrahim, A., Palsson, B. O., and Lewis, N. E. (2016). Bigg models: A platform for integrating, standardizing and sharing genome-scale models. Nucleic acids research , 44(D1):D515-D522.

Klopp, O., Tsybakov, A. B., and Verzelen, N. (2017). Oracle inequalities for network models and sparse graphon estimation. Annals of Statistics , 45(1):316-354.

Kshirsagar, M., Carbonell, J., and Klein-Seetharaman, J. (2013). Multitask learning for host-pathogen protein interactions. Bioinformatics , 29(13):i217-i226.

Lee, J., Kim, H., Lee, J., and Yoon, S. (2017). Transfer learning for deep learning on graph-structured data. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 31.

Lee, K.-M., Kim, J. Y., Lee, S., and Goh, K.-I. (2014). Multiplex networks. Networks of networks: The last frontier of complexity , pages 53-72.

Lee, K.-M., Min, B., and Goh, K.-I. (2015). Towards real-world complexity: an introduction to multiplex networks. The European Physical Journal B , 88:1-20.

Leng, Y., Dong, X., Wu, J., and Pentland, A. (2020). Learning quadratic games on networks. In International Conference on Machine Learning , pages 5820-5830. PMLR.

Leskovec, J. and Krevl, A. (2014). SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data .

Levin, K., Lodhia, A., and Levina, E. (2022). Recovering shared structure from multiple networks with unknown edge distributions. Journal of machine learning research , 23(3):1-48.

Li, L., Dannenfelser, R., Zhu, Y., Hejduk, N., Segarra, S., and Yao, V. (2022). Joint embedding of biological networks for cross-species functional alignment. bioRxiv , pages 2022-01.

Long, J., Shelhamer, E., and Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3431-3440.

Lovász, L. (2012). Large Networks and Graph Limits. , volume 60 of Colloquium Publications . American Mathematical Society.

Mansour, Y., Mohri, M., and Rostamizadeh, A. (2009). Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430 .

Mao, X., Chakrabarti, D., and Sarkar, P. (2021). Consistent nonparametric methods for network assisted covariate estimation. In International Conference on Machine Learning , pages 7435-7446. PMLR.

Mazumdar, A. and Saha, B. (2017a). Clustering with noisy queries. Advances in Neural Information Processing Systems , 30.

Mazumdar, A. and Saha, B. (2017b). Query complexity of clustering with side information. Advances in Neural Information Processing Systems , 30.

Mossel, E. and Xu, J. (2016). Local algorithms for block models with side information. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science , pages 71-80.

Mukherjee, S. S. and Chakrabarti, S. (2019). Graphon estimation from partially observed network data. arXiv preprint arXiv:1906.00494 .

Neyshabur, B., Sedghi, H., and Zhang, C. (2020). What is being transferred in transfer learning? In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems , volume 33, pages 512-523. Curran Associates, Inc.

Norsigian, C. J., Pusarla, N., McConn, J. L., Yurkovich, J. T., Dräger, A., Palsson, B. O., and King, Z. (2020). Bigg models 2020: multi-strain genome-scale models and expansion across the phylogenetic tree. Nucleic acids research , 48(D1):D402-D406.

Olhede, S. C. and Wolfe, P. J. (2014). Network histograms and universality of blockmodel approximation. Proceedings of the National Academy of Sciences , 111(41):14722-14727.

Paranjape, A., Benson, A. R., and Leskovec, J. (2017). Motifs in temporal networks. In Proceedings of the tenth ACM international conference on web search and data mining , pages 601-610.

Qiao, Z., Luo, X., Xiao, M., Dong, H., Zhou, Y., and Xiong, H. (2023). Semi-supervised domain adaptation in graph transfer learning. ArXiv , abs/2309.10773.

Ren, M., Zhang, S., and Wang, J. (2023). Consistent estimation of the number of communities via regularized network embedding. Biometrics , 79(3):2404-2416.

Rossi, E., Monti, F., Leng, Y., Bronstein, M., and Dong, X. (2022). Learning to infer structures of network games. In International Conference on Machine Learning , pages 18809-18827. PMLR.

Rubin-Delanchy, P., Cape, J., Tang, M., and Priebe, C. E. (2022). A statistical interpretation of spectral embedding: the generalised random dot product graph. Journal of the Royal Statistical Society Series B: Statistical Methodology , 84(4):1446-1473.

Sarkar, P., Chakrabarti, D., and Jordan, M. I. (2012). Nonparametric link prediction in dynamic networks. In Proceedings of the 29th International Coference on International Conference on Machine Learning , ICML'12, page 1897-1904, Madison, WI, USA. Omnipress.

Sarkar, P. and Moore, A. (2005). Dynamic social network analysis using latent space models. In Weiss, Y., Schölkopf, B., and Platt, J., editors, Advances in Neural Information Processing Systems , volume 18. MIT Press.

Sen, R., Tagore, S., and De, R. K. (2018). Asapp: Architectural similarity-based automated pathway prediction system and its application in host-pathogen interactions. IEEE/ACM Transactions on Computational Biology and Bioinformatics , 17(2):506-515.

Sewell, D. K. and Chen, Y. (2015). Latent space models for dynamic networks. Journal of the american statistical association , 110(512):1646-1657.

Simchowitz, M., Gupta, A., and Zhang, K. (2023). Tackling combinatorial distribution shift: A matrix completion perspective. In Neu, G. and Rosasco, L., editors, Proceedings of Thirty Sixth Conference on Learning Theory , volume 195 of Proceedings of Machine Learning Research , pages 3356-3468. PMLR.

Sweet, T. M., Thomas, A. C., and Junker, B. W. (2013). Hierarchical network models for education research: Hierarchical latent space models. Journal of Educational and Behavioral Statistics , 38(3):295-318.

Tang, J., Lou, T., Kleinberg, J., and Wu, S. (2016). Transfer learning to infer social ties across heterogeneous networks. ACM Trans. Inf. Syst. , 34(2).

Trifonova, N., Kenny, A., Maxwell, D., Duplisea, D., Fernandes, J., and Tucker, A. (2015). Spatiotemporal bayesian network models with latent variables for revealing trophic dynamics and functional networks in fisheries ecology. Ecological Informatics , 30:142-158.

Tripuraneni, N., Jordan, M. I., and Jin, C. (2020). On the theory of transfer learning: the importance of task diversity. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS '20, Red Hook, NY, USA. Curran Associates Inc.

Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation . Springer series in statistics. Springer, Dordrecht.

Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T. (2017a). Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7167-7176.

Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T. (2017b). Adversarial discriminative domain adaptation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2962-2971.

Vershynin, R. (2018). High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press.

Wang, H., Mao, Y., Sun, J., Zhang, S., Fan, Y., and Zhou, D. (2023). Dynamic transfer learning across graphs. arXiv preprint arXiv:2305.00664 .

Weiss, K. R., Khoshgoftaar, T. M., and Wang, D. (2016). A survey of transfer learning. Journal of Big Data , 3:1-40.

Wu, J., Ainsworth, L., Leakey, A., Wang, H., and He, J. (2024). Graph-structured gaussian processes for transferable graph learning. Advances in Neural Information Processing Systems , 36.

Wu, Y.-J., Levina, E., and Zhu, J. (2018). Link prediction for egocentrically sampled networks. Journal of Computational and Graphical Statistics , 32:1296 - 1319.

Xie, F. and Xu, Y. (2020). Optimal bayesian estimation for random dot product graphs. Biometrika , 107(4):875-889.

Xu, J. (2018). Rates of convergence of spectral methods for graphon estimation. In International Conference on Machine Learning , pages 5433-5442. PMLR.

Yan, H. and Levin, K. (2023). Minimax rates for latent position estimation in the generalized random dot product graph. arXiv preprint arXiv:2307.01942 .

Yang, L., Hanneke, S., and Carbonell, J. (2013). A theory of transfer learning with applications to active learning. Machine learning , 90:161-189.

Yu, B. (1997). Assouad, fano, and le cam. In Festschrift for Lucien Le Cam: research papers in probability and statistics , pages 423-435. Springer.

Zhang, Y., Levina, E., and Zhu, J. (2017). Estimating network edge probabilities by neighbourhood smoothing. Biometrika , 104(4):771-783.

Zhu, Q., Yang, C., Xu, Y., Wang, H., Zhang, C., and Han, J. (2021). Transfer learning of graph neural networks with ego-graph information maximization. Advances in Neural Information Processing Systems , 34:1766-1779.

Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y ., Zhu, H., Xiong, H., and He, Q. (2019). A comprehensive survey on transfer learning. Proceedings of the IEEE , 109:43-76.

Zou, J., Lin, F., Gao, S., Deng, G., Zeng, W., and Alterovitz, G. (2021). Transfer learning based multiobjective genetic algorithm for dynamic community detection. arXiv preprint arXiv:2109.15136 .

A Proofs

A.1 Preliminaries

Recall Hoeffding's inequality.

Lemma A.1 (Hoeffding (1994)) . Let X 1 , . . . , X n be independent random variables such that a i ≤ X i ≤ b i almost surely for all i ∈ [ n ] . Then

We also need Bernstein's inequality.

Lemma A.2 (Bernstein's Inequality) . Let X 1 , . . . , X n be independent mean-zero random variables with | X i | ≤ 1 for all i and n ≥ 5 . Then

A.2 Proof of Theorem 2.3

Throughout this section, let X = [0 , 1] d and µ : X → [0 , 1] be the normalized Lebesgue measure. We require the following Lemmata.

Lemma A.3. Let υ ∈ (0 , 1) and µ : X → [0 , 1] be the normalized Lebesgue measure. Then for all x ∈ X ,

Proof. Recall X = [0 , 1] d . Fix x ∈ X , υ > 0 . Note that µ (Ball( x , υ ) ∩ X ) is smallest when x is a vertex of the hypercube; therefore take x ∈ { 0 , 1 } d without loss of generality. Then, note that for each z ∈ Ball( x , υ ) ∩X , we can find (2 d -1) other points z ′ ∈ Ball( x , υ ) \X by reflecting subsets of coordinates of z about x . There are 2 d -1 such nonempty subsets of coordinates. This shows that µ (Ball( x , υ ) ∩ X ) ≥ µ (Ball( x , υ )) / 2 d for all x . Since µ (Ball( x , υ )) ≍ υ d , the conclusion follows.

We will repeatedly make use of the concentration of latent positions.

Lemma A.4 (Latent Concentration) . Let X = [0 , 1] d and µ denote the normalized Lebesgue measure on X . Suppose x 1 , . . . , x n ∼ X are sampled iid and uniformly at random from µ . Fix some T ⊂ X such that µ ( T ) = v . Then

Proof. Let X i be an indicator variable that equals 1 if x i ∈ T and zero otherwise. Notice the X i are iid and bounded within [0 , 1] . Moreover, ∑ i E [ X i ] = nµ ( T ) . Therefore by Hoeffding's inequality, for any t > 0 ,

Setting t = 10 √ log n n gives the result.

Corollary A.5. Let ϵ > 0 . For i ∈ [ n ] let ϵ ′ i > 0 be ϵ ′ i := sup { υ > 0 : µ (Ball( x i , υ ) ∩ X ) ≤ ϵ . Let T i := Ball( x i , ϵ ′ i ) ∩ X . Let u i ( S ) := |{ j ∈ S : x j ∈ T i }| denote the number of members of S landing in T i . Then

Proof. Notice that each T i has Lebesgue measure ϵ by definition. Therefore E [ u i ( S )] = n Q ϵ . Since S has n Q members, setting t = 10 √ log n n Q in the statement of Lemma A.4 and taking a union bound over all i ∈ [ n ] gives the conclusion.

We will decompose the error of Algorithm 1 into two parts.

Proposition A.6. Let ̂ Q ∈ [0 , 1] n × n be the estimator from Algorithm 1. Then

where J S , J B are the smoothing and Bernoulli errors respectively:

Controlling the Bernoulli errors is relatively straightforward.

Proposition A.7. Let h be the bandwidth of Algorithm 1. The Bernoulli error is at most O ( log n m ) with probability ≥ 1 -n -8 , where m = h 2 n 2 Q .

Proof. Fix i, j ∈ [ n ] . We will bound the maximum Bernoulli error J S ( i, j ) over i, j , which suffices to bound the average. Let m = | T i | | T j | . We want to bound:

Notice each summand is bounded within ± 1 m . Bernstein's inequality gives:

Setting t = C √ log n m for large enough C = O (1) , a union bound tells us that with probability ≥ 1 -n -8 , the Bernoulli error is bounded by t 2 .

Corollary A.8. The Bernoulli error is at most O ( √ log n Q n Q ) with probability ≥ 1 -n -4 Q .

The rest of this section is devoted to bounded the smoothing errors J S ( i, j ) .

A.2.1 Latent Distance to Graph Distance

We claim that if nodes are close in the latent space then they are close in graph distance.

Proposition A.9. Suppose that ∥ x i -x r ∥ ≤ ϵ and Q is β -smooth. Then d Q ( i, r ) ≤ C 2 β n 3 ϵ 2( β ∧ 1) .

Proof. We the use smoothness of Q . By definition there exists C β > 0 such that Q ki -Q kr ≤ C β ∥ x i -x r ∥ β ∧ 1 . Therefore,

̸

̸

Wecan now bound the minimum sizes of the neighborhoods using the concentration of latent positions and the smoothness of the graphon.

Lemma A.10 (Vershynin (2018)) . The volume of a ball of radius r > 0 in R d is √ π d Γ( d/ 2+1) r d , where Γ( · ) is the Γ function.

Proposition A.11. Let C d = (Γ( d 2 +1)) 1 /d . Let C 0 , C ′ be constants. If υ n ≥ C · C d ( √ log n n Q ) 1 /d for large enough constant C > 0 , and g n = C 0 C 2 β n 2 ( υ n ) 2( β ∧ 1) , then with probability ≥ 1 -n -6 for all i ∈ [ n ] the neighborhood size is |{ r : d Q ( i, r ) ≤ g n }| ≥ C ′ n Q √ log n n Q .

Proof. Fix i ∈ [ n ] and υ n > 0 . Let ϵ i denote the Lebesgue measure of Ball( x i , υ n ) ∩ X . By Lemma A.3 and Lemma A.10, for all i , ϵ i ≥ ( √ πυ n 2 C d ) d = ( 0 . 5 √ πυ n C d ) d . Let ϵ = min i ∈ [ n ] ϵ i .

By Corollary A.5, with probability ≥ 1 -n -8 , there are n Q ϵ -C √ log n n Q members j of S such that ∥ x i -x j ∥ ≤ υ n . A union bound over i gives the result simultaneously for all i with probability ≥ 1 -n -6 .

From Proposition A.9, it follows that for all i ∈ [ n ] ,

Choosing υ n ≥ C · C d ( log n n Q ) 1 2 d for large enough C > 0 gives the conclusion.

A.2.2 Graph Distance Concentration

Next, we show that the empirical graph distance concentrates to the population distance.

Proposition A.12. For any arbitrary symmetric P ∈ [0 , 1] n × n , we have, for all i, j simultaneously with probability at least ≥ 1 -O ( n -8 ) , that

̸

Proof. Fix i, j . Let C ij := ( A 2 P ) ij . By Mao et al. (2021) A.1, we have C ij = ( P 2 ) ij + t ij for an error term t ij such that P [ ∀ i, j : | t ij | ≤ 10 √ n log n ] ≥ 1 -n -10 . Then,

̸

2

̸

Finally, notice that all entries of P are of size O ( n ) , so the conclusion follows.

Finally, we will show that taking the restriction of the graph distance T P i to nodes in S ⊂ [ n ] does not incur too much error.

Proposition A.13. Suppose n = n O (1) Q . Then there exists a constant C such that if h 0 ≥ C √ log n n Q + ∆ n , then for all i, r simultaneously, r ∈ T A P i ( h 0 ) implies r ∈ T P i ( h 2 ) for some h 2 = O ( h ) with probability ≥ 1 -O ( n -5 ) .

Proof. Let us introduce the notation T P,S i ( h ) to denote the bottom h -quantile of { d P ( i, j ) : j ∈ S } . In this notation, T A P i ( h ) := T A P ,S i ( h ) since we restrict the quantile to nodes in S . From Proposition A.12 and Assumption 2.2, we know that if n ≥ n Q then for h 0 ≤ h 1 -20 √ log n n -∆ n we have T A P i ( h 0 ) ⊆ T P,S i ( h 1 ) simultaneously for all i ∈ [ n ] with probability ≥ 1 -O ( n -8 ) . It remains to compare T P,S i ( h 1 ) with T P i ( h 2 ) for some h 2 .

We claim that if h 2 ≥ 30 √ log n Q n Q then P [ ∀ i ∣ ∣ T P i ∩ S ∣ ∣ ≥ h 2 n Q -3 √ n Q log n Q ] ≥ 1 -O ( n -2 Q ) . To see this, fix i ∈ [ n ] and consider T P i ( h 2 ) . For j ∈ S , let X j be the indicator variable:

Notice that ∣ ∣ T P i ( h 2 ) ∩ S ∣ ∣ = ∑ j ∈ S X j . By Hoeffding's inequality, since E [ ∑ j ∈ S X j ] = h 2 n Q and | X j -h 2 | ≤ 1 for all j , we have

Taking a union bound over all i ∈ [ n ] shows the claim holds with probability ≥ 1 -O ( n -5 ) . Therefore we set h 1 ≤ h 2 -3 . 1 √ log n n Q then j ∈ T P,S i ( h 1 ) implies j ∈ T P i ( h 2 ) .

The conclusion follows with C = 24 √ log n log n Q = O (1) .

The ranking condition (Definition 1.3) then allows us to translate between graph distances in A P and Q .

Corollary A.14. Suppose that Definition 1.3 holds for ( P, Q ) at h n = c √ log n Q n Q +∆ n , for large enough constant c > 0 . Suppose n Q ≤ n ≤ n O (1) Q . Then for h > h n and r ∈ T A P i ( h ) , it follows that r ∈ T Q i ( h 3 ) for some h 3 = O ( h ) . The statement holds simultaneously for all i, r with probability ≥ 1 -O ( n -5 ) .

̸

A.2.3 Control of Smoothing Error

We will decompose smoothing error into a sum of two terms called E S, 1 and E S, 2 . The control of E S, 1 is relatively straightforward.

Lemma A.15. The total smoothing error can be bounded with two terms:

where

Proof. Note that

The second inner summand is precise E S, 2 . For E S, 1 , notice that | T i | = | T j | = h ( n Q -1) by definition. Therefore

We can now bound E S, 1 in terms of graph distances.

Lemma A.16. The smoothing error term E S, 1 can be bounded as follows:

for some constant c > 0 .

Proof. Fix i ∈ [ n ] and r ∈ T i . We have

Now we will pass to graph distances. Let e ab := (( Q 2 ) aa -( Q 2 ) ab ) 2 for a, b ∈ [ n ] . Notice that ∥ Q 2 ( e i -e r ) ∥ 2 = √ d Q ( i, r ) + e ir + e ri . Moreover, √ e ir + e ri ≤ 2 √ n since the entries of Q 2 are individually bounded by O ( n ) . The conclusion follows.

Proposition A.17. Suppose ∆ n = O ( √ log n n Q ) . Let C d be the constant of Proposition A.11. Then if the bandwidth of Algorithm 1 is h n = C √ log n n Q , for a constant C = O (1) , then the smoothing error E S, 1 is at most

for some C 2 = O (1) , with probability ≥ 1 -O ( n -6 ) .

Proof. Fix i ∈ [ n ] and r ∈ T A P i ( h n ) . By Corollary A.14, if h n ≥ C √ log n n Q +∆ n for a large enough constant C > 0 , then there exists constant C 2 > 0 such that the following holds. With probability ≥ 1 -O ( n -5 ) , for all i ∈ [ n ] and r ∈ S , r ∈ T Q i ( C 2 h n ) ,

Let υ n = CC d ( √ log n n Q ) 1 /d for C d as in Proposition A.11 and C > 0 large enough constant. Then by Proposition A.11 the set of s ∈ S such that d Q ( i, r ) ≤ C 0 C 2 β n 2 ( υ n ) 2( β ∧ 1) has size at least C 2 n Q √ log n n Q .The statement holds for all i simultaneously with probability at least 1 -O ( n -6 ) . Therefore for all i ∈ [ n ] and r ∈ T A P ( h ) , we have i n

for some C 0 , C β = O (1) , with probability ≥ 1 -O ( n -6 ) . By Lemma A.16 we conclude that E S, 1 is bounded by 2 υ β ∧ 1 n + 2 √ n with the same probability.

A.2.4 Control of the Second Smoothing Error

In this section, we show that the second smoothing error can be controlled in terms of E S, 1 . We will need to track the following quantity.

Definition A.18 (Membership Count) . For r ∈ S and bandwidth h , distance cutoff ϵ , the P -neighborhood count of r is ψ P ( r ) := ∣ ∣ { j ∈ [ n ] : r ∈ T P j ( h, ϵ ) } ∣ ∣ .

In words, ψ P ( r ) counts the number of nodes j ∈ [ n ] such that r lands in the neighborhood of j in our algorithm. While we know that ∣ ∣ T P j ( h ) ∣ ∣ ≤ hn Q always, simply applying the pigeonhole principle gives too weak of a bound on membership counts. The base case is that there may be a 'hub' node r lands in T P j ( h ) for all j . We will show that there can be no such hub node.

Supposing that we can control of the empirical count ψ A P , we show that the smoothing error can be bounded.

Proposition A.19. Let h n be the bandwidth. Then

Proof. Rearranging terms, we have

where the last step follows because j, s do not depend on i, r and because S ⊂ [ n ] is chosen uniformly at random. Now, we will control the expectation by passing to a row sum, which is handled by E S, 1 .

Recall that n 2 n Q h n E S, 1 = Ω ( ∑ j ∈ [ n ] ∑ s ∈ T j ∥ Q ( e j -e s ) ∥ 2 2 ) . Hence we conclude that

We therefore must show that max r ∈ S ψ A P ( r ) ≤ O ( hn ) with high probability.

Proposition A.20 (Population Version) . Suppose Assumption 2.2 holds for P with c 1 < c 2 and ∆ n = O (( log n n Q ) 1 2 ∨ α ∧ 1 d ) . Then if h ≤ C √ log n n Q for large enough constant C > 0 , then we have max r ∈ S ψ P ( r ) ≤ O ( hn ) with probability at least 1 -O ( n -8 Q ) .

Proof. Fix r ∈ S . Let C d be as in Proposition A.11.Suppose that ϵ = C d ( C +10) √ log n Q n Q 1 /d and h = C √ log n Q n Q . Now, we will claim that for large enough constant c > 0 , that ψ P ( r ) is at most the size of Ball( x r , cϵ ) ∩ { x 1 , . . . , x n } .

Suppose that c > 0 is a large enough constant. Now suppose that x j is such that ∥ x j -x r ∥ ≥ cϵ . We can lower bound the graph distance using Assumption 2.2, as:

On the other hand, suppose that i ∈ S is such that ∥ x i -x j ∥ ≤ ϵ . Then d P ( i, j ) ≤ C 2 α n 3 ϵ 2( α ∧ 1) by Proposition A.9. Therefore since ϵ = C d ( C +10) √ log n Q n Q 1 /d and ∆ n = O (( log n n Q ) 1 2 ∨ α ∧ 1 d ) , for large enough c 1 > 0 we have

Then, if we choose c > 0 such that c 2( α ∧ 1) > 2 C 2 α c 1 , then d P ( i, j ) < d P ( r, j ) .

Next, from our choices of h, ϵ , by Corollary A.5, simultaneously for all i ∈ [ n ] there are at least hn Q nodes in S that have distance ≤ ϵ in latent space from x i , with probablity ≥ 1 -O ( n -6 Q ) .

Therefore, if x r ̸∈ Ball( x j , cϵ ) ∩ { x 1 , . . . , x n } then r ̸∈ T P j ( h ) . This implies that ψ P ( r ) ≤ |{ Ball( x r , 2 cϵ ) ∩ { x 1 , . . . , x n }| . We can bound the size of this ball with Lemma A.4. Notice the Lebesgue measure of Ball( x r , 2 cϵ ) ∩ [0 , 1] is at most ( 4 cϵ C d ) d . Therefore, since x i are chosen iid from the Lebesgue measure on X , with probability at least ≥ 1 -O ( n -10 Q ) , we have

The right-hand side is bounded by O ( h ) if n ≥ n Q . Taking a union bound over all r ∈ S gives the conclusion.

We conclude with the desired upper bound.

Proposition A.21 (Bound on ψ A P ( r ) ) . Suppose Assumption 2.2 holds for P with c 1 < c 2 and ∆ n = O (( log n n Q ) 1 2 ∨ α ∧ 1 d ) . Then if h ≤ C 0 √ log n Q n Q for small enough constant C 0 , then we have max r ∈ S ψ A P ( r ) ≤ O ( hn ) with probability at least 1 -O ( n -8 Q ) .

Proof. By Proposition A.12, with probability at least 1 -O ( n -8 Q ) , we have for all r ∈ S, j ∈ [ n ] simultaneously that

Similarly, d A P ( r, j ) ≤ (1 + O ( 1 √ n )) d P ( r, j ) . We conclude that ψ A P ( r ) ≤ 2 ψ P ( r ) = O ( hn ) with probability ≥ 1 -O ( n -8 Q ) .

A.2.5 Overall Error

We can bound C d := Γ( d 2 +1) 1 /d with the elementary inequality.

Proof of Theorem 2.3. By Proposition A.21 and Prop A.19, we have that E S, 1 ≤ O ( E S, 1 ) with probability at least 1 -O ( n -8 Q ) . Therefore by Proposition A.17,

By Lemma A.22, C d ≤ √ d/ 2 . Finally, by Corollary A.8, the Bernoulli error is bounded by O ( √ log n Q n Q ) with probability ≥ 1 -O ( n -4 Q ) . Applying a union bound over the two kinds of error and Lemma A.15 gives the result.

A.3 Proof of Theorem 3.2

Recall the Gilbert-Varshamov code (Guruswami et al., 2019).

Theorem A.23 (Gilbert-Varshamov) . Let q ≥ 2 be a prime power. For 0 < ϵ < q -1 q there exists an ϵ -balanced code C ⊂ F n q with rate Ω( ϵ 2 n ) .

We will use the following version of Fano's inequality.

Theorem A.24 (Generalized Fano Method, Yu (1997)) . Let P be a family of probability measures, ( D , d ) a pseudo-metric space, and θ : P → D a map that extracts the parameters of interest. For a distinguished P ∈ P , let X ∼ P be the data and ̂ θ := ̂ θ ( X ) be an estimator for θ ( P ) .

̸

Let r ≥ 2 and P r ⊂ P be a finite hypothesis class of size r . Let α r , β r > 0 be such that for all i = j , and all P i , P j ∈ P r ,

Definition A.25 (Relative Hamming Distance) . For x , y ∈ { 0 , 1 } m , we define their relative Hamming distance as follows:

̸

We will need the following construction of coupled codes.

Proposition A.26. Let m P , m Q ≥ 2 and m Q divide m P . There exists a code C ⊂ { 0 , 1 } m P and a projection map Π : { 0 , 1 } m P →{ 0 , 1 } m Q such that if C ′ = { Π( w ) : w ∈ C } then C ′ is a code with relative Hamming distance Ω(1) . Moreover, | C | = | C ′ | ≥ 2 0 . 1 m Q

Throughout the proof, we will identify the community assignment function z : [ n ] → [ k ] of an SBM (Definition 3.1) with the matrix Z ∈ { 0 , 1 } n × k where Z ij = 1 if and only if z ( i ) = j .

Proof. Begin with a Gilbert-Varshamov code B ⊂ { 0 , 1 } m Q as in Theorem A.23. We can 'lift' B to a code on { 0 , 1 } m P simply by concatenation. If w ∈ B , then the corresponding w ′ ∈ C is just w ′ = ( w,w,... , w ) ∈ { 0 , 1 } m P . Let Π : { 0 , 1 } m P →{ 0 , 1 } m Q simply select the first m Q bits of a word. It is clear that B = { Π( w ) : w ∈ C } , so we are done.

Now we are ready to prove Theorem 3.2.

Then

Proof of Theorem 3.2. Let m P = ( n 2 ) , m Q = ( n Q 2 ) , and m = m P . Let C ⊂ { 0 , 1 } m P be the code and Π : { 0 , 1 } m P →{ 0 , 1 } m Q the projection map of Prop A.26. For each w ∈ C , we construct a pair of SBMs P w , Q w ∈ R n × n as follows.

Each P w , Q w is a stochastic block model with k P , k Q classes respectively. All the P w share the same community structure, namely the lexicographic assignment where nodes 1 , 2 , . . . , n k P are assigned to community 1 , and so on. Similarly all the Q w share the same lexicographic community structure with nodes 1 , 2 , . . . , n k Q assigned to community 1 , and so on. Therefore, there are fixed Z P ∈ { 0 , 1 } n × k P , Z Q ∈ { 0 , 1 } n × k Q , such that for all w ∈ C , there exist A w ∈ R k P × k P , B w ∈ R k Q × k Q with

The A w , B w are defined as follows. Let i, j ∈ [ k P ] and i ′ , j ′ ∈ [ k Q ] be such that i < j and i ′ < j ′ . Since m P = ( k P 2 ) and m Q = ( k Q 2 ) , we can identify ( i, j ) and ( i ′ , j ′ ) with indices of [ m P ] , [ m Q ] respectively. Then for fixed δ P , δ Q > 0 , the edge connectivity probabilities are

We can set the diagonals of A w , B w to be 1 / 2 as well.

Next, let P r be a family of r = | C | probability measures. For fixed w ∈ C , the corresponding measure is the distribution over data ( A P , A Q ) ∈ { 0 , 1 } n × n ×{ 0 , 1 } n Q × n Q sampled from ( P w , Q w [ S, S ]) . Note that we restrict S to be a fixed subset of [ n ] .

̸

Next, let θ (( P w , Q w )) := Q w , and let d ( θ (( P w , Q w )) , θ (( P w ′ , Q w ′ ))) := 1 n ∥ Q w -Q w ′ ∥ F . We will show that for all w,w ′ ∈ C with w = w ′ ,

For the β claim, by Proposition 4.2 of Gao et al. (2015), if δ P , δ Q ∈ (0 , 1 / 4) , we have

̸

Next, notice that A w ( i, j ) = A w ′ ( i, j ) if and only if w ij = w ′ ij . Then for distinct w,w ′ ∈ C , we have d H ( w,w ′ ) = Ω( m P ) , so

̸

The bound for Q w is similar, so this verifies the β claim.

Similarly, for the α claim, notice that

By Prop A.26, d H (Π( w ) , Π( w ′ )) = Ω( m Q ) = Ω( k 2 Q ) . Therefore α ≤ Ω( δ Q ) .

Next, by Prop A.27, the pair ( P w , Q w ) satisfies Definition 1.3 for all w ∈ C . Moreover, log | C | ≥ 0 . 1 m Q by Prop A.26.

Combining these results, by Theorem A.24 the overall lower bound is

If we choose δ P = 0 . 01( k Q n ) and δ Q = 0 . 01 k Q n Q , then

Note that k Q ≤ n Q ≤ n , so δ P , δ Q ∈ (0 , 1 / 4) as desired.

Proposition A.27. If h n = min { 1 k P , 1 k Q } then for all w ∈ C , the pair ( P w , Q w ) satisfies Defn 1.3 at h n .

̸

Proof. Consider h = h n and some node i ∈ [ n ] . Suppose that j = i is in the same P w -community as i , and that ℓ = i is in a different community. Then notice that d P w ( i, ℓ ) ≥ d P w ( i, j ) . Therefore j ∈ T P w i ( h ) . Moreover, since h ≤ 1 k P and since the nodes of S ⊂ [ n ] are equidistributed among the communities 1 , 2 , . . . , k P , it follows that all members of T P w i ( h ) must belong to the same P w -community as i .

̸

Therefore, since the communities of Q w are a coarsening of the communities of P w , j ∈ T Q w i ( 1 k Q ) . Since h ≤ 1 k Q , we are done.

A.4 SBM Clustering Error

In this section, we prove a minimax lower bound in the clustering regime for stochastic block models. Theorem A.28. Let Π denote the parameter space of pairs of SBMs ( P, Q ) on n nodes with k P , k Q communities respectively, such that the cluster structure of Q is a coarsening the cluster structure of P . Then

Proof. Let H m ∈ [0 , 1] m × m be the Hadamard matrix of order m modified to replace all entries -1 with 0 . If m is not a power of two, let H m be defined as follows. Let ℓ = ⌊ log 2 m ⌋ and let H m ′ ∈ R m/ 2 × m/ 2 contain H 2 ℓ -1 on its top left block and zeroes elsewhere. Let

Notice that at most 7 8 fraction of the entries of H m are zero-padded, for any m . Now, let B P = 1 2 11 T + δ P H k P and B Q = 1 2 11 T + δ Q H k Q for some δ P , δ Q ∈ (0 , 1 / 4) to be chosen later.

We will define two families of matrices indexed by a finite set T . For i ∈ T , there are some Z i ∈ { 0 , 1 } n × k P and Y i ∈ { 0 , 1 } n × k Q to be specified later. Then

̸

Now, we define Y i as follows. Let Z n,k Q denote the set of balanced clusterings z : [ n ] → [ k Q ] such that for all i, j ∈ [ k Q ] , ∣ ∣ z -1 ( { i } ) ∣ ∣ = ∣ ∣ z -1 ( { j } ) ∣ ∣ . Let Z ⊂ Z n,k Q select the z such that for all j ≤ k Q / 2 , z -1 ( j ) = { ⌊ n ( j -1) k Q ⌋ , . . . , ⌊ nj k Q ⌋ } . Define a distance on Z as follows. For y, y ′ ∈ Z let Y, Y ′ ∈ { 0 , 1 } n × k Q be the corresponding cluster matrices and let d ( y, y ′ ) := 1 n ∥ Y B Q Y T -Y ′ B Q ( Y ′ ) T ∥ F . By Theorem 2.2 of Gao et al. (2015), there exists a packing T 0 ⊂ Z with respect to d such that for all y, y ′ ∈ T 0 , we have |{ j : y ′ ( j ) = y ( j ) }| ≥ n/ 6 . Moreover, log | T 0 | ≥ 1 12 n log k Q . Set T = T 0 . For any y i ∈ T 0 , let Y i ∈ { 0 , 1 } n × k Q be the corresponding cluster matrix and then Q i = Y i B Q Y T i .

Now, to define Z i , take a ∈ [ k Q ] and partition y -1 i ( { a } ) ⊂ [ n ] into k P /k Q equally sized communities in a uniformly random way. Number these 1 , . . . , k P k Q . In this way, we split community 1 of y i into communities 1 , . . . , k P k Q of z i , and so on. Define Z i to be the matrix corresponding to z i . Notice that Z i , Y i are both balanced clusterings and that the clustering Y i coarsens that of Z i . Therefore ( P i , Q i ) are a pair of heterogeneous symmetric SBMs satisfying Definition 1.3 at h = 1 /k Q .

Next, we apply Fano's Inequality (Theorem A.24). Recall log | T | ≥ 1 12 n log k Q . Now, for i, j ∈ T distinct, Prop 4.2 of Gao et al. (2015) gives

Finally, we can bound:

̸

where c 0 > 1 is some constant. This follows because there are a constant fraction of j > n/ 2 such that y i ( j ) = y ′ i ( j ) , and any two rows of the Hadamard matrix differ on half their entries.

Now, set δ 2 Q = n Q log k Q 10 n 2 Q and δ 2 P = log k Q 10 n 2 . Since n ≥ n Q , we conclude that

A.5 Proof of Proposition 3.4

We first argue that Algorithm 2 perfectly recovers Z P , Z Q with high probability.

Theorem A.29 (Implicit in Chen et al. (2014)) . Let M = ZBZ T be an ( n, n min , s ) -HSBM. Then there exists absolute constant C > 0 such that the Algorithm of Chen et al. (2014) can recover Z , up to permutation, with zero error with probability ≥ 1 -O ( n -8 ) if

Proof. The algorithm of Chen et al. (2014) returns a matrix Y ∈ { 0 , 1 } n × n such that Y ij = 1 if and only if i, j are in the same community, with probability ≥ 1 -O ( n -8 ) . Therefore, to construct a clustering from Y , simply assign the cluster of node 1 to all j ∈ [ n ] such that Y 1 j = 1 , and so on. This returns the true Z ∈ { 0 , 1 } n × k up to permutation with probability ≥ 1 -O ( n -8 ) . Note that k is correctly chosen because Y is equal to a block-diagonal matrix of ones up to permutation, with k blocks.

Theorem A.29 implies the following.

Proposition A.30. Let ̂ Z P , ̂ Z Q be as in Algorithm 2. Let s P , s Q be the signal to noise ratios of P, Q respectively. If s P , s Q satisfy the conditions of Theorem A.29 with respect to ( n, n ( P ) min and ( n Q , n ( Q ) min ) respectively, then then with probability ≥ 1 -O ( n -8 Q ) , there are permutation matrices U P ∈ { 0 , 1 } k P × k P , U Q ∈ { 0 , 1 } k Q × k Q such that ̂ Z P = Z P U P and ̂ Z Q = Z Q U Q .

Next, we want to recover the clustering of Q on all n nodes, not just the n Q nodes that we observe in A Q . This is given by the following.

Proposition A.31. 1. If h n = 1 /k P and k Q ≤ k P then there exists a unique Π ∈ { 0 , 1 } k P × k Q such that Z P Π contains the Q -clustering of all nodes in [ n ] . Let ˜ Z Q := Z P Π .

2. Let ̂ Π be as in Algorithm 2 and U P , U Q be as in Proposition A.30. Then with probability 1 -O ( 1 n Q ) , Z P U P ̂ Π = ˜ Z Q U Q .

Proof. Part (1) follows immediately from the SBM structure of P, Q and definition of Definition 1.3. For Part (2), first notice that by Proposition A.30, with probability at least 1 -O ( 1 n Q ) , Algorithm 2 returns the true clusterings ̂ Z P = Z P ∈ { 0 , 1 } n × k P and ̂ Z Q = Z Q ∈ { 0 , 1 } n Q × k Q , up to permutation.

Now, Algorithm 2 simply takes unions of the clusters of Z P to learn ̂ Π . Therefore, let V : R n → R n Q project onto coordinates in S . Then V ̂ Z P ̂ Π = ̂ Z Q . Moreover, by Proposition A.30, ̂ Z P = Z P U P and ̂ Z Q = Z Q U Q . Hence V Z P U P ̂ Π = V ˜ Z Q U Q . To remove dependence on V , we need to argue that each Q -cluster has a reprensentatve in S .

Let E be the event that at least one Q -cluster has no representative in S . For a fixed j ∈ [ k Q ] , cluster j has no representative in S with probability ≤ ( 1 -n ( Q ) min n Q ) n Q . A union bound implies that

The last inequality holds because the condition of Theorem A.29 implies that n ( Q ) min ≥ Ω( √ n Q ) and k Q ≤ n Q n ( Q ) min .

Finally, we proceed by conditioning on ¬ E . Since ̂ Z P = Z P U P , we know that for all i ∈ S , the unique j P ∈ [ k P ] such that row i , column j P of Z P is nonzero contains its true P -community up to U P . Similarly since ̂ Z Q = Z Q U Q , the the unique j Q ∈ [ k Q ] such that row i , column j Q of Z P is nonzero contains its true Q -community up to U Q . Therefore the nodes in community j P in P are in community j Q in Q . So, up to permutations U P and U Q , we have Π j P ,j Q = 1 . Since we condition on ¬ E , each cluster of Q has at least one representative in S , so each columns of Π is nonzero. We conclude that Z P U P ̂ Π = ˜ Z Q U Q with probability at least 1 -O ( n -1 Q ) .

We are ready to give the overall error of Proposition 2.

Proposition A.32. Suppose that ̂ Z P = Z P , ̂ Π = Π in Algorithm 2. Then with probability ≥ 1 -O ( 1 n Q ) , Algorithm 2 returns a ̂ Q ∈ [0 , 1] n × n such that

Proof. By Proposition A.31, with probability ≥ 1 -O ( 1 n Q ) , we have ̂ Z P = Z P U P , ̂ Z Q = Z Q U Q , and ˜ Z Q U Q = Z P U P ̂ Π . We proceed by conditioning on these events.

Next, let W Q ∈ R k Q × k Q be the population version of ̂ W Q with W Q ; ii = ( 1 T Z Q e i ) -1 . Then since ̂ Z Q = Z Q U Q we have ̂ W Q = U T Q W Q U Q . Hence

Next, let z Q : [ n ] → [ k Q ] be the ground truth clustering map given by ˜ Z Q ∈ { 0 , 1 } n × k Q . Let B Q be defined analogously to ̂ B Q in Algorithm 2, but using W Q , Z Q , E [ A Q ] in place of ̂ W Q , ̂ Z Q , A Q . Let m i := W -1 Q ; ii be the the number of nodes in S belong to community i , and let n i be the the number of nodes in [ n ] belonging to community i of Q . Then the error of Algorithm 2 is then

Next, fix i, j ∈ [ k Q ] and let

If we condition on the clusterings of P, Q being correct then E [ B Q ; ij -A Q ; rs ] = 0 . Therefore by Hoeffding's inequality,

Setting t 2 = 10log( m i m j ) m i m j implies that with probability at least 1 -k 2 Q min i ( m i ) -20 , that the overall error is

Finally, note that there exists a constant c 0 > 0 such that for all i ∈ [ k Q ] , m i ≥ c 0 √ n Q and n i ≥ c 0 √ n , by assumption. Note that each m i is a random quantity depending on the choice of S ⊂ [ n ] such that E [ m i ] = n Q n n i . Hoeffding's inequality and a union bound over all i ∈ [ k Q ] imply that that with probability at least ≥ 1 -O ( n -8 Q ) that m i ≥ E [ m i ] -10 √ log n Q ≥ Ω( E [ m i ]) . We conclude that

B Additional Experiments

B.1 Ablation Experiments

In this section, we discuss additional experiments that quantify the dependence of our algorithms on all relevant parameters. Our experiments also include a new baseline adapted from the estimator of Levin et al. (2022).

Description of New Baseline. Levin et al. (2022) assumes that full edge data from both P and Q are observed, and P = Q . Since this is not true for us, we instead compute the following modified MLE based on their estimator from Section 3.3 of Levin et al. (2022).

Here w P , w Q are computed as in their paper, based on estimated sub-gamma parameters of the noise for A P , A Q . Akin to their adjacency spectral embedding, which assumes known rank of Q , we use Universal Singular Value Thresholding to obtain ̂ Q from ˜ Q Chatterjee (2015).

Oracle with p = 0 . 0 . In addition to testing the new baseline from Levin et al. (2022), we also test the Oracle baseline with p = 0 . 0 . As noted in Section 4, this corresponds to the non-transfer setting where all edges from the target graph Q are observed. Note that in this case, the value of n Q does not matter because edges incident to nodes outside of S never get flipped. The Oracle error for β -smooth graphons on d -dimensional latent variables will therefore be O ( n -2 β 2 β + d ) Xu (2018), which is less than the error bound of Theorem 2.3. Indeed, we will find that the Oracle our transfer algorithms in the regimes where its theoretical upper bound is better than our theoretical upper bounds.

Next, we describe the experimental results.

Figure 3 tests Algorithm 1 for general latent variable models. The error (Theorem 2.3) depends on the smoothness β of the target graph, the number of observed target nodes n Q , and the dimension of the latent variables d .

Figure 4 tests Algorithm 2 for Stochastic Block Models. The error (Proposition 3.4) depends on the number of communities k Q in the target graph, and the number of observed target nodes n Q . Note that Proposition 3.4 also depends logarithmically on the minimum community size of Q , but this is less significant.

Algorithm 1

Oracle (p=0.0)

Alg. 1 Error Bound (Thrm 2.2)

OracleErrorBound

Graphons,VaryingSmoothnessβ

Graphons,Varyingno

Graphons,VaryingDimensiond

100

10-1

MSE

10-2

10-3

0.2

0.4

0.6

0.8

0

50

100

150

200

1

2

3

5

9

β

no

d

β → 1 . Here f P ( x, y ) = x + y 2 , f Q ( x, y ) = x + y 2 with α = 0 . 01 and β varying. Middle : Testing number of observed target nodes n Q with n = 200 , d = 1 . Here f P ( x, y ) = x α + y α 2 , f Q ( x, y ) = x β + y β 2 with α = 0 . 01 , β = 0 . 1 . Note that the oracle does not depend on n Q because it observes the full adjacency matrix A Q ∈ { 0 , 1 } n × n .

Q Q α α β β

Right : Testing dimension d of latent positions x 1 , . . . , x n ∈ [0 , 1] d (i.i.d. Lebesgue) with n = 200 , n Q = 25 . Here f P ( x , y ) = exp( -6 ∥ x -y ∥ 2 ) and f Q ( x , y ) = exp( -| x 1 -y 1 | ) . Points are the median MSE across 50 trials, with with [5 , 95] percentile outcomes shaded.

Algorithm 2

Oracle (p=0.0)

Alg. 2 Error Bound (Prop 3.4)

Oracle Error Bound

StochasticBlockModels,n=200,Varyingko

10-1

MSE

10-2

10

15

20

25

30

35

40

4

5

6

7

8

6

10

11

12

no

KQ

Left : n = 200 , k P = 12 , k Q = 6 . Note that the oracle does not depend on n Q because it observes the full adjacency matrix A Q ∈ { 0 , 1 } n × n .

Right : n = 200 , n Q = 25 , k P = 2 k Q .

For both experiments, the intra-community edge probabilities are 0 . 2 , 0 . 9 for P, Q respectively, while the inter-community edge probabilities are 0 . 1 , 0 . 8 respectively. Points are the median MSE across 50 trials, with with [5 , 95] percentile outcomes shaded.

Note that while we can plot theoretical guarantees for the mean squared error 1 n 2 || ̂ Q -Q || 2 F of both our algorithms' ̂ Q and the oracle's ̂ Q , Levin et al. (2022) only give theoretical guarantees on the spectral norm || ̂ Q -Q || 2 for their estimator ̂ Q . Analyzing the stronger metric of mean-squared error would require different techniques than their paper.

B.2 Link Prediction Experiments

In this section, we present additional link prediction experiments on the EMAIL-EU and BIGG MODELS datasets. Unlike Section 4, we tune the sparsity estimate ̂ ρ ∈ (0 , 1) used in the Universal Singular Value Thresholding step of the Oracle baseline. In particular, we set ̂ ρ ∈ (0 , 1) to be the mean of the entries of the ground truth target matrix Q ∈ [0 , 1] n × n . Note that this value is inaccessible to other algorithms since it requires knowing all the edges of Q .

Figures 7 and 8 show the performance of our Algorithms on the EMAIL-EU dataset, and Figures 5 and 6 for the BIGG MODELS dataset. As in the mean-squared error setting (Figure 2), we find that Algorithm 1 outperforms Algorithm 2, and that the Oracle baseline outperforms both for small p . Moreover, we find that the choice of source & target affects the performance of both of our algorithms. Hence Figure 7 shows better performance than Figure 8 for the same source but different targets, and Figure 5 shows better performance than Figure 6 for the same target but different sources.

C Experimental Details

In this section, we give further details on the experiments of Section 4.

Compute Environment. We run all experiments on a personal Linux machine with 378GB of CPU/RAM. The total compute time across all results in the paper was less than 2 hours.

Functions for Figure 1. For the top row, the source is an ( n, 4) -SBM with 0 . 8 on the diagonal and 0 . 2 on the off-diagonal of B ∈ R 4 × 4 . The target is an ( n, 2) -SBM with 0 . 9 on the diagonal and 0 . 1 on the off-diagonal of B ∈ R 2 × 2 .

25

50

75

100

125

150

175

200

nQ

0.5

0.6

0.7

0.8

0.9

AUROC

Metabolic Network Link Prediction (Source = iWFL1372)

Algorithm 1

Algorithm 2

Oracle (p=0.0)

25

50

75

100

125

150

175

200

nQ

0.5

0.6

0.7

0.8

0.9

AUROC

Metabolic Network Link Prediction (Source = iPC815)

Algorithm 1

Algorithm 2

Oracle (p=0.0)

100

150

200

250

300

350

400

nQ

0.5

0.6

0.7

0.8

0.9

AUROC

Algorithm 1

Algorithm 2

Oracle (p=0.0)

For the second and third rows, the source function is Q ( x, y ) = 1+sin( π (1+3( x + y -1))) 2 (modified from Zhang et al. (2017)). The sources are P ( x, y ) = 1 -Q ( x, y ) and P ( x, y ) = Q ( ϕ ( x ) , y ) , where ϕ ( x ) = 0 . 5 + | x -0 . 5 | if x < 0 . 5 , and 0 . 5 -| x -0 . 5 | otherwise.

Metabolic Networks. We access metabolic models from King et al. (2016) at http://bigg.ucsd. edu . To construct a reasonable set of shared metabolites across the networks, we take the intersection of the node sets for the following BiGG models: iCHOv1, IJN1463, iMM1415, iPC815, iRC1080,

100

150

200

250

300

350

400

nQ

0.5

0.6

0.7

0.8

0.9

AUROC

Algorithm 1

Algorithm 2

Oracle (p=0.0)

iSDY1059, iSFxv1172, iYL1228, iYS1720, and Recon3D. We obtain a set of n = 251 metabolites that are present in all of the listed models.

The resulting networks are undirected, unweighted graphs on 251 nodes. We construct the matrix A P ∈ { 0 , 1 } n × n for species P by setting A P ; uv = 1 if and only if u and v co-occur in a metabolic reaction in the BiGG model for P .

EMAIL-EU. We use the 'email-EU-core-temporal' dataset at https://snap.stanford.edu/ data/email-Eu-core-temporal.html , as introduced in Paranjape et al. (2017). Note that we do not perform any node preprocessing, so we use all n = 1005 nodes present in the data, as opposed to Leskovec and Krevl (2014); Paranjape et al. (2017) who use only 986 nodes.

Data consist of triples ( u, v, t ) where u, v are anonymized individuals and t > 0 is a timestamp. We split the data into 10 bins based on equally spaced timestamp percentiles. For simplicity we refer to these time periods as consisting of 80 days each in Section 4, but technically there are 803 days total. The network at time period ℓ consists of an unweighted undirected graph with adjacency matrix entry A uv = 1 if and only if ( u, v, t ) or ( v, t, u ) occurred in the data for an appropriate timestamp t .

Hyperparameters. We do not tune any hyperparameters. For Algorithm 1 we use the quantile cutoff of h n = √ log n Q n Q in all experiments.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We discuss our main contributions in the abstract and introduction, and then develop them in the main body of the paper. Specifically, we point out Algorithm 1 and Algorithm 2, the upper bounds from Theorem 2.3 and Proposition 3.4, and the lower bound Theorem 3.2. Finally, Section 4 shows experimental results.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitation of lacking a minimax lower bound for network estimation in d -dimensions after Theorem 2.3, the need for a different graph distance for graphs with sparsity O ( log n n ) in the Conclusion, and adapting to multiple source distributions in the Conclusion.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We give self-contained theoretical statements with definitions and assumptions in the main body, and full proofs for all claims in Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We give experimental details in Section 4, and give further details on our exact data preparation methods in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We submit our code as a supplementary zip file in accordance with the NeurIPS code and data submission guidelines.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe our choices of source and target network, the network sizes and degrees, and the single hyperparameter (quantile cutoff) for Algorithm 1 in Section 4.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For each experiment we run 50 independent trials. We report ± 2 standard deviations for the mean-squared error in Table 1, and [1 , 99] percentile errors in Figure 2.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: As described in Appendix C, we run all experiments on a personal Linux machine with 378GB of CPU/RAM. The total compute time across all results in the paper was less than 2 hours.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We conform with the NeurIPS Code of Ethics with regards to potential harms caused by the research process, societal impact, and impact mitigation measures. Moreover, we have anonymized our code and manuscript.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper presents work whose goal is to advance the field of Machine Learning through foundational research. There are many possible societal consequences of our work, none which we feel must be specifically highlighted here.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We do not release data or models that have a high risk for misuse. We use only publicly available data from the academic literature.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We credit the authors of the BiGG metabolic models dataset (King et al., 2016; Norsigian et al., 2020) and the EMAIL-EU dataset (Leskovec and Krevl, 2014; Paranjape et al., 2017). Both datasets are publicly released for academic research.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release new assets. See Appendix C for the exact details regarding our use of existing datasets and assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We do not perform any crowdsourcing or research on human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: We do not perform any research on living subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of transfer learning for estimating the edge probabilities of random graphs under latent position models. In particular, given a fully observed graph on $n$ nodes generated from an independent-edge random graph model, with edge probabilty matrix $P$, and an $n_Q \\times n_Q$ submatrix of a graph generated from an independent-edge random graph with edge probability matrix $Q$, the authors consider the task of estimating $Q$, by leveraging information from both graphs. In order for transfer learning to be possible, the underlying graph models must share some structure. The authors assume that both $P$ and $Q$ are given by latent positions models, i.e.\n\n$$\nP_{ij} = f_P(x_i,x_j),\\qquad Q_{ij} = f_Q(x_i, x_j)\n$$\n\nwhere  $x_1,\\ldots,x_n \\in \\mathcal{X} \\subset \\R^d$ are a common set of latent positions, drawn i.i.d. from a uniform distribution on $\\mathcal{X}$, and that $f_P, f_Q$ are $\\alpha$​-Hölder-smooth symmetric functions. The authors quantify the similarity between two nodes in a graph model using an interpretable distance based on their expected number of common neighbours (Definition 1.2), and measure the distance between two graph models on the same node set by comparing the ranks of these distances for each node in the graphs.\n\nIn Section 2, a simple algorithm (Algorithm 1) is proposed which estimates $Q_{ij}$ by averaging entries of $A_Q$ which are similar in terms of graph distance to $(i,j)$ in $A_P$, and under some reasonable conditions, a high-probability tail bound from the Frobenius norm difference between $\\hat Q$ and $Q$ is dervived (Theorem 2.2).\n\nIn Section 3, information theoretic lower bounds for the problem are derived under the special case that $A_P$ and $A_Q$ are stochastic block model (SBM) graphs (Theorem 3.2), which are proved using Fano's lemma. An alternative algorithm (Algorithm 2) for estimating $Q$ under the SBM modelling assumption is proposed, based on the same ideas as Algorithm 1, and a high-probability tail bound (Proposition 3.4), analagous to Theorem 2.2, is derived which matches the information-theoretic lower bound up to a log factor.\n\nIn Section 4, Algorithms 1 and 2 are compared with an oracle method on a selection of simulated and real-world datasets.\n\nComplete proofs of all theoretical results are provided in the appendix.\n\nSTRENGTHS:\n- This paper was a joy to read. The problem of transfer-learning on graphs is well-motivated with real-world applications and the relevant literature is reviewed extensively. The reader is guided through each aspect of the problem; every definition feels motivated and mathematical statements are followed by intuition and illustrative examples. The paper is well organised, the writing is exemplary, and proofs are complete, accurate and clear.\n\n- This paper addresses a problem which is of great relevance to the machine-learning community, and yet provides what appears to be the first provably consistent algorithm for this problem. The proposed solution, is simple and elegant, and the derived high-probability tail bound and its proof provide a lens into the fundamentally important aspects of this problem. The lower bounds derived in Section 3 show the, despite the simplicity of the method, it is optimal (in the minimax sense) under the SBM, albeit under fairly strong signal-to-noise assumptions (see weaknesses).\n\n- The simulated and real data experiments in Section 4 are informative and demonstrate the effectiveness of the approach.\n\nWEAKNESSES:\n- One weakness of this paper, is that their setup only considers graphs whose expected node degrees grow linearly in the number of nodes in the graph. In practice, many networks are sparse and it is common in the literature to consider asymptotic regimes in which expected node degrees grow sub-linearly. However, there is sufficient novelty in this paper that I don't see that this can be held against it. The authors claim in their conclusion that they believe their algorithm will work for when node degree are of the order $\\Theta(n^{1/2})$, but not down to $\\Theta(\\log(n))$ (see Questions), but it is not clear *why* they believe this. \n- It is not clear to me what is being shown in Figure 1. I was left quite confused. Some additional explanation here would be helpful.\n\nQUESTIONS:\nMy only questions to the authors relate to addressing to points raised in the Weaknesses section. I should mention that the first point is a curiosity and I believe it is only the second point that needs addressing before this work is ready for publication.\n\n**Typos:**\n- I believe there is a typo in the conclusion, and that the big-Os relating to edge densities should be big-$\\Omega$s. Is this correct?\n- The first letter on line 111 should be capitalised.\n\nLIMITATIONS:\nThe authors are upfront the main limitations of their work, although I think it would be helpful to mention the first weakness I mention explicitly.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-01"}

comments: SUMMARY:
In this paper, the authors address two topics  in random network/graph models:
1. The transfer learning in latent variable network models. It proposed estimate of the distribution of target network from source network using a defined graph distance.
2. It also proves a minimax lower bound for Stochastic Block Models and show that a simple algorithm achieves this rate.

STRENGTHS:
This is a theoretical paper that addresses the transfer learning from source to target network in latent variable network models.

WEAKNESSES:
1. Theorem 1.1 is an informal form of Theorem 2.2, it seems there is no need to repeat.
2. The authors repeatedly use the vague word "suitable", "suitably" in the paper. Many phrases (containing this word) need to be clarified.

QUESTIONS:
In Page 3, line 105: the authors mentioned: relative, not absolute graph distances,  it seems to me this is not defined.

LIMITATIONS:
In the authors' reply to the question set, in line 950, the authors mentioned that they discussed  "the need for a different graph distance ..."
. It will benefit the readers if the authors explain further in this paper on the selection of of the distance, an explain why they prefer this graph distance that they are currently using.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-09

comments: SUMMARY:
The work explores transfer learning in latent variable network models. In particular the work focuses on the setting of observing samples from an n x n probability matrix from a source P and a submatrix of the adjacency of a target Q. The goal is then to estimate Q, using information from P.  

The authors propose an algorithm with vanishing error under certain conditions and a more simple algorithm for the special case of stochastic block models.

STRENGTHS:
The paper is well-written. I particularly enjoyed the introductory sections that help motivate the work. I believe that the problem the authors tackle has value, as it stands mostly as a theoretical contribution. 

I am unfortunately unable to judge the strength of the contribution of this work as the submission is far from my area of research.

WEAKNESSES:
The experimental part of the paper focuses on purely synthetic tasks. While I understand that, as the authors mention, they believe there aren't direct baselines or datasets for this, I still see this as a potential weakness.

A way to greatly strengthen the work would be to have experiments on domains mentioned in the introduction (i.e. metabolic networks).

QUESTIONS:
Would the authors be able to explain why they think there are no direct baselines? I am not familiar with the surrounding literature, but the problem of estimating a full matrix from a sub-matrix should be a well-studied area. It might be useful to include baselines that do this, even if they are not operating in the transfer learning regime. 

Would the authors be able to provide experiments on a real-world task -- or if not provide a justification?

LIMITATIONS:
None that I am aware.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates transfer learning in the context of estimating latent variable network models.  Specifically, the goal is to estimate the edge probability matrix $Q$ of the target graph using (1) edge data from a source graph $P$ given by its adjacency matrix, and (2) edge data from a vanishingly small subgraph of $Q$ that consists of an $o(1)$ fraction of the nodes in $Q$. The authors propose a transfer learning algorithm (Algorithm 1), which is based on matching quantiles, and demonstrate that it is possible to accurately estimate $Q$ if $P$ and $Q have similar quantile/ranking profiles (per Definition 1.3) and smooth latent variable representations (Assumption 2.1).  Furthermore, in Section 3, the authors focus on Stochastic Block Models, stating the minimax rate under the setup and providing an estimation algorithm (Algorithm 2) that achieves this rate (Proposition 3.4).  The proposed algorithms are then tested with numerical experiments on synthetic and real-world datasets.\n\nSTRENGTHS:\nThis work addresses the significant problem of learning graphs/networks from a small subset of edge data by leveraging knowledge from a source graph. This problem is fundamental and has numerous potential practical applications, such as in biological network estimation and social sciences. The paper considers a simple yet expressive mathematical model and presents key findings and supporting arguments clearly. Although primarily theoretical, the results are complemented by numerical experiments that substantiate the potential of the proposed approach.\n\nWEAKNESSES:\nWhile this manuscript makes substantial contributions, there are some areas for improvement to enhance its impact:\n\n**1. Model Assumptions:** The models and assumptions in this work —- latent variable models, H\\”older smoothness of the latent functions, and the ranking assumptions between source $P$ and target $Q$ —- are quite standard in theoretical literature. Nevertheless, including in-depth discussions and ablation studies to examine their relevance and applicability in real-world scenarios would help motivate and convince practically oriented readers.\n\n**2. Further Numerical Experiments:** Additional numerical experiments to verify and examine the theoretical results (Theorem 2.2 and Proposition 3.4) would be valuable. This would help validate whether the expected dependence on parameters such as $d, \\beta, n_Q$ are sharp or not. Additionally, comparing the performance of the proposed transfer learning algorithms against a true oracle with direct access to the full edge data from $Q$ (which would correspond to what the authors call “oracle” with $p_{flip} = 0$) would properly quantify the cost of transfer and evaluate the effectiveness of the proposed algorithms.  It would also be beneficial to compare the proposed algorithms to other existing algorithms in the literature to highlight the claimed advantages of the proposals.\n\nQUESTIONS:\n1. Based on the description of the Oracle algorithm in lines 246-252, it seems that its performance should be independent of $n_Q$. However, in Figure 2, the Oracle algorithm's performance appears to improve as $n_Q$ ​increases in the first two plots. Could the authors clarify if there is something I am missing?\n\n2. For the Email networks, Algorithm 1 seems to outperform the Oracle (with $p=0.01$ and $p=0.05$). Can the authors provide any insights on how a transfer learning algorithm (i.e., Algorithm 1) can outperform an Oracle algorithm that has full access to the edge data from $Q$? \n\n*Minor suggestions/typos:*\n- Line 71:  The order of $|S|$ and $n_Q$ should be swapped, i.e., $n_Q := |S| = o(n)$.\n- Lines 102 - 104: The quantifier for $i$ seems missing, e.g., \"... *for $i$,* and for all $j \\neq i$ ...\"\n- Line 111: Something should be off here.\n- Lines 183 - 184: In the second sentence of Theorem 2.2, it is unclear which parameter corresponds to which function.  I would suggest the authors write for example \"Let $f_P$ be $\\alpha$-H\\\"older-smooth and $f_Q$ be $\\beta$-H\\\"older-smooth for $\\beta \\geq \\alpha > 0$, ...\"\n- Lines 238 - 239: It seems a line break is needed before \"(1) Algorithm 2.\"\n- Lines 244 - 245: This line break should be removed.\n- Section 4: Please use $p_{flip}$ instead of $p$ for consistency (or vice versa).\n\nLIMITATIONS:\nThis is primarily a theoretical work, and the authors discussed the potential limitations of the work and potential future research directions.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-14"}