Iterative Connecting Probability Estimation for Networks

1 Introduction . Network analysis has been a promising research area in data science and widely applied in many fields , including medicine , social sciences . Objects such as genes , proteins , or people are presented as vertices in the network , and relationships between objects ( e.g. , protein interactions or friendships ) are presented as connections between pairs of vertices . In this paper , we focus on estimating the connecting probability between each pair of vertices based on the observed network . In the literature , the most fundamental assumption is the Aldous-Hoover theorem ( Aldous , 1981 ) . Assume that the network is vertex-exchangeable , then each vertex i is associated with a latent variable ξi , and the connecting probability between any two vertices depends on a graphon function f ( · ) of the corresponding latent variables . To estimate the connecting probabilities , some researchers focus on estimating f ( · ) under strong assumptions . Chan and Airoldi ( 2014 ) propose the sorting-and-smoothing ( SAS ) algorithm . It first sorts the adjacency matrix using the empirical degrees and smoothes the sorted matrix via total variation minimization . It requires strict monotonicity of the degrees , which is difficult to be satisfied in practice . Since the purpose of estimating f ( · ) is to estimate the connecting probabilities , another direction is to estimate the latter directly , such as stochastic block models approximation ( SBA ) ( Airoldi et al. , 2013 ; Holland et al. , 1983 ; Wang and Wong , 1987 ) . Under the SBM , vertices can be divided into several blocks , and the connecting probability between two vertices depends only on the blocks they belong to . Once the vertices are clustered correctly , the connecting probabilities can be estimated by a moment estimator . In recent years , methods based on SBM approximation and its variation , network histograms , have been widely studied ( Choi and Wolfe , 2014 ; Olhede ∗Correspondence Author , Center for Applied Statistics and School of Statistics , Renmin University of China 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and Wolfe , 2014 ) . Gao and Ma ( 2021 ) establish the framework of minimax error rate analysis for these methods . Since the adjacency matrix can be viewed as a noisy version of the connecting probability matrix , general matrix denoising methods can be applied too . Chatterjee ( 2015 ) provides a universal singular value thresholding method ( USVT ) , and Xu ( 2018 ) analyzes its error rate . Zhang et al . ( 2017 ) propose an efficient neighborhood smoothing method ( NS ) . They select those with similar connecting patterns as the neighbors for each vertex , then the connecting probabilities can be estimated by neighborhood averaging . However , the estimate is not reused to improve neighborhood selection , which may lead to information loss . Since the main purpose of estimating the graphon function is to obtain the connecting probabilities , we focus on estimating the connecting probability matrix and develop an iterative connecting probability estimation method ( ICE ) in this paper . We define a new vertex distance calculated directly on the probability matrix . In contrast to one-step neighborhood smoothing method , our method is iterative . In each iteration , with an estimate , it can update the pairwise vertex distances and the sets of similar vertices . Then , with better selected similar vertices , it can improve the performance on estimation , making it easier to discover complicated structure . The initial input of the iterations can be an estimate obtained by the existing methods as we introduce above . Even if it begins with a random neighborhood averaging estimate , it can converge quickly and perform well in many cases . We also propose a two-stage strategy for neighborhood selection , which uses different sizes of the similar vertex sets during and at the end of the iterations , aiming to cope with the trade-off between the smoothness of the estimate and the ability to discover local structure . The tuning parameters can be selected by network cross-validation . According to the results on simulated and real networks , our method is comparable to state-of-the-art methods in simple cases and outperforms them in the presence of complicated structure . 2 Framework . Suppose G = ( V , E ) is the observed undirected network where V = [ n ] denotes the vertex set and E denotes the edge set and E ⊆ { ( i , j ) : i , j ∈ V , i ̸= j } . The network can be represented by an adjacency matrix A = [ Aij ] n×n ∈ { 0 , 1 } n×n , where Aij = 1 if vertices i and j are connected and 0 otherwise . We only consider undirected networks without self-loops , so A is symmetric with Aii = 0 for i ∈ V . Assume that the network is vertex-exchangeable , according to the AldousHoover theorem ( Aldous , 1981 ) , we have the following representation : for each vertex i , there exists a latent random variable ξi i.i.d.∼ Uniform ( 0 , 1 ) , then for each vertex pair ( i , j ) , we have Aij ∼ Bernoulli ( Pij ) , where Pij = f ( ξi , ξj ) and f ( · ) is called the graphon function . Let P = [ Pij ] n×n ∈ [ 0 , 1 ] n×n denote the probability matrix . Let || · ||2 and || · ||F denote the ℓ2 and Frobenius norms respectively . Given the observed adjacency matrix A , our goal is to estimate the connecting probability matrix P. Since for each pair ( i , j ) , Aij is the only realization of Pij , it is hard to get a sample-based estimation like maximum likelihood estimation . However , if there exists another vertex i′ satisfying f ( ξi′ , · ) ≈ f ( ξi , · ) , then Ai′j can be roughly viewed as another realization of Pij for the reason that Pij = f ( ξi , ξj ) ≈ f ( ξi′ , ξj ) = Pi′j . Assume that there is such a group of vertices who share similar connecting patterns with vertex i ; we refer to them in the rest of this paper simply as its “ similar vertices ” . Let S∗i denote the set of similar vertices for vertex i , that is , S ∗ i = { i′ : Pi· ≈ Pi′· } . Then we could estimate Pij by simply averaging the entries Ai′j′ over i′ ∈ S∗i and j′ ∈ S∗j , that is , P̃ij = P̃ji = ( s ∗ ) −2 ∑ i′∈S∗i ∑ j′∈S∗j Ai′j′ , ( 1 ) where s∗ = |S∗i | is the number of similar vertices . Ideally , with known P , it is easy to define the set of similar vertices for vertex i as S∗i = { i′ : 0 < d∗ii′ ≤ ds∗ } , where d∗ii′ = ||Pi·−Pi′·||22/n denotes the distance between vertex i and vertex i′ and ds∗ is the s∗-th smallest elements of { d∗ii′ : i′ ̸= i } . 3 Methodology . 3.1 Iterative Connecting Probability Estimation . The neighborhood averaging method in Section 2 is only conceptually feasible . In practice , without knowledge of P , it is difficult to obtain the pairwise vertex distances and the sets of similar ver- tices . We propose an iterative connecting probability estimation method ( ICE ) to solve the problem . Assume that we have an initial estimate P̂ , the estimated vertex distance dii′ can be formulated as dii′ = ||P̂i· − P̂i′·||22/n . Then we can estimate S∗i with Si = { i′ : 0 < dii′ ≤ ds } , where s = |Si| is the estimated size of the similar vertex set and ds is the s-th smallest elements of { dii′ : i′ ̸= i } . Then a new estimate can be obtained by neighborhood averaging with P̂ij = P̂ji = s −2 ∑ i′∈Si ∑ j′∈Sj Ai′j′ . ( 2 ) Figure 1 : The relationship among D , S and P̂ . Using such an estimate , we can update the pairwise vertex distances and the sets of similar vertices again , and obtain a new P̂ . We repeat the process above until P̂ converges . Let D = [ dij ] n×n and S = { S1 , . . . , Sn } denote the pairwise vertex distances and the sets of similar vertices respectively . The iterative procedure is inspired by the interdependence among D , S and P̂ , as is shown in Figure 1 . We outline the details of our method in Algorithm 1 . Algorithm 1 Iterative connecting probability estimation method Input : observed adjacency matrix A ; initial connecting probability estimate P̂ ( 0 ) ; neighborhood size s ; threshold δ0 > 0 . Output : connecting probability estimate P̂ . 1 : Let δP = +∞ and m = 0 . 2 : while δP > δ0 do 3 : For each vertex pair i , i′ ∈ V , obtain their distance dii′ = ||P̂ ( m ) i· − P̂ ( m ) i′· ||22/n . 4 : For each vertex i ∈ V , obtain its set of similar vertices Si = { i′ : 0 < dii′ ≤ ds } . 5 : For each vertex pair i , j ∈ V , update their connecting probability estimate P̂ ( m+1 ) ij = s−2 ∑ i′∈Si ∑ j′∈Sj Ai′j′ . 6 : Let δP = ||P̂ ( m+1 ) − P̂ ( m ) ||F /||P̂ ( m ) ||F . 7 : Let m = m+ 1 . 8 : end while 9 : return P̂ = P̂ ( m ) . In Algorithm 1 , an initial estimate P̂ ( 0 ) is required . Any P̂ via existing methods can be used as P̂ ( 0 ) . Alternatively , a random initial value would also work as discussed in Section 3.2 , since our method is not sensitive to the initialization . For each vertex i , we get Si by randomly sampling s vertices from V \ { i } without replacement , then we can obtain such a random P̂ ( 0 ) by neighborhood averaging with these random selected vertices . As to the size of the similar vertex set s , Zhang et al . ( 2017 ) set s = C ( n log n ) 1/2 for each vertex i , where C is recommended set as 1 . In Section 3.3 , we develop a two-stage procedure using different levels of C in different stages , which improves the performance of our method significantly . It should be noted that although we discuss our estimate as defined in ( 2 ) for convenience , in practice , in case that Si and Sj may overlap , we modify it as P̂ij = P̂ji = ( s 2 − |Si ∩ Sj | ) −1 ∑ i′∈Si , j′∈Sj , i′ ̸=j′ Ai′j′ . 3.2 Iterative Estimation versus One-step Estimation . A main contribution of our work is to estimate P iteratively rather than output a one-step estimate . As for those one-step methods ( Zhang et al. , 2017 ) , they obtain the set of similar vertices based on the distances calculated from A , then obtain P̂ by neighborhood averaging . However , if we update the set of similar vertices based on P̂ , the previous set of similar vertices may be changed . The contradiction may cause inefficiency on making full use of the observed network . Indeed , if P̂ is reliable , we should update the set of similar vertices to get a better estimate until the sets remain unchanged . We compare the performances of our iterative estimation and one-step estimation with a simulated network which consists of 1000 vertices as an example . As shown in Figure 2 ( a ) , the 1000 × 1000 connecting probability matrix is full rank with complicated structure . We use root mean squared errors ( RMSE ) to evaluate the performance . We present the results of the 1st and the 10th iterations with two kinds of initial inputs and varying value of tuning parameter C in Figure 2 ( b ) . It is clear that the iterations improve the precision of the estimates with both initializations . The results also illustrate that our method is not sensitive to the initialization . Without iterations , the one-step estimate obtained by the NS method proposed by Zhang et al . ( 2017 ) outperforms random initialization significantly . However , after sufficient number of iterations , the performance of our method with different initial values becomes comparable . In practice , as the true network structure is unknown , we recommend using the random initial input .

Summary: This paper deals with the task of estimating the connection probabilities in a random graph, assuming that it is drawn from a graphon model; i.e. $P_{ij} = f(\xi_i, \xi_j)$, where $f$ is a smooth function and the $\xi_i$ are uniform random variables. The authors do so by exploiting the same ansatz as in Zhang et al: for $i \in [n]$, find several $i'$ with connection probabilities "close" to those of $i$, and average the $P_{i'j}$ to find an estimate for $P_{ij}$.  The main difference with the existing litterature is the choice of the $i'$: the authors choose an iterative procedure, building ever improving estimators of the $P_{ij}$ and using the current estimate to find the $i'$ and build the next estimator. The paper also presents a two-step procedure to choose the right number of "close" vertices.  The authors then show several theoretical properties of their algorithm, in a model including (but not limited to) the classical stochastic block model. Their bounds depend on the number of vertices correctly classified as "close" to each vertex in the graph.  Finally, they present experiments on both synthetic and real-world data; these seem to indicate that the ICE algorithm outperforms its counterparts when the graphon function $f$ becomes more complex.