Optimality of variational inference for stochasticblock model with missing links

1 Introduction . The analysis of network data poses both computational and theoretical challenges . Most results obtained in the literature concentrate on the stochastic block model ( SBM ) which is known to be a good proxy for more general models , such as the inhomogeneous random graph model , [ 35 ] . Recently , variational methods ( [ 28 , 49 ] ) have attracted considerable attention as they offer computationally tractable algorithms often combined with theoretical guarantees . Theoretical results that one can find for such variational methods provide asymptotic normality rates for parameter estimates of stochastic block data . For example , consistency has been shown for profile likelihood maximization [ 8 ] and variational approximation to the maximum likelihood estimator [ 13 ] , [ 7 ] . These results have been extended to the case of dynamic stochastic block model [ 34 ] and sampled data [ 48 ] . These work focus on parameter estimation , as in [ 44 ] and [ 53 ] , who establish the minimax optimality of variational methods in a large class of models ( which does however not include the stochastic block model ) . Variational inference has also been successfully applied to the problem of community detection , see , e.g. , [ 3 , 54 , 26 , 45 ] . In particular , the authors of [ 54 ] show that an iterative Batch Coordinate Ascent Variational Inference algorithm designed for the two-parameters , assortative stochastic block model achieves statistical optimality for community detection problem . Note that this algorithm can not be extended to the more general stochastic block model considered here . In parallel with this line of work , the problem of statistical estimation of model parameters , in particular , the question of minimax optimal convergence rates , has been actively studied in the statistical community . In the case of dense graphs , a pioneering paper [ 17 ] shows that , for the problem of estimating the matrix of connection probabilities , the least square estimator is minimax optimal and [ 18 ] provides optimal rate for Bayes estimation . For the more challenging case of 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . sparse graphs , the minimax optimal rates have been first obtained in [ 29 ] building on the restricted least square estimator . In [ 16 ] , the authors consider the least square estimator in the setting when observations about the presence or absence of an edge are missing independently at random with the same probability p. Unfortunately , least square estimation is too computationally expensive to be used in practice . Many other approaches have been proposed , for example , spectral clustering [ 39 , 22 , 46 ] , modularity maximization [ 42 , 8 ] , belief propagation [ 14 ] , neighbourhood smoothing [ 55 ] , convex relaxation of k-means clustering [ 20 ] and of likelihood maximization [ 4 ] , and universal singular value thresholding [ 11 , 30 , 51 ] . These approaches are computationally tractable but show sub-optimal statistical performances . So the question of possible computational gap when no polynomial time algorithm can achieve minimax optimal rate of convergence has been raised . The present work goes in these two directions . We study the statistical properties of the mean field variational Bayes method and show that it achieves the optimal statistical accuracy . In particular , these results close the open question on the possible existence of a computational gap for the problem of global parameter estimation . We built our analysis on the approach developed in [ 13 ] , [ 7 ] and [ 48 ] using the closeness of maximum likelihood and maximum variational likelihood and on the results that show the minimax optimality of the maximum likelihood estimator [ 19 ] . In the present paper , we deal with settings where the network is not fully observed , a common problem when studying real life networks . In many applications the network has missing data as detecting interactions can require significant experimental effort , see , [ 32 , 52 , 24 , 21 ] . For example , in biology graphs are used to model interactions between proteins . Discovery of these interactions can be costly and time-consuming [ 9 ] . On the other hand , the size of some networks from social media or genome sequencing may be so large that only subsamples of the data are considered [ 6 ] . It has been observed that incomplete observation of the network structure may considerably affect the accuracy of inference methods [ 31 ] and missing data must be taken into account while analyzing networks data . A popular approach consists in considering the edges with uncertain status as non-existing . In the present paper , we use a different framework by considering such edges as missing and introducing a separated data missing mechanism . A natural application of our method is link prediction [ 36 , 56 ] , the task of predicting whether two nodes in a network are connected . Our approach allows to deduce the pairs of nodes that are most likely to interact based on the known interactions in the network . Behind inference of the networks structure , our algorithms can be used to predict the links that may appear in the future if we consider networks evolving over the time . For example , in a social network , two users that are not yet connected but are likely to be connected can be recommended as promising friends . 1.1 Contribution and outline . The paper is organized as follows . After summarizing notations , we introduce our model and the maximum likelihood estimator for the stochastic block model with missing observations in Section 2 . In Section 3 , we introduce the mean field variational Bayes method and present a new estimator which combines the labels obtained using the variational method and the empirical mean for estimation of connection probabilities . In Section 3.2 , we show that our estimator is minimax optimal for dense stochastic block models with missing observations as well as for sparse stochastic block models . Finally , in Section 4 we provide an extensive numerical study both on synthetic and real-life data which shows clear advantages of our estimator over current methods . 1.2 Notations . We provide here a summary of the notations used throughout the paper . For all d 2 N⇤ , we denote by [ d ] the set { 1 , ... , d } . For z : [ k ] ! [ n ] and all ( a , b ) 2 [ d ] ⇥ [ d ] , we abuse notations and denote z 1 ( a , b ) = { ( i , j ) : z ( i ) = a , z ( j ) = b , i 6= j } . For any two label functions z , z0 , we write z ⇠ z0 if there exists a permutation of { 1 , ... , k } such that ( z ( ( a ) ) ) ak = ( z ( a ) ) ak . For any set S , we denote by |S| its cardinality . For any matrix A , we denote by Aij its entry on row i and column j . If A 2 [ 0 , 1 ] n⇥n and A is symmetric , we write A 2 [ 0 , 1 ] n⇥nsym . We denote by A B the Hadamard product of two matrices A and B . The Frobenius norm of a matrix A is denoted by kAk2 = qP i , j A 2 ij . We denote by C and C 0 positive constants that can vary from line to line . These are absolute constants unless otherwise mentioned . For any two positive sequences ( an ) n2N , ( bn ) n2N , we write an = ! ( bn ) if an/bn ! 1 . 2 Maximum likelihood estimation in the stochastic block model with missing links . 2.1 Network model and missing data scheme . In the simplest situation , a network can be represented as undirected , unweighted graph with n nodes indexed from 1 to n. Then , the network can be encoded by its adjacency matrix A = ( Aij ) . The adjacency matrix is a n⇥n symmetric matrix such that for any i < j , Aij = 1 if there exists an edge between node i and node j , Aij = 0 otherwise . We consider that there is no edge linking a node to itself , so Aii = 0 for any i . A common approach in network data analysis is to assume that the observations are random variables drawn from a probability distribution over the space of adjacency matrices . More precisely , for i < j the variables Aij are assumed to be independent Bernoulli random variables of parameter ⇥⇤ij , where ⇥ ⇤ = ( ⇥⇤ij ) 1i < jn is a n⇥ n symmetric matrix with zero diagonal entries . The matrix ⇥⇤ corresponds to the matrix of probabilities of observing an edge between nodes i and j . This model is known as the inhomogeneous random graph model : 81 i < j n , Aij |⇥⇤ij ind.⇠ Bernoulli ⇥⇤ij . ( 1 ) Our focus is on the problem of estimation of the generative matrix ⇥⇤ which determines the overall structure of the network . This question is of particular interest for the task of link prediction . Many of real-life networks are characterized by block structure . Loosely speaking , the block structure means that the nodes of the network are partitioned into groups called blocks , and that the distribution of the connections between nodes depends on the blocks to which the nodes belong . For example , when considering citation networks , where two articles are linked if one is cited by the other , it amounts to saying that the probability that two articles are linked only depends on their topic . Similarly , if one considers students of a school in a social network , it is a reasonable assumption to say that the probability that two students are linked only depends on their cohorts . A very popular model that formalizes this idea is the stochastic block model ( see , e.g. , [ 27 ] ) . In this model , nodes are classified into k communities : each node i is associated with a community z ⇤ ( i ) , where z⇤ : [ n ] ! [ k ] is called the label function . This label function can either be treated as a parameter to estimate , or as a latent variable . In this last case , it is assumed that the indexes follow a multinomial distribution : 8i , z⇤ ( i ) i.i.d⇠ Multinomial ( 1 ; ↵⇤ ) where 8a 2 [ k ] , ↵a is the probability that node i belongs to the community a . Given this label function , the probability that there exists an edge between nodes i and j depends only on the communities of i and j . Thus , the matrix of connection probabilities ⇥⇤ can be factorized as follows : ⇥⇤ij = Q ⇤ z⇤ ( i ) z⇤ ( j ) , with Q ⇤ a k⇥ k symmetric matrix such that Q⇤ab is the probability that there exists an edge between a given member of the community a and a given member of the community b . The conditional stochastic block model can be written as : 9Q⇤ 2 [ 0 , 1 ] k⇥ksym , 9z⇤ : [ n ] ! [ k ] 81 i < j n , Aij | ( Q⇤ , z⇤ ) ind.⇠ Bernoulli ⇣ Q ⇤ z⇤ ( i ) z⇤ ( j ) ⌘ , Aii = 0 . ( 2 ) Assuming that the network follows the stochastic block model , the problem of estimating the matrix of connection probabilities reduces to estimating the label function z⇤ and the matrix of probabilities of connections between communities Q⇤ . Note that the conditional stochastic block model is at best identifiable up to a simultaneous permutation of the communities and of the rows and columns of the parameters Q⇤ . The stochastic block model has attracted considerable interest from the learning community . An important line of work has focused on the problem of estimation of the latent variables z⇤ , see , for example , [ 38 , 10 , 1 , 41 ] . The best understood framework is the binary , balanced , symmetric , assortative block model . In this simpler model , the two communities have the same size , the same probability of intra-community connection ( Q⇤11 = Q ⇤ 22 = p ) , and nodes are assumed to be more connected with nodes of the same community ( p > q = Q⇤12 ) . Much work has been done on the precise characterisation of the conditions on p , q that allow for strong recovery of z⇤ , i.e . to estimate z⇤ exactly with high probability . Closest to model ( 2 ) is perhaps the setting considered in [ 15 ] . In this work , the authors consider the related problem of community recovery in the binary block model [ 23 ] , [ 2 ] , and provide tight bounds on the recovery threshold for the balanced , two communities stochastic block model with missing observations . They propose a computationally efficient algorithm for estimating z⇤ in regime where strong recovery is possible ; this , however , requires prior knowledge of the parameter Q⇤ . Missing observations scheme Usually , when working with network data , not all the edges are observed . To account for this situation we introduce X 2 { 0 , 1 } n⇥nsym the known sampling matrix where Xij = 1 if Aij is observed and Xij = 0 otherwise . We assume that X is random and independent from the adjacency matrix A and its expectation ⇥⇤ . For any 1 i < j n , its entries Xij are mutually independent and Xij ind.⇠ Bernoulli ( p ) for some sampling rate p ! 0 such that p = ! ( log ( n ) /n ) when n ! 1 .

Summary: The main contribution of this paper is to propose and analyze a variational approximation (VA) method for the inference of the SBM. The main result is to establish that VA leads to a tractable and minimax optimal algorithm, specifically the latter is based on the EM principle with a variational approximation scheme. Numerical experiments are also performed to verify the theory.