We study networks that display community structure-groups of nodes within which connections are unusually dense. Using methods from random matrix theory, we calculate the spectra of such networks in the limit of large size, and hence demonstrate the presence of a phase transition in matrix methods for community detection, such as the popular modularity maximization method. The transition separates a regime in which such methods successfully detect the community structure from one in which the structure is present but is not detected. By comparing these results with recent analyses of maximum-likelihood methods we are able to show that spectral modularity maximization is an optimal detection method in the sense that no other method will succeed in the regime where the modularity method fails. The problem of community detection in networks has attracted a substantial amount of attention in recent years [1, 2] . Communities in this context are groups of vertices within a network that have a high density of within-group connections but a lower density of betweengroup connections. The challenge is to find such groups accurately and efficiently in a given network-the ability to do so would have applications in the analysis of observational data, network visualization, and complexity reduction and parallelization of network problems. In this paper we focus on matrix methods for community detection, which are based on the properties of matrix representations of networks such as the adjacency matrix or the modularity matrix. While significant effort has been devoted to the development of practical algorithms using these methods, there has been less work on formal examination of their properties and implications for algorithm performance. Here we give an analysis of the spectral properties of the adjacency and modularity matrices using random matrix methods, and in the process uncover a number of results of practical importance. Chief among these is the presence of a sharp transition between a regime in which the spectrum contains clear evidence of community structure and a regime in which it contains none. In the former regime, community detection is possible and current algorithms should perform well; in the latter, any method relying on the spectrum to perform structure detection must fail. A similar phase transition has been reported recently in an analysis of a different class of detection methods, based on Bayesian inference [3] . By comparing the two analyses, we are able to demonstrate that methods such as modularity maximization are optimal, in the sense that no other method will succeed where they fail. For the formal analysis of community structured networks, we must define the particular network or networks we will study. In this paper we focus on the most widely studied model of community structure, the stochastic block model, although our methods could be applied to other models as well. The stochastic block model, in its simplest form, divides a network of n vertices into some number q of groups denoted by r = 1 . . . q and then places undirected edges between vertex pairs i, j with independent probabilities p rs , where r, s are respectively the groups to which vertices i, j belong. In other words, the probability of an edge between two vertices in this model depends only on the groups in which the vertices fall. If the diagonal elements of the matrix of probabilities p rs are greater than the off-diagonal elements, then the network displays classic community structure with a greater density of edges within groups than between them. Particular instances of the stochastic block model are commonly used as testbeds for assessing the performance of community detection algorithms-especially in the "four groups" test [1] and the planted partition model [4] . Let us first demonstrate our argument for the simplest possible case of a network with q = 2 groups of equal size 1 2 n each and just two different probabilities p in and p out for connections within and between groups. We focus particularly on the case of sparse networks, those for which the fraction of possible edges that are present in the network vanishes in the limit of large n, which appears to be representative of most networks observed in the real world, although our results apply in principle to dense networks as well. The adjacency matrix A of an undirected network is the n × n symmetric matrix with elements A ij = 1 if vertices i and j are connected by an edge and 0 otherwise. If we average the adjacency matrix over the ensemble of our stochastic block model the resulting matrix A has elements equal to p in for vertices in the same group and p out for vertices in different groups. Defining c in = np in and c out = np out , this matrix can be written in the form  where 1 and u are the unit vectors 1 = (1, 1, 1, . . .)/ √ n and u = (1, 1, . . . , −1, −1, . . .)/ √ n, the ±1 elements in the latter denoting the members of the two communities. Now the full adjacency matrix can be written in the form A = A + X where the matrix X is the deviation
The detectability of the modularity method is studied in REF when the two subnetworks are of equal size and each node pair in each subnetwork is randomly connected by the same edge connection probability.
Graph spectra and the detectability of community structure in networks
Phys. Rev. Lett. 108, 188701 (2012)
Computer Science Physics Medicine