Triangle and Four Cycle Counting with Predictions in Graph Streams

1 INTRODUCTION . Counting the number of cycles in a graph is a fundamental problem in the graph stream model ( e.g. , Atserias et al . ( 2008 ) ; Bera & Chakrabarti ( 2017 ) ; Seshadhri et al . ( 2013 ) ; Kolountzakis et al . ( 2010 ) ; Bar-Yossef et al . ( 2002 ) ; Kallaugher et al . ( 2019 ) ) . The special case of counting triangles is widely studied , as it has a vast range of applications . In particular , it provides important insights into the structural properties of networks ( Prat-Pérez et al. , 2012 ; Farkas et al. , 2011 ) , and is used to discover motifs in protein interaction networks ( Milo et al. , 2002 ) , understand social networks ( Foucault Welles et al. , 2010 ) , and evaluate large graph models ( Leskovec et al. , 2008 ) . See Al Hasan & Dave ( 2018 ) for a survey of these and other applications . Because of its importance , a large body of research has been devoted to space-efficient streaming algorithms for ( 1 + ) -approximate triangle counting . Such algorithms perform computation in one or few passes over the data using only a sub-linear amount of space . A common difficulty which arises in all previous works is the existence of heavy edges , i.e. , edges that are incident to many triangles ( four cycles ) . As sublinear space algorithms often rely on sampling of edges , and since a single heavy edge can greatly affect the number of triangles ( four cycles ) in a graph , sampling and storing these edges are often the key to an accurate estimation . Therefore , multiple techniques have been developed to determine whether a given edge is heavy or not . Recently , based on the observation that many underlying patterns in real-world data sets do not change quickly over time , machine learning techniques have been incorporated into the data stream model via the training of heavy-hitter oracles . Given access to such a learning-based oracle , a wide range of significant problems in data stream processing — including frequency estimation , estimating the number of distinct elements , Fp-Moments or ( k , p ) -Cascaded Norms — can all achieve space bounds that are better than those provided by “ classical ” algorithms , see , e.g. , Hsu et al . ( 2019a ) ; Cohen et al . ( 2020 ) ; Jiang et al . ( 2020 ) ; Eden et al . ( 2021 ) ; Du et al . ( 2021 ) . More gen- erally , learning-based approaches have had wide success in other algorithmic tasks , such as data structures ( Kraska et al. , 2018 ; Ferragina et al. , 2020 ; Mitzenmacher , 2018 ; Rae et al. , 2019 ; Vaidya et al. , 2021 ) , online algorithms ( Lykouris & Vassilvtiskii , 2018 ; Purohit et al. , 2018 ; Gollapudi & Panigrahi , 2019 ; Rohatgi , 2020 ; Wei , 2020 ; Mitzenmacher , 2020 ; Lattanzi et al. , 2020 ; Bamas et al. , 2020 ) , similarity search ( Wang et al. , 2016 ; Dong et al. , 2020 ) and combinatorial optimization ( Dai et al. , 2017 ; Balcan et al. , 2017 ; 2018a ; b ; 2019 ) . See the survey and references therein for additional works ( Mitzenmacher & Vassilvitskii , 2020 ) . Inspired by these recent advancements , we ask : is it possible to utilize a learned heavy edge oracle to improve the space complexity of subgraph counting in the graph stream model ? Our results demonstrate that the answer is yes . 1.1 OUR RESULTS AND COMPARISON TO PREVIOUS THEORETICAL WORKS . We present theoretical and empirical results in several graph streaming models , and with several notions of prediction oracles . Conceptually , it is useful to begin by studying perfect oracles that provide exact predictions . While instructive theoretically , such oracles are typically not available in practice . We then extend our theoretical results to noisy oracles that can provide inaccurate or wrong predictions . We validate the practicality of such oracles in two ways : by directly showing they can be constructed for multiple real datasets , and by showing that on those datasets , our algorithms attain significant empirical improvements over baselines , when given access to these oracles . We proceed to a precise account of our results . Let G = ( V , E ) denote the input graph , and let n , m , and T denote the number of vertices , edges , and triangles ( or four-cycles ) in G , respectively . There are two major graph edge streaming models : the adjacency list model and the arbitrary order model . We show that training heavy edge oracles is possible in practice in both models , and that such oracles make it possible to design new algorithms that significantly improve the space complexity of triangle and four-cycle counting , both in theory and in practice . Furthermore , our formalization of a heavy edge prediction framework makes it possible to show provable lower bounds as well . Our results are summarized in Table 1 . In our algorithms , we assume that we know a large-constant approximation of T for the purposes of setting various parameters . This is standard practice in the subgraph counting streaming literature ( e.g. , see ( Braverman et al. , 2013 , Section 1 ) , ( McGregor et al. , 2016 , Section 1.2 ) for an extensive discussions on this assumption ) . Moreover , when this assumption can not be directly carried over in practice , in Subsection F.3 we discuss how to adapt our algorithms to overcome this issue . 1.1.1 PERFECT ORACLE . Our first results apply for the case that the algorithms are given access to a perfect heavy edge oracle . That is , for some threshold ρ , the oracle perfectly predicts whether or not a given edge is incident to at least ρ triangles ( four cycles ) . We describe how to relax this assumption in Section 1.1.2 . Adjacency List Model All edges incident to the same node arrive together . We show : Theorem 1.1 . There exists a one-pass algorithm , Algorithm 1 , with space complexity1 Õ ( min ( −2m2/3/T 1/3 , −1m1/2 ) ) in the adjacency list model that , using a learning-based oracle , returns a ( 1± ) -approximation to the number T of triangles with probability at least2 7/10 . An overview of Algorithm 1 is given in Section 2 , and the full analysis is provided in Appendix B . Arbitrary Order Model In this model , the edges arrive in the stream in an arbitrary order . We present a one-pass algorithm for triangle counting and another one-pass algorithm for four cycle counting in this model , both reducing the number of passes compared to the currently best known space complexity algorithms . Our next result is as follows : Theorem 1.2 . There exists a one-pass algorithm , Algorithm 4 , with space complexity Õ ( −1 ( m/ √ T + √ m ) ) in the arbitrary order model that , using a learning-based oracle , returns a ( 1± ) -approximation to the number T of triangles with probability at least 7/10 . 1We use Õ ( f ) to denote O ( f · polylog ( f ) ) . 2For all of our algorithms , the success probability can be replaced by 1 − δ by running log ( 1/δ ) copies of the algorithm and taking the median . An overview of Algorithm 4 is given in Section 3 , and full details are provided in Appendix C. We also show non-trivial space lower bounds that hold even if appropriate predictors are available . In Theorem C.2 in Appendix C.3 , we provide a lower bound for this setting by giving a construction that requires Ω ( min ( m/ √ T , m3/2/T ) ) space even with the help of an oracle , proving that our result is nearly tight in some regimes . Therefore , the triangle counting problem remains non-trivial even when extra information is available . Four Cycle Counting . For four cycle counting in the arbitrary order model , we give Theorem 1.3 which is proven in Appendix D. Theorem 1.3 . There exists a one-pass algorithm , Algorithm 5 , with space complexity Õ ( T 1/3 + −2m/T 1/3 ) in the arbitrary order model that , using a learning-based oracle , returns a ( 1 ± ) - approximation to the number T of four cycles with probability at least 7/10 . To summarize our theoretical contributions , for the first set of results of counting triangles in the adjacency list model , our bounds always improve on the previous state of the art due to McGregor et al . ( 2016 ) for all values of m and T . For a concrete example , consider the case that T = Θ ( √ m ) . In this setting previous bounds result in an Õ ( m3/4 ) -space algorithm , while our algorithm only requires Õ ( √ m ) space ( for constant ) . For the other two problems of counting triangles and 4-cycles in the arbitrary arrival model , our space bounds have an additional additive term compared to McGregor et al . ( 2016 ) ( for triangles ) and Vorotnikova ( 2020 ) ( for 4-cycles ) but importantly run in a single pass rather than multiple passes . In the case where the input graph has high triangles density , T = Ω ( m/ 2 ) , our space bound is worse due to the additive factor . When T = O ( m/ 2 ) , our results achieve the same dependence on m and T as that of the previous algorithms with an improved dependency in . Moreover , the case T ≤ m/ 2 is natural for many real world datasets : for = 0.05 , this condition holds for all of the datasets in our experimental results ( see Table 2 ) . Regardless of the triangle density , a key benefit of our results is that they are achieved in a single pass rather than multiple passes . Finally , our results are for general graphs , and make no assumptions on the input graph ( unlike Pagh & Tsourakakis ( 2012 ) ; Kallaugher & Price ( 2017 ) ; Bera & Sheshadhri ( 2020 ) ) . Most of our algorithms are relatively simple and easy to implement and deploy . At the same time , some of our results require the use of novel techniques in this context , such as the use of exponential random variables ( see Section 2 ) . 1.1.2 NOISY ORACLES . The aforementioned triangle counting results are stated under the assumption that the algorithms are given access to perfect heavy edge oracles . In practice , this assumption is sometimes unrealistic . Hence , we consider several types of noisy oracles . The first such oracle , which we refer to as a K-noisy oracle , is defined below ( see Figure 4 in the Supplementary Section C.2 ) . Definition 1.1 . For an edge e = xy in the stream , define Ne as the number of triangles that contain both x and y . For a fixed constant K ≥ 1 and for a threshold ρ we say that an oracle Oρ is a K-noisy oracle if for every edge e , 1−K · ρNe ≤ Pr [ Oρ ( e ) = HEAVY ] ≤ K · Ne ρ . This oracle ensures that if an edge is extremely heavy or extremely light , it is classified correctly with high probability , but if the edge is close to the threshold , the oracle may be inaccurate . We further discuss the properties of this oracle in Section G. For this oracle , we prove the following two theorems . First , in the adjacency list model , we prove : Theorem 1.4 . Suppose that the oracle given to Algorithm 1 is a K-noisy oracle as defined in Definition 1.1 . Then with probability 2/3 , Algorithm 1 returns a value in ( 1 ± √ K· ) T , and uses space at most Õ ( min ( −2m2/3/T 1/3 , K· −1m1/2 ) ) . Hence , even if our oracle is inaccurate for edges near the threshold , our algorithm still obtains an effective approximation with low space in the adjacency list model . Likewise , for the arbitrary order model , we prove in Theorem C.1 that the O ( −1 ( m/ √ T + √ m ) ) 1-pass algorithm of Theorem 1.2 also works when Algorithm 4 is only given access to a K-noisy oracle . The proof of Theorem 1.4 is provided in Appendix B.1 , and the proof of Theorem C.1 is provided in Appendix C.2 . We remark that Theorems 1.4 and C.1 automatically imply Theorems 1.1 and 1.2 , since the perfect oracle is automatically a K-noisy oracle . ( Noisy ) Value Oracles In the adjacency list model , when we see an edge xy , we also have access to all the neighbors of either x or y , which makes it possible for the oracle to give a more accurate prediction . For an edge xy , let Rxy denote the number of triangles { x , z , y } so that x precedes z and z precedes y in the stream arrival order . Formally , Rxy = |z : { x , y , z } ∈ ∆ and x < s z < s y| where x < s y denotes that the adjacency list of x arrives before that of y in the stream . Motivated by our empirical results in Section F.7 , it is reasonable in some settings to assume we have access to oracles that can predict a good approximation to Rxy . We refer to such oracles as value oracles . In the first version of this oracle , we assume that the probability of approximation error decays linearly with the error from above but exponentially with the error from below . Definition 1.2 . Given an edge e , an ( α , β ) value-prediction oracle outputs a random value p ( e ) where E [ p ( e ) ] ≤ αRe + β , and Pr [ p ( e ) < Reλ − β ] ≤ Ke −λ for some constant K and any λ ≥ 1 . For this variant , we prove the following theorem . Theorem 1.5 . Given an oracle with parameters ( α , β ) , there exists a one-pass algorithm , Algorithm 2 , with space complexity O ( −2 log2 ( K/ ) ( α + mβ/T ) ) in the adjacency list model that returns a ( 1± ) -approximation to the number of triangles T with probability at least 7/10 . In the second version of this noisy oracle , we assume that the probability of approximation error decays linearly with the error from both above and below . For this variant , we prove that we can achieve the same guarantees as Theorem 1.5 up to logarithmic factors ( see Theorem B.1 ) . The algorithms and proofs for both Theorem 1.5 and Theorem B.1 appear in Appendix B.2 . Experiments We conduct experiments to verify our results for triangle counting on a variety of real world networks ( see Table 2 ) in both the arbitrary and adjacency list models . Our algorithms use additional information through predictors to improve empirical performance . The predictors are data dependent and include : memorizing heavy edges in a small portion of the first graph in a sequence of graphs , linear regression , and graph neural networks ( GNNs ) . Our experimental results show that we can achieve up to 5x decrease in estimation error while keeping the same amount of edges as other state of the art empirical algorithms . For more details , see Section 4 . In Section F.7 , we show that our noisy oracle models are realistic for real datasets . Related Empirical Works On the empirical side , most of the focus has been on triangle counting in the arbitrary order model for which there are several algorithms that work well in practice . We primarily focus on two state-of-the-art baselines , ThinkD ( Shin et al. , 2018 ) and WRS ( Shin , 2017 ) . In these works , the authors compare to previous empirical benchmarks such as the ones given in Stefani et al . ( 2017 ) ; Han & Sethu ( 2017 ) ; Lim & Kang ( 2015 ) and demonstrate that their algorithms achieve superior estimates over these benchmarks . There are also other empirical works such as Ahmed et al . ( 2017 ) and Ahmed & Duffield ( 2020 ) studying this model but they do not compare to either ThinkD or WRS . While these empirical papers demonstrate that their algorithm returns unbiased estimates , their theoretical guarantees on space is incomparable to the previously stated space bounds for theoretical algorithms in Table 1 . Nevertheless , we use ThinkD and WRS as part of our benchmarks due to their strong practical performance and code accessibility . Implicit Predictors in Prior Works The idea of using a predictor is implicit in many prior works . The optimal two pass triangle counting algorithm of McGregor et al . ( 2016 ) can be viewed as an implementation of a heavy edge oracle after the first pass . This oracle is even stronger than the Knoisy oracle as it is equivalent to an oracle that is always correct on an edge e if Ne either exceeds or is under the threshold ρ by a constant multiplicative factor . This further supports our choice of oracles in our theoretical results , as a stronger version of our oracle can be implemented using one additional pass through the data stream ( see Section G ) . Similarly , the optimal triangle counting streaming algorithm ( assuming a random order ) given in McGregor & Vorotnikova ( 2020 ) also implicitly defines a heavy edge oracle using a small initial portion of the random stream ( see Lemma 2.2 in McGregor & Vorotnikova ( 2020 ) ) . The random order assumption allows for the creation of such an oracle since heavy edges are likely to have many of their incident triangle edges appearing in an initial portion of the stream . We view these two prior works as theoretical justification for our oracle definitions . Lastly , the WRS algorithm also shares the feature of defining an implicit oracle : some space is reserved for keeping the most recent edges while the rest is used to keep a random sample of edges . This can be viewed as a specific variant of our model , where the oracle predicts recent edges as heavy . Preliminaries . G = ( V , E ) denotes the input graph , and n , m and T denote the number of vertices , edges and triangles ( or four-cycles ) inG , respectively . We useN ( v ) to denote the set of neighbors of a node v , and ∆ to denote the set of triangles . In triangle counting , for each xy ∈ E ( G ) , we recall that Nxy = |z : { x , y , z } ∈ ∆| is the number of triangles incident to edge xy , and Rxy = |z : { x , y , z } ∈ ∆ , x < s z < s y| is the number of triangles adjacent to xy with the third vertex z of the triangle between x and y in the adjacency list order . Table A summarizes the notation .

Summary: The line of research of improving sketching data structures/sampling with the help of learned models is becoming quite popular. This paper follows this line of research and applies this paradigm to the problem of cycle counting in graph streams ( specifically triangles and four-cycles).  It provides a theoretical framework for analyzing Oracle-based problems, analyzing the paradigm's space bounds in cycle counting, and providing lower bounds for the problems.