SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\boldsymbol {x} _ {v} ^ {(l)} = \sigma \left(\sum_ {u \in \mathcal {N} (v) \cup \{v \}} \boldsymbol {w} _ {u \rightarrow v} ^ {(l)} \boldsymbol {x} _ {u} ^ {(l - 1)} + \boldsymbol {b} _ {v} ^ {(l)}\right), \forall v \in V \tag {★}
$$
where  $\mathcal{N}(v)$  is the set of all neighbors of  $v$ ,  $\sigma$  could be identity or any activation function.
With linear aggregate functions such as sum and mean, many classic GNN architectures could be rewritten in form  $(\star)$ , for example: Spectral Network [4], ChebNet [5], GCN [6], Neural FPs [10], DCNN [11], PATCHY-SAN [12], GraphSAGE [13], and MPNN [16].
2.5.2 Mixed-integer optimization formulations for non-fixed graphs
If the graph structure for inputs is given and fixed, then  $(\star)$  is equivalent to a fully connected layer, whose MIP formulations are already well-established [46, 49]. But if the graph structure is non-fixed (i.e., the elements in adjacency matrix are also decision variables), two issues arise in  $(\star)$ :  $(1)\mathcal{N}(v)$  is not well-defined;  $(2)\pmb{w}_{u\to v}^{(l)}$  and  $\pmb{b}_v^{(l)}$  may be not fixed and contain the graph's information. Assuming that the weights and biases are constant, we can build two MIP formulations to handle the first issue.
The first formulation comes from observing that the existence of edge from node  $u$  to node  $v$  determines the contribution link from  $\boldsymbol{x}_u^{(l-1)}$  to  $\boldsymbol{x}_v^{(l)}$ . Adding binary variables  $e_{u \to v}$  for all  $u, v \in V$ , we can then formulate the GNNs with bilinear constraints:
$$
\boldsymbol {x} _ {v} ^ {(l)} = \sigma \left(\sum_ {u \in V} e _ {u \rightarrow v} \boldsymbol {w} _ {u \rightarrow v} ^ {(l)} \boldsymbol {x} _ {u} ^ {(l - 1)} + \boldsymbol {b} _ {v} ^ {(l)}\right), \forall v \in V \quad (\text {b i l i n e a r})
$$
Introducing this nonlinearity results in a mixed-integer quadratically constrained optimization problem (MIQCP), which can be handled by state-of-the-art solvers such as Gurobi [87].
The second formulation generalizes the big-M formulation for GraphSAGE in [71] to all GNN architectures satisfying  $(\star)$  and the assumption of constant weights and biases. Instead of using binary variables to directly control the existence of contributions between nodes, auxiliary variables  $z_{u\to v}^{(l - 1)}$  are introduced to represent the contribution from node  $u$  to node  $v$  in the  $l$ -th layer:
$$
\boldsymbol {x} _ {v} ^ {(l)} = \sigma \left(\sum_ {u \in V} \boldsymbol {w} _ {u \rightarrow v} ^ {(l)} \boldsymbol {z} _ {u \rightarrow v} ^ {(l - 1)} + \boldsymbol {b} _ {v} ^ {(l)}\right), \forall v \in V \quad (\text {b i g - M})
$$
$$
\boldsymbol {z} _ {u \to v} ^ {(l - 1)} = \left\{ \begin{array}{l l} 0, & e _ {u \to v} = 0 \\ \boldsymbol {x} _ {u} ^ {(l - 1)}, & e _ {u \to v} = 1 \end{array} \right.
$$
Assuming that each feature is bounded, the definition of  $\mathbf{z}_{u\to v}^{(l - 1)}$  can be reformulated using big-M:
$$
\begin{array}{l} \boldsymbol {x} _ {u} ^ {(l - 1)} - \boldsymbol {M} _ {u} ^ {(l - 1)} (1 - e _ {u \rightarrow v}) \leq \boldsymbol {z} _ {u \rightarrow v} ^ {(l - 1)} \leq \boldsymbol {x} _ {u} ^ {(l - 1)} + \boldsymbol {M} _ {u} ^ {(l - 1)} (1 - e _ {u \rightarrow v}) \\ - \boldsymbol {M} _ {u} ^ {(l - 1)} e _ {u \rightarrow v} \leq \boldsymbol {z} _ {u \rightarrow v} ^ {(l - 1)} \leq \boldsymbol {M} _ {u} ^ {(l - 1)} e _ {u \rightarrow v} \\ \end{array}
$$
where  $|\pmb{x}_u^{(l-1)}| \leq M_u^{(l-1)}, e_{u \to v} \in \{0, 1\}$ . By adding extra continuous variables and constraints, as well as utilizing the bounds for all features, the big- $\mathbf{M}$  formulation replaces the bilinear constraints with linear constraints. Section 3 numerically compares these two formulations.
3 Computational experiments
We performed all experiments on a 3.2 GHz Intel Core i7-8700 CPU with 16 GB memory. GNNs are implemented and trained in PyG [95]. MIP formulations for GNNs and CAMD are implemented based on OMLT [79], and are optimized by Gurobi 10.0.1 [87] with default relative MIP optimality gap (i.e.,  $10^{-4}$ ). The code is available at https://github.com/cog-imperial/GNN_MIP_CAMD. These are all engineering choices and we could have, for example, extended software other than OMLT to translate the trained GNNs into an optimization solver [96, 97].
3.1 Mixed-integer optimization formulation for molecular design
MIP formulations are well-established in the CAMD literature [98-103]. The basic idea is representing a molecule as a graph, creating variables for each atom (or group of atoms), using constraints to preserve graph structure and satisfy chemical requirements. A score function is usually given as the optimization target. Instead of using knowledge from experts or statistics to build the score function, GNNs (or other ML models) could be trained from datasets to replace score functions. Here we provide a general and flexible framework following the MIP formulation for CAMD in [71]. Moreover, by adding meaningful bounds and breaking symmetry, our formulation could generate more reasonable molecules with less redundancy. Due to space limitation, we briefly introduce the formulation here (see Appendix B for the full MIP formulation).
To design a molecule with  $N$  atoms, we define  $N \times F$  binary variables  $X_{v,f}, v \in [N], f \in [F]$  to represent  $F$  features for  $N$  atoms. Hydrogen atoms are not counted in since they can implicitly considered as node features. Features include types of atom, number of neighbors, number of hydrogen atoms associated, and types of adjacent bonds. For graph structure of molecules, we add three sets of binary variables  $A_{u,v}, DB_{u,v}, TB_{u,v}$  to denote the type of bond (i.e., any bond, double bond, and triple bond) between atom  $u$  and atom  $v$ , where  $A$  is the adjacency matrix.
To design reasonable molecules, constraints (C1) - (C21) handle structural feasibility following [71]. Additionally, we propose new constraints to bound the number of each type of atoms (C22), double bonds (C23), triple bonds (C24), and rings (C25). In our experiments, we calculate these bounds based on the each dataset itself so that each molecule in the dataset will satisfy these bounds (see Appendix C.1 for details). By setting proper bounds, we can control the composition of the molecule, and avoid extreme cases such as all atoms being set to oxygen, or a molecule with too many rings or double/triple bounds. In short, constraints (C22) - (C25) provide space for chemical expertise and practical requirements. Furthermore, our formulation could be easily applied to datasets with different types of atoms by only changing the parameters in Appendix B.1. Moreover, group representation of molecules [98, 102] is also compatible with this framework. The advantage is that all constraints can be reused without any modification.
Among constraints (C1) - (C25) (as shown in Appendix B.3), constraints (C5) are the realization of (S1). Except for (C5), these structural constraints are independent of the graph indexing. Therefore, we can compatibly implement constraints (S2) and (S3) to break symmetry.
Corresponding to (S2), we add the following constraints over features:
$$
\sum_ {f \in [ F ]} 2 ^ {F - f - 1} \cdot X _ {0, f} \leq \sum_ {f \in [ F ]} 2 ^ {F - f - 1} \cdot X _ {v, f}, \forall v \in [ N ] \backslash \{0 \} \tag {C26}
$$
where  $2^{F - f - 1}$ ,  $f\in [F]$  are coefficients to help build a bijective  $h$  between all possible features and all integers in  $[0,2^F -1]$ . These coefficients are also called "universal ordering vector" in [72, 78].
On graph level, constraints (S3) can be equivalently rewritten as:
$$
\sum_ {u \neq v, v + 1} 2 ^ {N - u - 1} \cdot A _ {u, v} \geq \sum_ {u \neq v, v + 1} 2 ^ {N - u - 1} \cdot A _ {u, v + 1}, \forall v \in [ N - 1 ] \backslash \{0 \} \tag {C27}
$$
Similarly, the coefficients  $2^{N - u - 1}$ ,  $u \in [N]$  are used to build a bijective mapping between all possible sets of neighbors and all integers in  $[0, 2^N - 1]$ .
For illustration purposes, we can view CAMD as two separate challenges, where the first one uses structural constraints to design reasonable molecules (including all symmetric solutions), and the second one uses symmetry-breaking constraints to remove symmetric solutions. Note that the diversity of solutions will not change after breaking symmetry, because each molecule corresponds to at least one solution (guaranteed by Section 2.3).
3.2 Counting feasible structures: performance of symmetry-breaking
We choose two datasets QM7 [104, 105] and QM9 [106, 107] from CAMD literature to test the proposed methods. See Appendix C.1 for more information about QM7 and QM9. To test the efficiency of our symmetry-breaking techniques, we build a MIP formulation for CAMD and count all feasible structures for different  $N$ . By setting PoolSearchMode=2, PoolSolutions=10<sup>9</sup>, Gurobi can find many (up to 10<sup>9</sup>) feasible solutions to fill in the solution pool.
Table 1 shows the performance of our symmetry-breaking constraints (S2) and (S3) comparing to the baseline of (S1). Without adding (S1), we need to count the number of any graph with compatible features. Even ignoring features, the baseline would be  $2^{\frac{N(N - 1)}{2}}$ . This number will be much larger after introducing features, which loses the meaning as a baseline, so we use (S1) as the baseline.
Table 1: Numbers of feasible solutions. The time limit is 48 hours. At least  $2.5 \times 10^{6}$  solutions are found for each time out (t.o.). For each dataset, the last column reports the percentage of removed symmetric solutions after adding (S2) and (S3) to the baseline of (S1). Higher percentage means breaking more symmetries.
N	QM7	QM9
(S1)	(S1) - (S2)	(S1) - (S3)	(%)	(S1)	(S1) - (S2)	(S1) - (S3)	(%)
2	17	10	10	41	15	9	9	40
3	112	37	37	67	175	54	54	69
4	3,323	726	416	87	4,536	1,077	631	86
5	67,020	11,747	3,003	96	117,188	21,441	5,860	95
6	t.o.	443,757	50,951	≥98	t.o.	527,816	59,492	≥98
7	t.o.	t.o.	504,952	*	t.o.	t.o.	776,567	*
8	t.o.	t.o.	t.o.	*	t.o.	t.o.	t.o.	*
3.3 Optimizing over trained GNNs for molecular design
For each dataset, we train a GNN with two GraphSAGE layers followed by an add pooling layer, and three dense layers. Details about their structures and training process are shown in Appendix C.2. For statistical consideration, we train 10 models with different random seeds and use the 5 models with smaller losses for optimization. Given  $N$  ( $\in \{4,5,6,7,8\}$ ), and a formulation ( $\in$  {bilinear, bilinear+BrS, big-M, big-M+BrS}), where "+" means adding symmetry-breaking constraints (C26) and (C27), we solve the corresponding optimization problem 10 times with different random seeds in Gurobi. This means that there are 50 runs for each  $N$  and formulation.
Figure 3 shows the significant improvement of symmetry-breaking. Considering the average solving time, big-M performs better. For the bilinear constraints, Gurobi 10.0.1 appears to transform bilinear constraints into linear constraints. Appendix C.3 shows that, after Gurobi's presolve stage, the big-M formulation has more continuous variables but fewer binary variables compared to the bilinear formulation. The fewer binary variables after presolve may explain the better big-M performance.
Figure 3: These graphs report the time  $time_{tol}$  to achieve relative MIP optimality gap  $10^{-4}$  averaged over the number of successful runs among 50 runs. We plot a bar for each formulation at different values of  $N$ . The left- and right-hand graphs correspond to datasets QM7 and QM9, respectively. The number of runs that achieve optimality is shown above each bar. Results with fewer than 20 successful runs do not have a bar because so few runs terminated. The time limit is 10 hours.
In addition to the average solving time, we compare the performance of two formulations with breaking symmetry in each run. In 341 runs out of 500 runs (i.e., 50 runs for each dataset  $\in$  {QM7, QM9} with each  $N\in \{4,5,6,7,8\}$ ), the big-M formulation achieves optimality faster than the bilinear formulation. Additionally, Gurobi uses much of the solving time to improve the bounds. Table 6 and 7 in Appendix C.3 report time<sup>opt</sup> that denotes the first time to find the optimal solution. In 292 runs, the big-M formulation finds the optimal solution earlier than the bilinear formulation.
4 Discussion & Conclusion
We introduce optimization over trained GNNs and propose constraints to break symmetry. We prove that there exists at least one indexing (resulting from Algorithm 1) satisfying these constraints. Numerical results show the significant improvement after breaking symmetry. These constraints are not limited to the problem (i.e., optimizing trained GNNs), technique (i.e., MIP), and application (i.e., CAMD) used in this work. For example, one can incorporate them into genetic algorithms instead of MIP, or replace the GNN by artificially-designed score functions or other ML models. In other graph-based decision-making problems, as long as the symmetry issue caused by graph isomorphism exists, these constraints could be used to break symmetry. Moreover, the proposed frameworks for building MIP for GNNs as well as CAMD provide generality and flexibility for more problems.
Limitations. Assuming constant weights and biases excludes some GNN types. This limitation is an artifact of our framework and it is possible to build MIP formulations for some other architectures. Using MIP may limit GNN size, for instance, edge features may enlarge the optimization problem.
Future work. One direction is to make the MIP-GNN framework more general, such as adding edge features, supporting more GNN architectures, and developing more efficient formulations. Another direction is using the symmetry-breaking constraints in graph-based applications beyond trained GNNs because we can consider any function that is invariant to graph isomorphism. To facilitate further CAMD applications, more features such as aromacity and formal charge could be incorporated. Also, optimizing lead structures in a defined chemical space is potential to design large molecules.
This work was supported by the Engineering and Physical Sciences Research Council [grant numbers EP/W003317/1 and EP/T001577/1], an Imperial College Hans Rausing PhD Scholarship to SZ, a BASF/RAEng Research Chair in Data-Driven Optimisation to RM, and an Imperial College Research Fellowship to CT.
We really appreciate the comments and suggestions from reviewers and meta-reviewers during the peer review process, which were very helpful to clarify and improve this paper. We also thank Guoliang Wang for the discussion about graph indexing.
[1] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI Open, 1:57-81, 2020.
[2] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S. Yu Philip. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4-24, 2020.
[3] Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Ré, and Kevin Murphy. Machine learning on graphs: A model and comprehensive taxonomy. Journal of Machine Learning Research, 23(89):1–64, 2022.
[4] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun. Spectral networks and locally connected networks on graphs. In ICLR, 2014.
[5] Michael Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In NIPS, 2016.
[6] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
[7] Ruoyu Li, Sheng Wang, Feiyun Zhu, and Junzhou Huang. Adaptive graph convolutional neural networks. In AAAI, 2018.
[8] Chenyi Zhuang and Qiang Ma. Dual graph convolutional networks for graph-based semi-supervised classification. In WWW, 2018.
[9] Bingbing Xu, Huawei Shen, Qi Cao, Yunqi Qiu, and Xueqi Cheng. Graph wavelet neural network. In ICLR, 2019.
[10] David K. Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams. Convolutional networks on graphs for learning molecular fingerprints. In NIPS, 2015.
[11] James Atwood and Don Towsley. Diffusion-convolutional neural networks. In NIPS, 2016.
[12] Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In ICML, 2016.
[13] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.
[14] Petar Velicković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2017.
[15] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model CNNs. In CVPR, 2017.
[16] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In ICML, 2017.
[17] Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. Large-scale learnable graph convolutional networks. In Knowledge Discovery & Data Mining, 2018.
[18] Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Dit-Yan Yeung. GaAN: Gated attention networks for learning on large and spatiotemporal graphs. In UAI, 2018.
[19] Hiroyuki Shindo and Yuji Matsumoto. Gated graph recursive neural networks for molecular property prediction. arXiv preprint arXiv:1909.00259, 2019.
[20] Xiaofeng Wang, Zhen Li, Mingjian Jiang, Shuang Wang, Shugang Zhang, and Zhiqiang Wei. Molecule property prediction based on spatial graph embedding. Journal of Chemical Information and Modeling, 59(9):3817-3828, 2019.
[77] Gustavo Dias and Leo Liberti. Orbital independence in symmetric mathematical programs. In Combinatorial Optimization and Applications, 2015.
[78] Christopher Hojny and Marc E. Pfetsch. Polytopes associated with symmetry handling. Mathematical Programming, 175(1-2):197-240, 2019.
[79] Francesco Ceccon, Jordan Jalving, Joshua Haddad, Alexander Thebelt, Calvin Tsay, Carl D. Laird, and Ruth Misener. OMLT: Optimization & machine learning toolkit. Journal of Machine Learning Research, 23(1):15829-15836, 2022.
[80] Alan Frisch, Brahim Hnic, Zeynep Kiziltan, Ian Miguel, and Toby Walsh. Global constraints for lexicographic orderings. In *Principles and Practice of Constraint Programming-CP* 2002, pages 93–108, 2002.
[81] Joao Marques-Silva, Josep Argelich, Ana Graça, and Inês Lynce. Boolean lexicographic optimization: algorithms & applications. Annals of Mathematics and Artificial Intelligence, 62:317-343, 2011.
[82] Hani A. Elgabou and Alan M. Frisch. Encoding the lexicographic ordering constraint in sat modulo theories. In Thirteenth International Workshop on Constraint Modelling and Reformulation, 2014.
[83] Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural networks. In Computer Aided Verification, 2017.
[84] Tobias Achterberg. SCIP: solving constraint integer programs. Mathematical Programming Computation, 1:1-41, 2009.
[85] Tommi Juntila and Petteri Kaski. Engineering an efficient canonical labeling tool for large and sparse graphs. In ALENEX, 2007.
[86] Tommi Junttila and Petteri Kaski. Conflict propagation and component recursion for canonical labeling. In TAPAS, 2011.
[87] Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2023. URL https://www.gurobi.com.
[88] Volker Kaibel, Matthias Peinhardt, and Marc E. Pfetsch. Orbitopal fixing. Discrete Optimization, 8(4):595-610, 2011.
[89] François Margot. Pruning by isomorphism in branch-and-cut. Mathematical Programming, 94:71-90, 2002.
[90] Hanif D. Sherali and J. Cole Smith. Improving discrete model representations via symmetry considerations. Management Science, 47(10):1396-1407, 2001.
[91] Ahmed Ghoniem and Hanif D. Sherali. Defeating symmetry in combinatorial optimization via objective perturbations and hierarchical constraints. IIE Transactions, 43(8):575-588, 2011.
[92] Volker Kaibel and Marc Pfetsch. Packing and partitioning orbitopes. Mathematical Programming, 114(1):1-36, 2008.
[93] Yuri Faenza and Volker Kaibel. Extended formulations for packing and partitioning orbitopes. Mathematics of Operations Research, 34(3):686-697, 2009.
[94] James Ostrowski, Miguel F. Anjos, and Anthony Vannelli. Symmetry in scheduling problems. Citeseer, 2010.
[95] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds, 2019.
[96] David Bergman, Teng Huang, Philip Brooks, Andrea Lodi, and Arvind U. Raghunathan. Janos: An integrated predictive and prescriptive modeling framework. INFORMS Journal on Computing, 34(2):807-816, 2022.
[97] Laurens Lueg, Bjarne Grimstad, Alexander Mitsos, and Artur M. Schweidtmann. *reluMIP: Open source tool for MILP optimization of ReLU neural networks*, 2021. URL https://github.com/ChemEngAI/ReLU ANN_MILP.
[98] O. Odele and Sandro Macchietto. Computer aided molecular design: A novel method for optimal solvent selection. Fluid Phase Equilibria, 82:47-54, 1993.
[99] Nachiket Churi and Luke E.K. Achenie. Novel mathematical programming model for computer aided molecular design. Industrial & Engineering Chemistry Research, 35(10):3788-3794, 1996.
[100] Kyle V. Camarda and Costas D. Maranas. Optimization in polymer design using connectivity indices. Industrial & Engineering Chemistry Research, 38(5):1884-1892, 1999.
[101] Manish Sinha, Luke E.K. Achenie, and Gennadi M. Ostrovsky. Environmentally benign solvent design by global optimization. Computers & Chemical Engineering, 23(10):1381-1394, 1999.
[102] Nikolaos V. Sahinidis, Mohit Tawarmalani, and Minrui Yu. Design of alternative refrigerants via global optimization. AChE Journal, 49(7):1761-1775, 2003.
[103] Lei Zhang, Stefano Cignitti, and Rafiqul Gani. Generic mathematical programming formulation and solution for computer-aided molecular design. Computers & Chemical Engineering, 78: 79–84, 2015.
[104] Lorenz C. Blum and Jean-Louis Reymond. 970 million druglike small molecules for virtual screening in the chemical universe database gdb-13. Journal of the American Chemical Society, 131(25):8732–8733, 2009.
[105] Matthias Rupp, Alexandre Tkatchenko, Klaus-Robert Müller, and O. Anatole Von Lilienfeld. Fast and accurate modeling of molecular atomization energies with machine learning. Physical Review Letters, 108(5):058301, 2012.
[106] Lars Ruddigkeit, Ruud Van Deursen, Lorenz C. Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of Chemical Information and Modeling, 52(11):2864–2875, 2012.
[107] Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and O. Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1(1):1-7, 2014.
A Theoretical guarantee: supplementary
A.1 Proofs of properties and lemma
Property 1. For any  $s = 1,2,\ldots ,N - 1,\mathcal{I}^s (v)\left\{ \begin{array}{ll} <   s, & \forall v\in V_1^s\\ \geq s, & \forall v\in V_2^s. \end{array} \right.$
Proof. At the  $s$ -th iteration, nodes in  $V_1^s$  have been indexed by  $0, 1, \ldots, s - 1$ . Therefore, if  $v \in V_1^s$ , then  $\mathcal{I}^s(v) = \mathcal{I}(v) < s$ . If  $v \in V_2^s$ , since  $\mathcal{I}^s(v)$  is the sum of  $s$  and  $\text{rank}^s(v)$  (which is non-negative), then we have  $\mathcal{I}^s(v) \geq s$ .
Proof. First,  $\mathcal{N}^{s_1}(\cdot)$  is well-defined on  $V_{2}^{s_{2}}$  since  $V_{2}^{s_{2}}\subset V_{2}^{s_{1}}$ . By definitions of  $\mathcal{N}^s (\cdot)$  and  $V_{1}^{s}$ , for any  $v\in V_2^{s_2}$  we have: