Paper: Empowering Active Learning for 3D Molecular Graphs with Geometric Graph Isomorphism (ICLR-2024)

Reviewer comment (Theory):
One aspect that appears to be lacking in the method section is a detailed discussion on the motivation behind choosing the specific three 3D isometrics proposed. While it’s evident that these constitute a complete isometry space, there exist various alternative approaches and descriptors to delineate the isometry space, such as using the first and second fundamental forms. Enhancing the section with additional references and a more thorough discussion that delves into the rationale behind the selection of these particular 3D isometrics would significantly bolster the clarity and comprehensiveness of the presentation.

Author response:
Thank you for your insightful feedback! **Based on the suggestion, we added a paragraph at the end of Sec 2.1.1 to provide a detailed rationale for developing the three 3D isometrics.**
In short,  the three 3D isometrics - Reference Distance, Triangular, and Cross Angular - were developed for `their natural use in molecular shape analysis and high computational efficiency`.

- First, our objective is to develop an accurate method for molecular diversity computing.
Existing mainstream paradigms for molecular diversity, like USR and Gaussian overlay-based methods (references see Related Work Sec 3.2 in paper), are all based on geometries (distances, angles) in Euclidean space. Through analysis in Sec 2.1.1 and 2.1.2, we can easily see our methods will render several geometries, including reference distances, triangles, and cross angles. Thus, our three 3D isometrics can be **naturally and easily integrated into existing pipelines for accurate molecular diversity computing**.

- Second, our method is more computationally efficient.
  - The mentioned *first and second fundamental forms* indeed provide a detailed capture of a surface's intrinsic and extrinsic geometries. By definition, the first fundamental form is the inner product on the tangent space of a surface, and the second fundamental form is a quadratic form on the tangent plane of a surface. However, they require the computation of differential properties such as gradients and curvatures from a point cloud, which is a discrete representation of a molecule's structure. When applied to non-smooth surfaces, such as molecular point clouds, this can lead to a substantial increase in computational effort. For instance, to approximate these forms, one might have to fit a local surface at every point and compute its curvature using second-order derivatives. If this process involves examining the interactions of every triplet of points to fit the surface, the complexity can escalate to $O(N^3)$ ($N$ is the number of points), which becomes computationally intensive for large molecules.

  - In contrast, the three isometrics we employ are designed for direct computation from point cloud data. The **Reference Distance Isometric** and **Triangular Isometric** both have a computational complexity of $O(N)$, as they involve operations over $N-1$ points. The **Cross Angular Isometric** has a complexity of $O(N^2)$, considering the angles between all pairs of points, which, while higher, is still significantly more tractable than $O(N^3)$.

  - In short, our approach avoids the computational burden, such that we maintain a tradeoff between descriptive power and computational feasibility for complete isometry space. This is crucial for active learning tasks, where high-throughput and computational speed are also paramount concerns.

Reviewer comment (Writing):
The paper keeps using "2D graph/molecule" to refer to "planar graph/molecule" or "topological graph". It is not quite concise, as "2D" already indicates geometric information is considered.

Author response:
Thanks for the value point. We agree that the word "2-dimensional" somehow infers geometric information. We think "planar graph/molecule" is truly precise, thus we changed "2D graph" to "planar graph" and  "2D molecule" to "planar molecule" in the revised version.

Reviewer comment (Writing):
\mathcal{SE}(3) -> $SE(3)$.

Author response:
Thanks for pointing this out. We have changed accordingly in the paper.

Reviewer comment (Reproducibility):
Page 5 Section 2.3, what is the specific "off-the-shelf QP solver" being used? It is important to reveal the info here since different frameworks can result in very different computational speeds, as is already demonstrated in the experimental section. For example, how do you project the continuous solution to the binary space, are you using branch-and-bound?

Author response:
We agree that the computational speed of different QP Solvers varies.
In this work, the Operator Splitting Quadratic Program (OSQP) [1] is used to solve the QP problem in Eq (4). We then employ a greedy approach to project the continuous solution back to the binary space, where the $k$ highest entries of the continuous solution vector are set to $1$ and the remaining to $0$. Such an approach is commonly used to convert continuous solutions obtained from a QP solver to binary solutions in AL research [2, 3].

**We have updated section 2.3 to include this text and the references to address your comment.**

[1] B. Stellato, G. Banjac, P. Goulart, A. Bemporad, and S. Boyd. OSQP: an operator splitting solver for quadratic programs. Mathematical Programming Computation, 12(4):637-672, 2020. doi: 10.1007/
s12532-020-00179-2. URL https://doi.org/10.1007/s12532-020-00179-2

[2] Rita Chattopadhyay, Wei Fan, Ian Davidson, Sethuraman Panchanathan, and Jieping Ye. Joint transfer and batch-mode active learning. In International Conference on Machine Learning (ICML), 2013.

[3] Zheng Wang and Jieping Ye. Querying discriminative and representative samples for batch mode active learning. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013.

Reviewer comment (Writing):
In the first paragraph in Sec 1, the 3D graph usually does not include chemical bonds.

Author response:
Thank you for pointing this out. Yes, you are right. Most existing 3D datasets do not contain information of chemical bonds. Based on the research purpose, different research works use different ways to determine the connections among atoms (if two atoms are connected or not). In most existing 3D GNN studies (SchNet, DimeNet, SphereNet, etc), when constructing 3D graphs, the connectivities among atoms are usually based on a predefined `cut-off distance`. For example, if we set the cut-off distance to be 6A as a threshold, then if the Euclidean distance between two atoms is smaller than or equal to 6A, there will an an edge between them in the 3D graph. Usually, edge information is critical as the message will be passing through edges (there is no message passing between two nodes if there is no edge between them).

**We have updated the first paragraph in Sec 1 to clarify this.**

Reviewer comment (Theory):
In the third paragraph in Sec 1, it should be SE(3)-equivariant, not SE(3)-invariant. If the model is SE(3)-invariant (as it is now), then the claim that “can be applied in conjunction with any 3D GNN architecture” is not correct because there are equivariant models not included in invariant modeling.

Author response:
Thank you for the valuable suggestion. The output of our proposed diversity computing module is a set of geometries, like distances and angles. Hence, our module can be incorporated into any existing equivariant geometric model (SchNet, EGNN, SphereNet, GemNet, TFN, Equiformer, etc).

**We have corrected this in the third paragraph in Sec 1.**

Reviewer comment (Writing):
In Sec 2.1.1, what do the curly brackets mean in “We define A to be mapped {-Reference Distance, … } isometrically to B, if …”? The reviewer can understand this sentence without the curly brackets. More clarifications are needed here.

Author response:
We admit that these statements in the original version are confusing, so we have revised this part in the paper.

In short, the statement *“There needs to exist a bijective function $f: A \rightarrow B$, such that $h_{f(a)} = h_a$ for any node $a \in A$.”* is a **common and minimal requirement** for all three isometrics. Each isometric then has **additional constraints**.

Specifically, we define three isometrics, including Reference Distance Isometric, Triangular Isometric, and Cross Angular Isometric. Each of the three isometrics is associated with several constraints. However, the above statement regarding a mapping $f$ is common for all three isometrics. You can also easily find that, in the definition of each isometric, we need such a mapping $f$. In summary, the above statement is the **common and minimal** requirement for all three isometrics. Hope the revised version is much clearer, and please let us know if you find any further issues regarding this.

Reviewer comment (Writing):
In Sec 2.1.1, what does “together with an additional constraint for each scenario” mean? The sentence is saying, “A is isometric to B, if … s.t. …”. Then the authors say “and with additional constraint for two scenarios: (1) If …, s.t. …; (2) If …, s.t. …; (3) If …, s.t.” The reviewer does not fully understand what these sentences mean. While they can guess the meaning from the context, such a description is not good enough and needs polishing.

Author response:
We admit that these statements in the original version are confusing, so we have revised this part in the paper.

In short, the statement *“There needs to exist a bijective function $f: A \rightarrow B$, such that $h_{f(a)} = h_a$ for any node $a \in A$.”* is a **common and minimal requirement** for all three isometrics. Each isometric then has **additional constraints**.

Specifically, we define three isometrics, including Reference Distance Isometric, Triangular Isometric, and Cross Angular Isometric. Each of the three isometrics is associated with several constraints. However, the above statement regarding a mapping $f$ is common for all three isometrics. You can also easily find that, in the definition of each isometric, we need such a mapping $f$. In summary, the above statement is the **common and minimal** requirement for all three isometrics. Hope the revised version is much clearer, and please let us know if you find any further issues regarding this.

Reviewer comment (Presentation):
Almost all the equivariant geometric models (SchNet, EGNN, SphereNet, GemNet, TFN, Equiformer, etc.) satisfy the ‘isometric’ condition. The reviewer is not sure why the authors highlight this in Sec 2.1.1 - 2.1.2. Besides, Sec 2.1.1-2.1.2 are disconnected from Sec 2.1.3 and Sec 2.2.

Author response:
Thanks for starting this insightful discussion!


- First, for the relationship between our method and existing equivariant geometric models - In short, our method is a `model-agnostic` solution by directly performing isomorphy study on 3D molecular conformations, **avoiding the need of a perfectly pretrained equivariant geometric model, as well as achieving a guaranteed upper bound of the expressiveness of all existing geometric models.**

  - The contribution of 'isometric' condition is that, we set an upper bound (with theoretical guarantee) of the expressiveness of ALL existing equivariant geometric models. Actually, we define isometrics at three levels of expressiveness. For example, a **well pretrained SchNet** (only considers distance information) is upper bounded by Reference Distance Isometric (but not Triangular Isometric or Cross Angular Isometric); a **well pretrained DimeNet** (more powerful than SchNet) is upper bounded by Triangular Isometric (but not Cross Angular Isometric).

  - The main purpose of this research is NOT to design a powerful equivariant geometric model, but to design a way to compute diversities between two 3D molecules of any size for active learning. For sure, a perfectly pretained equivariant geometric model may distinguish two molecules at a reasonably good accuracy level, but we can not guarantee this in practice. Furthermore, there could be domain shift issues (pretrained on one data and perform on another data with a different distribution) that harm the active learning performance. Hence, we design a `model-agnostic` method by directly performing an isomorphy study on 3D molecular conformations, avoiding these risks, as well as achieving a guaranteed upper bound of all existing equivariant geometric models.

  - **In the original submission, we had a paragraph at the end of Sec 2.1.2 to explain this. Based on your comments, we added more statements for further clarification.**

- Second, the relationships among several sections.

  - Regarding the relationships among Sec 2.1.1, Sec 2.1.2, and Sec 2.1.3:
  In short, through analysis and theoretical studies in **Sec 2.1.1**, we conclude $GR_{\text{ours}}$ should include all reference distances, triangles, and cross angles in a 3D graph in **Sec 2.1.2**. We also formally prove $GR_{\text{ours}}$ has greater expressive power than the GWL Test in **Sec 2.1.2**, so it can precisely capture the 3D shape diversity among different molecules. Then the final distributional representations in **Sec 2.1.3** are based on these three geometries expressed in $GR_{\text{ours}}$. **We have added several statements under Theorem 1 in Sec 2.1.2, as well as in the first paragraph of Sec 2.1.3 to clearly indicate such relationship.**

  - Regarding the relationships between Sec 2.1 and Sec 2.2: Diversity and uncertainty are two typical and different measures in active learning. **Sec 2.1 focuses on diversity and Sec 2.2 focuses on uncertainty.** These two components describe which samples should be selected from different perspectives; diversity describes how a subset can contain as much information as the original full set, and uncertainty indicates how the model is confident in a sample. Usually, combining them two would help select the most informative subset for active learning. **Our main novelty is designing new diversity and uncertainty components for 3D graphs.** The Experimental studies (especially the ablation study) also show both components contribute to the active learning performance a lot. **We have added several statements in the first paragraph of Sec 2.2 to clarify this.**