Paper: Graph2Tac: Learning hierarchical representations of math concepts in theorem proving (ICLR-2024)

Reviewer comment (Experiments):
K-NN outperforms all pure G2T variants in terms of pass rate/sec for the most part; as the author pointed out, maybe for more complicated tasks G2T will have an advantage, but it is not sufficiently demonstrated in the experiments.

Author response:
**On the empirical significance of our work**

As we mention in the common reply to all reviewers,
the k-NN solver was actually among state of the art in this field, better than existing works in this field.
We are the first paper to systematically compare it to other approaches and point this out.
We provide evidence that Graph2Tac outperforms any other neural or symbolic solvers available for Coq.

The k-NN is a strong approach with two advantages that many other solvers don't have.
Each call is fast, allowing it to explore an order of magnitude more states in a given time
(see Figure 9 in Appendix G _The Relative Speed of the Models_).
Also, it incorporates online proof information not seen during training
(see Figure 1).
Our G2T model not only rivals the state of the art k-NN despite being slower,
it is also a complementary approach to the online setting
of working in a package not seen during training.

Also, we see the main significance of our work is the introduction of a novel method that learns from the entire hierarchy of mathematical objects,
especially in an online setting. Our definition task improves G2T from 17.4% to 26.1% which is a 1.5x increase in theorems proved.

Reviewer comment (Experiments):
The only method that outperforms k-NN is the hybrid one G2T+k-NN, which cannot prove that G2T is a superior method as it is difficult to tell the contribution apart.

Author response:
**On the empirical significance of our work**

As we mention in the common reply to all reviewers,
the k-NN solver was actually among state of the art in this field, better than existing works in this field.
We are the first paper to systematically compare it to other approaches and point this out.
We provide evidence that Graph2Tac outperforms any other neural or symbolic solvers available for Coq.

The k-NN is a strong approach with two advantages that many other solvers don't have.
Each call is fast, allowing it to explore an order of magnitude more states in a given time
(see Figure 9 in Appendix G _The Relative Speed of the Models_).
Also, it incorporates online proof information not seen during training
(see Figure 1).
Our G2T model not only rivals the state of the art k-NN despite being slower,
it is also a complementary approach to the online setting
of working in a package not seen during training.

Also, we see the main significance of our work is the introduction of a novel method that learns from the entire hierarchy of mathematical objects,
especially in an online setting. Our definition task improves G2T from 17.4% to 26.1% which is a 1.5x increase in theorems proved.

Reviewer comment (Evaluation):
The empirical significance of this work is minor.

Author response:
**On the empirical significance of our work**

As we mention in the common reply to all reviewers,
the k-NN solver was actually among state of the art in this field, better than existing works in this field.
We are the first paper to systematically compare it to other approaches and point this out.
We provide evidence that Graph2Tac outperforms any other neural or symbolic solvers available for Coq.

The k-NN is a strong approach with two advantages that many other solvers don't have.
Each call is fast, allowing it to explore an order of magnitude more states in a given time
(see Figure 9 in Appendix G _The Relative Speed of the Models_).
Also, it incorporates online proof information not seen during training
(see Figure 1).
Our G2T model not only rivals the state of the art k-NN despite being slower,
it is also a complementary approach to the online setting
of working in a package not seen during training.

Also, we see the main significance of our work is the introduction of a novel method that learns from the entire hierarchy of mathematical objects,
especially in an online setting. Our definition task improves G2T from 17.4% to 26.1% which is a 1.5x increase in theorems proved.

Reviewer comment (Reproducibility):
It is unclear how an unknown definition is encoded and predicted without a text encoder. From section 3.1, definition nodes seen in the training set all get assigned an embedding, but when an unseen definition comes, there is supposedly no embedding in the table that matches this definition; it is unclear how it is encoded into an embedding in the first place.

Author response:
**On the definition task**

As for the definition task, we think there are some misunderstandings,
which we will try to make more clear in the paper:

> It is unclear to me how unknown definition is encoded and predicted without a text encoder.

For each new definition we have its definition body.
We uses graphs instead of text to encode new definitions, but otherwise it is similar to a text encoder.
The (graph of) the definition is put into our graph model, and the new definition embedding is the (pooled) output vector.
Graphs also let us build hierarchical representations, where the representation of a definition is
built from the embeddings of the component definitions.

When predicting a definition as an argument to a tactic, we select it from among all possible arguments in the global context.
This attention-like mechanism is step G in Figure 4.
(We will rename constant embeddings to definition embeddings in the Figure to be more clear.)

> Judging from 3.1, definition nodes that are seen in the training set all get assigned with an embedding,
> then if an unseen definition comes, supposedly there is no embedding in the table that matches this definition,
> then how does one encode it into an embedding in the first place?

Definitions seen during training use embeddings learned during training,
similar to how, say, a transformer model learns vocabulary embeddings.
Our definition task trains a graph model which can predict those learned embeddings.
If we encounter a new definition not seen during training, 
we use our definition graph model to predict its embedding.

Reviewer comment (Experiments):
The definition task seems essential for learning to represent unknown definitions; for the G2T-NoDef model, it is unclear how unseen definitions are handled if there was no definition task.

Author response:
**On the definition task**

As for the definition task, we think there are some misunderstandings,
which we will try to make more clear in the paper:

> Furthermore, it seems to me the definition task is essential for learning to represent unknown definitions,
> then for the G2T-NoDef model, how is unseen definition handled if there was not a definition task?

The G2T-NoDef model uses learned embeddings for definitions seen during training.
At inference, any new embeddings get assigned random vectors (the `-Frozen` setting described in Section 3.1).

Reviewer comment (Presentation):
It is difficult to comprehend the diagrams in Fig 2, 3, and 4. For example, while the caption says Fig 2 is the graph representation of Fig 1, it is difficult to see the connections: what do "@" and "up_arrow" correspond to? What do different colors mean in the graph? How is online update reflected in the graph representation? Why are there triangles with no symbols?

Author response:
**Regarding clarifications of the figures**

We agree that Figure 2 could use more explanations. Unfortunately, the page limit prevents us from adding this. 
However, the graph dataset has been extensively
documented in a separate publication. An anonymized version can be found
[here](http://64.71.146.254:8000/SFyud_C5TmEq7AwnK9jaLsfTFMzgBl54cQ0pl2FJB-x9o2Hk24F4jO_W75RqGdOJ/papers/web-paper.pdf).
Regarding Figures 3 and 4, we agree they are complex.  We will attempt to clean them up and align them more with the text.
To answer your questions:
The edge and node embeddings are used in step A to initialize the embeddings of the graphs,
but we don't see how to put them in the figure.
We will relabel "constant embeddings" to be "definition embeddings" to match the text.
The entry point of the definition graph is the node labeled `f`.  Do you think we should label that?
It is unclear to us how to improve Figure 3.  Do you have suggestions on what could make the figures more clear?

Reviewer comment (Reproducibility):
Section 3.1 misses formal definitions of many important components, making it difficult to track the architecture. Examples include:
- The RNN for local and global argument prediction
- The loss for definition and prediction task
- The formal representation of how new definitions are added to the current state, and how they are calculated into an embedding using the definition task.

Author response:
**More technical details in 3.1**

We will clean up Section 3.1 and add more details (to that section or the appendix).

In Appendix P: *Further details on the G2T architecture* we add more details about the model architecture including more descriptions about the graph inputs going into the model and the explicit formulas for the calculations.  We color coded it in dark red.

We also fixed a number of small inconsistencies between the images and the text in Section 3.1 describing the model.  For example, we renamed entry point nodes to root nodes, and in the text pointed out where to find them in Figure 4.  We also explicitly labeled the definition embedding table in Figures 3 and 4.  It will hopefully be easier to follow.

Reviewer comment (Presentation):
Fig 4 is not very helpful; notations and names do not match very well with the text: what is "const embs"? Are they trainable node labels? Which part corresponds to edge embeddings? Which parts correspond to "entry-point" nodes and "local context" nodes?

Author response:
**Regarding clarifications of the figures**

Regarding Figures 3 and 4, we agree they are complex.  We will attempt to clean them up and align them more with the text.
To answer your questions:
The edge and node embeddings are used in step A to initialize the embeddings of the graphs,
but we don't see how to put them in the figure.
We will relabel "constant embeddings" to be "definition embeddings" to match the text.
The entry point of the definition graph is the node labeled `f`.  Do you think we should label that?
It is unclear to us how to improve Figure 3.  Do you have suggestions on what could make the figures more clear?

In Appendix P: *Further details on the G2T architecture* we add more details about the model architecture including more descriptions about the graph inputs going into the model and the explicit formulas for the calculations.  We color coded it in dark red.

We also fixed a number of small inconsistencies between the images and the text in Section 3.1 describing the model.  For example, we renamed entry point nodes to root nodes, and in the text pointed out where to find them in Figure 4.  We also explicitly labeled the definition embedding table in Figures 3 and 4.  It will hopefully be easier to follow.

Reviewer comment (Evaluation):
It is unclear that the proposed technique is superior to others. For example, in Figure 7, most of the time k-NN performs better.

Author response:
As for performance of our G2T model versus the k-NN, see the common response to all the reviewers.
Our G2T rivals the already state-of-the-art k-NN, suggesting our model is also state-of-the-art.

Reviewer comment (Experiments):
Authors show that their technique in combination with k-NN is superior to others. However, what if we combine some other technique with k-NN?

Author response:
> But, what if we combine some other technique with k-NN?

This is highlighted in the Venn diagram in Figure 6.
The two approaches which are most disjoint are the k-NN and G2T solvers.
Nonetheless, we are happy to include an appendix highlighting additional combinations of models.
This will be added to a future revision of the paper.

Reviewer comment (Evaluation):
Have you noticed some patterns in theorems solved by different systems? You show a Venn diagram and note that different systems solve significantly different sets of theorems.

Author response:
> Have you noticed some patterns in theorems solved by different systems?
> You show a Venn diagram and note that different systems solve significantly different sets of theorems.

This is partially highlighted in Figure 7 where different approaches perform better on different packages.
Further in Appendix L _Impact of the Online Setting_ we show the significance of the online setting.
When there are more dependencies the k-NN and G2T solvers show their strength.
Time permitting, we will try to look into this more, but we also plan to release our theorem-level results,
so that others can also perform this analysis without having to rerun the benchmarks.

Reviewer comment (Evaluation):
Can you show some visualization of embeddings produced by your GNNs? Are there some interesting relations, groups, etc., in the embedding space?

Author response:
> Can you show some visualization of embeddings produced by your GNNs?
> Are there some interesting relations, groups, etc, in the embedding space?

Thank you for the suggestion.
We are currently doing an analysis with UMAP (similar to t-SNE) embeddings.
We will include it in the paper when complete.
Preliminary findings show that there are strong patterns in the embeddings.
The embeddings using the definition task are more structured and interpretable.

Reviewer comment (Experiments):
Can you run your tool on CoqGym, or is it not compatible?

Author response:
> Can you run your tool on CoqGym, or is it not compatible?

In the new Appendix B _Informal comparison on CoqGym test packages_ we make an informal comparison of our models to CoqGym,
which suggests the k-NN and G2T models are state of the art for Coq in the online setting
of a new project not seen during training.

Reviewer comment (Novelty):
The idea of using graph neural networks for automating interactive theorem proving is not new and the improvement is relatively minor. The general idea seems fairly incremental, however, on the other hand, the model architecture is quite complicated, consisting of GNN, several MLPs, argument RNNs, as well as bidirectional LSTM. It is difficult to see the effectiveness of different parts (e.g., are they all essential? Is GNN the most important component?).

Author response:
**Response regarding incrementality and novelty**

We believe that our work provides strong novelty in that we are one of the first
that learns from the entire hierarchy of mathematical objects, especially in an
online setting. Our definition task improves G2T from 17.4% to 26.1% which is a
1.5x increase in theorems proved.

As we mention in the common reply to all reviewers,
the k-NN solver was actually among state of the art in this field, better than existing works in this field.
We are the first paper to systematically compare it to other approaches and point this out.

**Response regarding concerns regarding architectural complexity**

Much of the complexity is needed for tactic prediction.
Since we do not use a language model to generate tactics,
we need to predict both a base tactic, e.g. `rewrite _ in _`
and a sequence of arguments to go with that tactic,
which is why we need the RNN (or some other sequence model).
As for the bidirectional LSTM,
it is only used to embed definition names in the G2T-Named model.
We indeed show in our ablations that it does not improve results
(both in the experimental results
and in Appendix H _Ablation of G2T solvers and symmetry breaking with names_).

Reviewer comment (Experiments):
The title and abstract highlight learning hierarchical representations, however, there is no evidence supporting that hierarchical representations really make a big difference.

Author response:
**Response regarding lack of evidence for effectiveness of hierarchical representations**

We didn't have an opportunity to explore in great detail the importance
of dependency depth in our hierarchical representations,
and we agree that would be interesting follow-up work.
Nonetheless, our ablations show that the hierarchical definition embedding task improves the G2T model from 17.4% to 26.1%,
highlighting the power of calculating new definitions embeddings from their definitions.
Further, in Figure 15, Appendix L _Impact of the Online Setting_,
we show that using the definition task doubles the pass rate if there are at least 10 new
dependencies in the global context.
This advantage increases as the number of unseen dependencies increases.
If your comment pertains to the fact that we are using learned embeddings for
definitions seen during training and computed hierarchical embeddings for definitions
not seen during training,
note that the learned definitions are trained to resemble the computed hierarchical embeddings
(and vice versa).
Additionally, we perform experiments
in Appendix H _Ablation of G2T solvers and symmetry breaking with names_
where for inference we recompute all embeddings to use the computed hierarchical embeddings.

Reviewer comment (Novelty):
The improvement of Graph2Tac is not very significant, although combining Graph2Tac and k-NN by running them in parallel does give some interesting improvements, which is more like a simple engineering solution rather than a technical one.

Author response:
**Response regarding incrementality and novelty**

We believe that our work provides strong novelty in that we are one of the first
that learns from the entire hierarchy of mathematical objects, especially in an
online setting. Our definition task improves G2T from 17.4% to 26.1% which is a
1.5x increase in theorems proved.

As we mention in the common reply to all reviewers,
the k-NN solver was actually among state of the art in this field, better than existing works in this field.
We are the first paper to systematically compare it to other approaches and point this out.

Reviewer comment (Experiments):
Can you elaborate a bit more on the objective of the definition task? Shouldn't it be a (separate) pre-training task for learning good representations?

Author response:
> Can you elaborate a bit more on the objective of the definition task?
> Shouldn't it be a (separate) pre-training task for learning good representations?

The definition task serves the purpose of aligning the learned definitions during training to the calculated definitions.
This way, during inference, if we encounter a new definition we can compute its embedding.
As for pretraining, would could have alternately learned our definition embedding computation as a pre-training task
via contrastive or non-contrastive self-supervised methods (like SimCLR, BYOL, VICReg, etc.).
However, by training them along with the final tactic prediction task it gives us a few advantages:
1. It lets us use learned embeddings when available and only swap to calculated embeddings for new definitions.
2. It ensures our calculated embedding is useful for the downstream task.
3. It prevents mode collapse where all embeddings become the same (since the embeddings have to be useful for the downstream task).
4. It allows us to share the weights of our GNN backbone on both the definition task and prediction task.
5. Since definitions are hierarchical (one definition depends on many previous ones),
   we need to store (or constantly recompute) embeddings of all intermediate definitions.
   (This is unique to our setting and not found in most other self-supervised learning papers.)
   The learned definition embedding table serves as a way to store and update intermediate embeddings.

Reviewer comment (Reproducibility):
Having an embedding for each definition might be too expensive. What is the size of the embedding table for definitions?

Author response:
> Having an embedding for each definition might be too expensive.
> What is the size of the embedding table for definitions?

Our table is indeed large, with about 459k entries (about an order of magnitude larger than the vocabulary table in small language model).
However, during argument selection we mask out all definition embeddings not in the current global context, significantly reducing the table size. Empirically, our approach is both fast and effective.
In Appendix H _Ablation of G2T solvers and symmetry breaking with names_,
we do show that if we used the G2T-Named model,
we could replace all the learned embeddings with calculated embeddings,
reducing the size of our model.
Other approaches in the literature based on premise selection also must compute embeddings for every theorem,
either beforehand or on-the-fly,
and the HOList models also learned embeddings for every definition.

Reviewer comment (Presentation):
Figure 4 is hard to follow for a new reader. It may help to remove some less important steps and intermediate results to make the figure more clear and helpful.

Author response:
Thank you for you thoughtful review.

Regarding Figure 4, we will work on cleaning it up as well as unifying it better with the surrounding text.
We agree it is complex, but we also have difficultly seeing what can be removed without adding more confusion.
Is there anything in particular you would suggest changing?

Reviewer comment (Experiments):
As mentioned in the paper, this new training objective utilizes the hierarchical structure of definitions. It is interesting to see if the learned definition embeddings contain any implicit structures compared to training to generate the next steps only. For example, try to plot the definition embeddings using t-SNE.

Author response:
>  As mentioned in the paper, this new training objective utilize the hierarchical structure of definitions.
> It is interesting to see if the learning definition embeddings contain any implicit structures compared to training to generate the next steps only. For example, try to plot the definition embeddings using t-sne.

Thank you for the suggestion.
We are currently doing an analysis with UMAP (similar to t-SNE) embeddings.
We will include it in the paper when complete.
Preliminary findings show that there are significant differences in the embeddings.
The embeddings using the definition task are more structured and interpretable.