Paper: Lyra: Orchestrating Dual Correction in Automated Theorem Proving (ICLR-2024)

Reviewer comment (Novelty):
Tool correction: in the DSP work [1], the authors already used sledgehammer + heuristics to close conjectures made. The understanding is that the Lyra method first tries a LLM-generated tactic to close conjectures, and if it doesn't work, tries sledgehammer. This is largely similar and should be noted.

Author response:
A1: The part is shown on the first page of the Appendix. Because of the paper page limit, we have added the sledgehammer+heuristic description to the Appendix.

Meanwhile, compared to sledgehammer+heuristics in DSP, our proposed Tool Correction has a different motivation and implementation. For example, if “by (simp add: div mult mod eq)” fails, the DSP will immediately stop validating, while our proposed Tool Correction will try all other potential tools. With Tool Correction, we would like to present one important point: the post-process of formal proof can significantly further improve performance, and future work could try to design a more powerful post-process method. And the following is the number of fixed wrong steps via Tool Correction.

|  Dataset | Sledgehammer+heuristics   | Tool Correction  | Number of Fixed Steps|
|  ---- |  ----  | ----  | ----  |
| miniF2F-valid  |2260  | 3486 |1226 |
| miniF2F-test  |2594  | 3887 |1293 |

For more details, please kindly check our answer in  **The difference between Tool Correction and sledgehammer+heuristic** in **Author Response to All Reviewers**.

Reviewer comment (Novelty):
In the Baldur work [2] from April 2023, the authors proposed to use the proof assistant error message to repair the proofs. This is very similar to the conjecture correction with error messages and should be noted.

Author response:
A2: We have added Baldur to our reference. And the difference between Baldur and Lyra Conjection Correction is the following.
Compared to Baldur which needs a training process, Lyra Conjection Correction is training-free, and it can reset the initial solution after several failure rounds. Please refer to the more detailed answer to **The difference between Lyra and other works** in the **Author response to all reviewers**.

Reviewer comment (Experiments):
Are the presented techniques overfit to miniF2F dataset and Isabelle? In particular, I am afraid this might be the case for TC.

Author response:
**Q4: Are the presented techniques overfit to miniF2F dataset and Isabelle? In particular, I am afraid this might be the case for TC.**

A4: The presented techniques are not overfit to miniF2F or Isabelle.

For Tool Correction, 
* We can easily adapt it to other provers, such as Lean. And this is illustrated in **Q2**.
* Also, Tool Correction implementation is dataset-agnostic, and it is not related to any datasets, including miniF2F.

For Conjecture Correction,
* Conjecture Correction needs formal proof and the corresponding error message, which can come from Isabelle (in this work) and other provers (such as Lean)
* Conjecture Correction is also not overfit to miniF2F, because its implementation is dataset-agnostic.

Reviewer comment (Experiments):
Would CC work with the error messages from a different ITP? How much does the availability of formal proofs and error messages for a particular ITP affect the effectiveness of CC? How much does the quality of the error message affect CC?

Author response:
**Q7: Would CC work with the error messages from a different ITP?**
 
A7: Yes, the CC can work with the error message from an ITP.
 
The following is the requirement of Conjecture Correction.
* The pair of (input, output). For DSP [1], Subgoa-Learning [2] and Lyra, the input is (problem, informal proof), and the output is (formal proof).
* The feedback from the ITP. Given formal proof, the ITP should be able to validate and give feedback, such as error messages.
* The refinement instruction. The refine instruction is employed to ask the LLM to refine the previous response. The refinement instruction is shown in Algorithm 2.
 
Moreover, we emphasize that Conjecture Correction can be an easy framework that can be extended to other tasks. It refines response with instruction and error messages but does not need several pairs of (input&error, output). Our work follows the previous experiment setting, such as DSP [1] and Subgoal-Learning [2], validating the proposed methods with Isabelle Prover on the miniF2F dataset.
 
**Q8: How much does the availability of formal proofs and error messages for a particular ITP affect the effectiveness of CC?**

A8: Excluding the first round, there is always a formal proof and the corresponding error message from a particular ITP. The proposed Conjecture Correction needs both previous formal proof and error messages from a particular ITP.
 
The Conjecture Correction process is the following:
* In the first round, LLM generates a formal proof (just like DSP [1], without an error message), and then sends it for prover validation to get error messages.
* In the following round, Conjecture Correction combines previous formal proof and error message to generate a better formal proof and then sends it for prover validation to get error messages. Conjecture Correction repeats the process until one formal proof passes the prover validation.
* Therefore, there always exists a formal proof and an error message, excluding the first round. And Conjecture Correction needs both a formal proof and an error message.

**Q9: How much does the quality of the error message affect CC?**

A9: Raw error message from Isabelle Prover is enough for Conjecture Correction.
 
In our experiment setting, we directly utilize the raw error message from Prover. The experiments have shown that raw error message is enough to improve performance. Future work may discuss how to post-process the raw error message from the prover for LLM to better understand.

Reviewer comment (Novelty):
Although I very much like the idea of CC, the innovation of TC appears slightly limited. At least to me, it does not involve much interaction with LLMs -- it is more like an exhaustive attempt on a set of heuristically chosen proof methods other than Sledgehammer.

Author response:
**Q1: the innovation of TC appears slightly limited.**

A1: TC implementation reveals an important point: post-process of formal proof can significantly further improve performance, which does not get enough attention in previous work. Future work can explore how to design better post-process rules or techniques to improve the formal proof quality. And this is one important point that we want to present and prove via Tool Correction. For more details, please refer to **The difference between Tool Correction and sledgehammer+heuristic** in **Author Response to All Reviewers**.

Reviewer comment (Experiments):
Page 1, 'However, they have not been able to post-process LLM generation or gradually refine previous generations.': The Baldur paper (https://arxiv.org/abs/2303.04910) has explored post-processing LLM-generated proofs. I would love to see a comparison between Baldur and Lyra if possible.

Author response:
**Q2: I would love to see a comparison between Baldur and Lyra if possible.**
 
A2: The following is our comparison between Baldur and Lyra. We have clarified this in our general response to all reviewers. Please kindly check our answer in **The difference between Lyra and other works** in **Author Response to All Reviewers**.

|  Method   | Training-Free  |Information Besides Response and Error  |Reset Initial solution| Control Indicator |
|  ----  | ----  | ----  | ----  | ----  | 
| Baldur  |&cross;  |&cross;  |&cross;  |&cross;  |
| Lyra  | &check; |&check; |&check; |&check; |

Compared to Baldur which needs a training process, the proposed Lyra is training-free. As Baldur does not release code or conduct experiments on miniF2F, currently we cannot directly compare their performance. However, we can compare Baldur, Thor [1] and Lyra to draw a conclusion between Baldur and Lyra.

|  Method   | AFP-Computer Science  | AFP-Logic  | AFP-Mathematics  | AFP-Tools|  miniF2F-valid|miniF2F-test| 
|  ----  | ----  | ----  | ----  | ----  | ----  | ----  |
| Baldur  | 50.0%  | 51.6%  | 41.9%  |53.9%  |-  |-  |
| Thor [1]  |57.5% | 53.6% |  50.5%  |51.8%  |28.3%  |29.9%  |
| Lyra  |- | - | -  |-  |55.3%  |51.2%  |

Fortunately, we can 1) first compare Baldur with Thor [1] on AFP topic classification; 2) then compare Thor with Lyra on miniF2F; 3) finally, conclude: that Lyra is significantly better than Thor, while Thor is better or comparable to Baldur.

Reviewer comment (Experiments):
This paper is mostly based on the previous DSP paper, where LLMs are used to produce proof skeletons (i.e., unproved conjectures). In the tool correction part, it appears that LLMs are prompted to produce a full mechanised proof including the tactic 'by (simp add: div mult mod eq)', which is considered as LLM hallucination by the authors. Is that the case? If so, it might be a good idea to make the distinction clear as this may affect the ablation study.

Author response:
**Q3: In the tool correction part, it appears that LLMs are prompted to produce a full mechanised proof including the tactic 'by (simp add: div mult mod eq)', which is considered as LLM hallucination by the authors. Is that the case? If so, it might be a good idea to make the distinction clear as this may affect the ablation study.**
 
A3: Yes, that is the case. And we update the following to make the distinction clear.

* Page 1: “ As shown in the observation in Figure 1, prover fails to prove conjecture x = 19 ∗ (x div 19) + 4 because LLM wrongly believes that by (simp add: div mult mod eq) can prove x = 19 ∗ (x div 19) + 4” —>  “ As shown in the observation in Figure 1, prover fails to prove conjecture x = 19 ∗ (x div 19) + 4 because by (simp add: div mult mod eq) generated by LLM cannot prove x = 19 ∗ (x div 19) + 4 (considered as LLM hallucination)”
* Figure 1: “The prover fails because LLM wrongly believes that by (simp add: div mult mod eq) can prove x = 19∗(x div 19)+4” —> “The prover fails because by (simp add: div mult mod eq) generated by LLM cannot prove x = 19∗(x div 19)+4, which is considered as the hallucination.” 
* Moreover, we have highlighted that this is the LLM hallucination in our paper in Page 4: “For instance, consider the statement x = 19 ∗ (x div 19) + 4, where LLM proposes to utilize the tactic by (simp add: div mult mod eq), leading to failure. This is the LLM hallucination…”

Reviewer comment (Theory):
Figure 3, the informal proof and the formal sketch are actually quite different. For example, the formal one does not cover continuity nor limit, which have been mentioned several times in the informal proofs. I was wondering if the authors could elaborate a bit on the discrepancy between the informal proof and the formal one.

Author response:
**Q4: Informal proof and the formal sketch are quite different. Elaborate a bit on the discrepancy between the informal proof and the formal one.**
 
A4: The informal proof is a guide to formal proof, and it is not necessary that the informal proof steps and formal proof steps are one-to-one. The detailed explanation is the following.
* Why Informal proof and the formal sketch are quite different? $Reason$: The informal proof is a guide to formal proof generation. According to the original DSP paper, the informal proof only needs to be useful for producing a sketch in the next stage.
* Why the formal one does not cover continuity or limit? $Reason$: Actually, the formal one covers the limit. For example, the formal proof has shown that "ultimately have "1 < ?S" by simp [ATPWithTC]", which covers the limit. The limit is proved via the ATP and Tool Correction.

Reviewer comment (Writing):
Page 3, 'conducted on LLLMs' -> 'conducted on LLMs'.

Author response:
**Q5: 'conducted on LLLMs' -> 'conducted on LLMs'**
 
A5: Thank you very much for your notice. We have fixed the typos.

Reviewer comment (Theory):
Page 5, 'As all formal proof begins with proof -': strictly speaking this is not quite true, as some Isabelle proofs start with 'proof (...)' or 'apply (...)', where ... can be some Isabelle tactics.

Author response:
**Q6: Some Isabelle proofs start with 'proof (...)' or 'apply (...)'**
 
A6: Thank you very much for your notice. The following is the new description: As most formal proofs of our prompts begin with “proof -”, we add “proof -” at the end of the instruction so that the LLM response is formal proof.
 
If you think our comment and update address your concerns, could you please consider raising your rating of the paper? Thank you!

Reviewer comment (Novelty):
The paper doesn't have much novelty in terms of the approach. It seems that the set of 11 tactics in TC have been proposed in DSP.

Author response:
**Q1: novelty in terms of the approach.**

A1: For Tool Correction, it provides this insight: post-process of formal proof can significantly further improve performance. Please refer to the **The difference between Tool Correction and sledgehammer+heuristic** in **Author Response to All Reviewers**. 

And, we also show the **The difference between Lyra and other works** in **Author Response to All Reviewers** to present the innovation of Conjecture Correction.

Reviewer comment (Experiments):
I think one good baseline would be replace Minerva and Codex in DSP with GPT4. So we still have GPT4 to generate the proof sketch (the intermediate conjectures) but use 11 tactics + sledgehammer to close the open goals.

Author response:
**Q2: I think one good baseline would be replace Minerva and Codex in DSP with GPT4. So we still have GPT4 to generate the proof sketch (the intermediate conjectures) but use 11 tactics + sledgehammer to close the open goals.**
 
A2: We have shown the result in Table 2 in our paper. Our proposed Lyra degrades to DSP if removes both Tool Correction and Conjecture Correction. That is, $DSP+Tool Correction + Conjecture Correction = Lyra$, while the original DSP has sledgehammer + 11 tactics  implementation (only works when the given tactic is "sledgehammer"), achieving 50.4% on miniF2F-valid and 42.6% on miniF2F-test. 

|  Method   | Model |TC  | CC  | miniF2F-valid|miniF2F-test|
|  ----  | ----  | ----  | ----  | ----  | ----  |
| DSP  | GPT-4|&cross; |&cross; |50.4% |42.6% |
| DSP+TC  | GPT-4|&check; |&cross; |52.8% |45.9% |
| DSP+CC  | GPT-4|&cross; |&check; |46.7% | 43.0% |
| DSP+TC+CC (Lyra)  | GPT-4|&check; |&check; |55.3% |51.2% |

Reviewer comment (Evaluation):
Could you calculate the number of wrong proof steps fixed by each tactic in TC?

Author response:
**Q3: Could you calculate the number of wrong proof steps fixed by each tactic in TC?**
 
A3: The following is the number of wrong proof steps fixed by each tactic in TC.
* Proof comes from: the miniF2F validation set (pass rate 55.3%) and test set result (pass rate 51.2%), with GPT-4, human informal proof, Conjecture Correction and Tool Correction. 
* The definition of proof step: a proof step is regarded as a tactic.
* Calculation protocol: if sledgehammer+heuristic or Tool Correction fails to validate the current tactic, then the current proving process will be terminated and we will turn to the next formal proof validation. Finally, we calculate how many correct tactics/proof steps.

|  Dataset | Sledgehammer+heuristics   | Tool Correction  | Number of Fixed Steps|
|  ---- |  ----  | ----  | ----  |
| miniF2F-valid  |2260  | 3486 |1226 |
| miniF2F-test  |2594  | 3887 |1293 |

On miniF2F-valid, sledgehammer+heuristics can help the prover successfully pass 2260 steps. After adding Tool Correction, the number increases to 3486 steps. Therefore, Tool Correction fixes 1226 wrong steps. On miniF2F-test, Tool Correction fixes 1293 wrong steps