HyperTree Proof Search for Neural Theorem Proving
Anonymous Author(s)
AfﬁliationAddress email
Abstract
We propose an online training procedure for a transformer-based automated theorem prover.
Our approach leverages a new search algorithm,
HyperTree Proof
Search (HTPS), inspired by the recent success of AlphaZero.
Our model learnsfrom previous proof searches through online training, allowing it to generalizeto domains far from the training distribution.
We report detailed ablations of ourpipeline’s main components by studying performance on three environments of increasing complexity.
In particular, we show that with HTPS alone, a model trainedon annotated proofs manages to prove 65 .
4% of a held-out set of Metamath theorems, signiﬁcantly outperforming the previous state of the art of 56 .
5% by GPT-f.
Online training on these unproved theorems increases accuracy to 82 .
With asimilar computational budget, we improve the state of the art on the Lean-basedminiF2F-curriculum dataset from 31% to 42% proving accuracy.
Introduction
The nineteenth century saw the emergence of proofs so involved that they could only be veriﬁedby a handful of specialists.
This limited peer review process inevitably led to invalid proofs, withmistakes sometimes remaining undiscovered for years (e g. the erroneous proof of the Four Colour
Conjecture [1]).
Some mathematicians argue that the frontier of mathematics has reached such a levelof complexity that the traditional review process is no longer sufﬁcient, envisioning a future wherearticles are submitted with formal proofs so that the correctness can be delegated to a computer [2].
Unfortunately, very few mathematicians have adopted formal systems in their work and as of today,only a fraction of existing mathematics has been formalized.
Several obstacles have hindered thewidespread adoption of formal systems.
First, formalized mathematics are quite dissimilar fromtraditional mathematics, rather closer to source code written in a programming language, whichmakes formal systems difﬁcult to use, especially for newcomers.
Second, formalizing an existingproof still involves signiﬁcant effort and expertise (the formalization of the Kepler conjecture tookover 20 person years to complete [3]) and even seemingly simple statements sometimes remainfrustratingly challenging to formalize.
To write a formal proof, mathematicians typically work with Interactive Theorem Provers (ITPs).
The most popular ITPs provide high-level “tactics” that can be applied on an input theorem (e.g. theinitial goal) to generate a set of subgoals, with the guarantee that proving all subgoals will resultin a proof of the initial goal (reaching an empty set means the tactic solves the goal).
An exampleof a proof in Lean [4], an interactive theorem prover, is given in Figure 1 and the correspondingproof hypertree 1 is illustrated in Figure 5 of the Appendix.
A tactic, induction k , is applied on theroot goal ( n + k  m + k ) to start a proof by induction. The formal system returns two subgoals:n + 0  m + 0 (the initial case) and n + k  m + k ) n + k + 1  m + k + 1 (the induction step) As the ﬁrst subgoal is our initial hypothesis, it can be solved using the exact tactic.
To prove thesecond subgoal, we ﬁrst rewrite it using nat.succ_le_succ_iff , a theorem from the Lean librarystating that m + 1  n + 1 () m  n . The new subgoal then becomes our induction hypothesis n + k  m + k , and can again be solvedusing the exact tactic, thereby solving the last remaining subgoal and completing the proof ofthe initial statement. Starting from a goal and reducing it to subgoals until these can be solved iscommonly referred to as backward proving.
[Figure]
In this paper, we aim at creating a prover that can automatically solve input theorems by generating asequence of suitable tactics without human interaction.
The backward procedure naturally suggestsa simple approach where a machine learning model trained to map goals to tactics interacts withan ITP to build the proof of an input goal in a backward fashion.
The automated prover builds ahypergraph with the theorem to be proved as the root node, tactics as edges and subgoals as nodes.
The prover recursively expands leaves by generating tactics with our model until we ﬁnd a proof ofthe initial theorem.
A proof is then a hypertree rooted in the initial theorem whose leaves are emptysets.
Search in this setting is analog to search in two player games.
However, particular challenges arise in theorem proving.
First, the action space, i.e. the amountof possible “moves” in a given state, is inﬁnite (there is an unlimited number of tactics that can beapplied to a given theorem).
This requires sampling possible actions from a language model forwhich training data is scarce.
Moreover, if all tactics sampled at a goal fail, we have no informationon what region of the probability space to sample next.
Second, in the context of theorem proving,we need to provide a proof of all subgoals created by a tactic, whereas AlphaZero[5] for two playergames is allowed to focus on the most likely adversary moves.
Third, in Chess or Go, playing asub-optimal move does not necessarily lead to losing the game, thus exploring these branches canprovide information.
In theorem proving, it is frequent to generate tactics that result in subgoals thatcan no longer be proved and on which the model will waste signiﬁcant search budget.
This paper presents an in-depth study of our approach to overcome these difﬁculties and the resultingmodel, Evariste.
In particular, we make the following contributions:
• A new MCTS-inspired search algorithm for ﬁnding proofs in unbalanced hypergraphs.•
A new environment (Equations) to easily prototype and understand the behavior of the
models we train and our proof search.
• A detailed ablation study and analysis of the different components used in our approach on three different theorem proving environments.
We study how data is selected for training the
policy model after a successful or failed proof-search, what target should be used to train
the critic model, and the impact of online training vs. expert iteration.
• State-of-the-art performance on all analyzed environments.
In particular, our model manages to prove over 82.6% of proofs in a held-out set of theorems from set.mm in Metamath, aswell as 58.6% on miniF2F-valid [6] in Lean.
We begin by introducing related work in Section 2 and present the three theorem proving environmentsthat we study in Section 3.
Then, we present our method in Section 4, and all experimental details in
Section 5.
Finally, we describe our main results and ablation studies in Section 6.
Related
Automated theorem proving has been a long-standing goal of artiﬁcial intelligence, with the earliestwork dating from the 1950s [7, 8].
We focus here on recent work closest to ours and defer additionalrelated work to Appendix B.
Neural theorem provers.
Recent work applying deep learning methods to theorem proving [9–11]are the closest to this work and obtained impressive results on difﬁcult held-out sets for Metamathand Lean.
The main differences between their approach and ours are the proof-search algorithm wepropose, the training data we extract from proof-searches and our use of online training compared totheir expert iterations.
Another similar approach is Holophrasm [12], which is based on a differenttree exploration technique.
Unlike previous studies that focus on a single proving environment(e.g. Metamath, Lean, or HOL-Light), we extensively study the performance of our prover onthree different formal languages, and found that some conclusions signiﬁcantly vary based on theconsidered environment.
MCTS and two player games.
AlphaZero [5] demonstrated good performances on two playergames, replacing the Monte-Carlo evaluations of MCTS [13] with evaluations from a deep neuralnetwork and guiding the search with an additional deep policy.
This recent success follows extensiveliterature in search methods for two player games, notably alpha-beta search [14].
Theorem provingcan be thought of as computing game-theoretic value for positions in a min/max tree: to prove a goal,we need one move (max) that leads to subgoals that are all proven (min).
Noticing heterogeneity inthe arities of min or max nodes, we propose a search method that goes down simultaneously in allchildren of min nodes, such that every simulation could potentially result in a full proof-tree.  Proving environments In this paper, we develop and test our methods on three theorem proving environments: Metamath, Lean and Equations.
Metamath [15] comes with a database of 30 k human-written theorems calledset.mm . We also evaluate our methods on the Lean proving environment, which provides a levelof automation that is helpful to solve more complex theorems.
Lean comes with a human-writtenlibrary of 27 k theorems called Mathlib [16]. Finally, since Metamath proofs can be quite difﬁcult tounderstand and Lean requires more computing resources, we developed our own environment called Equations, restricted to proofs of mathematical identities. Its simplicity makes it ideal for prototypingand debugging.
We brieﬂy introduce these environments below.
3.1 Metamath
Metamath’s only rule is string substitution.
Starting from a theorem to be proven, variables aresubstituted until we reach axioms.
In our setup, we consider a tactic to be the label of a theorem inset.mm , along with the necessary substitutions.
For instance, to show that 2 + 2 = 4 , we can usethe rule eqtr4i which states that
A = B ^ C = B )
A = C with substitutions:
A = (2 + 2) ,
B = (2 + (1 + 1)) , and C = 4 .
We are then left with two subgoals to prove: (2 + 2) =
{+(2 + (1 + 1))and 
+}
=
(2 + (1 + 1)) .
We give the corresponding proof-tree in Figure 4 of the Appendix.
The simplicity of Metamath makes it a great test bed for our algorithms.
However, its lack ofautomation leads to larger proof sizes and its syntax and naming conventions make each step difﬁcultto interpret for neophytes.
Similar to GPT-f[9], we implement a parser for Metamath in order toautomatically prove the syntactic correctness of statements.
Moreover, we use this parser to allowgenerating only substitutions that cannot be inferred from the goal.
3.2 Lean
Lean is a full-ﬂedged programming language and beneﬁts from more powerful automation than Metamath, with tactics such as ring (able to prove goals using manipulations in semirings), norm_num(able to prove numerical goals) or linarith (able to ﬁnd contradictions in a set of inequalities).
Unlike Polu and Sutskever [9],
3.3 Equations
We developed the Equations environment as a simpler analogue to existing proving environments.
Itsexpressivity is restricted to manipulating mathematical expressions (e.g. equalities or inequalities)with simple rules (e.g. A + B = B + A , or A < B )  B <  A ).
This reduced expressivity makesgoals and tactics easy to understand, helping with interpretability and debugging: plotting the set ofgoals explored during a Metamath proof search does not give a lot of insights on whether it is on trackto ﬁnd a proof.
In Section E of the appendix, we give an in-depth presentation of this environment.
Unlike in Metamath or Lean, we do not have access to a training set of human annotated proofsfor this environment.
Instead, we create a training set composed of randomly generated synthetictheorems and their proofs (see Section E.5 for details), and manually create an out-of-domain set ofnon-trivial mathematical identities for which we do not provide proofs.
We refer to this evaluationsplit as Identities , a set of 160 mathematical expressions.
As synthetic theorems randomly generatedare much simpler and signiﬁcantly differ from statements in the Identities split, we can evaluate theability of our model to generalize to complex and out of domains data.
An example proof-tree in
Equations is shown in Figure 6 of the Appendix.
Online training from proof searches
In this section, we introduce our Hypertree Proof Search (HTPS) algorithm and describe how it isused to generate training data for our model.
We then detail our online training method.
4.1 Hypertree Proof Search
Given a main goal g to automatically prove, HTPS is the algorithm that interacts with a policymodel P ✓ and a critic model c ✓ , and the theorem proving environment to ﬁnd a proof hypertree for g .
Proof search progressively grows a hypergraph starting from g , iteratively repeating the three stepsillustrated in Figure 2: selection, expansion and back-propagation.
The main difference with othersearch algorithms is our parallel descent in all subgoals of a tactic.
A proof is found when thereexists a hypertree from the root to leaves that are empty sets.
[-v T (g 0 ) =1 × 0. N(g 0 ,t 0 ) =
-]We explain HTPS in full details in
Appendix C and compare it to other search methods in Appendix C.
[Figure]
4.2 Online training
Both the policy model P ✓ and the critic model c ✓ are encoder-decoder transformers [17] with sharedweights ✓ , which are trained online on two different objectives.
The policy model P ✓ takes as input atokenized goal and generates tactics.
It is trained with a standard seq2seq objective [18], where weminimize the cross-entropy loss of predicted tactic tokens conditioned on the input goal.
Our critic model c ✓ is used to predict ﬂoating point values representing how likely a goal is to besolved.
We start decoding with a special token, restrict the output vocabulary to two tokens PROVABLEand UNPROVABLE , and evaluate the critic with c ✓ ( g )
=
P ( PROVABLE | g, CRITIC ) .
This objective isidentical to a seq2seq objective where the cross-entropy is minimized over the two special tokens.
Our online training uses a distributed learning architecture reminiscent of AlphaZero [19] or distributed reinforcement learning setups [20, 5].
A distributed data parallel trainer receives trainingdata from a set of asynchronous provers that run proof searches on theorems chosen by a controller.
Provers, in turn, continuously retrieve the latest model versions produced by the trainers in order toimprove the quality of their proof search.
This set-up is represented in Figure 7 of the Appendix.
Once a prover ﬁnishes a proof-search, we extract two types of training samples from its hypergraph:
Tactic samples.
At the end of a successful proof search, we extract (goal, tactic) pairs of a minimalproof hypertree of the root node as training samples for the policy model.
We show that this selectionhas a large impact on performances, other options such as selecting all solved nodes are investigatedin Section 6.2.1. We use a different minimality criteria depending on the environment: number ofproof steps for Metamath and Equations and total tactic CPU time for Lean.
It is trained with astandard seq2seq objective [18], where we minimize the cross-entropy loss of predicted tactic tokensconditioned on an input goal.
Critic samples.
In the proof search hypergraph, we select all nodes that are either solved, invalid(all tactics failed or led to invalid nodes), or with a visit count higher than a threshold.
Then, we usec ( g ) = 1 as the training target for solved nodes.
For internal nodes, we use the ﬁnal estimated actionvalue c ( g ) =
W ( g, t ⇤ ) /N
( g, t ⇤ ) where t ⇤ is the tactic that maximizes the search policy at g .
Finally,for invalid nodes, we use c ( g ) = 0 .
The trainers receive training samples that are stored into two separate ﬁnite-size queues, one for eachobjective.
When a queue is full, appending a new sample discards the oldest one.
In order to create abatch for a task, we uniformly select samples in the corresponding queue.
During online training, wecontinue sampling from the supervised tasks which provide high-quality data.
All training objectivesare weighted equally.
An overwiew of our full training pipeline is given in Appendix D.1.
Our proof-search depends on many hyper-parameters, and the optimal settings might not be the samefor all statements, making tuning impractical.
Thus, the controller samples these hyper-parametersfrom pre-deﬁned ranges (see Appendix D.3 for details) for each different proof-search attempt.
Experiments
In this section, we provide details about our experimental training and evaluation protocols.
We ﬁrstdescribe the supervised datasets used to ﬁne-tune our policy models, as well as the tokenization used.
Then, we discuss the evaluation datasets and methodology.
In Appendix D.2, we provide additionaldetails about our model pretraining and architecture.
5.1 Model ﬁne-tuning and supervised datasets
Starting the HTPS procedure described in Section 4 from a randomly initialized model would besub-optimal, as no valid tactic would ever be sampled from the policy model.
Thus, starting theonline training from a non-trivial model is critical.
To this end, we ﬁrst ﬁne-tune our policy model P ✓on a supervised dataset of theorems speciﬁc to each environment.
Metamath In Metamath, we extract all proofs from the set.mm library, composed of 37091theorems (c.f. Section D.6 for the version of set.mm ).
We ﬁrst derive a graph of dependenciesbetween statements, and generate a random train-valid-test split of theorems, with 1000 valid and testtheorems.
We use the DAG to ensure that each theorem in the valid or test set is not used to proveanother theorem. Moreover, this DAG is used to build a table of forbidden tokens : if the proof of Adepends on B , we set to zero the probability of generating the token A during a proof-search of B . More details on our seq2seq training objective and co-training tasks can be found in Appendix D.4.
The training set is composed of around 1M goal-tactic pairs; more statistics about the trainingdata are provided in Table 1.
Tokenization in Metamath is trivial, as statements are composed ofspace-separated tokens.
[Table] Lean Following [11], we extract a supervised dataset from the Mathlib library. The trainingset is composed of 24k theorems and 144k goal-tactic pairs. In addition, we co-train with thedataset of proof-artifacts of Han et al.
[10] to reduce overﬁtting.
To facilitate experimentation andreproducibility, we use ﬁxed versions of Lean, Mathlib, and miniF2F (c.f. Appendix D.6).
Finally,we add another supervised co-training task by converting to Lean a synthetic dataset of theoremsgenerated by the Equations environment (c.f. Appendix E.6).
In order to avoid hooking into the
Lean parser, we tokenize goals and tactics using byte-pair encoding (BPE [21]) following previouswork [9, 11].
Statistics about the training set are available in Table 1.
Equations Unlike Metamath or Lean, the Equations environment does not come with a dataset ofmanually annotated proofs of theorems.
Instead, we generate supervised data on the ﬂy using therandom graph generator described in Appendix E.5.
As the model quickly reaches a 100% provingaccuracy on these synthetic theorems, there would be no beneﬁt in using them during online training. Thus, we ﬁne-tune on the synthetic dataset, and only leverage statements from the Identities splitduring online training.
As in Metamath, tokenization of statements for this environment is natural, aseach statement can be tokenized using the list of symbols from its preﬁx decomposition [22].
5.2 Evaluation settings and protocol
In Polu et al.
[11], the model is ﬁne-tuned on theorems from the training set and expert iteration is doneon theorems from different sources: train theorems, synthetic statements, and an extra curriculum ofstatements without proofs (miniF2F-curriculum).
The produced model is then evaluated on unseenstatements, namely the validation and test splits of the miniF2F dataset [6].
In this work, we also consider the transductive setup: on a corpus of unproved statements availableat train time, how many proofs can our method learn to generate?
This protocol is also sensible, asallowing the model to learn from a failed proof-search can lead to more focused exploration on thenext attempt, proving more statements overall than a model that would not be trained online.
Following [9], we also evaluate the pass@k by running k proof searches on the evaluated statementswith the policy and critic obtained by online training.
Given the many evaluations presented in thiswork, we only run them once.
We give more details on the hyper-parameters used in Appendix D.3.
In the transductive setup, we also report the cumulative pass rate , i.e. the proportion of theoremssolved at least once during online training.
Results
In this section, we present our results and study the moving parts of our pipeline through ablations.
We compare our results with GPT-f which represents the state of the art on Metamath and Lean.
[Table]
6.1 Main results6.1.1 Lean
In Lean, we run our experiments on A100 GPUs with 32 trainers and 200 provers.
Each proverruns our Lean API on 48 CPU cores.
Unlike Polu et al.
[11], we sample statements equally frommathlib-train and miniF2F-curriculum, to avoid giving too much importance to statements from adifferent domain than the target.
Results can be found in Table 2.
After 1 day of training, eachstatement from miniF2F-curriculum has been sampled on average 250 times, and 110 out of the 327statements have been solved.
Our model outperforms GPT-f on miniF2F-test, with an approximately10 ⇥ training time speed-up.
After 7 days, we solve 139 statements of miniF2F-curriculum ( 100 for
GPT-f), and observe further improvements on miniF2F-valid or miniF2F-test.
For other evaluations, we depart from the set-up of Polu et al.
[11], directly using the statementsfrom the miniF2F-valid split in our online training, obtaining 58.6% cumulative pass rate.
We thenevaluate the ﬁnal model on miniF2F-test, reaching 41% pass@64, against 36.6% for GPT-f. Withoutthe synthetic data co-training task, the performance drops to 54 .
9% cumulative pass rate on theminiF2F-valid split, and 38 .
5% pass@64 on the miniF2F-test split.
Examples of proofs found by ourmodel can be found in Appendix F.
6.1.2 Metamath
On Metamath, we train our model on V100 GPUs, with 128 trainers and 256 provers, whereasablations are run on 16 trainers and 32 provers.
We report our results in Table 3 for the supervisedmodel and for a model trained with online training.
During online training, we sample statementsfrom the training and from the validation splits of set.mm equally.
Online training dramatically improves performances on valid statements, going from a 61% pass@8to a cumulative pass rate of 82 .
This improvement cannot solely be explained by the high numberof attempts on validation theorems during training.
Indeed, the ablation in Figure 3 (right) showsthat Evariste signiﬁcantly outperforms a supervised model with the same number of attempts.
Thesupervised model plateaus at 66% while Evariste keeps improving beyond 74% after 7 days oftraining, showing that the model is able to learn from previous proof searches through online training.
On test theorems, for which statements were not provided during online training, the accuracyincreased by 10% compared to the supervised model, from 55 . 8% to 65 .
6% accuracy.
The supervisedmodel obtains a pass@32 accuracy of 65 . 4% (resp.
2% ) on the validation (resp.
test) splits,compared to GPT-f’s 56 .
5% (resp.
2% ) after expert iteration.
[Table]
In Equations, we run our main experiment with 32 trainers and 64 provers, whereas ablations are runon 16 trainers and 32 provers.
In this environment, the model easily learns the training distribution ofour random generator, and solves all synthetically generated problems.
Thus, online training is runon the Identities statements only.
Our main experiment reaches a cumulative pass rate of 91 .
3% onthe Identities split, while a supervised model never exceeds 36% even after a similar number of proofattempts.
In Appendix 9, we give examples of Identities statements proved during online training, aswell as the size and depth of proofs found by the model.
In particular, Evariste managed to ﬁnd the proof of complex mathematical statements,such as sinh( x/ 2)
= sinh( x ) / p 2(1 + cosh( x )) and tan(3 x )(1  3(tan( x )) 2 ) =
3 tan( x ) (tan( x )) 2 tan( x ) that required 82 and 117 proof steps respectively, showing the abilities of HTPS to prioritize subgoals and guide the search in very large proof graphs.
This shows that online training isable to adapt our policy and critic models to a completely new domain, going from automaticallygenerated statements to identities found in math books.
Examples to understand the gap betweenthese two domains can be found in Appendix E.
6.2 Ablation study
In this section we present an ablation study on several components of our system.
Since Leanexperiments are CPU intensive, we run most of our ablations on the Equations and Metamathenvironments.
On Lean, we ran experiments on a smaller subset of hyper-parameters that consistentlyperformed well on the other environments.
6.2.1 Online training data for tactic objective
[Table]
The way we ﬁlter tactics sent to the trainers has a large impact on ﬁnal performances.
We investigatedseveral ﬁltering methods and report the results in Table 4.
The ﬁrst method is similar to the one usedin AlphaZero and exposed in [19]: we select all nodes of the proof search hypergraph where the visitcount is above a certain threshold and we ﬁlter tactics above a given search policy score.
At trainingtime, tactics are sampled according to the ﬁltered search policy.
With this method the model reaches51.6% pass@8 on the valid set of Metamath and 37.5% cumulative pass rate on Equations.
We then experimented with other ﬁltering criteria, selecting only goal-tactic pairs that are part ofproofs: either a proof of the root node, or of any solved node in the hypergraph.
Then, we learn fromall possible proofs, or only from proofs that are minimal according to a criteria (number of proofsteps for Equations and Metamath, cumulative CPU time for Lean).
Learning only from minimal proofs always leads to improved performance, regardless of the selectedroots.
Learning from the minimal proofs of all solved nodes, we reach a cumulative pass rate of78.1% on Equations, compared to 40.6% when learning from all proofs.
On Metamath, only learningfrom the root’s minimal proof gives the best result on the valid set, reaching a pass@8 of 68.6%. [Figure] [Figure]
2 Critic
[Figure]
[Table]
To measure the impact of our critic model, we run an experiment where the proof search is onlyguided by the policy model.
In particular, during the back-propagation phase, we set v T ( g ) = 0 .
5for leaves of T .
In that context, our model is no longer trained with a critic objective.
We report theresults in Table 5.
We ﬁnd that using a critic model improves the performance signiﬁcantly, by 5 .
2%and 12 .
5% on Metamath and Equations respectively.
As mentioned in Section 4, to train the critic objective, we set the training targets as c ( g ) =
1 forsolved nodes, c ( g ) = 0 for invalid nodes and c ( g ) =
W ( g, t ⇤ ) /N
( g, t ⇤ ) where t ⇤ is the tactic thatmaximizes the search policy at g , for internal nodes.
We also tested a hard critic estimation of thetarget values, following Polu and Sutskever [9], where c ( g ) = 1 for solved nodes and c ( g ) =
forboth invalid and internal nodes.
We report results in Table 5.
For both Metamath and Equations,
HTPS critic targets allow Evariste to reach its best performance.
In Equations, the model reaches acumulative pass rate of 78.1%, compared to 63.1% with hard critic estimates.
In Equations, usinghard critic targets gives worse performances than having no critic model at all, showing that thesetargets are a bad estimation: setting all internal nodes to zero is too pessimistic.
6.2.3 Fixed proof search parameters
We study the impact of sampling HTPS hyper-parameters for each attempt during online training.
Werun experiments with ﬁxed, chosen search parameters for Equations and Metamath to compare withrandom sampling, and report results in Table 5.
Evariste achieves better performances than the modeltrained with ﬁxed search parameters on Metamath test set and Equations Identities , reaching 78.1%pass rate compared to 73.8% in Equations Identities .
6.2.4 Model update frequency during online training
In our online training procedure, the policy and critic models are updated every ﬁve minutes on theprovers.
We measure the impact of the frequency of these updates by trying different refresh rates: 5minutes, 1 hour, 6 hours for Equations, and no updates at all for both Equations and Metamath.
Wereport the cumulative pass rate over training hours in Figure 3.
The higher the refresh rate, the betterthe cumulative pass rate over time, conﬁrming the beneﬁts of online training over expert iteration.
Conclusion
In this work, we introduce HTPS, an AlphaZero-inspired proof search algorithm for automatedtheorem proving, along with an online training procedure.
We run an extensive study of our pipeline,and present state-of-the-art results on multiple proving environments.
We show that online trainingprovides large speed-ups over expert iteration, and allows generalization of the policy and criticmodels to completely new domains.
Despite large number of attempts per theorem, proving theentirety of datasets like miniF2F remains elusive, and generating data from proof-search on thecurrently available corpora will likely be insufﬁcient in the long term.
As manually annotated formaldatasets are limited, another way of providing exploration and additional training data (in the spirit ofself-play for two player games) is required.
Automated generation of new theorems is likely to beone of the future milestones.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] Section 6.1 includes our results
as well as their limitations.
(c) Did you discuss any potential negative societal impacts of your work?
[No] Beyond
environmental cost of training models, we do not see any direct path from the research
presented in this paper to negative applications.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] The code for
[Table]
the Equations environment will be open-sourced.
We also plan to make our trained
[Table]
model publicly available to help people in the formal community.
Code for the overall
distributed training architecture is tied to our infrastructure and will be difﬁcult to
open-source. [Table]
(b) Did you specify all the training details (e.g., data splits, hyper-parameters, how they [Table] were chosen)?
[Yes] We provide all training details in Section D.2 of the Appendix. [Table]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] As stated in Section 5.2, we report the result of onetraining and one evaluation as running all evaluations multiple times would be too [Table] costly. [Table]
(d) Did you include the total amount of compute and the type of resources used (e.g., type [Table] of GPUs, internal cluster, or cloud provider)?
[Yes]
[Table]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes]
[Table]
(b) Did you mention the license of the assets?
[No] License is available on the repositories [Table]
associated with each asset.
[Table] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
[Table]
No new assets in this work.
[Table] (d) Did you discuss whether and how consent was obtained from people whose data you’re [Table]
using/curating?
[N/A]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
[Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table]
applicable?
[N/A
[Table]
b) Did you describe any potential participant risks, with [Table] to Institutional
[Table]
IRB) approvals, if applicable?
[N/A]
[Table]
c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A] [Table]