Original: M ATHEMATICAL R EASONING IN L ATENT S PACE
Revised: M ATHEMATICAL R EASONING IN L ATENT S PACE

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We design and conduct a simple experiment to study whether neural networks can perform several steps of approximate reasoning in a ﬁxed dimensional latent space.
Revised: We design and conduct a simple experiment to study whether neural networks can perform several steps of approximate reasoning in a ﬁxed dimensional latent space.

Original: The set of rewrites (i.e. transformations) that can be successfully performed on a statement represents essential semantic features of the statement.
Revised: The set of rewrites (i.e. transformations) that can be successfully performed on a statement represents essential semantic features of the statement.

Original: We can compress this information by embedding the formula in a vector space, such that the vector associated with a statement can be used to predict whether a statement can be rewritten by other theorems.
Revised: We can compress this information by embedding the formula in a vector space, such that the vector associated with a statement can be used to predict whether a statement can be rewritten by other theorems.

Original: Predicting the embedding of a formula generated by some rewrite rule is naturally viewed as approximate reasoning in the latent space.
Revised: Predicting the embedding of a formula generated by some rewrite rule is naturally viewed as approximate reasoning in the latent space.

Original: In order to measure the effectiveness of this reasoning, we performapproximate deduction sequences  in the latent space and use the resulting embedding to inform the semantic features of the corresponding formal statement (which is obtained by performing the corresponding rewrite sequence using real formulas).
Revised: In order to measure the effectiveness of this reasoning, we perform sequences of rewrite steps both in formula space and in latent space, and compare the quality of embeddings of the resulting formulas to their predicted embeddings.

Original: Our experiments show that graph neural networks can make non-trivial predictions about the rewrite-success of statements, even when they propagate predicted latent representations for several steps.
Revised: Our experiments show that graph neural networks can make non-trivial predictions about the rewrite-success of statements, even when they propagate predicted latent representations for several steps.

Original: Since our corpus of mathematical formulas includes a wide variety of mathematical disciplines, this experiment is a strong indicator for the feasibility of deduction in latent space in general.
Revised: Since our corpus of mathematical formulas includes a wide variety of mathematical disciplines, this experiment is a strong indicator for the feasibility of deduction in latent space in general.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Automated reasoning has long been considered to require development of logics and “hard” algorithms, such as backtracking search.
Revised: Automated reasoning has long been considered to require development of logics and “hard” algorithms, such as backtracking search.

Original: Recently, approaches that employ deep learning have also been applied, but these have focused on predicting the next step of a proof, which is again executed with a hard algorithm (Loos et al., 2017; Gauthier et al., 2017; Lederman et al., 2018; Bansal et al., 2019b).
Revised: Recently, approaches that employ deep learning have also been applied, but these have focused on predicting the next step of a proof, which is again executed with a hard algorithm (Loos et al., 2017; Gauthier et al., 2017; Lederman et al., 2018; Bansal et al., 2019b).

Original: We raise the question of whether hard algorithms could be omitted from this process and mathematical reasoning performed entirely in the latent space.
Revised: We raise the question of whether hard algorithms could be omitted from this process and mathematical reasoning performed entirely in the latent space.

Original: To this end, we investigate whether we can predict useful latent representations of the mathematical formulas that result from proof steps.
Revised: To this end, we investigate whether we can predict useful latent representations of the mathematical formulas that result from proof steps.

Original: Ideally, we could rely entirely on predicted latent representations to sketch out proofs and only go back to the concrete mathematical formulas to check if our intuitive reasoning was correct.
Revised: Ideally, we could rely entirely on predicted latent representations to sketch out proofs and only go back to the concrete mathematical formulas to check if our intuitive reasoning was correct.

Original: This would allow for more ﬂexible and robust system designs for automated reasoning.
Revised: This would allow for more ﬂexible and robust system designs for automated reasoning.

Original: In this work, we present a ﬁrst experiment indicating directly that theorem proving in the latent space might be possible.
Revised: In this work, we present a ﬁrst experiment indicating directly that theorem proving in the latent space might be possible.

Original: We build on HOList, an environment and benchmark for automated theorem provers based on deep learning (Bansal et al., 2019b), which makes use of the interactive theorem prover HOL Light (Harrison, 1996), an interactive proof assistant.
Revised: We build on HOList, an environment and benchmark for automated theorem provers based on deep learning (Bansal et al., 2019b), which makes use of the interactive theorem prover HOL Light (Harrison, 1996), an interactive proof assistant.

Original: The HOList theorem database comprisesof over 19 thousand theorems and lemmas from a variety of mathematical domains, including topology, multivariate calculus, real and complex analysis, geometric algebra, and measure theory.
Revised: The HOList theorem database comprises over 19 thousand theorems and lemmas from a variety of mathematical domains, including topology, multivariate calculus, real and complex analysis, geometric algebra, and measure theory.

Original: Concrete examples include basic properties of real and complex numbers such as ( x − 1 = y ) ⇔ ( x = y − 1 ) , and also well-known theorems, such as Pythagoras’ theorem,Skolem’s theorem, the fundamental theorem of calculus,  Abel’s theorem for complex power series and that the eigenvalues of a complex matrix are the roots of its characteristic polynomial.
Revised: Concrete examples include basic properties of real and complex numbers such as ( x − 1 = y ) ⇔ ( x = y − 1 ) , and also well-known theorems, such as Pythagoras’ theorem, the fundamental theorem of calculus, Skolem’s theorem, Abel’s theorem for complex power series, and that the eigenvalues of a complex matrix are the roots of its characteristic polynomial.

Original: We focus on rewrite rules (or rewrites in short).
Revised: We focus on rewrite rules (or rewrites in short).

Original: Rewrites are only one of several proof tactics in HOL Light, but they enable powerful transformations on mathematical formulas, as they can be given arbitrary theorems as parameters.
Revised: Rewrites are only one of several proof tactics in HOL Light, but they enable powerful transformations on mathematical formulas, as they can be given arbitrary theorems as parameters.

Original: For example, the formula 3 2 =
Revised: For example, the formula 3 2 =

Original: z can be rewritten to 3 · 3 = z by performing a rewrite with the parameter x 2 = x · x .
Revised: z can be rewritten to 3 · 3 = z by performing a rewrite with the parameter x 2 = x · x .

Original: Alternatively, a rewrite may diverge (as it operates recursively) or it may return the same formula – in both these cases we consider the rewrite  to fail .
Revised: Alternatively, a rewrite may diverge (as it operates recursively) or it may return the same formula – in both these cases we consider the rewrite [Figure] to fail .

Original: For instance, in the example above, the rewrite would fail if we used equation x + y
Revised: For instance, in the example above, the rewrite would fail if we used equation x + y

Original: = y
Revised: = y

Original: + x[Figure] as a rewrite parameter instead, since the expression 3 2 = z does not contain any + operators to match with.
Revised: + x as a rewrite parameter instead, since the expression 3 2 = z does not contain any + operators to match with.

Original: In our experiments, we ﬁrst train a neural network to map mathematical formulas into a latent space of ﬁxed dimension.
Revised: In our experiments, we ﬁrst train a neural network to map mathematical formulas into a latent space of ﬁxed dimension.

Original: This network is trained by predicting – based on the latent representation being trained – whether a given rewrite is going to succeed (i.e. returns with a new formula).
Revised: This network is trained by predicting – based on the latent representation being trained – whether a given rewrite is going to succeed (i.e. returns with a new formula).

Original: For successful rewrites we also predict the latent representation of the resulting formula.
Revised: For successful rewrites we also predict the latent representation of the resulting formula.

Original: To evaluate the feasibility of reasoning in latent space over two steps, we ﬁrst predict the latent representation of the result of a rewrite, then we evaluate whether the predicted latent representation still allows for accurate predictions of the rewrite success of the resulting formula.
Revised: To evaluate the feasibility of reasoning in latent space over two steps, we ﬁrst predict the latent representation of the result of a rewrite, then we evaluate whether the predicted latent representation still allows for accurate predictions of the rewrite success of the resulting formula.

Original: For multi-step reasoning beyond two steps, we predict the future latent representations based on the previous latent representation onlywithout seeing the intermediate formula (cf.
Revised: For multi-step reasoning beyond two steps, we predict the future latent representations based on the previous latent representation onlywithout seeing the intermediate formula (cf.

Original: Figure 1).
Revised: Figure 1).

Original: Our experiments suggest that even after nine steps of reasoning purely in latent space, neural networks show non-trivial reasoning capabilities, despite not being trained on this task directly.
Revised: Our experiments suggest that even after nine steps of reasoning purely in latent space, neural networks show non-trivial reasoning capabilities, despite not being trained on this task directly.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Our work is motivated by deep learning based automated theorem proving, but is also closely connected to model based reinforcement learning and approaches that learn to predict the future as part of reinforcement learning.
Revised: Our work is motivated by deep learning based automated theorem proving, but is also closely connected to model based reinforcement learning and approaches that learn to predict the future as part of reinforcement learning.

Original: Model based reinforcement learning is concerned with creating a model of the environment while maximizing the expected reward (e.g. Ha & Schmidhuber (2018)).
Revised: Model based reinforcement learning is concerned with creating a model of the environment while maximizing the expected reward (e.g. Ha & Schmidhuber (2018)).

Original: Early works have already shown that predicting the latent representations of reinforcement learning environments with deep learning is sometimes feasible - even over many steps (Oh et al., 2015; Chiappa et al., 2017; Burda et al., 2019).
Revised: Early works have already shown that predicting the latent representations of reinforcement learning environments with deep learning is sometimes feasible - even over many steps (Oh et al., 2015; Chiappa et al., 2017; Burda et al., 2019).

Original: This can enable faster training, since it can preempt the need for performing expensive simulations of the environment.
Revised: This can enable faster training, since it can preempt the need for performing expensive simulations of the environment.

Original: Predicting latent representation was also proposed in Brunner et al.
Revised: Predicting latent representation was also proposed in Brunner et al.

Original: (2018) as a regularization method for reinforcement learning.
Revised: (2018) as a regularization method for reinforcement learning.

Original: One recent successful example of model based reinforcement learning is Kaiser et al.
Revised: One recent successful example of model based reinforcement learning is Kaiser et al.

Original: (2019), where the system learns to predict the pixel-wise output of the Atari machine.
Revised: (2019), where the system learns to predict the pixel-wise output of the Atari machine.

Original: However this approach is based on actually simulating the environment directly in the “pixel space” as opposed to performing predictions in a low dimensional semantic embedding space.
Revised: However this approach is based on actually simulating the environment directly in the “pixel space” as opposed to performing predictions in a low dimensional semantic embedding space.

Original: More related to our work is Piotrowski et al.
Revised: More related to our work is Piotrowski et al.

Original: (2019), which attempts to learn to rewrite simple formulas.
Revised: (2019), which attempts to learn to rewrite simple formulas.

Original: The goal is there again is to predict the actual outcome of the rewrite rather than a latent representation of it.
Revised: The goal is there again is to predict the actual outcome of the rewrite rather than a latent representation of it.

Original: In Dosovitskiy & Koltun (2017), they predict “expected measurements” as an extra supervision in addition to the reward signal.
Revised: In Dosovitskiy & Koltun (2017), they predict “expected measurements” as an extra supervision in addition to the reward signal.

Original: Graph neural networks have been used for premise selection in higher order logic (Wang et al., 2017) and more recently by Paliwal et al.
Revised: Graph neural networks have been used for premise selection in higher order logic (Wang et al., 2017) and more recently by Paliwal et al.

Original: (2019) as the core component of DeepHOL, a neural theorem prover for higher order logic.
Revised: (2020) as the core component of DeepHOL, a neural theorem prover for higher order logic.

Original: In this work, we build upon their neural network architecture, but utilize it for a different task.
Revised: In this work, we build upon their neural network architecture, but utilize it for a different task.

Original:  HOL L IGHT
Revised:  HOL L IGHT

Original: HOL Light (Harrison, 1996) is an interactive proof assistant (or interactive theorem prover) for higher-order logic reasoning.
Revised: HOL Light (Harrison, 1996) is an interactive proof assistant (or interactive theorem prover) for higher-order logic reasoning.

Original: Traditionally, proof assistants have been used by human users for creating formalized proofs of mathematical statements manually.
Revised: Traditionally, proof assistants have been used by human users for creating formalized proofs of mathematical statements manually.

Original: While proof assistants have some limited forms of automation, it is still a cumbersome process to formalize proofs, even when they are already available in natural language.
Revised: While proof assistants have some limited forms of automation, it is still a cumbersome process to formalize proofs, even when they are already available in natural language.

Original: Some large scale formalization efforts were conducted successfully in HOL Light and Coq (Coq), for example the formal proofs of the Kepler conjecture (Hales et al., 2017) and that of the four color theorem (Gonthier, 2008).
Revised: Some large scale formalization efforts were conducted successfully in HOL Light, Coq (Coq) and Isabelle/HOL and other theorem provers, for example the formal proof of the Kepler conjecture (Hales et al., 2017), the four color theorem (Gonthier, 2008), and the correctness of a microkernel (Klein et al., 2009).

Original: They required signiﬁcant meticulous manual work and expert knowledge of the system.
Revised: They required signiﬁcant meticulous manual work and expert knowledge of interactive proof assistants.

Original: Lately, there have been several attempts to improve the automation of the proof assistants signiﬁcantly by so called “hammers” (Kaliszyk & Urban, 2015).
Revised: Lately, there have been several attempts to improve the automation of the proof assistants signiﬁcantly by so called “hammers” (Kaliszyk & Urban, 2015).

Original: Still, traditional proof automation lacks the mathematical intuition of human mathematicians who can perform complicated intuitive arguments.
Revised: Still, traditional proof automation lacks the mathematical intuition of human mathematicians who can perform complicated intuitive arguments.

Original: The quest for modelling and automating fuzzy, “human style” reasoning is one of the main motivations for this work.
Revised: The quest for modelling and automating fuzzy, “human style” reasoning is one of the main motivations for this work.

Original: 3. R EWRITE T ACTIC IN HOL L IGHT
Revised: 3. R EWRITE T ACTIC IN HOL L IGHT

Original: The HOL Light system allows the user to specify a goal to prove, and then offers a number of tactics to apply to the goal.
Revised: The HOL Light system allows the user to specify a goal to prove, and then offers a number of tactics to apply to the goal.

Original: A tactic application consumes the goal and returns a list of subgoals.
Revised: A tactic application consumes the goal and returns a list of subgoals.

Original: Proving all of the subgoals is equivalent to proving the goal itself.
Revised: Proving all of the subgoals is equivalent to proving the goal itself.

Original: Accordingly, if a tactic application returns the empty list of subgoals, the parent goal is proven.
Revised: Accordingly, if a tactic application returns the empty list of subgoals, the parent goal is proven.

Original: In this work, we focus on the rewrite tactic ( REWRITE TAC ) of HOL Light, which is a particularly common and versatile tactic.
Revised: In this work, we focus on the rewrite tactic ( REWRITE TAC ) of HOL Light, which is a particularly common and versatile tactic.

Original: It takes a list of theorems as parameters (though in this work we only consider applications of rewrite with a single parameter).
Revised: It takes a list of theorems as parameters (though in this work we only consider applications of rewrite with a single parameter).

Original: Parameters must be an equation or a conjunction of equations; possibly guarded by a condition.
Revised: Parameters must be an equation or a conjunction of equations, possibly guarded by a condition.

Original: Given a goal statement T and parameter P i , the rewrite tactic searches for subexpressions in T that match the left side of one of the equations in P i and replaces it with the right side of the equation.
Revised: Given a goal statement T and parameter P i , the rewrite tactic searches for subexpressions in T that match the left side of one of the equations in P i and replaces it with the right side of the equation.

Original: The matching process takes care of variable names and types, such that minor differences can be bridged.
Revised: The matching process takes care of variable names and types, such that minor differences can be bridged.

Original: The rewrite procedure is recursive, and hence tries to rewrite the formula until no opportunities for rewrites are left.
Revised: The rewrite procedure is recursive, and hence tries to rewrite the formula until no opportunities for rewrites are left.

Original: The rewrite tactic also has a set of built-in rewrite rules, representing trivial simpliﬁcations, such as
Revised: The rewrite tactic also has a set of built-in rewrite rules, representing trivial simpliﬁcations, such as

Original: FST ( x, y ) = x .
Revised: FST ( x, y ) = x .

Original: Note that REWRITE TAC uses “big step semantics”, meaning that the application of each individual operation can perform multiple elementary rewrite steps recursively.
Revised: Note that REWRITE TAC uses “big step semantics”, meaning that the application of each individual operation can perform multiple elementary rewrite steps recursively.

Original: For more details on REWRITE TAC , refer to the manual (HOL Light Rewrite Tactic Reference).
Revised: For more details on REWRITE TAC , refer to the manual (HOL Light Rewrite Tactic Reference).

Original:  R EASONING IN L ATENT S PACE
Revised:  R EASONING IN L ATENT S PACE

Original: In this section we describe our training data, a model architecture, and an evaluation setup that allows us to demonstrate mathematical reasoning in latent space.
Revised: In this section we describe our training data, a model architecture, and an evaluation setup that allows us to demonstrate mathematical reasoning in latent space.

Original: We use the embedding model by Paliwal et al.
Revised: We use the embedding model by Paliwal et al.

Original: (2019) to map formulas into a ﬁxed dimensional latent space, and train the model on two prediction tasks: (1) on the outcome (success or failure) of a large number of possible formula rewrite operations, and (2) on the embedding of the outcome of the rewrite.
Revised: (2020) to map formulas into a ﬁxed dimensional latent space, and train the model on two prediction tasks: (1) on the outcome (success or failure) of a large number of possible formula rewrite operations, and (2) on the embedding of the outcome of the rewrite.

Original: For evaluation, we use the embedding prediction not only for predicting the embedding of the next formula, but also the formulas that result from several successive rewriting steps.
Revised: For evaluation, we use the embedding prediction not only for predicting the embedding of the next formula, but also the formulas that result from several successive rewriting steps.

Original: The challenge in the design of the model is preventing the embeddings from collapsing.
Revised: The challenge in the design of the model is preventing the embeddings from collapsing.

Original: Trained na ¨ ıvely to minimize the l 2 distance between the predicted and the true embedding of the result of the rewrite, the embedding model quickly learns to map most formulas to the same embeddingdestroying the information needed for performing multiple rewrite steps in latent space.
Revised: Trained na¨ıvely to minimize the l 2 distance between the predicted and the true embedding of the result of the rewrite, the embedding model quickly learns to map most formulas to the same embeddingdestroying the information needed for performing multiple rewrite steps in latent space.

Original: To address this problem, we stop the gradient from updating the embedding of the outcome formula directly (but we share the embedding network for the source expression).
Revised: To address this problem, we stop the gradient from updating the embedding of the outcome formula directly (but we share the embedding network for the source expression).

Original: We have also found that adding extra noise to the embedding during training makes the formulas more robust to prediction errors for multi-step step reasoning.
Revised: We have also found that adding extra noise to the embedding during training makes the formulas more robust to prediction errors for multi-step step reasoning.

Original: We hypothesize that training with noisy embeddings acts as a fuzzy error-correction mechanism by preventing errors from accumulating as quickly during multiple steps of reasoning in the latent space.
Revised: We hypothesize that training with noisy embeddings acts as a fuzzy error-correction mechanism by preventing errors from accumulating as quickly during multiple steps of reasoning in the latent space.

Original: Both ideas are described in detail in Subsection 4.3.
Revised: Both ideas are described in detail in Subsection 4.3.

Original: 4. T RAINING D ATA
Revised: 4. T RAINING D ATA

Original: We generate the training data from the theorem database of the HOList environment (Bansal et al., 2019b), which contains 19591 theorems and deﬁnitions ordered such that later theorems use only earlier statements in their human proof.
Revised: We generate the training data from the theorem database of the HOList environment (Bansal et al., 2019b), which contains 19591 theorems and deﬁnitions ordered such that later theorems use only earlier statements in their human proof.

Original: The theorems are split into 11655 training, 3668 validation, and 3620 testing theorems.
Revised: The theorems are split into 11655 training, 3668 validation, and 3620 testing theorems.

Original: To generate our training data, we generate all pairs ( T, P ) of theorems from the training set, where P must occur before T in the database (to avoid circular dependencies).
Revised: To generate our training data, we generate all pairs ( T, P ) of theorems from the training set, where P must occur before T in the database (to avoid circular dependencies).

Original: We then interpret theorem T as a goal to prove and try to rewrite T with P using the REWRITE TAC of HOL Light.
Revised: We then interpret theorem T as a goal to prove and try to rewrite T with P using the REWRITE TAC of HOL Light.

Original: This can result in three different outcomes:
Revised: This can result in three different outcomes:

Original: T is rewritten by theorem P successfully, and the result differs from T .
Revised: T is rewritten by theorem P successfully, and the result differs from T .

Original: The rewrite operation terminates, but fails to change the input theorem T .
Revised: The rewrite operation terminates, but fails to change the input theorem T .

Original: The rewrite operation times out or diverges (becomes too big).
Revised: The rewrite operation times out or diverges (becomes too big).

Original: In our experiments, we consider only the ﬁrst outcome as successful, i.e. when the application ﬁnishes within the speciﬁed time limit and changes the target, as a successful rewrite attempt.
Revised: In our experiments, we consider only the ﬁrst outcome as successful, i.e. when the application ﬁnishes within the speciﬁed time limit and changes the target, as a successful rewrite attempt.

Original: Each training example, therefore, consists of the pair ( T, P ) , the success/fail-bit of the rewrite (1 for successful rewrites, 0 for failed rewrites), and, for successful tactic applications, the new formula that results from the rewrite, which we denote with R ( T, P ) .
Revised: Each training example, therefore, consists of the pair ( T, P ) , the success/fail-bit of the rewrite (1 for successful rewrites, 0 for failed rewrites), and, for successful tactic applications, the new formula that results from the rewrite, which we denote with R ( T, P ) .

Original: Of the 115 million possible pairs of ( T, P ) where T falls in the training split, there are 1.6 million successful rewrites that are used as training examples.
Revised: Of the 115 million possible pairs of ( T, P ) where T falls in the training split, there are 1.6 million successful rewrites that are used as training examples.

Original: 4. M ODEL A RCHITECTURE AND T RAINING M ETHODOLOGY
Revised: 4. M ODEL A RCHITECTURE AND T RAINING M ETHODOLOGY

Original: π γ c p e γ
Revised: π γ c p e γ

Original: L
Revised: L

Original: P
Revised: P

Original: T
Revised: T

Original: R(T, P)
Revised: R(T, P)

Original: εε
Revised: ε

Original: Let S denote the set of syntactically correct higher-order logic formulas in HOL Light.
Revised: Let S denote the set of syntactically correct higher-order logic formulas in HOL Light.

Original: Additionally, we deﬁne two latent spaces, L = R k and M = R k ( k = 1024) , corresponding to embeddings of theorems used as goals, and the statements used as parameters, respectively.
Revised: Additionally, we deﬁne two latent spaces, L = R k and M = R k ( k = 1024) , corresponding to embeddings of theorems used as goals, and the statements used as parameters, respectively.

Original: We use two embedding networks γ : S −→ L and π : S −→ M , one for each of the two formulas T and P .
Revised: We use two embedding networks γ : S −→ L and π : S −→ M , one for each of the two formulas T and P .

Original: For both embedding networks, we follow the GNN approach by Paliwal et al.
Revised: For both embedding networks, we follow the GNN approach by Paliwal et al.

Original: (2019): We interpret the formulas’ abstract syntax trees as graphs, feed the graphs into graph neural networks, and let the networks exchange messages between neighboring nodes for a ﬁxed number of hops.
Revised: We interpret the formulas’ abstract syntax trees as graphs, feed the graphs into graph neural networks, and let the networks exchange messages between neighboring nodes for a ﬁxed number of hops.

Original: The ﬁnal embeddings of the nodes are max-pooled to form a representation of the entire formula in a ﬁxed dimensional embedding space R k .
Revised: The ﬁnal embeddings of the nodes are max-pooled to form a representation of the entire formula in a ﬁxed dimensional embedding space R k .

Original: The embeddings are then concatenated and processed by a three-layer perceptron c : L × M −→ C , which is followed by p : C −→ R , a single output linear function producing a logit predicting whether theorem P rewrites T .
Revised: The embeddings are then concatenated and processed by a three-layer perceptron c : L × M −→ C , which is followed by p : C −→ R , a single output linear function producing a logit predicting whether theorem P rewrites T .

Original: The perceptron p is trained with logistic regression on the success/fail-bit of the rewrite.
Revised: The perceptron p is trained with logistic regression on the success/fail-bit of the rewrite.

Original: Formally, this output is p ( c ( γ ( T ) , π ( P ))) , which we also refer to as p ( T, P ) or p ( γ ( T ) , π ( P )) .
Revised: Formally, this output is p ( c ( γ ( T ) , π ( P ))) , which we also refer to as p ( T, P ) or p ( γ ( T ) , π ( P )) .

Original: The output of c is also used as input to another linear function e : C −→ L , which predicts the latent representation of the result γ ( R ( T, P )) by minimizing the l 2 distance.
Revised: The output of c is also used as input to another linear function e : C −→ L , which predicts the latent representation of the result γ ( R ( T, P )) by minimizing the l 2 distance.

Original: Formally, this output is e ( c ( γ ( T ) , π ( P ))) , but we also use e ( T, P ) or e ( γ ( T ) , π ( P )) .
Revised: Formally, this output is e ( c ( γ ( T ) , π ( P ))) , but we also use e ( T, P ) or e ( γ ( T ) , π ( P )) .

Original: 4. S TOP G RADIENTS AND N OISE
Revised: 4. S TOP G RADIENTS AND N OISE

Original: Note that we stop gradient backpropagation between the l 2 loss and the second embedding tower γ
Revised: Note that we stop gradient backpropagation between the l 2 loss and the second embedding tower γ

Original: ( R ( T, P )) .
Revised: ( R ( T, P )) .

Original: The purpose of this is to prevent the second tower from making the magnitude of the embedding of the rewrite outcome very small or “collapse”.
Revised: The purpose of this is to prevent the second tower from making the magnitude of the embedding of the rewrite outcome very small or “collapse”.

Original: We discuss alternative solutions to this problem in Section 5.
Revised: We discuss alternative solutions to this problem in Section 5.

Original: In the case of the  goal embedding tower γ , we add Gaussian noise ε with 0 mean and standard deviation β = 10 − 3 during training only.
Revised: In the case of the second goal embedding tower γ , we add Gaussian noise ε with 0 mean and standard deviation β = 10 − 3 during training only.

Original: We can see in the experimental section that training with noise has the effect of more accurate downstream predictions in multiple step reasoning.
Revised: We can see in the experimental section that training with noise has the effect of more accurate downstream predictions in multiple step reasoning.

Original: 4. R EASONING
Revised: 4. R EASONING

Original: After we have trained our model on the training set theorems (and theorem pairs) of the HOList benchmark, we can use it to perform rewritesin the latent space alone.
Revised: After we have trained our model on the training set theorems (and theorem pairs) of the HOList benchmark, we can use it to perform rewrites

Original: We use p : S × S −→ R as a quality metric for the propagated embedding vector.
Revised: We use p : S × S −→ R as a quality metric for the propagated embedding vector.

Original: Given an approximation E (cid:48)
Revised: Given an approximation E (cid:48)

Original: T of the latent representation γ ( T ) , we can evaluate p ( E (cid:48) T , π ( P )) for a large number of tactic parameters P .
Revised: T of the latent representation γ ( T ) , we can evaluate p ( E (cid:48) T , π ( P )) for a large number of tactic parameters P .

Original: This is compared with true rewrite successes of T by P to assess the quality of the approximation.
Revised: This is compared with true rewrite successes of T by P to assess the quality of the approximation.

Original: To evaluate multiple steps of reasoning in the latent space, start with the embedding of formula Tand predict the embeddings that result from rewriting by theorems P 1 , · · · P k in that order.
Revised: To evaluate multiple steps of reasoning in the latent space, start with the embedding of formula Tand predict the embeddings that result from rewriting by theorems P 1 , · · · P k in that order.

Original: To assess the quality of the overall reasoning, the same sequence of rewrites is performed with the actual formulas and the ﬁnal predicted embedding is evaluated with respect to the formula resulting from the sequence of formal rewrites.
Revised: To assess the quality of the overall reasoning, the same sequence of rewrites is performed with the actual formulas and the ﬁnal predicted embedding is evaluated with respect to the formula resulting from the sequence of formal rewrites.

Original: In latent space L = R k we start from some initial theorem T 1 ∈ S .
Revised: In latent space L = R k we start from some initial theorem T 1 ∈ S .

Original: The following schema indicates the sequence of predictions performed in latent space L :
Revised: The following schema indicates the sequence of predictions performed in latent space L :

Original: [Equation]
Revised: [Equation]

Original: This is compared with the following formal sequence of rewrites:
Revised: This is compared with the following formal sequence of rewrites:

Original: [Equation]
Revised: [Equation]

Original: That is, l 2 approximates the latent vector of T 2 , that is γ ( R ( T 1 , P 1 )) and l 3 approximates the latent vector of T 3 , that is γ ( R ( R ( T 1 , P 1 ) , P 2 ) , by construction.
Revised: That is, l 2 approximates the latent vector of T 2 , that is γ ( R ( T 1 , P 1 )) and l 3 approximates the latent vector of T 3 , that is γ ( R ( R ( T 1 , P 1 ) , P 2 ) , by construction.

Original:  A LTERNATIVE M ODEL A RCHITECTURES
Revised:  A LTERNATIVE M ODEL A RCHITECTURES

Original: Learning a high quality embedding space while simultaneously learning to make predictions in that space proved to be some challenge.
Revised: Learning a high quality embedding space while simultaneously learning to make predictions in that space proved to be some challenge.

Original: As we described above, the embeddings quickly approach 0 if trained na¨ıvely, minimizing the least squares distance between the e ( T, P ) and γ ( R ( T, P )) without improving the quality of p .
Revised: As we described above, the embeddings quickly approach 0 if trained na ¨ ıvely, minimizing the least squares distance between the e ( T, P ) and γ ( R ( T, P )) without improving the quality of p .

Original: Through the course of this work we evaluated several alternative approaches to tackling this problem.
Revised: Through the course of this work we evaluated several alternative approaches to tackling this problem.

Original: The empirical comparison between these architectures is shown in Section 6.4.
Revised: The empirical comparison between these architectures is shown in Section 6.4.

Original: 5. F IXED L ATENT S PACE
Revised: 5. F IXED L ATENT S PACE

Original: [Equation] p p' e' [Figure]
Revised: π T γ γ' π' c c' p p' e' α σ ω

Original:  Our ﬁrst, straightforward approach to this problem was to ﬁx the embedding space L and learn predictions from a separate space L (cid:48) .
Revised: P Our ﬁrst, straightforward approach to this problem was to ﬁx the embedding space L and learn predictions from a separate space L (cid:48) .

Original: The advantage of this approach is that it does not suffer from embedding collapse during the prediction of the embedding of the rewrite outcome.
Revised: The advantage of this approach is that it does not suffer from embedding collapse during the prediction of the embedding of the rewrite outcome.

Original: On the other hand, this approach incurs training of three different models, which is awkward and cumbersome.
Revised: On the other hand, this approach incurs training of three different models, which is awkward and cumbersome.

Original: The three models are shown in
Revised: The three models are shown in

Original: Figure 3.
Revised: Figure 3.

Original: Rewrite success prediction σ : S × S −→ R , 2.
Revised: Rewrite success prediction σ : S × S −→ R , 2.

Original: Rewrite outcome prediction ω : S × S −→ R × L , 3.
Revised: Rewrite outcome prediction ω : S × S −→ R × L , 3.

Original: Embedding alignment prediction α : L −→ L (cid:48) .
Revised: Embedding alignment prediction α : L −→ L (cid:48) .

Original: The rewrite success prediction model σ ( T, P ) is identical to the function p described above.
Revised: The rewrite success prediction model σ ( T, P ) is identical to the function p described above.

Original: Training σ ﬁxes the embedding space L , from which we predict rewrite success.
Revised: Training σ ﬁxes the embedding space L , from which we predict rewrite success.

Original: In addition to σ , we train a separate rewrite outcome prediction ω ( T, P ) .
Revised: In addition to σ , we train a separate rewrite outcome prediction ω ( T, P ) .

Original: This model has an identical two-tower embedding architecture as σ , but with an extra prediction layer e (cid:48) to predict the embedding vector of the outcome of the rewritten formulas in L .
Revised: This model has an identical two-tower embedding architecture as σ , but with an extra prediction layer e (cid:48) to predict the embedding vector of the outcome of the rewritten formulas in L .

Original: Here the embedding towers are denoted by γ (cid:48) and π (cid:48) , the combiner network is c (cid:48) and the two linear prediction layers are p (cid:48) and e (cid:48) .
Revised: Here the embedding towers are denoted by γ (cid:48) and π (cid:48) , the combiner network is c (cid:48) and the two linear prediction layers are p (cid:48) and e (cid:48) .

Original: That is: ω ( T, P ) =
Revised: That is: ω ( T, P ) =

Original: ( p (cid:48)
Revised: ( p (cid:48)

Original: ( c (cid:48)
Revised: ( c (cid:48)

Original: ( γ (cid:48) ( T ) , π (cid:48) ( P ))) , e (cid:48)
Revised: ( γ (cid:48) ( T ) , π (cid:48) ( P ))) , e (cid:48)

Original: ( c (cid:48)
Revised: ( c (cid:48)

Original: ( γ (cid:48) ( T ) , π (cid:48) ( P ))) .
Revised: ( γ (cid:48) ( T ) , π (cid:48) ( P ))) .

Original: We denote the embedding space learned by ω , L (cid:48) .
Revised: We denote the embedding space learned by ω , L (cid:48) .

Original: Since σ and ω produce latent vectors γ ( T ) ∈ L and γ (cid:48)
Revised: Since σ and ω produce latent vectors γ ( T ) ∈ L and γ (cid:48)

Original: ( T ) ∈ L (cid:48) in different spaces, we need to align those spaces to enable deduction purely in the embedding space.
Revised: ( T ) ∈ L (cid:48) in different spaces, we need to align those spaces to enable deduction purely in the embedding space.

Original: In particular, γ ( R ( R ( T, P 0 ) , P 1 )) is approximated by e ( γ (cid:48)
Revised: In particular, γ ( R ( R ( T, P 0 ) , P 1 )) is approximated by e ( γ (cid:48)

Original: ( R ( T, P 0 )) , π (cid:48) ( P )) , and so we need γ (cid:48)
Revised: ( R ( T, P 0 )) , π (cid:48) ( P )) , and so we need γ (cid:48)

Original: ( R ( T, P 0 )) .
Revised: ( R ( T, P 0 )) .

Original: For that, we train a translation model α : L −→ L (cid:48) , which is a three layer fully connected network with 1024 units in[Figure] each layer and rectiﬁed linear activations, which predicts γ (cid:48) ( T ) given an embedding of γ ( T ) .
Revised: For that, we train a translation model α : L −→ L (cid:48) , which is a three layer fully connected network with 1024 units in each layer and rectiﬁed linear activations, which predicts γ (cid:48) ( T ) given an embedding of γ ( T ) .

Original: Note that α does not see T ∈ S as an input, it makes its prediction based on the latent space representation γ ( T ) of T alone.
Revised: Note that α does not see T ∈ S as an input, it makes its prediction based on the latent space representation γ ( T ) of T alone.

Original: Reasoning with this model can be summarized by:
Revised: Reasoning with this model can be summarized by:

Original: [Figure]
Revised: [Equation]

Original: 5. C ENTERING AND N ORMALIZATION
Revised: 5. C ENTERING AND N ORMALIZATION

Original: A different approach to learning a good embedding prediction is normalizing the embeddings to a ﬁxed length.
Revised: A different approach to learning a good embedding prediction is normalizing the embeddings to a ﬁxed length.

Original: The na ¨ ıve implementation of this leads both γ and e to predict an almost-constant embedding vector independent of T , with the relevant information encoded in the noise.
Revised: The na¨ıve implementation of this leads both γ and e to predict an almost-constant embedding vector independent of T , with the relevant information encoded in the noise.

Original: Centering the embeddings before normalization mitigates this issue.
Revised: Centering the embeddings before normalization mitigates this issue.

Original: In our experiments, we found that using a variation of batch normalization without learned mean and scale parameters, followed by l 2 normalization, is sufﬁcient to force the model to predict meaningful embeddings from e .
Revised: In our experiments, we found that using a variation of batch normalization without learned mean and scale parameters, followed by l 2 normalization, is sufﬁcient to force the model to predict meaningful embeddings from e .

Original: However, this model is sensitive to hyperparameters, in particular the scaling of losses between p and e , and even in the best conﬁguration we found performance was worse than our ﬁnal model.
Revised: However, this model is sensitive to hyperparameters, in particular the scaling of losses between p and e , and even in the best conﬁguration we found performance was worse than our ﬁnal model.

Original: 5. S TOCHASTIC E MBEDDINGS
Revised: 5. S TOCHASTIC E MBEDDINGS

Original: Another natural approach is to replace the mean squared error objective of the embedding prediction output e with a KL divergence objective, which is scale invariant.
Revised: Another natural approach is to replace the mean squared error objective of the embedding prediction output e with a KL divergence objective, which is scale invariant.

Original: To do this, we attempted to substitute the deterministic embedding tower γ with a stochastic variant, where the ﬁnal layer of the tower is used to parameterize a diagonal Gaussian distribution.
Revised: To do this, we attempted to substitute the deterministic embedding tower γ with a stochastic variant, where the ﬁnal layer of the tower is used to parameterize a diagonal Gaussian distribution.

Original: The ﬁnal layer of the embedding network e is similarly used to paramaterize a diagonal Gaussian.
Revised: The ﬁnal layer of the embedding network e is similarly used to paramaterize a diagonal Gaussian.

Original: KL ( e ( T, P ) || γ
Revised: KL ( e ( T, P ) || γ

Original: ( R ( T, P ))) is the objective to minimize.
Revised: ( R ( T, P ))) is the objective to minimize.

Original: This model proved difﬁcult to optimize: the na¨ıve solution fails to converge.
Revised: This model proved difﬁcult to optimize: the na ¨ ıve solution fails to converge.

Original: A carefully tuned information bottleneck mitigates this problem, but the performance of theresulting model remains poor.
Revised: A carefully tuned information bottleneck mitigates this problem, but the performance of the model remains poor.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: This section provides an experimental evaluation that aims to answer the following question:
Revised: This section provides an experimental evaluation that aims to answer the following question:

Original: Is our model capable of predicting useful embedding vectors multiple steps ahead?
Revised: Is our model capable of predicting useful embedding vectors multiple steps ahead?

Original: We explore the prediction quality of the embedding vectors (for rewrite success) and see how the quality of predicted embedding vectors degrades after e is used for predicting multiple steps in the latent space alone.
Revised: We explore the prediction quality of the embedding vectors (for rewrite success) and see how the quality of predicted embedding vectors degrades after e is used for predicting multiple steps in the latent space alone.

Original: 6. N EURAL N ETWORK A RCHITECTURE D ETAILS
Revised: 6. N EURAL N ETWORK A RCHITECTURE D ETAILS

Original: Our network has two independent towers, which are each 16 -hop graph neural networks with internal node representations in R 128 .
Revised: Our network has two independent towers, which are each 16 -hop graph neural networks with internal node representations in R 128 .

Original: The output of each of the two towers is fed into a layer that expands the dimension of the node representation to R 1024 with a fully connected layer with shared weights for each node.
Revised: The output of each of the two towers is fed into a layer that expands the dimension of the node representation to R 1024 with a fully connected layer with shared weights for each node.

Original: This is followed by maximum pooling over all nodes of the network.
Revised: This is followed by maximum pooling over all nodes of the network.

Original: We add Gaussian noise with β = 10 − 3 to the embedding of the goal theorem.
Revised: We add Gaussian noise with β = 10 − 3 to the embedding of the goal theorem.

Original: The two resulting embedding vectors are concatenated along with their element-wise multiplication, and are processed by a three layer perceptron with 1024 hidden units and rectiﬁed linear activation between the layers.
Revised: The two resulting embedding vectors are concatenated along with their element-wise multiplication, and are processed by a three layer perceptron with 1024 hidden units and rectiﬁed linear activation between the layers.

Original: Our model is trained with g = 16 choices of T in each batch.
Revised: Our model is trained with g = 16 choices of T in each batch.

Original: For each T i , there are l + 1 =choices of P : one successful example in each group and l random negatives.
Revised: For each T i , there are l + 1 =choices of P : one successful example in each group and l random negatives.

Original: However all choices of P corresponding to other T k
Revised: However all choices of P corresponding to other T k

Original: (cid:54)
Revised: (cid:54)

Original: =
Revised: =

Original: i are used as negative instances as well, regardless of whether they would rewrite it – this is justiﬁed by the fact that only a few theorems rewrite any given T
Revised: i are used as negative instances as well, regardless of whether they would rewrite it – this is justiﬁed by the fact that only a few theorems rewrite any given T

Original: i so this introduces only a small amount of uncorrelated label noise.
Revised: i so this introduces only a small amount of uncorrelated label noise.

Original: This training methodology is motivated by the fact that evaluating the combiner network is much cheaper than computing the embedding using the graph neural network.
Revised: This training methodology is motivated by the fact that evaluating the combiner network is much cheaper than computing the embedding using the graph neural network.

Original: Based on Alemi et al.
Revised: Based on Alemi et al.

Original: , we expect that hard negative mining, or iteratively replacing random negative examples with examples that p ( T, P ) classiﬁes incorrectly, would improve our results signiﬁcantly, but it is left for future work.
Revised: , we expect that hard negative mining, or iteratively replacing random negative examples with examples that p ( T, P ) classiﬁes incorrectly, would improve our results signiﬁcantly, but it is left for future work.

Original: 6. E VALUATION D ATASET
Revised: 6. E VALUATION D ATASET

Original: In order to measure the performance of our models after multiple deduction steps are performed, we generate datasets D 0 , D 1 , . . .
Revised: In order to measure the performance of our models after multiple deduction steps are performed, we generate datasets D 0 , D 1 , . . .

Original: , D r successively by applying rewrites to a randomly selected set of examples.
Revised: , D r successively by applying rewrites to a randomly selected set of examples.

Original: We start with all theorems from the validation set of HOList, denoted by T 0 .
Revised: We start with all theorems from the validation set of HOList, denoted by T 0 .

Original: For each theorem T i,j ∈ T i , we perform all possible rewrites R ( T i,j , P k ) , k < j and include them in
Revised: For each theorem T i,j ∈ T i , we perform all possible rewrites R ( T i,j , P k ) , k < j and include them in

Original: D i .
Revised: D i .

Original: We then randomly select a new goal to include in T i + 1 .
Revised: We then randomly select a new goal to include in T i + 1 .

Original: This method ensures that the goals in
Revised: This method ensures that the goals in

Original: T i accurately represent the full range of complexity of theorems present in the database, while the distribution of parameters P and the ratio of successful rewrites by each parameter matches that of the space of all possible rewrites.
Revised: T i accurately represent the full range of complexity of theorems present in the database, while the distribution of parameters P and the ratio of successful rewrites by each parameter matches that of the space of all possible rewrites.

Original: 6. E VALUATION OF R EWRITE P REDICTION M ODEL
Revised: 6. E VALUATION OF R EWRITE P REDICTION M ODEL

Original: To evaluate a model m : S × S −→ R on dataset D
Revised: To evaluate a model m : S × S −→ R on dataset D

Original: i , we compute m ( T, P ) for each pair ( T, P ) in the dataset.
Revised: i , we compute m ( T, P ) for each pair ( T, P ) in the dataset.

Original: We expect pairs where R ( T, P ) succeeds to score highly relative to pairs where
Revised: We expect pairs where R ( T, P ) succeeds to score highly relative to pairs where

Original: R ( T, P ) fails.
Revised: R ( T, P ) fails.

Original: For a concrete metric, we compute the ROC curve of the predictions and use the area under the curve (AUC) as our main metric.
Revised: For a concrete metric, we compute the ROC curve of the predictions and use the area under the curve (AUC) as our main metric.

Original: Higher curves and higher AUC values represent more accurate predictions of rewrite success.
Revised: Higher curves and higher AUC values represent more accurate predictions of rewrite success.

Original: We measure how ROC curves change as we use different approximations of γ ( T ) .
Revised: We measure how ROC curves change as we use different approximations of γ ( T ) .

Revised: [Figure]

Original: As we have trained our model only on pairs of theorems from the original database, the model exhibits increasing generalization error as we evaluate it on formulas with increasing number of rewrites.
Revised: As we have trained our model only on pairs of theorems from the original database, the model exhibits increasing generalization error as we evaluate it on formulas with increasing number of rewrites.

Original: The accuracy of the the model p evaluated directly on theorems after multiple rewrites is referred to as the “True” measurement.
Revised: The accuracy of the the model p evaluated directly on theorems after multiple rewrites is referred to as the “True” measurement.

Original: This gives an upper bound on the rewrite success prediction since noisier embedding vectors generated by e will produce worse results on average.T
Revised: This gives an upper bound on the rewrite success prediction since noisier embedding vectors generated by e will produce worse results on average.

Original: T 1 R(T 0 , P 0 )

Original: T 2 R(T 1 , P 1 )

Original: E 1 '

Original: E 2 '

Original: E

Original: E

Original: E

Original: E 2 ''

Original: T

Original: E

Original: R(T 2 , P 2 )

Original: E 3 '

Original: E 3 ''

Original: Theorem

Original: True

Original: One Step

Original: Multi Step γ(T 0 ) γ(T 1 ) γ(T 2 ) γ(T 3 ) e(E 1 , π(P 1 )) e(E 0 , π(P 0 ))

Original: e(E 2 , π(P 2 ))

Original: e(E 1' , π(P 1 ))

Original: e(E 2'' , π(P 1 ))

Original: Additionally, we want to measure the rate at which the latent vectors degrade as we propagate them in embedding space as described in Subsection 4.4.
Revised: Additionally, we want to measure the rate at which the latent vectors degrade as we propagate them in embedding space as described in Subsection 4.4.

Original: Embedding predictions generated directly from theorem statements, e ( γ ( T ) , π ( P )) , are referred to as “One Step”, while predictions produced through propagating are referred to as “Multi Step”.
Revised: Embedding predictions generated directly from theorem statements, e ( γ ( T ) , π ( P )) , are referred to as “One Step”, while predictions produced through propagating are referred to as “Multi Step”.

Original: A comparison of these generation methods is shown in Figure 5.
Revised: A comparison of these generation methods is shown in Figure 4.

Original: In order to evaluate the performance of our model in isolation we need to compare it with carefully selected baselines:
Revised: In order to evaluate the performance of our model in isolation we need to compare it with carefully selected baselines:

Original: As Figure 4 shows , a
Revised: [Figure]

Original:  few theorems are much more frequently applicable than others
Revised: As Figure 5 shows, a few theorems are much more frequently applicable than others.

Original: We want to see how the prediction of rewrite success performs based on the rewrite parameter alone if we ignore the theorem to be rewritten.
Revised: We want to see how the prediction of rewrite success performs based on the rewrite parameter alone if we ignore the theorem to be rewritten.

Original: One way to establish such a baseline is to feed a randomly selected theorem T (cid:48) to p instead of T to predict its rewrite success.
Revised: One way to establish such a baseline is to feed a randomly selected theorem T (cid:48) to p instead of T to predict its rewrite success.

Original: This is referred to as the “Random” baseline.
Revised: This is referred to as the “Random” baseline.

Original: A stronger “baseline” is achieved by utilizing the ground-truth rewrite statistics to rank each parameter P by the number of theorems it successfully rewrites.
Revised: A stronger “baseline” is achieved by utilizing the ground-truth rewrite statistics to rank each parameter P by the number of theorems it successfully rewrites.

Original: This is the best achievable prediction that does not depend on T .
Revised: This is the best achievable prediction that does not depend on T .

Original: We refer to this as the “Usage” baseline.
Revised: We refer to this as the “Usage” baseline.

Original: [Figure]
Revised: [Figure]

Original: Figure 6 shows the distribution of the theorem pair prediction score logits of p ( c ( γ ( T ) , π ( P ))) , for the those “positive” pairs that rewrite and the “negative” pairs that do not rewrite.
Revised: Figure 6 shows the distribution of the theorem pair prediction score logits of p ( c ( γ ( T ) , π ( P ))) , for the those “positive” pairs that rewrite and the “negative” pairs that do not rewrite.

Original: Note that the ratio is normalized separately for the positive and negative pairs as negative pairs occur much more frequently than positive pairs.
Revised: Note that the ratio is normalized separately for the positive and negative pairs as negative pairs occur much more frequently than positive pairs.

Original: One can see that the quality of the rewrite success prediction degrades somewhat after nine steps of reasoning purely in the latent space [Figure] but it is still much better than the random baseline.
Revised: One can see that the quality of the rewrite success prediction degrades somewhat after nine steps of reasoning purely in the latent space, but it is still much better than the random baseline.

Original: This gives clear evidence that the embedding prediction manages to propagate much useful information over multiple steps in the latent space alone
Revised: This gives clear evidence that the embedding prediction manages to propagate much useful information over multiple steps in the latent space alone.

Original: [Figure]
Revised: [Figure]

Original: In Figure 7 we make further measurements and comparisons on the quality of reasoning in the embedding space.
Revised: In Figure 7 we make further measurements and comparisons on the quality of reasoning in the embedding space.

Original: One can see that our model could perform reasoning for nine steps in the embedding space and still retain a lot of the predictive power of the original model.
Revised: One can see that our model could perform reasoning for nine steps in the embedding space and still retain a lot of the predictive power of the original model.

Original: In order to appreciate the above results one should keep in mind that our model is not trained on statements that were already rewritten.
Revised: In order to appreciate the above results one should keep in mind that our model is not trained on statements that were already rewritten.

Original: All the training was done only on the theorems present in the initial database.
Revised: All the training was done only on the theorems present in the initial database.

Original: The reduction of prediction performance is apparent from downward trajectory of the “Pred (Multi step)” curve, which isolates this effect from that of the error accumulated by the embedding predictions, the effect of which is measured indirectly by the “True” curve
Revised: The reduction of prediction performance is apparent from downward trajectory of the “Pred (Multi step)” curve, which isolates this effect from that of the error accumulated by the embedding predictions, the effect of which is measured indirectly by the “True” curve.

Original: [Figure]
Revised: [Figure]

Original: In Figure 8 we measure the l 2 distance from the predicted embedding vectors to the true embedding vectors γ ( T ) of formulas after multiple rewrite steps in the latent space.
Revised: In Figure 8 we measure the l 2 distance from the predicted embedding vectors to the true embedding vectors γ ( T ) of formulas after multiple rewrite steps in the latent space.

Original: These results are consistent with our earlier ﬁndings on success of rewrite predication after rewrite steps in the latent space: while there is some divergence of the predicted embedding vectors from the true embedding vectors (as computed from the rewritten statements directly), the predicted embedding vectors are signiﬁcantly closer to the true embedding vectors than randomly selected embeddings
Revised: These results are consistent with our earlier ﬁndings on success of rewrite predication after rewrite steps in the latent space: while there is some divergence of the predicted embedding vectors from the true embedding vectors (as computed from the rewritten statements directly)

Original: [Figure]
Revised: [Figure]

Revised: the predicted embedding vectors are signiﬁcantly closer to the true embedding vectors than randomly selected embeddings.

Original: 6. C OMPARISON OF A LTERNATIVES
Revised: 6. C OMPARISON OF A LTERNATIVES

Original: [Figure]
Revised: [Figure]

Original: Our ﬁnal model adds noise to the embeddings produced by γ , during training time only.
Revised: Our ﬁnal model adds noise to the embeddings produced by γ , during training time only.

Original: After applying noise, the network must produce embeddings sufﬁciently far apart to distinguish between them.
Revised: This forces the network to produce embeddings sufﬁciently far apart to distinguish between them.

Original: We found that this method provided the best experimental results compared to the models discussed in Section 5 as shown in Figure 9.
Revised: We found that this method provided the best experimental results compared to the models discussed in Sectionas shown in Figure 9.

Original: We hypothesize that the noise makes the network robust to future prediction mistakes as the prediction of the next rewrite is trained on noisy versions of the theorem embedding.
Revised: We hypothesize that the noise makes the network robust to future prediction mistakes as the prediction of the next rewrite is trained on noisy versions of the theorem embedding.

Original: This may act as a natural error correction mechanism.
Revised: This may act as a natural error correction mechanism.

Original: Figure 9 supports this hypothesis, since the difference between the noisy and noiseless versions of the model increases with the number of rewrite steps performed in the latent space.
Revised: Figure 9 supports this hypothesis, since the difference between the noisy and noiseless versions of the model increases with the number of rewrite steps performed in the latent space.

Revised: However, our experiments indicate that adding noise alone without stop gradients or other approaches is not sufﬁcient to learn a diverse latent space.

Original: 6. E VALUATION OF R EWRITEABILITY
Revised: 6. E VALUATION OF R EWRITEABILITY

Original: [Figure] Particularly interesting rewrites are those that change the rewritability of a theorem
Revised: Particularly interesting rewrites are those that change the rewritability of a theorem, or rewrites where R ( T, P ) fails and R ( R ( T, P 0 ) , P ) succeeds, or vice versa.

Original: [Figure]
Revised: [Figure]

Original: or rewrites where R ( T, P ) fails and R ( R ( T, P 0 ) , P ) succeeds, or vice versa.

Original: In Figure
Revised: In Figure

Original: [Figure]
Revised: [Figure]

Original: we evaluate our model on these cases.
Revised: we evaluate our model on these cases.

Original: We created a database of statements that are the result of R ( R ( T, P 0 ) , P ) and split it into two groups, depending on whether R ( T, P ) succeeds or fails.
Revised: We created a database of statements that are the result of R ( R ( T, P 0 ) , P ) and split it into two groups, depending on whether R ( T, P ) succeeds or fails.

Original: Within each group, we score each rewrite and plot the resulting ROC curves.
Revised: Within each group, we score each rewrite and plot the resulting ROC curves.

Original: From this ﬁgure, we can see that the model is capable of detecting when the rewriteability of a theorem does not change (First Rewrite Succeeds and Second Rewrite Succeeds, or First Rewrite Fails and Second
Revised: From this ﬁgure, we can see that the model is capable of detecting when the rewriteability of a theorem does not change (First Rewrite Succeeds and Second Rewrite Succeeds, or First Rewrite Fails and Second

Original: Rewrite Fails).
Revised: Rewrite Fails).

Original: Most interestingly, cases where the ﬁrst rewrite fails and the second rewrite succeeds are also scored correctly.
Revised: Most interestingly, cases where the ﬁrst rewrite fails and the second rewrite succeeds are also scored correctly.

Original: This demonstrates that the predicted embeddings are precise enough to encode the appearance of new possibilities.
Revised: This demonstrates that the predicted embeddings are precise enough to encode the appearance of new possibilities.

Original: However, it is noteworthy that the model often fails to detect when rewrites are no longer possible.
Revised: However, it is noteworthy that the model often fails to detect when rewrites are no longer possible.

Original: This may be due to label noise caused by randomly sampling negative examples, as some “negative” examples are actually positives, leading to generalization error.
Revised: This may be due to label noise caused by randomly sampling negative examples, as some “negative” examples are actually positives, leading to generalization error.

Original: A deeper investigation of these failures is an interesting subject for further work.
Revised: A deeper investigation of these failures is an interesting subject for further work.

Original: 6. V ISUALIZATION OF L ATENT S PACES

Original: In addition to numerically evaluating the performance of our latent embedding models, it is interesting to visually examine the embedding spaces produced.

Original: Figure 11a depicts the predicted embeddings of the datasets D 1 · · · D 4 by using PCA to reduce the dimension of each point in the joint dataset to 50, then using tSNE to produce a 2D visualization which is ﬁltered by original step.

Original: The points are colored by their l 2 -distance to the true embedding.

Original: From this graph it can be seen that the distribution of embeddings shifts meaningfully over multiple rewrites, further demonstrating that the rewrite task produces non-trivial changes.

Original: We also wished to compare the embeddings produced by our model to human understanding of mathematics.

Original: In the HOList database, each theorem comes with tags describing the area of mathematics it originates from, such as topology or set theory.

Original: In Figure 11b we separate the embeddings of theorems from these categories by color, and examine the their evolution across multiple rewrites.

Original: It can be seen that theorems originating from similar human categories correspond to similar embeddings in the latent space.

Original: [Figure]

Original: [Figure]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper we studied the feasibility of performing complex reasoning for mathematical formulas in a ﬁxed (1024) dimensional embedding space.
Revised: In this paper we studied the feasibility of performing complex reasoning for mathematical formulas in a ﬁxed (1024) dimensional embedding space.

Original: We proposed a new evaluation metric that measures the preservation semantic information under multiple reasoning steps in the embedding space.
Revised: We proposed a new evaluation metric that measures the preservation semantic information under multiple reasoning steps in the embedding space.

Original: Although our models were not trained for performing rewrites on rewritten statements, nor were they trained for being able to deduce multiple steps in the embedding space, our approximate rewrite prediction model p has demonstrated signiﬁcant prediction power on as many as nine rewrite steps in latent space alone.
Revised: Although our models were not trained for performing rewrites on rewritten statements, nor were they trained for being able to deduce multiple steps in the embedding space, our approximate rewrite prediction model p has demonstrated signiﬁcant prediction power on as many as nine rewrite steps in latent space alone.

Original: Our methods showcase a simple but general methodology for reasoning in latent space.
Revised: Our methods showcase a simple but general methodology for reasoning in latent space.

Original: This work can also be seen as an easy to use, fast to train, and crisp evaluation methodology for representing mathematical statements by neural networks.
Revised: This work can also be seen as an easy to use, fast to train, and crisp evaluation methodology for representing mathematical statements by neural networks.

Original: We hope that this will lead to improved neural network architectures for mathematics.
Revised: We hope that this will lead to improved neural network architectures for mathematics.

Original: It is likely that such representations prove helpful for faster learning to prove without imitating human proofs like that in DeepHOL-Zero Bansal et al. (2019a), given that premise selection is a closely related task to predicting the rewrite success of statements.
Revised: It is likely that such representations prove helpful for faster learning to prove without imitating human proofs like that in DeepHOL-Zero Bansal et al. (2019a), given that premise selection is a closely related task to predicting the rewrite success of statements.

Original: Self-supervised pre-training or even co-training such models with premise selection could prove useful as a way of learning more semantic feature representations of mathematical formulas
Revised: Self-supervised pre-training or even co-training such models with premise selection could prove useful as a way of learning more semantic feature representations of mathematical formulas. A V ISUALIZATION OF L ATENT S PACES [Figure] [Figure]

Revised: In addition to numerically evaluating the performance of our latent embedding models, it is interesting to visually examine the embedding spaces produced.

Revised: Figure 11a depicts the predicted embeddings of the datasets D 1 · · · D 4 by using PCA to reduce the dimension of each point in the joint dataset to 50, then using tSNE to produce a 2D visualization which is ﬁltered by original step.

Revised: The points are colored by their l 2 -distance to the true embedding.

Revised: From this graph it can be seen that the distribution of embeddings shifts meaningfully over multiple rewrites, further demonstrating that the rewrite task produces non-trivial changes.

Revised: We also wished to compare the embeddings produced by our model to human understanding of mathematics.

Revised: In the HOList database, each theorem comes with tags describing the area of mathematics it originates from, such as topology or set theory.

Revised: In Figure 11b we separate the embeddings of theorems from these categories by color, and examine the their evolution across multiple rewrites.

Revised: It can be seen that theorems originating from similar human categories correspond to similar embeddings in the latent space.