Original: G RAMMAR -I NDUCED G EOMETRY FOR D ATAE FFICIENT M OLECULAR P ROPERTY P REDICTION
Revised: G RAMMAR -I NDUCED G EOMETRY FOR D ATAE FFICIENT M OLECULAR P ROPERTY P REDICTION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The prediction of molecular properties is a crucial task in the field of material and drug discovery.
Revised: The prediction of molecular properties is a crucial task in the field of material and drug discovery.

Original: The potential benefits of using deep learning techniques are reflected in the wealth of recent literature.
Revised: The potential benefits of using deep learning techniques are reflected in the wealth of recent literature.

Original: Still, these techniques are faced with a common challenge in practice: Labeled data are limited by the cost of manual extraction from literature and laborious experimentation.
Revised: Still, these techniques are faced with a common challenge in practice: Labeled data are limited by the cost of manual extraction from literature and laborious experimentation.

Original: In this work, we propose a data-efficient property predictor by utilizing a learnable hierarchical molecular grammar that can generate molecules from grammar production rules.
Revised: In this work, we propose a data-efficient property predictor by utilizing a learnable hierarchical molecular grammar that can generate molecules from grammar production rules.

Original: Such a grammar induces an explicit geometry of the space of molecular graphs, which provides an informative prior on molecular structural similarity.
Revised: Such a grammar induces an explicit geometry of the space of molecular graphs, which provides an informative prior on molecular structural similarity.

Original: The property prediction is performed using graph neural diffusion over the grammar-induced geometry.
Revised: The property prediction is performed using graph neural diffusion over the grammar-induced geometry.

Original: On both small and large datasets, our evaluation shows that this approach outperforms a wide spectrum of baselines, including supervised and pre-trained graph neural networks.
Revised: On both small and large datasets, our evaluation shows that this approach outperforms a wide spectrum of baselines, including supervised and pre-trained graph neural networks.

Original: We include a detailed ablation study and further analysis of our solution, showing its effectiveness in cases with extremely limited data (only ∼ 100 samples), and its extension to application in molecular generation.
Revised: We include a detailed ablation study and further analysis of our solution, showing its effectiveness in cases with extremely limited data (only ∼ 100 samples), and its extension to application in molecular generation.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Molecular property prediction is an essential step in the discovery of novel materials and drugs, as it applies to both high-throughput screening and molecule optimization.
Revised: Molecular property prediction is an essential step in the discovery of novel materials and drugs, as it applies to both high-throughput screening and molecule optimization.

Original: Recent advances in machine learning, particularly deep learning, have made tremendous progress in predicting complex property values that are difficult to measure in reality due to the associated cost.
Revised: Recent advances in machine learning, particularly deep learning, have made tremendous progress in predicting complex property values that are difficult to measure in reality due to the associated cost.

Original: Depending on the representation form of molecules, various methods have been proposed, including recurrent neural networks (RNN) for SMILES strings (Lusci et al., 2013; Goh et al., 2017), feed-forward networks (FFN) for molecule fingerprints (Tao et al., 2021b;a), and, more dominantly, graph neural networks (GNN) for molecule graphs (Yang et al., 2019; Bevilacqua et al., 2022; Aldeghi & Coley, 2022; Yu & Gao, 2022).
Revised: Depending on the representation form of molecules, various methods have been proposed, including recurrent neural networks (RNN) for SMILES strings (Lusci et al., 2013; Goh et al., 2017), feed-forward networks (FFN) for molecule fingerprints (Tao et al., 2021b;a), and, more dominantly, graph neural networks (GNN) for molecule graphs (Yang et al., 2019; Bevilacqua et al., 2022; Aldeghi & Coley, 2022; Yu & Gao, 2022).

Original: They have been employed to predict biological and mechanical properties of both polymers and drug-like molecules.
Revised: They have been employed to predict biological and mechanical properties of both polymers and drug-like molecules.

Original: Typically, these methods learn a deep neural network that maps the molecular input into an embedding space, where molecules are represented as latent features and then transformed into property values.
Revised: Typically, these methods learn a deep neural network that maps the molecular input into an embedding space, where molecules are represented as latent features and then transformed into property values.

Original: Despite their promising performance on common benchmarks, these deep learning-based approaches require a large amount of training data in order to be effective (Audus & de Pablo, 2017; Wieder et al., 2020).
Revised: Despite their promising performance on common benchmarks, these deep learning-based approaches require a large amount of training data in order to be effective (Audus & de Pablo, 2017; Wieder et al., 2020).

Original: In practice, however, scientists often have small datasets at their disposal, in which case deep learning fails, particularly in the context of polymers (Subramanian et al., 2016; Altae-Tran et al., 2017).
Revised: In practice, however, scientists often have small datasets at their disposal, in which case deep learning fails, particularly in the context of polymers (Subramanian et al., 2016; Altae-Tran et al., 2017).

Original: For example, due to the difficulty of generating and acquiring data—which usually entails synthesis, wet-lab measurement, and mechanical testing— state-of-the-art works on polymer property prediction using real data are limited to only a few hundred samples (Menon et al., 2019; Chen et al., 2021).
Revised: For example, due to the difficulty of generating and acquiring data—which usually entails synthesis, wet-lab measurement, and mechanical testing— state-of-the-art works on polymer property prediction using real data are limited to only a few hundred samples (Menon et al., 2019; Chen et al., 2021).

Original: To compensate for the scarcity of experimental data, applied works often rely on labeled data generated by simulations, such as density functional theory and molecular dynamics (Aldeghi & Coley, 2022; Antoniuk et al., 2022).
Revised: To compensate for the scarcity of experimental data, applied works often rely on labeled data generated by simulations, such as density functional theory and molecular dynamics (Aldeghi & Coley, 2022; Antoniuk et al., 2022).

Original: Yet, these techniques suffer from high computational costs, tedious parameter optimization, and a considerable discrepancy between simulations and experiments (Afzal et al., 2020; Chen et al., 2021), which limit their applicability in practice.
Revised: Yet, these techniques suffer from high computational costs, tedious parameter optimization, and a considerable discrepancy between simulations and experiments (Afzal et al., 2020; Chen et al., 2021), which limit their applicability in practice.

Original: Recent deep learning research has recognized the scarcity of molecular data in several domains and has developed methods handling small datasets, including self-supervised learning (Zhang et al., 2021;
Revised: Recent deep learning research has recognized the scarcity of molecular data in several domains and has developed methods handling small datasets, including self-supervised learning (Zhang et al., 2021;

Original: Rong et al., 2020; Wang et al., 2022; Ross et al., 2021), transfer learning (Hu et al., 2020),
Revised: Rong et al., 2020; Wang et al., 2022; Ross et al., 2021), transfer learning (Hu et al., 2020),

Original: Cl Cl Cl Cl
Revised: Cl Cl Cl Cl

Original: Cl
Revised: Cl

Original: Cl Cl Cl
Revised: Cl Cl Cl

Original: Cl
Revised: Cl

Original: Cl
Revised: Cl

Original: Hierarchical Molecular Grammar Grammar-induced Geometry
Revised: Hierarchical Molecular Grammar Grammar-induced Geometry

Original: GrammarLearning
Revised: GrammarLearning

Original: Graph Neural Diffusion 0.
Revised: Graph Neural Diffusion 0.

Original: 0.63 Propertyvalues
Revised: 0.63 Propertyvalues

Original: MetaGrammar
Revised: MetaGrammar

Original: MolecularGrammar
Revised: MolecularGrammar

Original: Cl
Revised: Cl

Original: Cl Cl
Revised: Cl Cl

Original: Cl
Revised: Cl

Original: Cl Cl Cl Cl … and few-shot
Revised: Cl Cl Cl Cl … and few-shot

Original: (Guo et al., 2021b; Stanley et al., 2021).
Revised: (Guo et al., 2021b; Stanley et al., 2021).

Original: These methods involve pre-training networks on large molecular datasets before being applied to domain-specific, small-scale target datasets.
Revised: These methods involve pre-training networks on large molecular datasets before being applied to domain-specific, small-scale target datasets.

Original: However, when applied to datasets of very small size (e.g., ∼ 300 ), most of these methods are prone to perform poorly and are statistically unstable (Hu et al., 2020).
Revised: However, when applied to datasets of very small size (e.g., ∼ 300 ), most of these methods are prone to perform poorly and are statistically unstable (Hu et al., 2020).

Original: Moreover, as we will show in our experiments, these methods are less reliable when deployed on target datasets that contain significant domain gaps from the pre-trained
Revised: Moreover, as we will show in our experiments, these methods are less reliable when deployed on target datasets that contain significant domain gaps from the pre-trained

Original: (e.g., inconsistency in molecule sizes).
Revised: (e.g., inconsistency in molecule sizes).

Original: As an alternative to pure deep learning-based methods, formal grammars over molecular structures offer an explicit, explainable representation for molecules and have shown their great potential in addressing molecular tasks in a data-efficient manner (Kajino, 2019; Krenn et al., 2019; Guo et al., 2021a; 2022).
Revised: As an alternative to pure deep learning-based methods, formal grammars over molecular structures offer an explicit, explainable representation for molecules and have shown their great potential in addressing molecular tasks in a data-efficient manner (Kajino, 2019; Krenn et al., 2019; Guo et al., 2021a; 2022).

Original: A molecular grammar consists of a set of production rules that can be chained to generate molecules.
Revised: A molecular grammar consists of a set of production rules that can be chained to generate molecules.

Original: The production rules, which can either be manually defined (Krenn et al., 2019;
Revised: The production rules, which can either be manually defined (Krenn et al., 2019;

Original: Guo et al., 2021a) or learned from data (Kajino, 2019; Guo et al., 2022), encode necessary constraints for generating valid molecular structures, such as valency restrictions.
Revised: Guo et al., 2021a) or learned from data (Kajino, 2019; Guo et al., 2022), encode necessary constraints for generating valid molecular structures, such as valency restrictions.

Original: A molecular grammar has the combinatorial capacity to represent a large amount of molecules using a relatively small number of production rules.
Revised: A molecular grammar has the combinatorial capacity to represent a large amount of molecules using a relatively small number of production rules.

Original: It has thus been adapted as a data-efficient generative model (Kajino, 2019; Guo et al., 2022).
Revised: It has thus been adapted as a data-efficient generative model (Kajino, 2019; Guo et al., 2022).

Original: While molecular generation based on grammars has been widely studied and is relatively straightforward, extending the data-efficiency advantage of grammars to property prediction has not yet been well-explored.
Revised: While molecular generation based on grammars has been widely studied and is relatively straightforward, extending the data-efficiency advantage of grammars to property prediction has not yet been well-explored.

Original: Motivation.
Revised: Motivation.

Original: In this paper, we propose a framework for highly data-efficient property prediction based on a learnable molecular grammar.
Revised: In this paper, we propose a framework for highly data-efficient property prediction based on a learnable molecular grammar.

Original: The intuition behind is that the production rule sequences for molecule generation provide rich information regarding the similarity of molecular structures.
Revised: The intuition behind is that the production rule sequences for molecule generation provide rich information regarding the similarity of molecular structures.

Original: For instance, two molecular structures that share a common substructure would use the same sequence of grammar production rules.
Revised: For instance, two molecular structures that share a common substructure would use the same sequence of grammar production rules.

Original: As it is widely recognized in cheminformatics that molecules with similar structures have similar properties (Johnson & Maggiora, 1990; Martin et al., 2002), grammar production sequences can thus be used as a strong structural prior to predict molecular properties.
Revised: As it is widely recognized in cheminformatics that molecules with similar structures have similar properties (Johnson & Maggiora, 1990; Martin et al., 2002), grammar production sequences can thus be used as a strong structural prior to predict molecular properties.

Original: We aim to develop a model that explicitly represents grammar production sequences and captures the structure-level similarity between molecules.
Revised: We aim to develop a model that explicitly represents grammar production sequences and captures the structure-level similarity between molecules.

Original: Even from a few molecules, this model is expected to reveal a wealth of information regarding the structural relationship between molecules, providing the key to a data efficient property predictor.
Revised: Even from a few molecules, this model is expected to reveal a wealth of information regarding the structural relationship between molecules, providing the key to a data efficient property predictor.

Original: Framework.
Revised: Framework.

Original: Figure 1 outlines our approach.
Revised: Figure 1 outlines our approach.

Original: At the heart of our method is a grammar-induced geometry (in the form of a graph) for the space of molecules.
Revised: At the heart of our method is a grammar-induced geometry (in the form of a graph) for the space of molecules.

Original: In the geometry, every path tracing from the root to a leaf represents a grammar production sequence that generates a particular molecule.
Revised: In the geometry, every path tracing from the root to a leaf represents a grammar production sequence that generates a particular molecule.

Original: Such a geometry explicitly captures the intrinsic closeness between molecules, where structurally similar molecules are closer in distance along the geometry.
Revised: Such a geometry explicitly captures the intrinsic closeness between molecules, where structurally similar molecules are closer in distance along the geometry.

Original: In contrast to the embedding space used in most deep learning methods, our grammar-induced geometry is entirely explicit , which can be integrated with the property predictor by considering all involved molecules simultaneously.
Revised: In contrast to the embedding space used in most deep learning methods, our grammar-induced geometry is entirely explicit , which can be integrated with the property predictor by considering all involved molecules simultaneously.

Original: To construct the geometry, we propose a hierarchical molecular grammar consisting of two parts: a pre-defined meta grammar at the top and a learnable molecular grammar at the bottom.
Revised: To construct the geometry, we propose a hierarchical molecular grammar consisting of two parts: a pre-defined meta grammar at the top and a learnable molecular grammar at the bottom.

Original: We provide both theoretical and experimental evidence to demonstrate that the hierarchical molecular grammar is compact yet complete.
Revised: We provide both theoretical and experimental evidence to demonstrate that the hierarchical molecular grammar is compact yet complete.

Original: To predict properties, we exploit the structural prior captured by the grammar-induced geometry using a graph neural diffusion model over the geometry.
Revised: To predict properties, we exploit the structural prior captured by the grammar-induced geometry using a graph neural diffusion model over the geometry.

Original: A joint optimization framework is proposed to simultaneously learn both the geometry and the diffusion.
Revised: A joint optimization framework is proposed to simultaneously learn both the geometry and the diffusion.

Original: Results.
Revised: Contributions.

Original: Our evaluation covers two small datasets of polymers (each containing ∼ 300 samples) and two large datasets of drug-like molecules (containing ∼ 600 and ∼ 1 , 000 samples, respectively).
Revised: Our evaluation covers two small datasets of polymers (each containing ∼ 300 samples) and two large datasets of drug-like molecules (containing ∼ 600 and ∼ 1 , 000 samples, respectively).

Original: Our approach outperforms both competitive state-of-the-art GNN approaches and pre-trained methods by large margins.
Revised: Our approach outperforms both competitive state-of-the-art GNN approaches and pretrained methods by large margins.

Original: A further analysis shows that when trained on only a subset of the training data ( ∼ 100 samples), our method can achieve performance comparable to pre-trained GNNs fine-tuned on the whole training set of the downstream prediction task, thus demonstrating the effectiveness of our method on extremely small datasets.
Revised: A further analysis shows that when trained on only a subset of the training data ( ∼ 100 samples), our method can achieve performance comparable to pre-trained GNNs fine-tuned on the whole training set of the downstream prediction task, thus demonstrating the effectiveness of our method on extremely small datasets.

Original: We also show that our method can be integrated with molecular generation to guide the discovery of molecules with optimal properties.
Revised: We also show that our method can be integrated with molecular generation to guide the discovery of molecules with optimal properties.

Original: Related Work.
Revised: Related Work.

Original: Our method is mainly related to three areas: 1) machine learning, particularly graph neural networks for molecular property prediction, in which we target the same problem but propose an entirely new approach; 2) grammars for molecular machine learning, which are used in existing works for molecular generation, while we use grammars to induce a geometry for molecular property prediction; and 3) geometric deep learning, where deep networks are applied to non-Euclidean domains.
Revised: Our method is mainly related to three areas: 1) machine learning, particularly graph neural networks for molecular property prediction, in which we target the same problem but propose an entirely new approach; 2) grammars for molecular machine learning, which are used in existing works for molecular generation, while we use grammars to induce a geometry for molecular property prediction; and 3) geometric deep learning, where deep networks are applied to non-Euclidean domains.

Original: We briefly discuss the latter two in Section 2 as necessary background for our approach and refer to Appendix H for an in-depth discussion of related works.
Revised: We briefly discuss the latter two in Section 2 as necessary background for our approach and refer to Appendix G for an in-depth discussion of related works.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Molecular Hypergraph Grammar (MHG).
Revised: Molecular Hypergraph Grammar (MHG).

Original: In MHGs, molecules are represented as hypergraphs
Revised: In MHGs, molecules are represented as hypergraphs

Original: H =
Revised: H =

Original: ( V, E ) , where nodes represent chemical atoms and hyperedges represent chemical bonds or ring-like substructures.
Revised: ( V, E ) , where nodes represent chemical atoms and hyperedges represent chemical bonds or ring-like substructures.

Original: A MHG G = ( N , Σ , P , X ) contains a set N of non-terminal nodes, a set Σ of terminal nodes representing chemical atoms, and a starting node X .
Revised: A MHG G = ( N , Σ , P , X ) contains a set N of non-terminal nodes, a set Σ of terminal nodes representing chemical atoms, and a starting node X .

Original: It describes how molecular hypergraphs are generated using a set of production rules P
Revised: It describes how molecular hypergraphs are generated using a set of production rules P

Original: = { p i | i = 1 , ..., k } of form p
Revised: = { p i | i = 1 , ..., k } of form p

Original: i : LHS →
Revised: i : LHS →

Original: RHS , where the left- and right-hand sides ( LHS and RHS ) are hypergraphs.
Revised: RHS , where the left- and right-hand sides ( LHS and RHS ) are hypergraphs.

Original: Starting at X , a molecule is generated by iteratively selecting a rule whose LHS matches a sub-hypergraph in the current hypergraph and replacing it with RHS until only terminal nodes remain.
Revised: Starting at X , a molecule is generated by iteratively selecting a rule whose LHS matches a sub-hypergraph in the current hypergraph and replacing it with RHS until only terminal nodes remain.

Original: For each production rule, the LHS contains only non-terminal nodes and no terminal nodes, whereas the RHS can contain both non-terminal and terminal nodes.
Revised: For each production rule, the LHS contains only non-terminal nodes and no terminal nodes, whereas the RHS can contain both non-terminal and terminal nodes.

Original: Both sides of the production rule have the same number of anchor nodes, which indicate correspondences when LHS is replaced by RHS in a production step.
Revised: Both sides of the production rule have the same number of anchor nodes, which indicate correspondences when LHS is replaced by RHS in a production step.

Original: For a formal definition, see Guo et al.
Revised: For a formal definition, see Guo et al.

Original: Graph Diffusion models the information propagation between nodes on a graph using heat equations.
Revised: Graph Diffusion models the information propagation between nodes on a graph using heat equations.

Original: The node features are updated following a diffusion PDE as follows:
Revised: The node features are updated following a diffusion PDE as follows:

Original: [Equation]
Revised: [Equation]

Original: where matrix U t represents the features of all nodes in the graph at time t and the matrix L α , which has the same sparsity structure as a graph Laplacian, represents the diffusivity defined on all edges in the graph.
Revised: where matrix U t represents the features of all nodes in the graph at time t and the matrix L α , which has the same sparsity structure as a graph Laplacian, represents the diffusivity defined on all edges in the graph.

Original: L α is calculated using a ( · , · ; α ) parameterized by α , i.e. L ij = a ( U ( i ) t , U ( j ) t ; α ) for all connected node pairs ( i, j ) .
Revised: L α is calculated using a ( · , · ; α ) parameterized by α , i.e. L ij = a ( U ( i ) t , U ( j ) t ; α ) for all connected node pairs ( i, j ) .

Original: For more details, see Chamberlain et al.
Revised: For more details, see Chamberlain et al.

Original: Notation.
Revised: Notation.

Original: For a graph grammar G =
Revised: For a graph grammar G =

Original: ( N , Σ , P , X ) , we say a graph H can be derived from the grammar if there is a sequence of production rules from P that generates this graph, denoted by
Revised: ( N , Σ , P , X ) , we say a graph H can be derived from the grammar if there is a sequence of production rules from P that generates this graph, denoted by

Original: X ∗ ⇒ P H .
Revised: X ∗ ⇒ P H .

Original: H 1 p ⇒ H 2 denotes one-step grammar production that transforms graph H 1 to graph H 2 using rule p .
Revised: H 1 p ⇒ H 2 denotes one-step grammar production that transforms graph H 1 to graph H 2 using rule p .

Original: As a special form of general graph, a tree is denoted as T and the set of all possible trees is denoted as T .
Revised: As a special form of general graph, a tree is denoted as T and the set of all possible trees is denoted as T .

Original: Note that all trees discussed in this paper are unrooted without order , i.e. connected acyclic undirected graphs (Bender & Williamson, 2010).
Revised: Note that all trees discussed in this paper are unrooted without order , i.e. connected acyclic undirected graphs (Bender & Williamson, 2010).

Original: ∆( T ) denotes the maximal degree of T .
Revised: ∆( T ) denotes the maximal degree of T .

Original: TED ( T 1 , T 2 ) denotes the tree edit distance between tree T 1 and tree T 2 as defined in Zhang (1996) and Paaßen (2018).
Revised: TED ( T 1 , T 2 ) denotes the tree edit distance between tree T 1 and tree T 2 as defined in Zhang (1996) and Paaßen (2018).

Original: |P| denotes the number of rules in a production rule set P and | T | denotes the size of a tree T .
Revised: |P| denotes the number of rules in a production rule set P and | T | denotes the size of a tree T .

Original:  G RAMMAR - INDUCED GEOMETRY FOR PROPERTY PREDICTION 3. G RAMMAR - INDUCED G EOMETRY
Revised:  G RAMMAR - INDUCED GEOMETRY FOR PROPERTY PREDICTION 3. G RAMMAR - INDUCED G EOMETRY

Original: Problem Formulation.
Revised: Problem Formulation.

Original: A molecular property predictor can be expressed as a function π ( · ) :
Revised: A molecular property predictor can be expressed as a function π ( · ) :

Original: H → R that maps molecules formulated as hypergraphs H
Revised: H → R that maps molecules formulated as hypergraphs H

Original: =
Revised: =

Original: ( V, E ) ∈ H into scalar values.
Revised: ( V, E ) ∈ H into scalar values.

Original: The mapping
Revised: The mapping

Original: [Figure]
Revised: [Figure]

Original: function π = g ◦ f contains two components: an embedding function f ( · ) :
Revised: function π = g ◦ f contains two components: an embedding function f ( · ) :

Original: H → R n that maps input hypergraphs into an Euclidean latent feature space (also known as the embedding space ), and a transformation function g ( · ) : R n → R (usually a simple linear function) which maps latent features into property values.
Revised: H → R n that maps input hypergraphs into an Euclidean latent feature space (also known as the embedding space ), and a transformation function g ( · ) : R n → R (usually a simple linear function) which maps latent features into property values.

Original: The embedding function is designed to capture the similarity between molecules, such that molecules with similar properties are closer in the embedding space in terms of
Revised: The embedding function is designed to capture the similarity between molecules, such that molecules with similar properties are closer in the embedding space in terms of

Original: Euclidean distance (Cayton, 2005).
Revised: Euclidean distance (Cayton, 2005).

Original: The key reason why most supervised machine learning methods fail in data-sparse cases is that the embedded function fails to capture molecular similarity since it only has access to a limited number of samples with property labels.
Revised: The key reason why most supervised machine learning methods fail in data-sparse cases is that the embedded function fails to capture molecular similarity since it only has access to a limited number of samples with property labels.

Original: To address this issue, we propose an additional, explicit prior on molecular similarity that does not depend on data labels, but provides valuable information for property prediction.
Revised: To address this issue, we propose an additional, explicit prior on molecular similarity that does not depend on data labels, but provides valuable information for property prediction.

Original: Grammar-induced Geometry.
Revised: Grammar-induced Geometry.

Original: Our efforts are motivated by the fact that structure-level similarity between molecules is known to be strongly correlated with their properties, e.g., organic molecules sharing a functional group are proven to have similar chemical properties (Johnson & Maggiora, 1990; Martin et al., 2002).
Revised: Our efforts are motivated by the fact that structure-level similarity between molecules is known to be strongly correlated with their properties, e.g., organic molecules sharing a functional group are proven to have similar chemical properties (Johnson & Maggiora, 1990; Martin et al., 2002).

Original: We explicitly model this structure-level similarity as an informative prior for data-efficient property prediction.
Revised: We explicitly model this structure-level similarity as an informative prior for data-efficient property prediction.

Original: Inspired by the fact that grammars are generative models for molecules based on combinations of molecular substructures, we exploit the molecular hypergraph grammar to capture the structural similarity between molecules.
Revised: Inspired by the fact that grammars are generative models for molecules based on combinations of molecular substructures, we exploit the molecular hypergraph grammar to capture the structural similarity between molecules.

Original: Figure 2 outlines how a molecular hypergraph grammar
Revised: Figure 2 outlines how a molecular hypergraph grammar

Original: G = ( N , Σ , P , X ) can be used to construct a geometry for the space of molecules in the form of a graph G =
Revised: G = ( N , Σ , P , X ) can be used to construct a geometry for the space of molecules in the form of a graph G =

Original: ( V , E ) 1 .
Revised: ( V , E ) 1 .

Original: The geometry enumerates all possible production sequences, where every possible intermediate or molecular hypergraph is represented as a node in V = { v | v = H v ( V v , E v ) } , and each production step is represented as an edge in E
Revised: The geometry enumerates all possible production sequences, where every possible intermediate or molecular hypergraph is represented as a node in V = { v | v = H v ( V v , E v ) } , and each production step is represented as an edge in E

Original: = { ( s, t ) | H s p ⇒ H t , p ∈ P} .
Revised: = { ( s, t ) | H s p ⇒ H t , p ∈ P} .

Original: In this geometry, every leaf node represents a molecule.
Revised: In this geometry, every leaf node represents a molecule.

Original: Every path tracing from the root H root =
Revised: Every path tracing from the root H root =

Original: ( X , ∅ ) to a leaf represents a production sequence that generates the corresponding molecule.
Revised: ( X , ∅ ) to a leaf represents a production sequence that generates the corresponding molecule.

Original: Since all the molecules can be derived from the initial node X , any two leaves representing two molecules are connected in this geometry, where the path between them represents the sequence of transforming from one molecular structure to the other.
Revised: Since all the molecules can be derived from the initial node X , any two leaves representing two molecules are connected in this geometry, where the path between them represents the sequence of transforming from one molecular structure to the other.

Original: More structurally similar molecules have a smaller shortest path as their molecular hypergraphs share a common intermediate hypergraph.
Revised: More structurally similar molecules have a smaller shortest path as their molecular hypergraphs share a common intermediate hypergraph.

Original: The distance between two molecules is defined as the shortest-path distance between them along the geometry with unit weight at each edge.
Revised: The distance between two molecules is defined as the shortest-path distance between them along the geometry with unit weight at each edge.

Original: As a result, structurally similar molecules are closer in distance along the geometry.
Revised: As a result, structurally similar molecules are closer in distance along the geometry.

Original: We use this geometry as an additional input to the molecular property predictor: π ′ ( · , · ) : H × G → R , where G ∈ G is the grammar-induced geometry.
Revised: We use this geometry as an additional input to the molecular property predictor: π ′ ( · , · ) : H × G → R , where G ∈ G is the grammar-induced geometry.

Original: The geometry can be optimized in conjunction with the property predictor in order to minimize the prediction loss.
Revised: The geometry can be optimized in conjunction with the property predictor in order to minimize the prediction loss.

Original: As the geometry is determined by the grammar, the optimization of the geometry can be converted into the learning of production rules, where the latter can be achieved using the method described in Guo et al.
Revised: As the geometry is determined by the grammar, the optimization of the geometry can be converted into the learning of production rules, where the latter can be achieved using the method described in Guo et al.

Original: The crucial remaining question is how to construct this geometry G =
Revised: The crucial remaining question is how to construct this geometry G =

Original: ( V , E ) from a given molecular hypergraph grammar?
Revised: ( V , E ) from a given molecular hypergraph grammar?

Original: A key characteristic of the grammar-induced geometry is that each node in the geometry represents a unique intermediate hypergraph or molecular hypergraph.
Revised: A key characteristic of the grammar-induced geometry is that each node in the geometry represents a unique intermediate hypergraph or molecular hypergraph.

Original: This ensures that the geometry includes the minimal path between two molecules.
Revised: This ensures that the geometry includes the minimal path between two molecules.

Original: To satisfy this characteristic, we use breadth-first search (BFS) originating from the root H root =
Revised: To satisfy this characteristic, we use breadth-first search (BFS) originating from the root H root =

Original: ( X , ∅ ) to iteratively expand the geometry.
Revised: ( X , ∅ ) to iteratively expand the geometry.

Original: At each iteration when we visit a node v , we enumerate all the production rules that could be applied to the intermediate hypergraph H v represented by the current node.
Revised: At each iteration when we visit a node v , we enumerate all the production rules that could be applied to the intermediate hypergraph H v represented by the current node.

Original: Then we produce a new hypergraph H ( i ) new for each applicable rule p i : { H ( i ) new | H v p i ⇒ H ( i ) new , p i ∈ P} .
Revised: Then we produce a new hypergraph H ( i ) new for each applicable rule p i : { H ( i ) new | H v p i ⇒ H ( i ) new , p i ∈ P} .

Original: For each new hypergraph H ( i ) new , if H ( i ) new ̸ = H v ′ for all v ′ ∈ V , we create a new node v ( i ) new = H ( i ) new , add it to the node set V , and create a new edge ( v, v ( i ) new ) that gets added to the edge set E of the geometry.
Revised: For each new hypergraph H ( i ) new , if H ( i ) new ̸ = H v ′ for all v ′ ∈ V , we create a new node v ( i ) new = H ( i ) new , add it to the node set V , and create a new edge ( v, v ( i ) new ) that gets added to the edge set E of the geometry.

Original: If there is an isomorphic hypergraph in the existing node set, i.e. H ˆ v = H ( i ) new , ˆ v ∈ V , we do not create any new node; instead, we add an edge ( v, ˆ v ) between the matched node ˆ v and the currently visited node v .
Revised: If there is an isomorphic hypergraph in the existing node set, i.e. H ˆ v = H ( i ) new , ˆ v ∈ V , we do not create any new node; instead, we add an edge ( v, ˆ v ) between the matched node ˆ v and the currently visited node v .

Original: The algorithm terminates once all the molecules involved in the property prediction task have been visited in the geometry.
Revised: The algorithm terminates once all the molecules involved in the property prediction task have been visited in the geometry.

Original: In practice, the construction of grammar-induced geometry is very costly, and often computationally intractable for more complex grammars.
Revised: In practice, the construction of grammar-induced geometry is very costly, and often computationally intractable for more complex grammars.

Original: By testing with random grammars learned from Guo et al.
Revised: By testing with random grammars learned from Guo et al.

Original: (2022), we find that it is infeasible to construct the geometry when there are more than ten production rules.
Revised: (2022), we find that it is infeasible to construct the geometry when there are more than ten production rules.

Original: The major bottleneck comes from the combinatorial complexity of production rules: The more rules a grammar has, the more intermediate hypergraphs it can generate.
Revised: The major bottleneck comes from the combinatorial complexity of production rules: The more rules a grammar has, the more intermediate hypergraphs it can generate.

Original: As the depth of the geometry increases, the number of intermediate hypergraphs increases exponentially.
Revised: As the depth of the geometry increases, the number of intermediate hypergraphs increases exponentially.

Original: This incurs a significant computational cost, as each newly expanded hypergraph requires pair-wise isomorphism tests.
Revised: This incurs a significant computational cost, as each newly expanded hypergraph requires pair-wise isomorphism tests.

Original: This cost also poses a serious obstacle to the optimization of the geometry, since the geometry must be rebuilt at each optimization iteration.
Revised: This cost also poses a serious obstacle to the optimization of the geometry, since the geometry must be rebuilt at each optimization iteration.

Original: 3. H IERARCHICAL M OLECULAR G RAMMARS
Revised: 3. H IERARCHICAL M OLECULAR G RAMMARS

Original: We propose a hierarchical molecular grammar to address the computational challenge mentioned above.
Revised: We propose a hierarchical molecular grammar to address the computational challenge mentioned above.

Original: Our insight stems from the fact that every hypergraph can be represented as a tree-structure object, i.e. a junction tree using tree decomposition (Diestel, 2005).
Revised: Our insight stems from the fact that every hypergraph can be represented as a tree-structure object, i.e. a junction tree using tree decomposition (Diestel, 2005).

Original: The tree representation brings two benefits: 1) It allows us to measure the underlying structural similarity between molecules by reflecting the scaffold of the substructure components (Jin et al., 2018); and 2) as a more compact representation, the possible junction tree structures are considerably fewer than molecular hypergraphs, and therefore can be enumerated.
Revised: The tree representation brings two benefits: 1) It allows us to measure the underlying structural similarity between molecules by reflecting the scaffold of the substructure components (Jin et al., 2018); and 2) as a more compact representation, the possible junction tree structures are considerably fewer than molecular hypergraphs, and therefore can be enumerated.

Original: By carefully designing the form of grammar rules, we divide the generation process of a molecular hypergraph into two parts: first generating a tree with homogeneous tree nodes, then converting the tree into a molecular hypergraph by specifying a subhypergraph per tree node.
Revised: By carefully designing the form of grammar rules, we divide the generation process of a molecular hypergraph into two parts: first generating a tree with homogeneous tree nodes, then converting the tree into a molecular hypergraph by specifying a subhypergraph per tree node.

Original: The grammar-induced geometry can thus be divided into two parts: a top part where geometry nodes represent only trees and a bottom part where the leaves of the geometry represent molecular hypergraphs.
Revised: The grammar-induced geometry can thus be divided into two parts: a top part where geometry nodes represent only trees and a bottom part where the leaves of the geometry represent molecular hypergraphs.

Original: Due to this hierarchical decomposition, the top part of the geometry is data-independent and can be computed offline to apply to any molecular datasets.
Revised: Due to this hierarchical decomposition, the top part of the geometry is data-independent and can be computed offline to apply to any molecular datasets.

Original: At run-time, we only need to construct the bottom part of the geometry, which is formulated as a byproduct of grammar learning in our approach.
Revised: At run-time, we only need to construct the bottom part of the geometry, which is formulated as a byproduct of grammar learning in our approach.

Original: As a result of the geometry construction, each molecule is connected to one junction tree in the top part.
Revised: As a result of the geometry construction, each molecule is connected to one junction tree in the top part.

Original: Molecular structure similarity is determined by the distance between their corresponding junction trees along the geometry.
Revised: Molecular structure similarity is determined by the distance between their corresponding junction trees along the geometry.

Original: A hierarchical molecular grammar consists of two sub-grammars that generate two parts of the geometry respectively.
Revised: A hierarchical molecular grammar consists of two sub-grammars that generate two parts of the geometry respectively.

Original: We refer to the sub-grammar that only generates trees as a meta grammar and the top-part geometry it constructs as a meta geometry .
Revised: We refer to the sub-grammar that only generates trees as a meta grammar and the top-part geometry it constructs as a meta geometry .

Original: We provide formal definitions of meta grammar and demonstrate that despite the additional hierarchical constraint, our hierarchical molecular grammar is as expressive as the general molecular hypergraph grammar in Guo et al.
Revised: We provide formal definitions of meta grammar and demonstrate that despite the additional hierarchical constraint, our hierarchical molecular grammar is as expressive as the general molecular hypergraph grammar in Guo et al.

Original: Figure 3 shows an overview of a hierarchical molecular grammar and its induced geometry.
Revised: Figure 3 shows an overview of a hierarchical molecular grammar and its induced geometry.

Original: Meta Grammars.
Revised: Meta Grammars.

Original: The formal definition of a meta grammar is as follows.
Revised: The formal definition of a meta grammar is as follows.

Original: Definition 1.
Revised: Definition 1.

Original: A meta grammar G =
Revised: A meta grammar G =

Original: ( N , ∅ , P G , X ) is a hypergraph grammar, which only contains non-terminal nodes and only generates trees, i.e. ∀ w ∈ { w |X ∗ ⇒
Revised: ( N , ∅ , P G , X ) is a hypergraph grammar, which only contains non-terminal nodes and only generates trees, i.e. ∀ w ∈ { w |X ∗ ⇒

Original: P G w } , w ∈ T .
Revised: P G w } , w ∈ T .

Original: A meta grammar G is k
Revised: A meta grammar G is k

Original: -degree
Revised: -degree

Original: if, for all trees T of maximal degree ∆( T ) ≤ k , we have X ∗ ⇒ P G T .
Revised: if, for all trees T of maximal degree ∆( T ) ≤ k , we have X ∗ ⇒ P G T .

Original: A meta grammar G is edit-complete if, for any tree pair ( T, T ′ ) with | T | < | T ′ | , tree edit distance
Revised: A meta grammar G is edit-complete if, for any tree pair ( T, T ′ ) with | T | < | T ′ | , tree edit distance

Original: TED ( T, T ′ )
Revised: TED ( T, T ′ )

Original: = 1 , and X ∗ ⇒ P G T, T ′ , there is a rule p ∈ P G such that T p ⇒ T ′ .
Revised: = 1 , and X ∗ ⇒ P G T, T ′ , there is a rule p ∈ P G such that T p ⇒ T ′ .

Original: A k-degree, edit-complete meta grammar G is minimal if there is no other such meta grammar G ′ with |P G ′ | < |P G | .
Revised: A k-degree, edit-complete meta grammar G is minimal if there is no other such meta grammar G ′ with |P G ′ | < |P G | .

Original: OH
Revised: OH

Original: OH N O
Revised: OH N O

Original: N O
Revised: N O

Original: Molecular Rule Learning MolecularRuleLearning
Revised: MolecularRuleLearning MolecularRuleLearning

Original: MolecularRuleLearning …
Revised: MolecularRuleLearning …

Original: HO
Revised: HO

Original: HO
Revised: HO

Original: M e t a G e o m e t r y
Revised: M e t a G e o m e t r y

Original: Meta tree node
Revised: Meta tree node

Original: Molecular leaf(a) Geometry induced by hierarchical molecular grammar (b) Meta grammar (
Revised: Molecular leaf

Original: [Figure]
Revised: [Figure]

Original: Molecular grammar… … / / /
Revised: induced by hierarchical molecular grammar

Original: The three additional attributes in the above definition can be used to deduce several desirable properties of a meta grammar. “Degree k ” ensures the meta grammar is expressive and complete, covering all possible trees under a simple tree degree constraint. “Edit completeness” enables the explicit capture of transformations between two trees with edit distance one and therefore between two arbitrary trees with arbitrary distances.
Revised: [Figure]

Original: “Minimality” guarantees that the meta grammaris compact. Based on these three attributes, we can construct a generic meta grammar generating trees of nonterminal nodes by using only a small but expressive set of production rules. Figure 3(b) shows a 3 -degree meta grammar. Each rule has one non-terminal node R ∗ on the LHS and two R ∗ s on the
Revised: grammar

Original: RHS . Depending on the number of anchor nodes on the LHS , these rules can transform tree nodes of different degrees by attaching a new node using different schemes indicated by the RHS . It is evident that we can generate any possible tree with a degree smaller than 4 by adopting a sequence of rules from this 3 -degree meta grammar. In Appendix B, we provide the formal construction of meta grammars for generating trees with arbitrary degree.
Revised: [Figure]

Original: Proposition 1.
Revised: Proposition 1.

Original: A meta grammar G with a meta rule set P G constructed in Appendix B is of degree k , edit-complete, and minimal.
Revised: A meta grammar G with a meta rule set P G constructed in Appendix H is of degree k , edit-complete, and minimal.

Original: Appendix B also provides the proof that the three attributes are satisfied.
Revised: Appendix H also provides the proof that the three attributes are satisfied.

Original: Generally speaking, we construct meta rule sets of arbitrary degrees by induction from 1 -degree meta grammar which consists of only one meta rule.
Revised: Generally speaking, we construct meta rule sets of arbitrary degrees by induction from 1 -degree meta grammar which consists of only one meta rule.

Original: In practice, a meta grammar with a reasonable degree should be chosen to ensure the coverage of most molecules.
Revised: In practice, a meta grammar with a reasonable degree should be chosen to ensure the coverage of most molecules.

Original: In our experiments, we find it sufficient to use a meta rule set of degree 4 , which contains 8 rules in total.
Revised: In our experiments, we find it sufficient to use a meta rule set of degree 4 , which contains 8 rules in total.

Original: Hierarchical Molecular Grammars.
Revised: Hierarchical Molecular Grammars.

Original: We define hierarchical molecular grammar as follows,
Revised: We define hierarchical molecular grammar as follows,

Original: Proposition 2.
Revised: Proposition 2.

Original: A hierarchical molecular grammar G =
Revised: A hierarchical molecular grammar G =

Original: ( G, G mol ) consisting of two hypergraph grammars: 1) a k -degree, edit-complete, and minimal meta grammar G defined in Definition 1; and 2) a molecular grammar G mol =
Revised: ( G, G mol ) consisting of two hypergraph grammars: 1) a k -degree, edit-complete, and minimal meta grammar G defined in Definition 1; and 2) a molecular grammar G mol =

Original: ( N , Σ , P mol , ∅ ) where the RHS of any molecular rule in P mol only contains terminal nodes (i.e. atoms), is a complete grammar.
Revised: ( N , Σ , P mol , ∅ ) where the RHS of any molecular rule in P mol only contains terminal nodes (i.e. atoms), is a complete grammar.

Original: “Completeness” states that any molecular graph can be derived from a hierarchical molecular grammar with a set of appropriate molecular rules. This can be demonstrated by the fact that we can generate an arbitrary molecule by first using the meta grammar to generate a tree of non-terminal nodes (which has the same tree structure as a junction tree decomposed from the molecule), and then using the molecular grammar to transform the tree into a molecular hypergraph by specifying a molecular fragment for each non-terminal node.
Revised: More discussion on the proposition is in Appendix H.

Original: The completeness of our hierarchical molecular grammar also shows that there is no loss of representation capacity as compared to general molecular hypergraph grammars.
Revised: The completeness of our hierarchical molecular grammar also shows that there is no loss of representation capacity as compared to general molecular hypergraph grammars.

Original: Geometry Construction and Molecular Grammar Learning.
Revised: Geometry Construction and Molecular Grammar Learning

Original: The geometry induced by the hierarchical molecular grammar is constructed in two phases, as shown in Figure 3(a): the top for the meta grammar and the bottom for the molecular grammar.
Revised: The geometry induced by the hierarchical molecular grammar is constructed in two phases, as shown in Figure 3(a): the top for the meta grammar and the bottom for the molecular grammar.

Original: Since the meta grammar is pre-defined, the top meta geometry can be pre-computed offline following the BFS procedure described in Section 3.1 with a given maximum BFS depth.
Revised: Since the meta grammar is pre-defined, the top meta geometry can be pre-computed offline following the BFS procedure described in Section 3.1 with a given maximum BFS depth.

Original: Each node in the meta geometry represents a tree of non-terminal nodes.
Revised: Each node in the meta geometry represents a tree of non-terminal nodes.

Original: We call the tree generated using the meta grammar a meta tree and the node in the meta geometry a meta tree node .
Revised: We call the tree generated using the meta grammar a meta tree and the node in the meta geometry a meta tree node .

Original: We find it sufficient to use a maximum depth of 10 in practice.
Revised: We find it sufficient to use a maximum depth of 10 in practice.

Original: The bottom part of the geometry determines how each molecule is connected to the meta geometry as a molecular leaf through its junction tree structure.
Revised: The bottom part of the geometry determines how each molecule is connected to the meta geometry as a molecular leaf through its junction tree structure.

Original: We obtain the junction trees of molecules simultaneously with the learning of molecular grammar rules.
Revised: We obtain the junction trees of molecules simultaneously with the learning of molecular grammar rules.

Original: Figure 3(c) illustrates the process.
Revised: Figure 3(c) illustrates the process.

Original: Thus, the geometry optimization can be achieved by learning the molecular grammar.
Revised: Thus, the geometry optimization can be achieved by learning the molecular grammar.

Original: We follow the grammar learning from Guo et al.
Revised: We follow the grammar learning from Guo et al.

Original: but constrain the learned rules to contain only terminal nodes on the RHS , so as to satisfy the molecular grammar.
Revised: but constrain the learned rules to contain only terminal nodes on the RHS , so as to satisfy the molecular grammar.

Original: Each molecule is considered as a hypergraph, and a set of hyperedges is iteratively sampled from the hypergraph until all hyperedges are visited.
Revised: Each molecule is considered as a hypergraph, and a set of hyperedges is iteratively sampled from the hypergraph until all hyperedges are visited.

Original: We construct the molecular rules along with the junction tree.
Revised: We construct the molecular rules along with the junction tree.

Original: At each iteration, a molecular rule is constructed for each connected component, where the LHS contains a single non-terminal node and the RHS contains the connected component.
Revised: At each iteration, a molecular rule is constructed for each connected component, where the LHS contains a single non-terminal node and the RHS contains the connected component.

Original: For the junction tree, we add a tree node representing the connected component and create a tree edge between two nodes if their corresponding connected components share a common hypergraph node.
Revised: For the junction tree, we add a tree node representing the connected component and create a tree edge between two nodes if their corresponding connected components share a common hypergraph node.

Original: The iterative sampling follows an i.i.d.
Revised: The iterative sampling follows an i.i.d.

Original: Bernoulli distribution based on a probability function ϕ
Revised: Bernoulli distribution based on a probability function ϕ

Original: ( e ; θ ) defined on each hyperedge e with learnable parameters θ .
Revised: ( e ; θ ) defined on each hyperedge e with learnable parameters θ .

Original: At the conclusion of the sampling process, we can obtain a junction tree of the molecular hypergraph along with a set of molecular rules for converting this junction tree back to the original hypergraph.
Revised: At the conclusion of the sampling process, we can obtain a junction tree of the molecular hypergraph along with a set of molecular rules for converting this junction tree back to the original hypergraph.

Original: The molecule is then connected to the meta tree node in the meta geometry that represents the isomorphic meta tree to the junction tree.
Revised: The molecule is then connected to the meta tree node in the meta geometry that represents the isomorphic meta tree to the junction tree.

Original: We provide a detailed formulation of the constructed molecular rule and the resulting junction tree in Appendix C, and more details of the grammar-induced geometry construction in Appendix D.
Revised: We provide a detailed formulation of the constructed molecular rule and the resulting junction tree in Appendix I, and more details of the grammar-induced geometry construction in Appendix J.

Original: 3. G RAPH D IFFUSION OVER G RAMMAR - INDUCED G EOMETRY
Revised: 3. G RAPH D IFFUSION OVER G RAMMAR - INDUCED G EOMETRY

Original: Property prediction requires a model that can operate on our grammar-induced geometry and is suitable for scarce data.
Revised: Property prediction requires a model that can operate on our grammar-induced geometry and is suitable for scarce data.

Original: We choose the graph neural diffusion model GRAND
Revised: We choose the graph neural diffusion model GRAND

Original: (Chamberlain et al., 2021) for its effectiveness in overcoming the oversmoothing that plagues most GNNs.
Revised: (Chamberlain et al., 2021) for its effectiveness in overcoming the oversmoothing that plagues most GNNs.

Original: Three learnable components are used in a graph diffusion process: an encoder function φ defined on all the nodes in the grammar-induced geometry, a decoder function ψ defined on molecular leaves, and a graph diffusion process given in Equation 1.
Revised: Three learnable components are used in a graph diffusion process: an encoder function φ defined on all the nodes in the grammar-induced geometry, a decoder function ψ defined on molecular leaves, and a graph diffusion process given in Equation 1.

Original: Specifically, the encoder φ yields the initial state of the diffusion process U 

Revised: Specifically, the encoder φ yields the initial state of the diffusion process U

Original: =
Revised: =

Original: φ ( U in ) , where U in is the matrix form of the input features of all the nodes.
Revised: φ ( U in ) , where U in is the matrix form of the input features of all the nodes.

Original: The decoder produces predicted property values of all molecular leaves u T =
Revised: The decoder produces predicted property values of all molecular leaves u T =

Original: ψ ( M ⊙ U T ) , where
Revised: ψ ( M ⊙ U T ) , where

Original: U T is the node-feature matrix from the final diffusion state, M is a binary mask that masks out the rows corresponding to non-molecule nodes, ⊙ is the Hadamard product, and u T is the resulting vector containing property values of all molecular leaves.
Revised: U T is the node-feature matrix from the final diffusion state, M is a binary mask that masks out the rows corresponding to non-molecule nodes, ⊙ is the Hadamard product, and u T is the resulting vector containing property values of all molecular leaves.

Original: Our optimization objective encodes the learning for both the grammar-induced geometry and the diffusion model:
Revised: Our optimization objective encodes the learning for both the grammar-induced geometry and the diffusion model:

Original: [Equation]
Revised: [Equation]

Original: where ˆ u represents the vector of ground-truth property values for all leaves and l ( · , · ) is a regression loss.
Revised: where ˆ u represents the vector of ground-truth property values for all leaves and l ( · , · ) is a regression loss.

Original: Recall that the geometry is only determined by the molecular rules, so the molecular grammar learning parameters θ are the only parameters relevant for obtaining the geometry.
Revised: Recall that the geometry is only determined by the molecular rules, so the molecular grammar learning parameters θ are the only parameters relevant for obtaining the geometry.

Original: Since θ and ( φ, ψ, α ) are two groups of independent variables, we exploit block coordinate descent (Wright, 2015) to optimize the objective.
Revised: Since θ and ( φ, ψ, α ) are two groups of independent variables, we exploit block coordinate descent (Wright, 2015) to optimize the objective.

Original: A detailed derivation is provided in Appendix E.
Revised: A detailed derivation is provided in Appendix K.

Original:  E VALUATION
Revised:  E VALUATION

Original: The experiments demonstrate the generality of our approach and answer the following questions:
Revised: The experiments demonstrate the generality of our approach and answer the following questions:

Original: • Does our approach outperform existing methods on small datasets ?
Revised: • Does our approach outperform existing methods on small datasets ?

Original: • How well does our approach perform on large, widely-studied datasets ?
Revised: • How well does our approach perform on large, widely-studied datasets ?

Original: • To what extent is our approach effective on extremely small datasets ?
Revised: • To what extent is our approach effective on extremely small datasets ?

Original: • Can our approach guide generative models toward molecules with optimal properties?[Table]
Revised: • Can our approach guide generative models toward molecules with optimal properties?

Original: 4. E XPERIMENT S ETUP
Revised: 4. E XPERIMENT S ETUP

Original: Data.
Revised: Data.

Original: We evaluate our approach on four datasets. The two small datasets of polymers are CROW (a curated dataset from literature) and Permeability (Yuan et al., 2021), which contain 234 and 353 samples, respectively.
Revised: We evaluate our approach on eight datasets: CROW (a curated dataset from literature), Permeability (Yuan et al., 2021), FreeSolv (Mobley & Guthrie, 2014), Lipophilicity (Wang et al., 2015),

Original: The two large datasets of drug-like molecules are FreeSolv (Mobley & Guthrie, 2014) and Lipophilicity (Wang et al., 2015), which contain 643 and 1 , 130 samples, respectively.
Revised: HOPV (Lopez et al., 2016), DILI (Ma et al., 2020), and PTC (Xu et al., 2018).

Original: All four datasets have continuous molecular property values and are suitable for regression models. For each dataset, we randomly split the data into 4 : 1 training and testing sets and create five such splits using 5 random seeds. Each of five splits is used to train and test a separate instance of each benchmark model.
Revised: These datasets cover: 1) commonly used benchmark datasets including MolecuNet (FreeSolv, Lipophilicity, HOPV, ClinTox) and TUDataset (PTC), 2) both classification (DILI, PTC, ClinTox) and regression tasks (CROW, Permeability, FreeSolv, Lipophilicity, HOPV), and 3) sizes that are small (CROW, Permeability, HOPV, PTC) and large (FreeSolv, Lipophilicity, ClinTox).

Original: We report theaverage and standard deviation of both mean absolute error (MAE) and the coefficient of determination ( R 2 ) across all testing sets.
Revised: We report the mean absolute error (MAE) and the coefficient of determination ( R 2 ) for regression, and Accuracy and ROC-AUC scores for classification.

Original:  Appendix F provides more details.
Revised: See Appendix E for more details.

Original: Baselines.
Revised: Baselines.

Original: We compare our approach with various approaches: Random Forest, FNN, wD-MPNN (D-MPNN), ESAN, HM-GNN, PN, and Pre-trained GIN.
Revised: We compare our approach with various approaches: Random Forest, FNN, wD-MPNN (D-MPNN), ESAN, HM-GNN, PN, and Pre-trained GIN.

Original: For descriptions, see Appendix G. To show the generalizability of our pipeline (called Geo-DEG), we implement two versions, each with a different diffusion encoder, GIN and MPNN.
Revised: For descriptions, see Appendix F. To show the generalizability of our pipeline (called Geo-DEG), we implement two versions, each with a different diffusion encoder, GIN and MPNN.

Original: Appendix G provides the implementation details. 
Revised: Appendix F provides the implementation details. [Table]

Original: 4. R ESULTS ON S MALL D ATASETSOF P OLYMERS
Revised: 4. R ESULTS ON S MALL D ATASETS

Revised: To answer the first question posted at the beginning

Revised: [Table]

Revised: we conduct experiments on five small datasets:

Revised: CROW, Permeability, HOPV, DILI

Original: Results & Discussion.
Revised: [Table] and PTC.

Original: Table 1 shows the results on the  two small polymer datasets.
Revised: Table 1 shows the results on the first two datasets, which contain polymers.

Original: In general,
Revised: Regarding MAE, the baselines ESAN and HM-GNN are only comparable with simple, non-deep learning methods (e.g., random forest), while pre-trained GIN and wDMPNN perform reasonably well. Both variants of our method outperform all the other methods by a large margin. Results and discussion on the other datasets are given in Appendix A.

Original:  CROW is a more challenging dataset than Permeability, as it has fewer samples and a broader range of property values (with a standard derivation of 86 .
Revised: Discussions. In general, CROW is a more challenging dataset than Permeability, as it has fewer samples and a broader range of property values (with a standard derivation of 86 .

Original: 8 versus 1 .
Revised: 8 versus 1 .

Original: 24 for Permeability).
Revised: 24 for Permeability).

Original: On CROW, traditional machine learning methods such as Random Forest and FNN are quite competitive and even outperform modern GNN architectures such as ESAN and HM-GNN.
Revised: On CROW, traditional machine learning methods such as Random Forest and FNN are quite competitive and even outperform modern GNN architectures such as ESAN and HM-GNN.

Original: wD-MPNN achieves reasonable performance thanks to the special graph representation for polymer ensembles.
Revised: wD-MPNN achieves reasonable performance thanks to the special graph representation for polymer ensembles.

Original: PN and Pre-trained GIN perform exceptionally well on CROW due to their pre-training on large datasets.
Revised: PN and Pre-trained GIN perform exceptionally well on CROW due to their pre-training on large datasets.

Original: However, this pre-training does not help on Permeability, which has a much larger average molecule size (with an average molecular weight of 391 . 8 versus 153 . 8 for CROW).
Revised: However, this pre-training does not help on Permeability, which has a much larger average molecule size (with an average molecular weight of 391 . 8 versus 153 . 8 for CROW).

Original: Thus, there is a domain gap between the dataset where PN and Pre-trained GIN are pre-trained and the Permeability dataset, resulting in poor performance of both methods.
Revised: Thus, there is a domain gap between the dataset where PN and Pre-trained GIN are pre-trained and the Permeability dataset, resulting in poor performance of both methods.

Original: ESAN benefits from the subgraph selection scheme on large molecular graphs and performs well on Permeability.
Revised: ESAN benefits from the subgraph selection scheme on large molecular graphs and performs well on Permeability.

Original: The poor performance of HM-GNN on both datasets shows that it is not as effective in regression tasks as it is in classification tasks.
Revised: The poor performance of HM-GNN on both datasets shows that it is not as effective in regression tasks as it is in classification tasks.

Original: The overall results show that: 1) our method is capable of handling polymers with varying molecular sizes, and 2) the superior performance of our method when coupled with either GIN or MPNN confirms its generalizability to different feature extractors.
Revised: The overall results show that: 1) our method is capable of handling polymers with varying molecular sizes, and 2) the superior performance of our method when coupled with either GIN or MPNN confirms its generalizability to different feature extractors.

Original: 4. R ESULTS ON L ARGE D ATASETSOF S MALL M OLECULES
Revised: 4. R ESULTS ON L ARGE D ATASETS

Revised: To answer the second question, we conduct experiments on three large datasets: FreeSolv,

Original: Results & Discussion.
Revised: Lipophilicity, and ClinTox. Tables 2 and 5 in Appendix A show the results. For FreeSolv and Lipophilicity, simple machine learning methods including random forest and FFN perform poorly.

Original: Table 2 in Appendix A shows the results on the two large datasets of druglike molecules.
Revised: Pre-trained GNNs cannot perform as well as they do on small polymer datasets. The performance of advanced GNN architectures, especially D-MPNN, is reasonably good on both datasets. Our method equipped with MPNN diffusion encoder performs the best among all approaches. Results and discussion on the other datasets are given in Appendix A.

Original:  On large datasets, random forest and simple FFN become less competitive than on small datasets.
Revised: Discussions. On large datasets, random forest and simple FFN become less competitive than on small datasets.

Original: This is reasonable since larger datasets require larger model capacities to represent the relationship between molecules and their properties.
Revised: This is reasonable since larger datasets require larger model capacities to represent the relationship between molecules and their properties.

Original: Pre-training does not significantly help on either dataset due to the inherent domain gap between datasets used for pre-training versus those[Figure] used for fine-tuning.
Revised: Pre-training does not significantly help on either dataset due to the inherent domain gap between datasets used for pre-training versus those used for fine-tuning.

Original: Among the three GNN-based baselines, D-MPNN performs the best and exhibits better stability than ESAN and HM-GNN.
Revised: Among the three GNN-based baselines, D-MPNN performs the best and exhibits better stability than ESAN and HM-GNN.

Original: Our approach can further improve the performance of D-MPNN and outperforms all the baselines.
Revised: Our approach can further improve the performance of D-MPNN and outperforms all the baselines.

Original: These results demonstrate that our method is scalable and competitive with state-of-the-art methods for large molecule datasets. 
Revised: These results demonstrate that our method is scalable and competitive with state-of-the-art methods for large molecule datasets. [Figure]

Original: 4. A NALYSIS
Revised: 4. A NALYSIS

Original: Combination with Generative Models.
Revised: This section examines the effectiveness of our method on extremely small datasets, its combination with generative models that answer the last two questions posted at the beginning and an ablation study on the meta grammar. Performance on Extremely Small Datasets. We conduct a study on the minimum viable number of samples for our method. We randomly sample 20% of CROW as a fixed testing set.

Revised: Using the remaining data, we construct eight randomly selected training sets, each consisting of a portion of all remaining data, ranging from 20% to 100% .

Revised: These eight training sets are used to train our method using GIN as the feature extractor.

Revised: Figure 4(b) shows the performance on the testing set.

Revised: Even when the training set is halved (with only 94 samples), our approach still achieves results that are comparable to those of the Pre-trained GIN fine-tuned on the whole training set.

Revised: Appendix B includes a more detailed study for the effect of changing dataset training size on model performance.

Original:  In our optimization framework, grammar metrics can be considered as additional objectives, allowing us to jointly optimize both generative models and property predictors.
Revised: Combination with Generative Models. In our optimization framework, grammar metrics can be considered as additional objectives, allowing us to jointly optimize both generative models and property predictors.

Original: Following Guo et al.
Revised: Following Guo et al.

Original: (2022), we use diversity and Retro ∗ score as our grammar metrics and then perform joint optimization on the Permeability dataset.
Revised: (2022), we use diversity and Retro ∗ score as our grammar metrics and then perform joint optimization on the Permeability dataset.

Original: After training, we generate 400 molecules and predict their property values using our approach, including all six property types from Permeability.
Revised: After training, we generate 400 molecules and predict their property values using our approach, including all six property types from Permeability.

Original: Figure 4(a) shows two out of six predicted property values with the Pareto front highlighted.
Revised: Figure 4(a) shows two out of six predicted property values with the Pareto front highlighted.

Original: Clearly, our approach can be combined with generative models to provide a comprehensive pipeline for the discovery of optimal molecules.
Revised: Clearly, our approach can be combined with generative models to provide a comprehensive pipeline for the discovery of optimal molecules.

Original: The use of Retro ∗ also enables finding synthesis paths of generated molecules as shown in Appendix I.
Revised: The use of Retro ∗ also enables finding synthesis paths of generated molecules as shown in Appendix L.

Original: Performance on Extremely Small Datasets.

Original: We conduct a study on the minimum viable number of samples for our method.

Original: We randomly sample 20% of CROW as a fixed testing set.

Original: Using the remaining data, we construct eight randomly selected training sets, each consisting of a portion of all remaining data, ranging from 20% to 100% .

Original: These eight training sets are used to train our method using GIN as the feature extractor.

Original: Figure 4(b) shows the performance on the testing set.

Original: Even when the training set is halved (with only 94 samples), our approach still achieves results that are comparable to those of the Pre-trained GIN fine-tuned on the whole training set.

Original: Ablation Study on Meta Grammar.
Revised: Ablation Study on Meta Grammar.

Original: In this study, we examine the necessity of the meta rules in our hierarchical grammar.
Revised: In this study, we examine the necessity of the meta rules in our hierarchical grammar.

Original: We remove two rules that have degree 4 on the LHS from the 4 -degree meta grammar, resulting in a meta geometry with same number of nodes but 10% fewer edges than the one used in our main experiments.
Revised: We remove two rules that have degree 4 on the LHS from the 4 -degree meta grammar, resulting in a meta geometry with same number of nodes but 10% fewer edges than the one used in our main experiments.

Original: With this modified meta grammar, we run the pipeline for all four datasets and compare the results with the original meta grammar in Figure 4(c).
Revised: With this modified meta grammar, we run the pipeline for all four datasets and compare the results with the original meta grammar in Figure 4(c).

Original: All four datasets exhibit a performance drop when using the modified meta grammar.
Revised: All four datasets exhibit a performance drop when using the modified meta grammar.

Original: The results provide experimental evidence for the necessity of a complete meta grammar.
Revised: The results provide experimental evidence for the necessity of a complete meta grammar.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We propose a data-efficient molecular property predictor based on a hierarchical molecular grammar.
Revised: We propose a data-efficient molecular property predictor based on a hierarchical molecular grammar.

Original: The grammar induces an explicit geometry describing the space of molecular graphs, such that a graph neural diffusion on the geometry can be used to effectively predict property values of molecules on small training datasets.
Revised: The grammar induces an explicit geometry describing the space of molecular graphs, such that a graph neural diffusion on the geometry can be used to effectively predict property values of molecules on small training datasets.

Original: One avenue of future work is to extend our pipeline to model 3D molecular structures and to address general graph design problems.
Revised: One avenue of future work is to extend our pipeline to model 3D molecular structures and to address general graph design problems.

Original: A A DDITIONAL R ESULTS
Revised: A A DDITIONAL R ESULTS

Original: [Table]
Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: Results & Discussion.

Revised: For HOPV, since it is a small regression dataset, traditional methods (random forest and FFN) achieve competitive performance and are even better than several graph neural networks including ESAN and HM-GNN.

Revised: Both variants of our method outperform other baselines by a large margin, which aligns with the results in the other two small regression datasets.

Revised: In the two small classification datasets, DILI and PTC, there is no significant difference between baseline models regarding performance.

Revised: The reason for this is that the test dataset is small, and the accuracy of the model can be limited by a few hard examples that cannot be classified correctly.

Revised: It is therefore more informative to use the ROC-AUC score.

Revised: Among the baselines, HM-GNN performs the best due to its motif-based representation.

Revised: Our method can still outperform all other methods regarding both accuracy and ROC-AUC score.

Revised: For ClinTox in Table 5, we compare our approach with a wide range of highly performant baseline methods from a recent paper Zhou et al.

Revised: which

Revised: [Table]

Revised: by itself proposes one of the SOTA methods on ClinTox.

Revised: Note that these are very strong baseline methods, including many methods pre-trained using 3D molecular data.

Revised: The overall results show that our method achieves significantly better performance on challenging small regression datasets and outperforms a wide spectrum of baselines on various common benchmarks.

Revised: B A NALYSIS ON T RAINING D ATASET S IZE

Revised: We further extend our study in Section 4.4 for the effect of changing dataset training size on model performance by including the analysis of all three small regression datasets with more baselines compared.

Revised: Figure 5 illustrates the results of Pre-trained GIN, wD-MPNN/D-MPNN, and two variants of our model on CROW, Permeability, and HOPV.

Revised: We report the performance of each model trained using different numbers of training samples, randomly sampled from the original training dataset.

Revised: Across different training dataset sizes, our proposed method consistently outperforms other baselines.

Revised: A smaller training dataset leads to a larger performance improvement, demonstrating the data-efficiency of our model.

Revised: [Figure]

Revised: C A NALYSIS ON CONSTRUCTION COST REDUCTION OF HIERARCHICAL MOLECULAR GRAMMAR

Revised: In this section, we provide empirical evidence on the construction cost reduction of our hierarchical molecular grammar compared with non-hierarchical molecular grammar from Guo et al.

Revised: We conduct ten groups of experiments by randomly sampling five molecules from CROW dataset by ten times.

Revised: For each group of experiment, we sample ten grammars for both hierarchical and non-hierarchical versions respectively and construct the geometry to cover the five molecules.

Revised: The non-hierarchical grammar is sampled using the algorithm from Guo et al.

Revised: The geometry construction is achieved following the BFS procedure described in Section 3.1 for both kinds of

Revised: 10 Experiment ID

Revised: A v e r a g e R unn i ng

Revised: T i m e ( s )   1.8 × 101 2 3 4 5 6 7 8 91 2 3 4 5 6 7 8 9 10 Experiment ID

Revised: Experiment ID

Original: B M ETA G RAMMAR C ONSTRUCTION
Revised: A v e r a g e N u m b e r o f R u l e s A v e r a g e

Revised: N u m b e r o f

Revised: I n t e r m e d i a t e G r a ph s

Revised: [Figure]

Revised: grammar.

Revised: In the experiments, we find that for non-hierarchical grammar, even for five molecules, it occurs frequently that the size of the geometry grows extremely large but still cannot cover the five molecules.

Revised: Therefore, we stop the construction when the size of the geometry reaches 2 × 10 5 .

Revised: Figure 6 illustrates (a) the average running time, (b) the average number of production rules, and (c) the average number of intermediate graphs of the final constructed geometry.

Revised: It can be noted that our hierarchical molecular grammar greatly reduces the running time of the geometry construction.

Revised: For non-hierarchical grammars, it is always intractable to construct the geometry since the number of intermediate graphs is enormous (some approaching 1 . 5 × 10 5 ).

Revised: Our hierarchical molecular grammar provides a practical framework for grammar-induced geometry with theoretical foundations.

Revised: D A NALYSIS ON C OMPUTATIONAL C OMPLEXITY

Revised: The computation of our approach consists of three parts: 1) the construction of grammar-induced geometry, 2) diffusion encoder, and 3) graph diffusion over the geometry.

Revised: Following Blakely et al.

Revised: (2021), suppose we have K samples of molecules, each with N nodes and |E| edges.

Revised: To construct the grammar-induced geometry, we need to sample all the edges in order to contract molecules into junction trees, so the computational complexity is O ( K |E| ) .

Revised: For diffusion encoder, the complexity is the same with general graph neural networks, which is O ( KLd ( Nd + |E| )) , where L is the number of layeres and d is the feature dimension.

Revised: As indicated in Chamberlain et al.

Revised: (2021), the complexity of graph diffusion is O ( |E ′ | d )( E b + E f ) , where E ′ is the edge set of grammar-induced geometry, and E b , E f are the numbers of function evaluations for forward and backward pass, respectively.

Revised: Therefore, the overall computational cost of our method is O

Revised: ( K |E| + KLd ( Nd + |E| )+ |E ′ |

Revised: d ( E b +

Revised: E f ))

Revised: =

Revised: O ( KLd ( Nd + |E| ) + |E ′ |

Revised: d ( E b + E f )) .

Revised: Compared with general graph neural networks which have a computational complexity of O ( KLd ( Nd + |E| )) , our approach entails additional computations O ( |E ′ | d ( E b + E f )) , where |E ′ | is the sum of the number of edges in the meta geometry |E meta | and the number of samples K , since each molecule is connected to the meta geometry by an edge.

Revised: In our experiments, we use a 4 -degree meta grammar with depth 10 , which gives us a meta geometry of 455 edges, i.e. 455 =

Revised: |E meta | ≪ K |E| .

Revised: We also find in our experiments that

Revised: E b + E f ≪ LNd .

Revised: As a result, the overall computational complexity of our method is the same as that of general graph neural networks in the big O sense.

Revised: In practice, geometry can be constructed in parallel, which reduces the running time of our entire method.

Revised: E D ETAILS ON D ATASETS

Revised: The Chemical Retrieval on the Web (CROW) polymer database 2 curates a database of thermophysical data for over 250 different polymers, focusing primarily on the most commonly used plastics and resins in industry.

Revised: CROW distinguishes from other digital polymer property databases due to its heavy usage of real, experimental data individually sourced via extensive literature search.

Revised: In the CROW polymer database, approximately 95% of reported polymers offer some experimental data from literature, with the remaining 5% derived purely from simulation.

Revised: We only use those data with labels from real experiments in our evaluation.

Revised: The CROW polymer database reports several properties including the glass transition temperature, Hildebrand solubility parameter, molar heat capacity, refractive index, and molar cohesive energy.

Revised: Of these, we choose glass transition temperature as our benchmark performance comparison since it is one of the best documented polymer properties in the literature (Tao et al., 2021b).

Revised: For each dataset except ClinTox, we randomly split the data into 4 : 1 training and testing sets and create five such splits using 5 random seeds.

Revised: For

Revised: ClinTox, we follow the settings in Zhou et al.

Revised: (2022) to train our model and report the results of other baselines directly from the paper.

Revised: Each of five splits is used to train and test a separate instance of each benchmark model.

Revised: For the other three datasets, we refer the reader to the original papers:

Revised: Permeability (Yuan et al., 2021), FreeSolv (Mobley & Guthrie, 2014), Lipophilicity (Wang et al., 2015), HOPV (Lopez et al., 2016), DILI (Ma et al., 2020), and PTC (Xu et al., 2018).

Revised: F D ETAILS ON THE I MPLEMENTATION

Revised: Baselines.

Revised: We compare our approach with various baselines: 1) Random Forest and FFN, two of the best-performing machine learning models for polymer informatics benchmarking (Tao et al., 2021b); 2) wD-MPNN, a state-of-the-art method specifically designed for polymer property prediction (Aldeghi & Coley, 2022); 3) ESAN, a general GNN architecture with enhanced expressive power (Bevilacqua et al., 2022); 4) HM-GNN, a motif-based GNN for molecular feature representation (Yu & Gao, 2022); 5) PN and Pre-trained GIN, two pre-trained GNNs with state-of-the-art performance on few-shot learning (Stanley et al., 2021) and transfer learning (Hu et al., 2020).

Revised: As

Revised: HM-GNN and Pre-trained GIN only provide code for graph classification, we modify the final layer of their networks and use l 1 loss for training.

Revised: For the other methods, we follow the same implementation as their original papers.

Revised: Since wD-MPNN cannot be depolyed on general molecules other than polymers, we report the results of D-MPNN (Yang et al., 2019) instead for two large datasets.

Revised: Our System.

Revised: For our approach, we use 4 -degree meta grammar, which contains eight rules.

Revised: The meta geometry contains all the meta trees whose size is smaller than 11 , resulting in 149 nodes and 455 edges.

Revised: For the molecular rule learning of θ , we follow all the hyperparamters used in Guo et al.

Revised: For the graph diffusion, the input feature of each meta tree node is the Weisfeiler Lehman graph hashing feature (Shervashidze et al., 2011).

Revised: The encoder for meta tree nodes is an embedding layer that maps hashing features into a 300 -dimension continuous vector.

Revised: For molecular leaves, we use two different encoders: GIN from Xu et al.

Revised: (2018) and MPNN from Yang et al.

Revised: (2019), both of which output a feature vector of dimension 300 .

Revised: For the decoder, we use a one-layer fully connected network with size 300 .

Revised: For the graph diffusion process, we follow Chamberlain et al.

Revised: and use

Revised: Dormand–Prince adaptive step size scheme (DIORI5) with adjoint method.

Revised: The diffusivity function a ( · , · ; α ) is an attention function.

Revised: We use Adam optimizer for the training of θ and ( φ, ψ, α ) , with learning rate 0 . 01 and 0 .

Revised: 001 , respectively.

Revised: We train θ for ten epochs.

Revised: For each training epoch of θ , we train ( φ, ψ, α ) for 50 epochs.

Revised: G R ELATED WORKS

Revised: Machine Learning for Molecular Property Prediction.

Revised: The use of machine learning methods to predict molecular properties has a long history.

Revised: Many early methods use SMILES strings or handcrafted fingerprints as input and rely on traditional machine learning methods, such as random forest and Gaussian processes, which are still competitive today in many applications (Tao et al., 2021b; Joo et al., 2022).

Revised: Recently, graph-based representations of molecules have gained increasing popularity with the development of GNNs (Feinberg et al., 2018;

Revised: Xu et al., 2018; Wu et al., 2021;

Revised: Bevilacqua et al., 2022; Yu & Gao, 2022; Aldeghi & Coley, 2022; Alon & Yahav, 2020).

Revised: We refer the reader to Wieder et al.

Revised: (2020) for a detailed review of GNN-based property predictors.

Revised: Current state-of-the-art GNN-based methods provide more advanced representations of molecules based on graphs.

Revised: Bevilacqua et al.

Revised: (2022) represents each individual graph as a set of subgraphs, increasing the expressive power of GNNs.

Revised: Aldeghi & Coley (2022) tailors molecular graphs by adding stochastic edges and designs GNN specifically for polymers based on Yang et al.

Revised: Yu & Gao (2022) and Zhang et al.

Revised: (2021) leverage motifs to model molecular relationships.

Revised: All these methods require large training datasets to achieve reasonable performance.

Revised: It is common to use self-supervised learning (Rong et al., 2020;

Revised: Wang et al., 2022) and transfer learning (Hu et al., 2020) to deal with sparse data, where neural networks are pre-trained on large datasets and then fine-tuned on the target small dataset.

Revised: Most of these methods, however, focus solely on molecular classification, while our approach can also address regression problems with extremely sparse data, which are considerably more challenging.

Revised: Molecular Grammars.

Revised: As an interpretable and compact design model, grammar has recently gained increasing attention in the field of molecule discovery (Dai et al., 2018; Kajino, 2019; Krenn et al., 2019; Xu et al., 2020; Nigam et al., 2021;

Revised: Guo et al., 2021a; 2022).

Revised: A molecular grammar can be considered as a generative model that uses production rules to generate molecules.

Revised: The rules of a grammar can be either manually constructed (Dai et al., 2018; Krenn et al., 2019;

Revised: Guo et al., 2021a) or automatically learned from molecular datasets (Kajino, 2019; Xu et al., 2020; Nigam et al., 2021;

Revised: Guo et al., 2022).

Revised: Recent works have demonstrated that a learnable hypergraph grammar can be effective in capturing hard chemical constraints, such as valency restrictions (Kajino, 2019; Guo et al., 2022).

Revised: Guo et al.

Revised: (2022) further proposes a learning pipeline for constructing a hypergraph grammar-based generative model that can incorporate domain-specific knowledge from very small datasets with dozens of samples.

Revised: Despite the inherent advantages of molecular grammars, such as explicitness, explanatory power, and data efficiency, existing works mainly use them for molecular generation.

Revised: Prediction and optimization of molecular properties can only be accomplished by using a separate model based on the grammar representation in a latent space, which is learned individually (Kajino, 2019; Xu et al., 2020).

Revised: We integrate molecular grammar into property prediction tasks by constructing a geometry of molecular graphs based on the learnable grammar, which allows us to optimize both the generative model and the property predictor simultaneously.

Revised: On extremely small datasets, our approach benefits especially from the data efficiency of grammar and achieves superior performance over existing property predictors.

Revised: Hierarchical Molecular Generation.

Revised: The decomposition of molecular structures in our hierarchical molecular grammar is related to substructure-based methods in molecular generation (Jin et al., 2018; 2020; Maziarz et al., 2021).

Revised: Jin et al.

Revised: (2018) employs encoders and decoders to generate a junction tree-structured scaffold and then combine chemical substructures into a molecule.

Revised: Jin et al.

Revised: (2020) uses a hierarchical graph encoder-decoder to generate molecules in a coarse-to-fine manner, from atoms to connected motifs.

Revised: Maziarz et al.

Revised: (2021) integrates molecule fragments and atomby-atom construction to generate new molecules.

Revised: Our approach is fundamentally different from all these existing hierarchical representations in two respects: 1) Without the need for any training, our proposed meta grammar (the coarse level) can enumerate all possible junction tree structures by using a compact set of meta production rules, which can be theoretically guaranteed .

Revised: Our method only requires learning at the fine level, which are molecular fragments determined by molecular rules, whereas existing methods require learning two models for both coarse and fine levels.

Revised: As opposed to existing methods that use latent spaces, our meta geometry induced by meta grammar explicitly models the similarity between molecules by using graph distance along the geometry.

Revised: Due to the edit-completeness of meta grammar in Definition 1, the graph distance is an explainable measurement of minimal editing distance between molecular graphs, whereas the distance in latent spaces used in existing methods lacks explainability.

Revised: Geometric Deep Learning applies deep neural networks to non-Euclidean domains such as graphs and manifolds with a wide range of applications (Bronstein et al., 2017; Cao et al., 2020; Bronstein et al., 2021).

Revised: Related to our method, a series of recent works use graph neural diffusion by treating GNNs as a discretisation of an underlying heat diffusion PDE (Chamberlain et al., 2021;

Revised: Elhag et al., 2022; Bodnar et al., 2022).

Revised: These methods work on large graph data where the graph connectivity of individual nodes is provided.

Revised: Additionally, there are also existing works on inferring the underlying geometry of graph data.

Revised: Ganea et al.

Revised: (2018) and Cruceru et al.

Revised: (2021) embed graphs into non-Euclidean manifolds such as hyperbolic and elliptical spaces and optimize their embeddings within these Riemannian spaces.

Revised: Cosmo et al.

Revised: (2020) learns a latent graph to model the underlying relationship between data samples and applies GNN to it.

Revised: Different from these methods, our approach uses explicit and learnable intrinsic geometry based on graph grammar to model the relationship between molecular data.

Revised: Since graph grammar is a generative model, our framework is capable of optimizing both molecular generation and property prediction simultaneously.

Revised: H M ORE D ETAILS OF H IERARCHICAL M OLECULAR G RAMMAR

Revised: Discussion on Hierarchical Molecular Grammar.

Revised: The three additional attributes in Definitioncan be used to deduce several desirable properties of a meta grammar.

Revised: “Degree k ” ensures the meta grammar is expressive and complete, covering all possible trees under a simple tree degree constraint.

Revised: “Edit completeness” enables the explicit capture of transformations between two trees with edit distance one and therefore between two arbitrary trees with arbitrary distances.

Original: Meta Rules Construction.
Revised: “Minimality” guarantees that the meta grammar is compact. Based on these three attributes, we can construct a generic meta grammar generating trees of non-terminal nodes by using only a small but expressive set of production rules. Figure 3(b) shows a 3 -degree meta grammar. Each rule has one non-terminal node R ∗ on the LHS and two R ∗ s on the RHS .

Revised: Depending on the number of anchor nodes on the

Revised: LHS , these rules can transform tree nodes of different degrees by attaching a new node using different schemes indicated by the RHS .

Revised: It is evident that we can generate any possible tree with a degree smaller than 4 by adopting a sequence of rules from this 3 -degree meta grammar.

Revised: In Proposition 2 “Completeness” states that any molecular graph can be derived from a hierarchical molecular grammar with a set of appropriate molecular rules.

Revised: This can be demonstrated by the fact that we can generate an arbitrary molecule by first using the meta grammar to generate a tree of non-terminal nodes (which has the same tree structure as a junction tree decomposed from the molecule), and then using the molecular grammar to transform the tree into a molecular hypergraph by specifying a molecular fragment for each non-terminal node.

Original:  We refer to the production rule set of k -degree, edit-complete, and minimal meta grammar as a k -degree meta rule set .
Revised: Meta Rules Construction. We refer to the production rule set of k -degree, edit-complete, and minimal meta grammar as a k -degree meta rule set .

Original: We visualize 4 -degree meta rules in Figure 5a, where { p 1 } , { p 1 , p 2 , p 3 } , and { p 1 , p 2 , p 3 , p 4 , p 5 } correspond to meta rule sets of degree 1 to 3 , respectively.
Revised: We visualize 4 -degree meta rules in Figure 7a, where { p 1 } , { p 1 , p 2 , p 3 } , and { p 1 , p 2 , p 3 , p 4 , p 5 } correspond to meta rule sets of degree 1 to 3 , respectively.

Original: 1 -degree meta rule set P (1) G is constructed as
Revised: 1 -degree meta rule set P (1) G is constructed as

Original: [Equation]
Revised: [Equation]

Original: When using the meta grammar for production, we treat R ∗ 1 , R ∗ 2 as the same type of non-terminal node, i.e. R ∗ =
Revised: When using the meta grammar for production, we treat R ∗ 1 , R ∗ 2 as the same type of non-terminal node, i [Equation]

Original:  R ∗ i , i = 1 , 2 , despite the use of indices in the above formulation.
Revised: e. R ∗ = R ∗ i , i = 1 , 2 , despite the use of indices in the above formulation.

Original: For k -degree meta rule set P ( k ) G ( k > 1 ), the construction is achieved by induction:
Revised: For k -degree meta rule set P ( k ) G ( k > 1 ), the construction is achieved by induction:

Original: [Equation]
Revised: [Equation]

Original: [-∗ , V anc,j ) } , V ( k ) R, 
 = {R ∗ 1 , R ∗ 2 }-] ∪ k − 1 (cid:91)
Revised: [-∗ , V anc,j ) } , V ( k ) R, 
 = {R ∗ 1 , R ∗ 2 }-] ∪ k − 1 (cid:91)

Original: j[-= { V anc,j } , E ( k ) R, 
-]
Revised: j[-= { V anc,j } , E ( k ) R, 
-]

Original: = {  ( R ∗ 1 , R ∗ 2 ) } ∪ k − 1 (cid:91) 
Revised: = { V anc,j } , E ( k ) R, 0 = { ( R ∗ 1 , R ∗ 2 ) } ∪ k − 1 (cid:91) j

Original: j = { ( V anc,j , R ∗ 2 ) } ,
Revised: = { ( V anc,j , R ∗ 2 ) } ,

Original: V ( k ) L,i =
Revised: V ( k ) L,i =

Original: {R ∗ } ∪ k (cid:91)
Revised: {R ∗ } ∪ k (cid:91)

Original: j = { V anc,j } , E ( k ) L,i =
Revised: j = { V anc,j } , E ( k ) L,i =

Original: k (cid:91)
Revised: k (cid:91)

Original: j = { ( R ∗ , V anc,j ) } ,
Revised: j = { ( R ∗ , V anc,j ) } ,

Original: V ( k ) R,i =
Revised: V ( k ) R,i =

Original: {R ∗ 1 , R ∗ 2 } ∪ k (cid:91)
Revised: {R ∗ 1 , R ∗ 2 } ∪ k (cid:91)

Original: j = { V anc,j } , E ( k ) R,i =
Revised: j = { V anc,j } , E ( k ) R,i =

Original: { ( R ∗ 1 , R ∗ 2 ) } ∪ i (cid:91)
Revised: { ( R ∗ 1 , R ∗ 2 ) } ∪ i (cid:91)

Original: j = { ( V anc,j , R ∗ 1 ) } ∪ k (cid:91) j
Revised: j = { ( V anc,j , R ∗ 1 ) } ∪ k (cid:91) j

Original: = i
Revised: = i

Original: + { ( V anc,j , R ∗ 2 ) } .
Revised: + { ( V anc,j , R ∗ 2 ) } .

Original: Specifically, the k -degree meta rule set P ( k ) G contains all the rules from ( k − 1) -degree meta rule set
Revised: Specifically, the k -degree meta rule set P ( k ) G contains all the rules from ( k − 1) -degree meta rule set

Original: P ( k − 1) G as well as other newly introduced rules P ( k ) .
Revised: P ( k − 1) G as well as other newly introduced rules P ( k ) .

Original: Each rule contains one non-terminal node R ∗ on the LHS and two non-terminal nodes R ∗ 1 and R ∗ 2 on the RHS .
Revised: Each rule contains one non-terminal node R ∗ on the LHS and two non-terminal nodes R ∗ 1 and R ∗ 2 on the RHS .

Original: The LHS of p ( k ) 0 has a degree of k − 1 as it contains k − 1 anchor nodes V anc .
Revised: The LHS of p ( k ) 0 has a degree of k − 1 as it contains k − 1 anchor nodes V anc .

Original: The RHS of p ( k )attaches one non-terminal node R
Revised: The RHS of p ( k )attaches one non-terminal node R

Original: ∗to the other non-terminal node R ∗ 1 which connects all the anchor nodes.
Revised: ∗to the other non-terminal node R ∗ 1 which connects all the anchor nodes.

Original: Therefore, the RHS has a degree of k which is larger than the degree of the LHS by one.
Revised: Therefore, the RHS has a degree of k which is larger than the degree of the LHS by one.

Original: In all the other rules, the LHS has k anchor nodes and is of degree k while the RHS keeps a maximal degree of k .
Revised: In all the other rules, the LHS has k anchor nodes and is of degree k while the RHS keeps a maximal degree of k .

Original: The k anchor nodes on the RHS are combinatorially distributed to R ∗ 1 and R ∗ 2 , i.e. if R ∗ 1 is attached to i anchor nodes,
Revised: The k anchor nodes on the RHS are combinatorially distributed to R ∗ 1 and R ∗ 2 , i.e. if R ∗ 1 is attached to i anchor nodes,

Original: R ∗ 2 is attached to the rest of k − i anchor nodes.
Revised: R ∗ 2 is attached to the rest of k − i anchor nodes.

Original: Since we consider R ∗ 1 and R ∗ 2 to be the same during grammar production, the range of i is { 1 , ...,
Revised: Since we consider R ∗ 1 and R ∗ 2 to be the same during grammar production, the range of i is { 1 , ...,

Original: ⌊ k 2 ⌋} .
Revised: ⌊ k 2 ⌋} .

Original: In total, there are 1+ ⌊ k 2 ⌋ rules in P ( k ) .
Revised: In total, there are 1+ ⌊ k 2 ⌋ rules in P ( k ) .

Original: Proof of Edit Completeness.
Revised: Proof of Edit Completeness.

Original: According to the definition in Zhang (1996); Paaßen (2018), tree edit distance TED ( T, T ′ ) between two trees T and T ′ is the minimum number of operations required to transform one tree into the other.
Revised: According to the definition in Zhang (1996); Paaßen (2018), tree edit distance TED ( T, T ′ ) between two trees T and T ′ is the minimum number of operations required to transform one tree into the other.

Original: There are three types of available operations: inserting, deleting, and relabeling.
Revised: There are three types of available operations: inserting, deleting, and relabeling.

Original: Consider two arbitrary trees T and T ′ that satisfy the definition of edit completeness (being derived from the meta grammar, TED ( T , T ′ )
Revised: Consider two arbitrary trees T and T ′ that satisfy the definition of edit completeness (being derived from the meta grammar, TED ( T , T ′ )

Original: = 1 , and | T | < | T ′ | ).
Revised: = 1 , and | T | < | T ′ | ).

Original: Since all the trees generated from the meta grammar have homogeneous tree node labels (i.e. R ∗ ), there is no relabeling operation in TED calculation.
Revised: Since all the trees generated from the meta grammar have homogeneous tree node labels (i.e. R ∗ ), there is no relabeling operation in TED calculation.

Original: Furthermore, the tree size relation between T and T ′ admits only one type of tree edit operation: a one-step inserting.
Revised: Furthermore, the tree size relation between T and T ′ admits only one type of tree edit operation: a one-step inserting.

Original: Figure 5b illustrates the one-step inserting operation.
Revised: Figure 7b illustrates the one-step inserting operation.

Original: It operates on a certain node v in T whose subtree is T sub ( v ) .
Revised: It operates on a certain node v in T whose subtree is T sub ( v ) .

Original: A new node v ′ , which takes a subset of T sub ( v ) as the children, is added as a new child of node v .
Revised: A new node v ′ , which takes a subset of T sub ( v ) as the children, is added as a new child of node v .

Original: The rest of T sub ( v ) stays as the children of node v .
Revised: The rest of T sub ( v ) stays as the children of node v .

Original: Suppose the degree of node v in T is m and the degree related to subtree T sub ( v ) is n .
Revised: Suppose the degree of node v in T is m and the degree related to subtree T sub ( v ) is n .

Original: Without loss of generality, we suppose 0 ≤ n ≤ m/ 2 .
Revised: Without loss of generality, we suppose 0 ≤ n ≤ m/ 2 .

Original: We can treat v in T as the R ∗ on the LHS , v ′ and v in T ′
Revised: We can treat v in T as the R ∗ on the LHS , v ′ and v in T ′

Original: as R ∗ 1 and R ∗ 2 on the RHS , respectively.
Revised: as R ∗ 1 and R ∗ 2 on the RHS , respectively.

Original: Then the one-step inserting operation can be achieved using the rule p ( m +1) n ∈ P ( m +1) from ( m +1)
Revised: Then the one-step inserting operation can be achieved using the rule p ( m +1) n ∈ P ( m +1) from ( m +1)

Original: -degree meta rule set P ( m +1) G as constructed above.
Revised: -degree meta rule set P ( m +1) G as constructed above.

Original: Since ∆( T ′ ) ≥ m +1 , the meta grammar P ( k ) G that generates T ′ has k ≥ m +1 .
Revised: Since ∆( T ′ ) ≥ m +1 , the meta grammar P ( k ) G that generates T ′ has k ≥ m +1 .

Original: So p ( m +1) n ∈ P ( m +1) ⊆
Revised: So p ( m +1) n ∈ P ( m +1) ⊆

Original: P ( k ) G ; thus finish the proof.
Revised: P ( k ) G ; thus finish the proof.

Original: [Figure]
Revised: [Figure]

Original: Proof of Degree k .
Revised: Proof of Degree k .

Original: For an arbitrary tree T with maximal degree k , we prove that meta grammar P ( k ) G can generate
Revised: For an arbitrary tree T with maximal degree k , we prove that meta grammar P ( k ) G can generate

Original: T by finding a reverse path of production rule from the tree T to the initial node X .
Revised: T by finding a reverse path of production rule from the tree T to the initial node X .

Original: We first find all the nodes { v i } ni =1 in T that have a degree of k .
Revised: We first find all the nodes { v i } ni =1 in T that have a degree of k .

Original: For each node, by edit completeness, we can use the inverse operation of inserting and make the node to have a degree of k − 1 .
Revised: For each node, by edit completeness, we can use the inverse operation of inserting and make the node to have a degree of k − 1 .

Original: It can be shown from the proof for edit completeness that each inverse operation of inserting corresponds to an inverse deployment of a production rule.
Revised: It can be shown from the proof for edit completeness that each inverse operation of inserting corresponds to an inverse deployment of a production rule.

Original: By using n reverse production steps on all the k -degree nodes, we can obtain a
Revised: By using n reverse production steps on all the k -degree nodes, we can obtain a

Original: T ′ with ∆( T ′ )
Revised: T ′ with ∆( T ′ )

Original: =
Revised: =

Original: k − 1 .
Revised: k − 1 .

Original: We can continue the process until the tree is 0 -degree, i.e. the X ; thus finish the proof.
Revised: We can continue the process until the tree is 0 -degree, i.e. the X ; thus finish the proof.

Original: Proof of Minimality.
Revised: Proof of Minimality.

Original: From edit completeness, we know each production rule in the meta grammar corresponds to at least one case of one-step inserting operation.
Revised: From edit completeness, we know each production rule in the meta grammar corresponds to at least one case of one-step inserting operation.

Original: As constrainted by degree k , arbitratry trees that are of degree smaller than k need to be covered by the meta grammar.
Revised: As constrainted by degree k , arbitratry trees that are of degree smaller than k need to be covered by the meta grammar.

Original: We thus need to handle all the possible cases of one-step inserting operation when finding the reverse production path from the tree to X .
Revised: We thus need to handle all the possible cases of one-step inserting operation when finding the reverse production path from the tree to X .

Original: Note that one-step inserting opeartion involves the division of a subtree, which has combinatorial ways.
Revised: Note that one-step inserting opeartion involves the division of a subtree, which has combinatorial ways.

Original: Then every combinatorial distribution of anchor nodes corresponds to one case of subtree division.
Revised: Then every combinatorial distribution of anchor nodes corresponds to one case of subtree division.

Original: This proves each production rule in the meta rule set is necessary.
Revised: This proves each production rule in the meta rule set is necessary.

Original: Since there are no duplicate rules, it can be concluded that the meta grammar constructed above is minimal.
Revised: Since there are no duplicate rules, it can be concluded that the meta grammar constructed above is minimal.

Original: 4 -degree Meta Rules.
Revised: 4 -degree Meta Rules.

Original: Figure 5a shows the 4 -degree meta grammar we use in our approach.
Revised: Figure 7a shows the 4 -degree meta grammar we use in our approach.

Original: Since the edit completeness is the foundation to prove all the other attributes of the meta grammar, we use a tree edit distance toolkit 2 and verify its correctness for the 4 -degree meta grammar.
Revised: Since the edit completeness is the foundation to prove all the other attributes of the meta grammar, we use a tree edit distance toolkit 3 and verify its correctness for the 4 -degree meta grammar.

Original: C M OLECULAR R ULES C ONSTRUCTION
Revised: I M OLECULAR R ULES C ONSTRUCTION

Revised: 1 Initialize P

Revised: T

Revised: 2 Create set nodes V unv ← V a set for unvisited hyperedges E

Revised: ←

Revised: 3 while E unv ̸ = ∅ do e

Original: k = ;
Revised: [ e e K ] e i ∈ E unv i K = | E V

Original:  E anc ← E anc ∪ { ( s, v ( i ) anc ) | ( s, v ) ∈
Revised: V anc ∪ { v ) anc i = | V anc | + k } | V n ∩ V sub | k = ; E anc ← E anc ∪ { ( s, v ( i ) anc ) | ( s, v ) ∈

Original: E sub , i = | V anc |
Revised: E sub , i = | V anc |

Original: + k } | V n ∩ V sub | k
Revised: + k } | V n ∩ V sub | k

Original: = ; /* Construct a production rule */ p =
Revised: = ; /* Construct a production rule */ p =

Original: RuleConstruction( ˆ V sub , ˆ E sub , V anc , E anc ) ;
Revised: RuleConstruction( ˆ V sub , ˆ E sub , V anc , E anc ) ;

Original: P mol ← P mol ∪ { p } ; /* Construct junction tree */ v T =
Revised: P mol ← P mol ∪ { p } ; /* Construct junction tree */ v T =

Original: ( V sub ) ; for each { v t ∈ V T | v t ∩ v T ̸ = ∅ } do
Revised: ( V sub ) ; for each { v t ∈ V T | v t ∩ v T ̸ = ∅ } do

Original: E T ← E T ∪ { ( v t , v T ) } ;
Revised: E T ← E T ∪ { ( v t , v T ) } ;

Original: V T ← V T ∪ { v T } ; /* update visited status
Revised: V T ← V T ∪ { v T } ; /* update visited status

Original: V unv ← V unv ∪ V sub , E unv ← E unv ∪ E sub ; 26 return P mol , T =
Revised: V unv ← V unv ∪ V sub , E unv ← E unv ∪ E sub ; 26 return P mol , T =

Original: ( V T , E T ) ;
Revised: ( V T , E T ) ;

Original: To obtain the input for the molecular rule construction, we first convert the molecule into a molecular hypergraph H =
Revised: To obtain the input for the molecular rule construction, we first convert the molecule into a molecular hypergraph H =

Original: ( V, E H ) .
Revised: ( V, E H ) .

Original: A node v ∈ V represents an atom of the molecule.
Revised: A node v ∈ V represents an atom of the molecule.

Original: A hyperedge e ∈ E H corresponds to either a bond that joins only two nodes or a ring (including aromatic ones) that joins all nodes in the ring.
Revised: A hyperedge e ∈ E H corresponds to either a bond that joins only two nodes or a ring (including aromatic ones) that joins all nodes in the ring.

Original: An illustration is provided in Figure 2 of Guo et al.
Revised: An illustration is provided in Figure 2 of Guo et al.

Original: The probability function ϕ ( · ; θ ) is defined on each hyperedge: ϕ ( e ; θ )
Revised: The probability function ϕ ( · ; θ ) is defined on each hyperedge: ϕ ( e ; θ )

Original: =
Revised: =

Original: σ ( −F θ ( f ( e ))) , where σ ( · ) is the sigmoid function, F θ ( · ) is a two-layer fully connected network whose final output dimension is 1 , and f ( · ) is a feature extractor using a pre-trained GNN (Hu et al., 2020).
Revised: σ ( −F θ ( f ( e ))) , where σ ( · ) is the sigmoid function, F θ ( · ) is a two-layer fully connected network whose final output dimension is 1 , and f ( · ) is a feature extractor using a pre-trained GNN (Hu et al., 2020).

Original: Algorithm 1 illustrates the detailed process of constructing molecular rules for a single molecule.
Revised: Algorithm 1 illustrates the detailed process of constructing molecular rules for a single molecule.

Original: Note that in our approach, the molecular rule construction is performed simultaneously for all input molecules.
Revised: Note that in our approach, the molecular rule construction is performed simultaneously for all input molecules.

Original: For each molecule, we first perform an i.i.d.
Revised: For each molecule, we first perform an i.i.d.

Original: sampling on all the hyperedges following a Bernoulli distribution which takes the value 1 with probability indicated by ϕ ( e ; θ ) .
Revised: sampling on all the hyperedges following a Bernoulli distribution which takes the value 1 with probability indicated by ϕ ( e ; θ ) .

Original: We then obtain a binary vector X that indicates whether each hyperedge is sampled (line 4 - 5 ).
Revised: We then obtain a binary vector X that indicates whether each hyperedge is sampled (line 4 - 5 ).

Original: Next, all connected components are extracted with respect to the sampled hyperedges (line 6 ).
Revised: Next, all connected components are extracted with respect to the sampled hyperedges (line 6 ).

Original: A production rule is constructed for each connected component H sub
Revised: A production rule is constructed for each connected component H sub

Original: = ( V sub , E sub ) (line 7 - 19 ).
Revised: = ( V sub , E sub ) (line 7 - 19 ).

Original: Specifically, a production rule needs two components: a hypergraph ˆ H sub =
Revised: Specifically, a production rule needs two components: a hypergraph ˆ H sub =

Original: ( ˆ V sub , ˆ E sub ) for RHS and anchor nodes V anc indicating the correspondence between the LHS and the RHS .
Revised: ( ˆ V sub , ˆ E sub ) for RHS and anchor nodes V anc indicating the correspondence between the LHS and the RHS .

Original: ˆ H sub is obtained by removing visited nodes and in-ring nodes from H sub .
Revised: ˆ H sub is obtained by removing visited nodes and in-ring nodes from H sub .

Original: V anc contains nodes from V that are connected to nodes from H sub but do not appear in H sub themselves.
Revised: V anc contains nodes from V that are connected to nodes from H sub but do not appear in H sub themselves.

Original: E anc are the edges connecting anchor nodes to ˆ H sub following the same connectivity in the original graph H .
Revised: E anc are the edges connecting anchor nodes to ˆ H sub following the same connectivity in the original graph H .

Original: Following Guo et al.
Revised: Following Guo et al.

Original: (2022), the function RuleConstruction( ˆ V sub , ˆ E sub , V anc , E anc ) returns a production rule p : LHS →
Revised: (2022), the function RuleConstruction( ˆ V sub , ˆ E sub , V anc , E anc ) returns a production rule p : LHS →

Original: RHS constructed as
Revised: RHS constructed as

Original: [Equation]
Revised: [Equation]

Original: In contrast to Guo et al. (2022), our molecular rule construction does not replace the connected component with the non-terminal node R ∗ at every iteration since according to our definition, the molecular rule does not contain any non-terminal nodes on the RHS .
Revised: In contrast to Guo et al. (2022), our molecular rule construction does not replace the connected component with the non-terminal node R ∗ at every iteration since according to our definition, the molecular rule does not contain any non-terminal nodes on the RHS .

Original: The junction tree of the molecule is constructed along with the construction of molecular rules (line 21 - 24 ).
Revised: The junction tree of the molecule is constructed along with the construction of molecular rules (line 21 - 24 ).

Original: We simply treat each connected component V sub sampled at each iteration as a node v T ∈ V T in the junction tree T =
Revised: We simply treat each connected component V sub sampled at each iteration as a node v T ∈ V T in the junction tree T =

Original: ( V T , E T ) .
Revised: ( V T , E T ) .

Original: Two nodes are connected by an edge e T ∈ E T if their corresponding connected components share hypergraph nodes from H .
Revised: Two nodes are connected by an edge e T ∈ E T if their corresponding connected components share hypergraph nodes from H .

Original: Since each hyperedge in H is only visited once, the constructed junction tree then contains all the hyperedges and nodes of H without redundancy (Kajino, 2019).
Revised: Since each hyperedge in H is only visited once, the constructed junction tree then contains all the hyperedges and nodes of H without redundancy (Kajino, 2019).

Original: D G RAMMAR - INDUCED G EOMETRY C ONSTRUCTION
Revised: J G RAMMAR - INDUCED G EOMETRY C ONSTRUCTION

Original: Input : meta production rules P G = { p i } Ni =1 , maximum BFS depth D , a set of molecular hypergraphs { H i } Mi =1 and their corresponding junction trees J =
Revised: Input : meta production rules P G = { p i } Ni =1 , maximum BFS depth D , a set of molecular hypergraphs { H i } Mi =1 and their corresponding junction trees J =

Original: { T i } Mi =
Revised: { T i } Mi =

Original: Output: geometry in the form of a graph G =
Revised: Output: geometry in the form of a graph G =

Original: ( V , E ) where each v =
Revised: ( V , E ) where each v =

Original: H v =
Revised: H v =

Original: V v , E v ) ∈
Revised: V v , E v ) ∈

Original: V represents a meta tree or a molecular hypergraph and E is the edge set of the geometry geometry
Revised: V represents a meta tree or a molecular hypergraph and E is the edge set of the geometry geometry

Original: Algorithm 2 illustrates the detailed algorithm to construct the grammar-induced geometry.
Revised: Algorithm 2 illustrates the detailed algorithm to construct the grammar-induced geometry.

Original: It contains two parts: the construction of the meta geometry (line 4 - 14 ) and the construction of the molecular leaves (line 15 - 19 ).
Revised: It contains two parts: the construction of the meta geometry (line 4 - 14 ) and the construction of the molecular leaves (line 15 - 19 ).

Original: The meta geometry construction follows the standard breadth-first search
Revised: The meta geometry construction follows the standard breadth-first search

Original: (BFS) starting from the root H root =
Revised: (BFS) starting from the root H root =

Original: ( X , ∅ ) (line 1 ).
Revised: ( X , ∅ ) (line 1 ).

Original: Every time when we visit a node v ∈ V in
Revised: Every time when we visit a node v ∈ V in

Original: G , we find all the rules that are applicable to H v from the meta rule set (line 6 - 7 ).
Revised: G , we find all the rules that are applicable to H v from the meta rule set (line 6 - 7 ).

Original: Each applicable rule is applied to H v in order to create a new meta tree H new (line 8 ).
Revised: Each applicable rule is applied to H v in order to create a new meta tree H new (line 8 ).

Original: Depending on whether there is a node in V that represents a isomorphic tree to H new in the current geometry, we either create a new node v new (line 9 ) or find the existing matched node ˆ v (line 14 ).
Revised: Depending on whether there is a node in V that represents a isomorphic tree to H new in the current geometry, we either create a new node v new (line 9 ) or find the existing matched node ˆ v (line 14 ).

Original: We then add an edge between v and v new , or between v and ˆ v (line 11 and 14 ).
Revised: We then add an edge between v and v new , or between v and ˆ v (line 11 and 14 ).

Original: The function GetIsomorphicGraph( H new , V ) enumerates every node in V , checks if the tree represented by the node is isomorphic to H new , and returns the matched node.
Revised: The function GetIsomorphicGraph( H new , V ) enumerates every node in V , checks if the tree represented by the node is isomorphic to H new , and returns the matched node.

Original: We use the algorithm from Cordella et al.
Revised: We use the algorithm from Cordella et al.

Original: (2001) which is implemented in networkx 3 for graph isomorphism test.
Revised: (2001) which is implemented in networkx 4 for graph isomorphism test.

Original: To increase the speed of GetIsomorphicGraph( H new ,
Revised: To increase the speed of GetIsomorphicGraph( H new ,

Original: V ) , we use Weisfeiler Lehman graph hash
Revised: V ) , we use Weisfeiler Lehman graph hash

Original: (Shervashidze et al., 2011) and only perform isomorphism test for graph pairs that share the same hashing code.
Revised: (Shervashidze et al., 2011) and only perform isomorphism test for graph pairs that share the same hashing code.

Original: The molecular leaves are added during run-time (line 15 - 19 ).
Revised: The molecular leaves are added during run-time (line 15 - 19 ).

Original: For each input molecular hypergraph, we check if there is a meta tree node that is isomorphic to its junction tree.
Revised: For each input molecular hypergraph, we check if there is a meta tree node that is isomorphic to its junction tree.

Original: If so, we add an edge connecting the matched meta tree node to the molecular hypergraph.
Revised: If so, we add an edge connecting the matched meta tree node to the molecular hypergraph.

Original: The function IsIsomorphic( H v ,
Revised: The function IsIsomorphic( H v ,

Original: T i ) is implemented using the same package for GetIsomorphicGraph( H new , V ) and returns true if H v and T i are isomorphic to each other.
Revised: T i ) is implemented using the same package for GetIsomorphicGraph( H new , V ) and returns true if H v and T i are isomorphic to each other.

Original: E O PTIMIZATION
Revised: [Equation]

Original: X θ u T φ ψ α l ( · , · ) ˆ u
Revised: X θ u T φ ψ α l ( · , · ) ˆ u

Original: Figure 6 shows a graphical model of dependency in our optimization problem.
Revised: Figure 8 shows a graphical model of dependency in our optimization problem.

Original: The two sets of parameters θ and ( φ, ψ, α ) are independent and thus can be iteratively optimized using block coordinate descent.
Revised: The two sets of parameters θ and ( φ, ψ, α ) are independent and thus can be iteratively optimized using block coordinate descent.

Original: Since the objective in Equation 2 is differentiable with respect to ( φ, ψ, α ) , we can use gradient descent for the optimization iterations related to the graph neural diffusion.
Revised: Since the objective in Equation 2 is differentiable with respect to ( φ, ψ, α ) , we can use gradient descent for the optimization iterations related to the graph neural diffusion.

Original: The geometry learning, however, is non-differentiable due to the fact that we perform sampling to construct molecular rules and construct the geometry.
Revised: The geometry learning, however, is non-differentiable due to the fact that we perform sampling to construct molecular rules and construct the geometry.

Original: Hence, we rewrite the objective for optimizing θ in an expectation form and apply REINFORCE (Williams, 1992) to obtain a stochastic gradient, as done in Guo et al.
Revised: Hence, we rewrite the objective for optimizing θ in an expectation form and apply REINFORCE (Williams, 1992) to obtain a stochastic gradient, as done in Guo et al.

Original: [Equation]
Revised: [Equation]

Original: where X is a concatenation of binary vectors indicating how hyperedges are sampled in molecular rule construction in Appendix C.
Revised: where X is a concatenation of binary vectors indicating how hyperedges are sampled in molecular rule construction in Appendix I.

Original: F D ETAILS ON D ATASETS

Original: The Chemical Retrieval on the Web (CROW) polymer database 4 curates a database of thermophysical data for over 250 different polymers, focusing primarily on the most commonly used plastics and resins in industry.

Original: CROW distinguishes from other digital polymer property databases due to its heavy usage of real, experimental data individually sourced via extensive literature search.

Original: In the CROW polymer database, approximately 95% of reported polymers offer some experimental data from literature, with the remaining 5% derived purely from simulation.

Original: We only use those data with labels from real experiments in our evaluation.

Original: The CROW polymer database reports several properties including the glass transition temperature, Hildebrand solubility parameter, molar heat capacity, refractive index, and molar cohesive energy.

Original: Of these, we choose glass transition temperature as our benchmark performance comparison since it is one of the best documented polymer properties in the literature (Tao et al., 2021b).

Original: For the other three datasets, we refer the reader to the original papers:

Original: Permeability (Yuan et al., 2021), FreeSolv (Mobley & Guthrie, 2014), and Lipophilicity (Wang et al., 2015).

Original: G D ETAILS ON THE I MPLEMENTATION

Original: Baselines.

Original: We compare our approach with various baselines: 1) Random Forest and FFN, two of the best-performing machine learning models for polymer informatics benchmarking (Tao et al., 2021b); 2) wD-MPNN, a state-of-the-art method specifically designed for polymer property prediction (Aldeghi & Coley, 2022); 3) ESAN, a general GNN architecture with enhanced expressive power (Bevilacqua et al., 2022); 4) HM-GNN, a motif-based GNN for molecular feature representation (Yu & Gao, 2022); 5) PN and Pre-trained GIN, two pre-trained GNNs with state-of-the-art performance on few-shot learning (Stanley et al., 2021) and transfer learning (Hu et al., 2020).

Original: As

Original: HM-GNN and Pre-trained GIN only provide code for graph classification, we modify the final layer of their networks and use l 1 loss for training.

Original: For the other methods, we follow the same implementation as their original papers.

Original: Since wD-MPNN cannot be depolyed on general molecules other than polymers, we report the results of D-MPNN (Yang et al., 2019) instead for two large datasets.

Original: Our System.

Original: For our approach, we use 4 -degree meta grammar, which contains eight rules.

Original: The meta geometry contains all the meta trees whose size is smaller than 11 , resulting in 149 nodes and 455 edges.

Original: For the molecular rule learning of θ , we follow all the hyperparamters used in Guo et al.

Original: For the graph diffusion, the input feature of each meta tree node is the Weisfeiler Lehman graph hashing feature (Shervashidze et al., 2011).

Original: The encoder for meta tree nodes is an embedding layer that maps hashing features into a 300 -dimension continuous vector.

Original: For molecular leaves, we use two different encoders: GIN from Xu et al.

Original: (2018) and MPNN from Yang et al.

Original: (2019), both of which output a feature vector of dimension 300 .

Original: For the decoder, we use a one-layer fully connected network with size 300 .

Original: For the graph diffusion process, we follow Chamberlain et al.

Original: and use

Original: Dormand–Prince adaptive step size scheme (DIORI5) with adjoint method.

Original: The diffusivity function a ( · , · ; α ) is an attention function.

Original: We use Adam optimizer for the training of θ and ( φ, ψ, α ) , with learning rate 0 . 01 and 0 .

Original: 001 , respectively.

Original: We train θ for ten epochs.

Original: For each training epoch of θ , we train ( φ, ψ, α ) for 50 epochs.

Original: H R ELATED WORKS

Original: Machine Learning for Molecular Property Prediction.

Original: The use of machine learning methods to predict molecular properties has a long history.

Original: Many early methods use SMILES strings or handcrafted fingerprints as input and rely on traditional machine learning methods, such as random forest and Gaussian processes, which are still competitive today in many applications (Tao et al., 2021b; Joo et al., 2022).

Original: Recently, graph-based representations of molecules have gained increasing popularity with the development of GNNs (Feinberg et al., 2018;

Original: Xu et al., 2018; Wu et al., 2021;

Original: Bevilacqua et al., 2022; Yu & Gao, 2022; Aldeghi & Coley, 2022; Alon & Yahav, 2020).

Original: We refer the reader to Wieder et al.

Original: (2020) for a detailed review of GNN-based property predictors.

Original: Current state-of-the-art GNN-based methods provide more advanced representations of molecules based on graphs.

Original: Bevilacqua et al.

Original: (2022) represents each individual graph as a set of subgraphs, increasing the expressive power of GNNs.

Original: Aldeghi & Coley (2022) tailors molecular graphs by adding stochastic edges and designs GNN specifically for polymers based on Yang et al.

Original: Yu & Gao (2022) and Zhang et al.

Original: (2021) leverage motifs to model molecular relationships.

Original: All these methods require large training datasets to achieve reasonable performance.

Original: It is common to use selfsupervised learning (Rong et al., 2020;

Original: Wang et al., 2022) and transfer learning (Hu et al., 2020) to deal with sparse data, where neural networks are pre-trained on large datasets and then fine-tuned on the target small dataset.

Original: Most of these methods, however, focus solely on molecular classification, while our goal is to address regression problems, which are considerably more challenging.

Original: Grammars for Molecular Generation.

Original: As an interpretable and compact design model, graph grammar has recently gained increasing attention in the field of molecular generation (Dai et al., 2018; Kajino, 2019; Krenn et al., 2019; Xu et al., 2020; Nigam et al., 2021;

Original: Guo et al., 2021a; 2022).

Original: Recent works have demonstrated that a learnable hypergraph grammar can be effective in capturing hard chemical constraints, such as valency restrictions (Kajino, 2019; Guo et al., 2022).

Original: Guo et al.

Original: (2022) further proposes a learning pipeline for constructing a hypergraph grammar-based generative model that can incorporate domain-specific knowledge from very small datasets with dozens of samples.

Original: Our approach leverages the data-efficiency of hypergraph grammar for property prediction tasks.

Original: We describe more details of hypergraph grammar in Section 2.

Original: Geometric Deep Learning applies deep neural networks to non-Euclidean domains such as graphs and manifolds with a wide range of applications (Bronstein et al., 2017; Cao et al., 2020; Bronstein et al., 2021).

Original: Related to our method, a series of recent works use graph neural diffusion by treating GNNs as a discretisation of an underlying heat diffusion PDE (Chamberlain et al., 2021;

Original: Elhag et al., 2022; Bodnar et al., 2022).

Original: These methods work on large graph data where the graph connectivity of individual nodes is provided.

Original: Additionally, there are also existing works on inferring the underlying geometry of graph data.

Original: Ganea et al.

Original: (2018) and Cruceru et al.

Original: (2021) embed graphs into non-Euclidean manifolds such as hyperbolic and elliptical spaces and optimize their embeddings within these Riemannian spaces.

Original: Cosmo et al.

Original: (2020) learns a latent graph to model the underlying relationship between data samples and applies GNN to it.

Original: Different from these methods, our approach uses explicit and learnable intrinsic geometry based on graph grammar to model the relationship between molecular data.

Original: Since graph grammar is a generative model, our framework is capable of optimizing both molecular generation and property prediction simultaneously.

Original: I E XAMPLES OF R ETRO -S YNTHESIS P ATHS
Revised: L E XAMPLES OF R ETRO -S YNTHESIS P ATHS

Original: Figure 7 shows the retro-synthesis paths of three molecules generated using our pipeline.
Revised: Figure 9 shows the retro-synthesis paths of three molecules generated using our pipeline.

Original: Since
Revised: Since

Original: Retro ∗ score is used as one of the grammar metrics, our approach is capable of retro-synthesis planning for all the generated molecules, providing a complete pipeline for novel molecule discovery.
Revised: Retro ∗ score is used as one of the grammar metrics, our approach is capable of retro-synthesis planning for all the generated molecules, providing a complete pipeline for novel molecule discovery.

Original: O
Revised: O

Original: OH
Revised: OH

Original: Br
Revised: Br

Original: O
Revised: O

Original: O
Revised: O

Original: OSO O OS O O
Revised: OSO O OS O O

Original: N P O N N
Revised: N P O N N

Original: S O
Revised: S O

Original: O Cl
Revised: O Cl

Original: O
Revised: O

Original: OH OSO O
Revised: OH OSO O

Original: O
Revised: O

Original: HO
Revised: HO

Original: OH
Revised: OH

Original: S O
Revised: S O

Original: O Cl
Revised: O Cl

Original: O
Revised: O

Original: OH O HO
Revised: OH O HO

Original: O
Revised: O

Original: O HO
Revised: O HO

Original: O
Revised: O

Original: O O
Revised: O O

Original: O
Revised: O

Original: OH O - O OOH
Revised: OH O - O OOH

Original: Br
Revised: Br

Original: O
Revised: O

Original: OSO O OS O O
Revised: OSO O OS O O

Original: N P O N N
Revised: N P O N N

Original: S O
Revised: S O

Original: O Cl
Revised: O Cl

Original: O
Revised: O

Original: OH OSO O
Revised: OH OSO O

Original: O
Revised: O

Original: O HO
Revised: O HO

Original: O S
Revised: O S

Revised: O O Cl

Original: [Figure]
Revised: O OH

Original: OSO O 
Revised: OSO O O O

Original: O
Revised: O

Original:  O  OOH
Revised: OH O - O OO O

Original: [Figure] [Figure] O
Revised: O O

Original: OO O O
Revised: O

Original: O
Revised: O

Original: OH
Revised: OH

Original: O
Revised: O

Original: OH
Revised: OH

Original: O
Revised: O

Original: O O O
Revised: O O O

Original: O
Revised: O

Original: O O
Revised: O O

Original: OH
Revised: OH

Original: F F F

Original: Cl

Original: Cl

Original: F F F Cl Cl F F F
Revised: [Figure]