Original: A NISOTROPIC M ESSAGE P ASSING : G RAPH N EURAL N ETWORKS WITH D IRECTIONAL AND L ONG -R ANGE I NTERACTIONS
Revised: A NISOTROPIC M ESSAGE P ASSING : G RAPH N EURAL N ETWORKS WITH D IRECTIONAL AND L ONG -R ANGE I NTERACTIONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Graph neural networks have shown great potential for the description of a variety of chemical systems.
Revised: Graph neural networks have shown great potential for the description of a variety of chemical systems.

Original: However, standard message passing does not explicitly account for long-range and directional interactions, for instance due to electrostatics.
Revised: However, standard message passing does not explicitly account for long-range and directional interactions, for instance due to electrostatics.

Original: In this work, an anisotropic state based on Cartesian multipoles is proposed as an addition to the existing hidden features.
Revised: In this work, an anisotropic state based on Cartesian multipoles is proposed as an addition to the existing hidden features.

Original: With the anisotropic state, message passing can be modified to explicitly account for directional interactions.
Revised: With the anisotropic state, message passing can be modified to explicitly account for directional interactions.

Original: Compared to existing models, this modification results in relatively little additional computational cost.
Revised: Compared to existing models, this modification results in relatively little additional computational cost.

Original: Most importantly, the proposed formalism offers as a distinct advantage the seamless integration of (1) anisotropic long-range interactions, (2) interactions with surrounding fields and particles that are not part of the graph, and (3) the fast multipole method.
Revised: Most importantly, the proposed formalism offers as a distinct advantage the seamless integration of (1) anisotropic long-range interactions, (2) interactions with surrounding fields and particles that are not part of the graph, and (3) the fast multipole method.

Original: As an exemplary use case, the application to quantum mechanics/molecular mechanics (QM/MM) systems is demonstrated.
Revised: As an exemplary use case, the application to quantum mechanics/molecular mechanics (QM/MM) systems is demonstrated.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Message passing graph neural networks (GNN) have shown great potential for the description of a wide range of chemical systems (Scarselli et al., 2009; Battaglia et al., 2016; 2018).
Revised: Message passing graph neural networks (GNN) have shown great potential for the description of a wide range of chemical systems (Scarselli et al., 2009; Battaglia et al., 2016; 2018).

Original: Particularly the description of quantum molecular (QM) systems with machine learning (ML) potentials has received a lot of interest (Gilmer et al., 2017; Unke et al., 2021b).
Revised: Particularly the description of quantum molecular (QM) systems with machine learning (ML) potentials has received a lot of interest (Gilmer et al., 2017; Unke et al., 2021b).

Original: However, in its general form, message passing does not explicitly account for directionality, which plays an important role in many physical interactions (Glotzer & Solomon, 2007; Kramer et al., 2014).
Revised: However, in its general form, message passing does not explicitly account for directionality, which plays an important role in many physical interactions (Glotzer & Solomon, 2007; Kramer et al., 2014).

Original: In recent years, a range of models which include directional information have been proposed (Anderson et al., 2019;
Revised: In recent years, a range of models which include directional information have been proposed (Anderson et al., 2019;

Original: Klicpera et al., 2020; Miller et al., 2020; Sch¨utt et al., 2021).
Revised: Klicpera et al., 2020; Miller et al., 2020; Sch¨utt et al., 2021).

Original: Especially models based on ClebschGordan tensor products have shown superior data efficiency (Batzner et al., 2022; Batatia et al., 2022b; Musaelian et al., 2022).
Revised: Especially models based on ClebschGordan tensor products have shown superior data efficiency (Batzner et al., 2022; Batatia et al., 2022b; Musaelian et al., 2022).

Original: However, the relatively high cost of these operations might hinder the application to larger systems such as biomolecules in solution.
Revised: However, the relatively high cost of these operations might hinder the application to larger systems such as biomolecules in solution.

Original: This difficulty is compounded by growing evidence that message passing cannot accurately resolve long-range interactions (Alon & Yahav, 2020; Dwivedi et al., 2022).
Revised: This difficulty is compounded by growing evidence that message passing cannot accurately resolve long-range interactions (Alon & Yahav, 2020; Dwivedi et al., 2022).

Revised: Note that we consider here interactions as long-range if convergence in real space is slow or non-existent, e.g. electrostatic interactions or polarization.

Original: In light of these challenges, which are exemplary illustrated in Figure 1, we propose a model with the aim to (1) include directional information while (2) retaininghigh computational efficiency and (3) incorporating (anisotropic) long-range interactions, including polarization.
Revised: In light of these challenges, which are exemplary illustrated in Figure 1, we propose a model with the aim to (1) include directional information while (2) retaining computational efficiency and (3) incorporating (anisotropic) long-range interactions.

Original: Specifically, the addition of an anisotropic state to the existing hidden features is proposed.
Revised: Specifically, the addition of an anisotropic state to the existing hidden features is proposed.

Original: This anisotropic state is based on Cartesian multipoles and expressed as a linear combination of local frames.
Revised: This anisotropic state is based on Cartesian multipoles and expressed as a linear combination of local frames.

Original: As a result, these multipoles are equivariant under rotations and can be used to describethe anisotropy of interactions.
Revised: As a result, these multipoles are equivariant under rotations and can be used to describe anisotropy of interactions.

Original: The formalism is developed analogously to the concept of atomic multipoles commonly used in computational chemistry (Stone, 2013).
Revised: The formalism is developed analogously to the concept of atomic multipoles commonly used in computational chemistry (Stone, 2013).

Original: The proposed modification results in relatively little computational overhead compared to standard message passing models.
Revised: The proposed modification results in relatively little computational overhead compared to standard message passing models.

Original: Most importantly, the formulation based on multipoles allows for the hybrid treatment of particles and fields.
Revised: Most importantly, the formulation based on multipoles allows for the hybrid treatment of particles and fields.

Original: This is of particular interest for two cases: (1) Systems of particles embedded in an external field, and (2) systems with large numbers of particles where long-range interactions may be treated with the fast multipole method (Rokhlin, 1985).
Revised: This is of particular interest for two cases: (1) Systems of particles embedded in an external field, and (2) systems with large numbers of particles where long-range interactions may be treated with the fast multipole method (Rokhlin, 1985).

Original: As a practical use case, we consider quantum mechanics/molecular mechanics (QM/MM) simulations (Warshel & Levitt, 1976).
Revised: As a practical use case, we consider quantum mechanics/molecular mechanics (QM/MM) simulations (Warshel & Levitt, 1976).

Original: In QM/MM simulations, a QM system is embedded in an external electrostatic field generated by MM particles, allowing for an efficient description of large systems, for instance protein-ligand complexes or enzymatic reactions in solution.
Revised: In QM/MM simulations, a QM system is embedded in an external electrostatic field generated by MM particles, allowing for an efficient description of large systems, for instance protein-ligand complexes or enzymatic reactions in solution.

Original: As such, the QM/MM formalism might be ideally suited to apply ML potentials to extended systems.
Revised: As such, the QM/MM formalism might be ideally suited to apply ML potentials to extended systems.

Original: Consequently, a
Revised: Consequently, a

Original: ML/MM formalism within the message passing framework is formulated analogous to QM/MM.
Revised: ML/MM formalism within the message passing framework is formulated analogous to QM/MM.

Original: [Figure]
Revised: [Figure]

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: ML potentials have sparked interest as a possible solution to the steeply scaling computational cost of quantum chemical methods (Schuch & Verstraete, 2009; Unke et al., 2021b), with high-dimensional neural network potentials being some of the earliest proposed methods (Behler, 2011).
Revised: ML potentials have sparked interest as a possible solution to the steeply scaling computational cost of quantum chemical methods (Schuch & Verstraete, 2009; Unke et al., 2021b), with high-dimensional neural network potentials being some of the earliest proposed methods (Behler, 2011).

Original: With the introduction of GNN (Scarselli et al., 2009; Battaglia et al., 2016; 2018), the development of ML potentials has shifted away from handcrafted features to end-to-end differentiable models which learn features solely from distances and element types (Gilmer et al., 2017; Unke et al., 2021b).
Revised: With the introduction of GNN (Scarselli et al., 2009; Battaglia et al., 2016; 2018), the development of ML potentials has shifted away from handcrafted features to end-to-end differentiable models which learn features solely from distances and element types (Gilmer et al., 2017; Unke et al., 2021b).

Original: Recognizing that GNN may fail to distinguish certain graphs has spurred the development of modified message passing schemes (Morris et al., 2019; Maron et al., 2019; Pozdnyakov & Ceriotti, 2022).
Revised: Recognizing that GNN may fail to distinguish certain graphs has spurred the development of modified message passing schemes (Morris et al., 2019; Maron et al., 2019; Pozdnyakov & Ceriotti, 2022).

Original: As a solution, the explicit integration of many-body interactions in the message passing (Kondor, 2018; Klicpera et al., 2020; Shui & Karypis, 2020; Zhao et al., 2021), directional messages (Anderson et al., 2019; Sch¨utt et al., 2021), and combinations thereof (Gasteiger et al., 2021; Batatia et al., 2022b) have been proposed.
Revised: As a solution, the explicit integration of many-body interactions in the message passing (Kondor, 2018; Klicpera et al., 2020; Shui & Karypis, 2020; Zhao et al., 2021), directional messages (Anderson et al., 2019; Sch¨utt et al., 2021), and combinations thereof (Gasteiger et al., 2021; Batatia et al., 2022b) have been proposed.

Original: Building on the concept of equivariant CNNs (Cohen & Welling, 2016;
Revised: Building on the concept of equivariant CNNs (Cohen & Welling, 2016;

Original: Weiler et al., 2018), equivariant message passing models based on Clebsch-Gordan tensor products have received increased attention (Anderson et al., 2019; Miller et al., 2020;
Revised: Weiler et al., 2018), equivariant message passing models based on Clebsch-Gordan tensor products have received increased attention (Anderson et al., 2019; Miller et al., 2020;

Original: Satorras et al., 2021;
Revised: Satorras et al., 2021;

Original: Brandstetter et al., 2021; Batzner et al., 2022; Musaelian et al., 2022).
Revised: Brandstetter et al., 2021; Batzner et al., 2022; Musaelian et al., 2022).

Original: A categorization based on employed architectural features was recently proposed by Batatia et al.
Revised: A categorization based on employed architectural features was recently proposed by Batatia et al.

Original: (2022a), offering an overview of essential ingredients of message passing models for ML potentials.
Revised: (2022a), offering a succinct overview.

Original: Deficiencies in the description of long-range interactions have led to models that include explicit interaction terms (Unke & Meuwly, 2019;
Revised: Deficiencies in the description of long-range interactions have led to models that include explicit interaction terms (Unke & Meuwly, 2019;

Original: Ko et al., 2021), as well as models that integrate this information in the representation itself (Grisafi & Ceriotti, 2019; Grisafi et al., 2021; Unke et al., 2021a).
Revised: Ko et al., 2021), as well as models that integrate this information in the representation itself (Grisafi & Ceriotti, 2019; Grisafi et al., 2021; Unke et al., 2021a).

Original: As an alternative to the ML potentials mentioned so far, ML within the QM/MM formalism might facilitate the description of condensed phase systems.
Revised: As an alternative to the ML potentials mentioned so far, ML within the QM/MM formalism might facilitate the description of condensed phase systems.

Original: Recent work has highlighted the challenges and potential of ML for the simulation of QM/MM systems (Zhang et al., 2018; B¨oselt et al., 2021; Pan et al., 2021;
Revised: Recent work has highlighted the challenges and potential of ML for the simulation of QM/MM systems (Zhang et al., 2018; B¨oselt et al., 2021; Pan et al., 2021;

Original: Hofstetter et al., 2022; Giese et al., 2022).
Revised: Hofstetter et al., 2022; Giese et al., 2022).

Original:  M ETHODS 3. M ESSAGE P ASSING G RAPH N EURAL N ETWORKS
Revised:  M ETHODS 3. M ESSAGE P ASSING G RAPH N EURAL N ETWORKS

Original: Message passing GNN process graph-structured data.
Revised: Message passing GNN process graph-structured data.

Original: In its commonly used form, node, edge, and/or global features are iteratively refined.
Revised: In its commonly used form, node, edge, and/or global features are iteratively refined.

Original: Various models have been proposed in recent years with differences found in the construction of the graph and the employed message passing mechanism (Scarselli et al., 2009; Battaglia et al., 2016;
Revised: Various models have been proposed in recent years with differences found in the construction of the graph and the employed message passing mechanism (Scarselli et al., 2009; Battaglia et al., 2016;

Original: Gilmer et al., 2017; Battaglia et al., 2018).
Revised: Gilmer et al., 2017; Battaglia et al., 2018).

Original: Considering a graph G =
Revised: Considering a graph G =

Original: ( V , E ) with nodes v i ∈ V and edges e ij ∈ E , message passing can be defined as
Revised: ( V , E ) with nodes v i ∈ V and edges e ij ∈ E , message passing can be defined as

Original: [Equation]
Revised: [Equation]

Original: with h li ∈ R n describing the hidden-feature vector of node v i after l iterations, u ij ∈ R n the edge feature of the edge e ij between node i and j , N ( i ) denoting the set of neighbours of v i , and ϕ e and ϕ h describing edge and node update functions.
Revised: with h li ∈ R n describing the hidden-feature vector of node v i after l iterations, u ij ∈ R n the edge feature of the edge e ij between node i and j , N ( i ) denoting the set of neighbours of v i , and ϕ e and ϕ h describing edge and node update functions.

Original: The superscript l denotes the current message passing iteration with n being the total number of message passing layers.
Revised: The superscript l denotes the current message passing iteration with n being the total number of message passing layers.

Original: In the following paragraphs, we will refer to QM and MM particles.
Revised: In the following paragraphs, we will refer to QM and MM particles.

Original: In the context of this work, QM particles are nodes v i of a graph G , which are labelled with a hidden feature
Revised: In the context of this work, QM particles are nodes v i of a graph G , which are labelled with a hidden feature

Original: h
Revised: h

Original: i .
Revised: i .

Original: MM particles, on the other hand, are not part of the graph, and therefore do not hold a hidden feature.
Revised: MM particles, on the other hand, are not part of the graph, and therefore do not hold a hidden feature.

Original: Instead, only a scalar partial charge q i (monopole) is attached to MM particles.
Revised: Instead, only a scalar partial charge q i (monopole) is attached to MM particles.

Original: More generally, MM particles can be understood as a specific example of a source of an external field in which the graph is embedded.
Revised: More generally, MM particles can be understood as a specific example of a source of an external field in which the graph is embedded.

Original: In addition, both QM and MM particles hold a position r i ∈ R 3 in Cartesian space.
Revised: In addition, both QM and MM particles hold a position r i ∈ R 3 in Cartesian space.

Original: 3. T HE A NISOTROPIC S TATE
Revised: 3. T HE A NISOTROPIC S TATE

Original: The present work proposes the introduction of an internal anisotropic state M i composed of Cartesian multipoles of order k to the existing hidden features.
Revised: The present work proposes the introduction of an internal anisotropic state M i composed of Cartesian multipoles of order k to the existing hidden features.

Original: Cartesian multipoles are ideally suited for this task as they result in relatively little additional computational costs and conceptual complexity.
Revised: Cartesian multipoles are ideally suited for this task as they result in relatively little additional computational costs and conceptual complexity.

Original: Specifically, M ki ∈ R 3 n refers to the k -th moment of the anisotropic state of node i composed of the monopole M 

Revised: Specifically, M ki ∈ R 3 n refers to the k -th moment of the anisotropic state of node i composed of the monopole M

Original: i , the dipole M 

Revised: i , the dipole M

Original: i
Revised: i

Original: and so on.
Revised: and so on.

Original: For clarity, the iteration superscript l will be omitted.
Revised: For clarity, the iteration superscript l will be omitted.

Original: However, in practice, the anisotropic state can be updated during each iteration much like the hidden feature h li .
Revised: However, in practice, the anisotropic state can be updated during each iteration much like the hidden feature h li .

Original: The anisotropic state is expressed as a linear combination of a local basis (Th ¨ urlemann et al., 2022).
Revised: The anisotropic state is expressed as a linear combination of a local basis (Th ¨ urlemann et al., 2022).

Original: This formulation preserves equivariance under rotations and invariance to translations.
Revised: This formulation preserves equivariance under rotations and invariance to translations.

Original: As the local basis, the traceless tensor product of order k of the unit vector ˆ r ij pointing from the position of node i to the position of its neighbour j is used
Revised: As the local basis, the traceless tensor product of order k of the unit vector ˆ r ij pointing from the position of node i to the position of its neighbour j is used

Original: [Equation]
Revised: [Equation]

Original: with k referring to the order of the multipole and r 0 ij = 1 .
Revised: with k referring to the order of the multipole and R 0 ij = 1 .

Original: Each component of order k of the anisotropic state M ki is then expressed as
Revised: Each component of order k of the anisotropic state M ki is then expressed as

Original: [Equation]
Revised: [Equation]

Original: with coefficients c kij predicted for each interaction as
Revised: with coefficients c kij predicted for each interaction as

Original: [Equation]
Revised: [Equation]

Original: where ϕ M ( k ) refers to a learnable function.
Revised: where ϕ M ( k ) refers to a learnable function.

Original: The scalar coefficient c kij is based on the hidden features of the interacting nodes i and j as well as an edge feature u ij .
Revised: The scalar coefficient c kij is based on the hidden features of the interacting nodes i and j as well as an edge feature u ij .

Original: The coefficient c kij can be interpreted as the contribution of the node j to the polarization of order k of node i analogous to a (hyper- )polarizability.
Revised: The coefficient c kij can be interpreted as the contribution of the node j to the polarization of order k of node i analogous to a (hyper- )polarizability.

Original: We note that the multipoles discussed here serve as a tool to introduce anisotropy.
Revised: We note that the multipoles discussed here serve as a tool to introduce anisotropy.

Original: Albeit possible, they do not necessarily carry physical meaning.
Revised: Albeit possible, they do not necessarily carry physical meaning.

Original: 3. A NISOTROPIC I NTERACTION F EATURES
Revised: 3. A NISOTROPIC I NTERACTION F EATURES

Original: Given the anisotropic state M i proposed in the previous section, multipole-multipole interactions can be defined.
Revised: Given the anisotropic state M i proposed in the previous section, multipole-multipole interactions can be defined.

Original: Formulation of the multipole interaction follows the notation and formalism described in (Smith, 1998; Lin, 2015; Burnham & English, 2020) which decomposes the interaction between multipoles on two distinct sites into a product of a radial component b k ( || r ij || ) and coefficients of the multipole interactions g kij ( r ij ) , resulting in the following terms
Revised: Formulation of the multipole interaction follows the notation and formalism described in (Smith, 1998; Lin, 2015; Burnham & English, 2020) which decomposes the interaction between multipoles on two distinct sites into a product of a radial component b k ( || r ij || ) and coefficients of the multipole interactions g kij ( r ij ) , resulting in the following terms

Original: [Equation]
Revised: [Equation]

Original: with b k ( || r ij || )
Revised: with b k ( || r ij || )

Original: = (2 k − 1)!! /
Revised: = (2 k − 1)!! /

Original: || r ij
Revised: || r ij

Original: ||
Revised: ||

Original: 2 k +1 .
Revised: 2 k +1 .

Original: In the context of machine learning, b k can be reinterpreted as filters analogous to the continuous-filter convolutions introduced by Sch¨utt et al.
Revised: In the context of machine learning, b k can be reinterpreted as filters analogous to the continuous-filter convolutions introduced by Sch¨utt et al.

Original: Consequently, the coefficients g kij encode the angular information based on the relative orientation of the multipoles centered on the interacting nodes.
Revised: Consequently, the coefficients g kij encode the angular information based on the relative orientation of the multipoles centered on the interacting nodes.

Original: In the notation of Burnham & English (2020), the coefficients can be generated with
Revised: In the notation of Burnham & English (2020), the coefficients can be generated with

Original: [Equation]
Revised: [Equation]

Original: given multipoles M ( d i + d c ) i of order ( d j + d c ) on site
Revised: given multipoles M ( d i + d c ) i of order ( d j + d c ) on site

Original: i and the previously defined traceless tensor products of the vector ˆ r ij .
Revised: i and the previously defined traceless tensor products of the vector ˆ r ij .

Original: (cid:76) refers to the concatenation of the scalar coefficients.
Revised: (cid:76) refers to the concatenation of the scalar coefficients.

Original: All d i , d j , d c where d i + d j + d c = k and d i , d j , d c ≥ 0 are included.
Revised: All d i , d j , d c where d i + d j + d c = k and d i , d j , d c ≥ 0 are included.

Original: d denotes the number of contractions, with d i being the number of contractions over the indices of the first bracket, d j the number of contractions in the second bracket and d c the number of contractions between the two brackets, indicated by ⟨ ., . ⟩ .
Revised: d denotes the number of contractions, with d i being the number of contractions over the indices of the first bracket, d j the number of contractions in the second bracket and d c the number of contractions between the two brackets, indicated by ⟨ ., . ⟩ .

Original: Analogous to the multipole interaction defined in Eq.
Revised: Analogous to the multipole interaction defined in Eq.

Original: 5, the anisotropic feature a ij is defined as
Revised: 5, the anisotropic feature a ij is defined as

Original: [Equation]
Revised: [Equation]

Original: where we use ◦ to denote the aforementioned element-wise multiplication with radial weights generated by ϕ b and G (ˆ r ij , M i , M j ) for the concatenated multipole interaction coefficients g kij of all considered orders k .
Revised: where we use ◦ to denote the aforementioned element-wise multiplication with radial weights generated by ϕ b and G (ˆ r ij , M i , M j ) for the concatenated multipole interaction coefficients g kij of all considered orders k .

Original: 3. A NISOTROPIC M ESSAGE P ASSING (AMP)
Revised: 3. A NISOTROPIC M ESSAGE P ASSING (AMP)

Original: Based on the anisotropic feature defined in the previous section, the modified message passing is obtained as
Revised: Based on the anisotropic feature defined in the previous section, the modified message passing is obtained as

Original: [Equation]
Revised: [Equation]

Original: Thus, two additional steps are added to the standard message passing formalism: (1) Expansion of the multipoles on each node, and (2) contraction of the multipoles for each pair of interacting nodes.
Revised: Thus, two additional steps are added to the standard message passing formalism: (1) Expansion of the multipoles on each node, and (2) contraction of the multipoles for each pair of interacting nodes.

Original: The resulting feature is used to incorporate directional information in the message m ij .
Revised: The resulting feature is used to incorporate directional information in the message m ij .

Original: For the use as a ML potential, the total potential energy is expressed as a sum of atom-based contributions of each QM particle, i.e. V QM
Revised: For the use as a ML potential, the total potential energy is expressed as a sum of atom-based contributions of each QM particle, i.e. V QM

Original: =
Revised: =

Original: (cid:80) QMi ϕ
Revised: (cid:80) QMi ϕ

Original: V ( h ni ) .
Revised: V ( h ni ) .

Original: We will refer to this model as AMP( k ) where k denotes the highest order of the multipoles used and n is the number of graph layers.
Revised: We will refer to this model as AMP( k ) where k denotes the highest order of the multipoles used and n is the number of graph layers.

Original: 3. Q UANTUM M ECHANICS /M OLECULAR M ECHANICS (QM/MM)
Revised: 3. Q UANTUM M ECHANICS /M OLECULAR M ECHANICS (QM/MM)

Original: QM/MM has become an integral part of computational chemistry since its inception in 1976 (Warshel & Levitt, 1976).
Revised: QM/MM has become an integral part of computational chemistry since its inception in 1976 (Warshel & Levitt, 1976).

Original: QM/MM offers an efficient way to incorporate interactions with an environment, for instance solvent molecules and/or proteins, through combination of a QM and a classical (MM) description of a system.
Revised: QM/MM offers an efficient way to incorporate interactions with an environment, for instance solvent molecules and/or proteins, through combination of a QM and a classical (MM) description of a system.

Original: As such it retains the fidelity of QM-based methods for the region of interest, e.g. the solute, and the computational efficiency of classical force fields for the surroundings.
Revised: As such it retains the fidelity of QM-based methods for the region of interest, e.g. the solute, and the computational efficiency of classical force fields for the surroundings.

Original: Here, a brief description of the electrostatic embedding, one of several possible
Revised: Here, a brief description of the electrostatic embedding, one of several possible

Original: QM/MM schemes, is given.
Revised: QM/MM schemes, is given.

Original: We refer the interested reader to Senn & Thiel (2007) for further details.
Revised: We refer the interested reader to Senn & Thiel (2007) for further details.

Original: Within an electrostatic embedding, a designated QM zone interacts with the partial charges of the surrounding MM particles.
Revised: Within an electrostatic embedding, a designated QM zone interacts with the partial charges of the surrounding MM particles.

Original: For this purpose, the molecular Hamiltonian ˆ H QM is extended by an interaction with the MM zone
Revised: For this purpose, the molecular Hamiltonian ˆ H QM is extended by an interaction with the MM zone

Original: [Equation]
Revised: [Equation]

Original:  where q i are the partial charges of the MM zone and Z i refer to the core charges of the nuclei.
Revised: ) where q i are the partial charges of the MM zone and Z i refer to the core charges of the nuclei.

Original: MM refers to the MM particles, EL to the electrons in the QM zone and QM to the QM nuclei.
Revised: MM refers to the MM particles, EL to the electrons in the QM zone and QM to the QM nuclei.

Original: While the first term incorporates the interaction of the surrounding partial charges analogous to the coreelectron interaction, the second term describes the Coulomb interaction between the nuclei within the QM zone and the partial charges.
Revised: While the first term incorporates the interaction of the surrounding partial charges analogous to the coreelectron interaction, the second term describes the Coulomb interaction between the nuclei within the QM zone and the partial charges.

Original: In other words, the QM system interacts with point charges representing the MM particles.
Revised: In other words, the QM system interacts with point charges representing the MM particles.

Original: The point charges polarize the QM system and interact with the resulting charge distribution.
Revised: The point charges polarize the QM system and interact with the resulting charge distribution.

Original: Remaining interactions between QM and MM particles, i.e. exchangerepulsion and dispersion, are described with classical force-field terms such as the Lennard-Jones potential (Jones & Chapman, 1924).
Revised: Remaining interactions between QM and MM particles, i.e. exchangerepulsion and dispersion, are described with classical force-field terms such as the Lennard-Jones potential (Jones & Chapman, 1924).

Original: 3. QM/MM FOR M ESSAGE P ASSING : D IRECT P OLARIZATION
Revised: 3. QM/MM FOR M ESSAGE P ASSING : D IRECT P OLARIZATION

Original: As a synthesis of the two previous sections, we propose a further modification to anisotropic message passing, which incorporates interactions with a surrounding field or particles that are not part of the graph.
Revised: As a synthesis of the two previous sections, we propose a further modification to anisotropic message passing, which incorporates interactions with a surrounding field or particles that are not part of the graph.

Original: [Equation]
Revised: [Equation]

Original: Here, N QM ( i ) refers to the neighbours of i in the QM zone and N MM ( i ) to the neighbours of i in the MM zone.
Revised: Here, N QM ( i ) refers to the neighbours of i in the QM zone and N MM ( i ) to the neighbours of i in the MM zone.

Original: In addition to the previously described polarization due to neighbouring atoms that are part of the graph, a second term is introduced, which adds a contribution caused by the MM particles.
Revised: In addition to the previously described polarization due to neighbouring atoms that are part of the graph, a second term is introduced, which adds a contribution caused by the MM particles.

Original: The second term is conditioned on the hidden feature of the QM particle and the anisotropic feature a ij , which is based on the multipoles on the QM particle i and the partial charge on the MM particle.
Revised: The second term is conditioned on the hidden feature of the QM particle and the anisotropic feature a ij , which is based on the multipoles on the QM particle i and the partial charge on the MM particle.

Original: Evidently, this formulation permits also the application to polarizable embeddings schemes, use with the fast multipole method, and use of higher-order multipoles.
Revised: Evidently, this formulation permits also the application to polarizable embeddings schemes, use with the fast multipole method, and use of higher-order multipoles.

Original: We note that MM particles contribute only to the polarization of multipoles with order k ≥ 1 , analogous to the definition of (hyper-)polarizabilities (Stone, 2013).
Revised: We note that MM particles contribute only to the polarization of multipoles with order k ≥ 1 , analogous to the definition of (hyper-)polarizabilities (Stone, 2013).

Original: Since MM particles are only labelled with a charge and not with a hidden feature h li , the interaction depends solely on the feature of the QM particle and the anisotropic interaction feature a ij , which encodes the strength of the interaction and the relative orientation of the MM particle with respect to the polarized QM particle.
Revised: Since MM particles are only labelled with a charge and not with a hidden feature h li , the interaction depends solely on the feature of the QM particle and the anisotropic interaction feature a ij , which encodes the strength of the interaction and the relative orientation of the MM particle with respect to the polarized QM particle.

Original: The advantages of the proposed interaction mechanism are twofold: First, the evaluation of messages between MM particles and between QM and MM particles is avoided.
Revised: The advantages of the proposed interaction mechanism are twofold: First, the evaluation of messages between MM particles and between QM and MM particles is avoided.

Original: Instead, only the cheaper polarization term is evaluated for the more numerous QM–MM interactions.
Revised: Instead, only the cheaper polarization term is evaluated for the more numerous QM–MM interactions.

Original: Second, directional and long-range interactions are efficiently incorporated in addition to the proper cancellation of opposing polarization terms.
Revised: Second, directional and long-range interactions are efficiently incorporated in addition to the proper cancellation of opposing polarization terms.

Original: The total potential energy is then obtained as the atom-based contribution due to the QM particles described previously and a contribution due to the Coulomb interaction between 
Revised: The total potential energy is then obtained as the atom-based contribution due to the QM particles described previously and a contribution due to the Coulomb interaction between QM and

Original: QM and MM particles,
Revised: MM particles,

Original: [Equation]
Revised: [Equation]

Original:  where q j denotes the partial charge of the respective MM particle and ϕ ρ ( h ni ) refers to a scalar charge density localized on the respective QM particle, analogous to a discretized Coulomb interaction.
Revised: [Equation] where q j denotes the partial charge of the respective MM particle and ϕ ρ ( h ni ) refers to a scalar charge density localized on the respective QM particle, analogous to a discretized Coulomb interaction.

Original: In other words, a pairwise interaction between each QM atom and its neighbours in the MM zone is added.
Revised: In other words, a pairwise interaction between each QM atom and its neighbours in the MM zone is added.

Original: In principle, ϕ ρ ( h ni , a ij ) may be used to incorporate anisotropy.
Revised: In principle, ϕ ρ ( h ni , a ij ) may be used to incorporate anisotropy.

Original: Here, only the hidden feature h ni is used.
Revised: Here, only the hidden feature h ni is used.

Original: Thus, asymptotically correct long-range behaviour can be guaranteed by enforcing that the resulting scalar charges conserve total charge within the QM zone by subtraction of the mean excess charge, i.e. (cid:80) QMi ϕ
Revised: Thus, asymptotically correct long-range behaviour can be guaranteed by enforcing that the resulting scalar charges conserve total charge within the QM zone by subtraction of the mean excess charge, i.e. (cid:80) QMi ϕ

Original: ρ ( h ni ) =
Revised: ρ ( h ni ) =

Original: for neutral systems.
Revised: for neutral systems.

Original: Expansion of ϕ ρ ( h ni ) itself may be an alternative route to obtain anisotropic QM/MM electrostatic interactions, as shown by Grisafi & Ceriotti (2019).
Revised: Expansion of ϕ ρ ( h ni ) itself may be an alternative route to obtain anisotropic QM/MM electrostatic interactions, as shown by Grisafi & Ceriotti (2019).

Original: The forces F i acting on the QM and MM particles are obtained as the derivative of the negative total energy ( V QM/MM,ESP + V QM ) with respect to the positions r i of the QM and MM particles, respectively.
Revised: The forces F i acting on the QM and MM particles are obtained as the derivative of the negative total energy ( V QM/MM,ESP + V QM ) with respect to the positions r i of the QM and MM particles, respectively.

Original: We will refer to models that include this direct polarization terms as AMP( k )-D. Use of the pairwise long-range potential in Eq.
Revised: We will refer to models that include this direct polarization terms as AMP( k )-D. Use of the pairwise long-range potential in Eq.

Original: 11 will be denoted by ‘-P’.
Revised: 11 will be denoted by ‘-P’.

Original:  R ESULTS
Revised:  R ESULTS

Original: Results are split into three parts: First, two model systems are investigated to explore the proposed modifications.
Revised: Results are split into three parts: First, two model systems are investigated to explore the proposed modifications.

Original: Second, the model is applied to an existing dataset of QM/MM systems.
Revised: Second, the model is applied to an existing dataset of QM/MM systems.

Original: Finally, results for QM9 are shown to allow comparison with other models.
Revised: Finally, results for QM9 are reported as a comparison with existing models.

Original: 4. W ATER D IMERS : D IRECTIONAL F EATURES
Revised: 4. W ATER D IMERS : D IRECTIONAL F EATURES

Original: As an illustrative example of the discussed concepts, a water dimer as shown in Figure 1B is investigated.
Revised: As an illustrative example of the discussed concepts, a water dimer as shown in Figure 1B is investigated.

Original: Configurations were generated by rotating one molecule in steps of one degree around the axis described by the aligned dipoles, resulting in a total of 360 configurations.
Revised: Configurations were generated by rotating one molecule in steps of one degree around the axis described by the aligned dipoles, resulting in a total of 360 configurations.

Original: The molecules were separated by a distance of 4 ˚A.
Revised: The molecules were separated by a distance of 4 ˚A.

Original: For such a system, three cutoff regimes are significant: (1) The cutoff r cut used to construct the graph is larger than the maximal distance r max in the system, resulting in a fully-connected graph ( r max < r cut ).
Revised: For such a system, three cutoff regimes are significant: (1) The cutoff r cut used to construct the graph is larger than the maximal distance r max in the system, resulting in a fully-connected graph ( r max < r cut ).

Original: The cutoff is larger than the distance r OO between the two opposing oxygen atoms but smaller than the maximal distance ( r OO <
Revised: The cutoff is larger than the distance r OO between the two opposing oxygen atoms but smaller than the maximal distance ( r OO <

Original: r cut < r max ).
Revised: r cut < r max ).

Original: The cutoff is smaller than the distance between the oxygen atoms, resulting in two disconnected graphs ( r cut < r OO ).
Revised: The cutoff is smaller than the distance between the oxygen atoms, resulting in two disconnected graphs ( r cut < r OO ).

Original: Particularly case (2) is interesting as it can be related to torsional degrees of freedom, which play an important role in molecular dynamics.
Revised: Particularly case (2) is interesting as it can be related to torsional degrees of freedom, which play an important role in molecular dynamics.

Original: [Figure]
Revised: [Figure]

Original: For all results shown in this section, a minimal 1 -layer model was used.
Revised: For all results shown in this section, a minimal 1 -layer model was used.

Original: While the k = 1 model is still able to describe the interaction using a cutoff that results in a fully connected graph (i.e. case (1) shown in Figure 2A), the learning task seems to be more challenging for the AMP(1) compared to the AMP(2) model, indicated by marginally larger errors (MAE of 0 .
Revised: While the k = 1 model is still able to describe the interaction using a cutoff that results in a fully connected graph (i.e. case (1) shown in Figure 2A), a marginally larger error MAE ( 0 . 32 AMP(1) versus 0 . J mol − AMP(2)) is found for the AMP(1) compared to the AMP(2) model. This observation could be an indication that the present task is also more challenging for AMP(1) than AMP(2).

Original: 32 versus 0 .
Revised: Learning curves shown in Figure 5 add further evidence to this interpretation.

Original: J mol − 1 ). For case (2) shown in Figure 2B, the k = 1 model (orange line) fails to describe the interaction, as the dipole-dipole interaction remains invariant under the considered rotation.
Revised: For case (2) shown in Figure 2B, the k = 1 model (orange line) fails to describe the interaction, as the dipole-dipole interaction remains invariant under the considered rotation.

Original: This issue can be traced back to the quadrupole–quadrupole interaction of the opposing oxygen atoms, motivating the need for k ≥ 2 .
Revised: This issue can be traced back to the quadrupole– quadrupole interaction of the opposing oxygen atoms, motivating the need for k ≥ 2 .

Original: We note that this case is also interesting as it exemplary demonstrates the limitations of two existing directional message passing architectures, i.e. Dime and PaiNN Klicpera et al.
Revised: We note that this case is also interesting as it exemplary demonstrates the limitations of two existing directional message passing architectures, i.e. DimeNet and PaiNN Klicpera et al.

Original: (2020); Sch¨utt et al.
Revised: ; Sch¨utt et al.

Original: These two models seem to fail to describe this quadrupole-quadrupole interaction despite the use of directional information.
Revised: These two models seem to fail to describe this quadrupole-quadrupole interaction despite the use of directional information.

Original: Our interpretation is that for Dime this is due to the invariance of bond angles under the considered rotation, and for PaiNN because it only uses vectorial information (i.e. k = 1 ).
Revised: Our interpretation is that for DimeNet this is due to the invariance of bond angles under the considered rotation, and for PaiNN because it only uses vectorial information (i.e. k = 1 ).

Original: Finally for case (3) shown in Figure 2C, all message passing models without additional long-range interactions fail (red line), motivating the proposed pairwise interaction and contributions to the polarization (models with suffix ‘-D’ and ‘-P’, shown in blue).
Revised: Finally for case (3) shown in Figure 2C, all message passing models without additional long-range interactions fail (red line), motivating the proposed pairwise interaction and contributions to the polarization (models with suffix ‘-D’ and ‘-P’, shown in blue).

Original: We note that the long-range interaction would in principle also include dispersion, which we do not address here due to the availability of robust and efficient classical models.
Revised: We note that the long-range interaction would in principle also include dispersion, which we do not address here due to the availability of robust and efficient classical models.

Original: 4. W ATER C LUSTERS : E XTENSIVITY
Revised: 4. W ATER C LUSTERS : E XTENSIVITY

Original: For this task, clusters of water molecules, ranging in size from six to 20 molecules, were investigated.
Revised: For this task, clusters of water molecules, ranging in size from six to 20 molecules, were investigated.

Original: For each cluster, one molecule was randomly selected as the QM zone while all other molecules were treated as MM particles.
Revised: For each cluster, one molecule was randomly selected as the QM zone while all other molecules were treated as MM particles.

Original: The goal of this task is the prediction of the difference in energy and gradients caused by placing the QM system in the external field generated by the MM particles.
Revised: The goal of this task is the prediction of the difference in energy and gradients caused by placing the QM system in the external field generated by the MM particles.

Original: A successful model must be able to discriminate between (1) the polarization of the QM zone by the external field, which does not directly scale with cluster size, and (2) the electrostatic interaction, which increases with cluster size.
Revised: A successful model must be able to discriminate between (1) the polarization of the QM zone by the external field, which does not directly scale with cluster size, and (2) the electrostatic interaction, which increases with cluster size.

Original: All models were trained on 1 ′ 289 samples from clusters of size 10 .
Revised: All models were trained on 1 ′ 289 samples from clusters of size 10 .

Original: All other cluster sizes were used for testing.
Revised: All other cluster sizes were used for testing.

Original: For clarity, the cluster sizes are grouped in the results (Table 1). In the following paragraphs, the role of the proposed model modifications is probed.
Revised: Results are shown in Figure 3.

Original: Baseline: Message Passing To establish a baseline, a standard message passing model based on
Revised: Baseline – Message Passing: As a baseline message passing model, the PaiNN architecture is used.

Original: Eq. 1 was trained on the data.
Revised: PaiNN was shown to perform accurately on a wide variety of tasks through inclusion of directional information (Sch¨utt et al., 2021).

Original: In this case, the MM particles are part of the graph. We also consider a second baseline model that uses thepairwise Coulomb interaction between QM and MM particles as described in Eq. We refer to these two models as GNN and GNN-P.
Revised: For our QM/MM task, the MM particles are part of the same graph as the QM particles. However, energies are only predicted for the three QM particles of the water molecule in the QM zone.

Original: Ablation Studies: Multipole Order and Polarization
Revised: Ablation Studies – Multipole Order and Polarization:

Original: In addition, the role of the multipole order k was investigated.
Revised: In addition, the role of the multipole order k was investigated.

Original: To compensate for different sizes of a ij due to the multipole order, the number of radial weights was adjusted such that the total number of trainable parameters ( ∼ 1 M) remained constant.
Revised: To compensate for different sizes of a ij due to the multipole order, the number of radial weights was adjusted such that the total number of trainable parameters ( ∼ 1 M) remained constant.

Original: The resulting models are labelled as AMP( k )-D, where k is the highest multipole order used. The additional suffix ‘-P’ indicates the use of the pairwise Coulomb interaction between QM and MM particles given in Eq. In addition, the role of the contribution of the MM particles to the polarization defined in Eq. 10 was considered.
Revised: [Figure]

Original: The model without this additional polarization term is referred to as AMP( k )-P.
Revised: M A E E n e r g y [ k J / m o l]

Original: [Table]
Revised: [Figure]

Original: As expected, both models that do not incorporate polarization, i.e. AMP(3)-P and AMP(0)-DP, show the largest errors (Table 1).
Revised: clusters of size 10 . Energies are reported in kJ mol − 1 and forces in kJ mol − 1 ˚A − 1 . Not surprisingly, AMP( 0 ) fails at this task as it only interacts with the MM particles through the pairwise Coulomb potential of Eq. The results for AMP( 0 ) provide a rough estimate of the role of both polarization and anisotropy in the present task.

Original: Adding the direct polarization, i.e., AMP(0)-DP versus AMP(1)-DP, results in the largest relative improvement.
Revised: Including the direct polarization (i.e., AMP( 0 ) versus AMP( 1 )), results in the largest relative improvement.

Original: This is no surprise as there is no way for the information to flow from the MM particles to the QM particles in these models.

Original: This result demonstrates that the proposed modification allows the model to incorporate information about the surrounding particles without explicit message passing.
Revised: This result demonstrates that the proposed modification allows the model to incorporate information about the surrounding particles without explicit message passing between the QM and MM zone.

Original: Surprisingly, increasing k  to 3 , i.e. AMP(2)-DP versus AMP(3)-DP, seems to diminish the extrapolation capabilities, most notably for the largest clusters.
Revised: Interestingly, increasing k from 2 to 3 (i.e., AMP( 2 ) versus AMP( 3 )), seems to diminish the extrapolation capabilities again, most notably on the largest clusters.

Original: As previously mentioned, a diminished ability to generalize to different cluster sizes is observed for both GNN modelscompared to the AMP-DP models. The fact that GNN-P and AMP(2)-DP perform comparable on the training set adds further evidence that this issue is, at least for this specific case, not caused by the lack of explicit directional information.
Revised: Overall, we find that all models that interact with the MM particles (i.e., AMP( 1 ), AMP( 2 ), AMP( 3 ) and PaiNN), perform comparably on the training set and clusters with similar size, whereas differences in performance arise for much larger and smaller clusters. While the performance of the PaiNN model is comparable to the AMP( 2 ) and AMP( 3 ) models on gradients, larger deviations are observed for energies.

Original: Instead, this observation may indicate that the GNN model cannot distinguish between thetwo involved interactions. Assuming that ‘-P’ captures the electrostatic  interaction for both models equally well, results for the GNN-P model add further evidence that the error is mostly caused by a misrepresentation of polarization.
Revised: This result could be an indication that message passing models like PaiNN may have difficulties to differentiate between the electrostatic and polarization interaction in the present task.

Original: Finally, it is worth mentioning that even for these relatively small systems, one training step with the GNN-P model takes approximately three times longer than the AMP(3)-DP model.
Revised: Building models that differentiate between interaction types might be an important step for ML potentials to improve transferability to large and/or condensed-phase systems.

Original: 4. QM/MM S YSTEMS
Revised: 4. QM/MM S YSTEMS

Original: As a more realistic example, the model was tested on a previously published datasets of QM/MM systems introduced by B¨oselt et al.
Revised: As a more realistic example, the model was tested on a previously published datasets of QM/MM systems introduced by B¨oselt et al.

Original: These datasets include QM energies, QM gradients, and MM gradients of five molecules of varying size solvated in water.
Revised: These datasets include QM energies, QM gradients, and MM gradients of five molecules of varying size solvated in water.

Original: In this work, we focus on the three largest and most challenging systems: uracil, retinoic acid (RA) and the S-adenosyl methionine/cysteine dimer (SAM) with 12 , 50 and 49 + 14 QM atoms, respectively.
Revised: In this work, we focus on the three largest and most challenging systems: uracil, retinoic acid (RA) and the S-adenosyl methionine/cysteine dimer (SAM) with 12 , 50 and 49 + 14 QM atoms, respectively.

Original: On average, each molecule is surrounded by approximately 1 ′ 491 (uracil), 2 ′ 391 (RA) and 2 ′ 553 (SAM) MM particles.
Revised: On average, each molecule is surrounded by approximately 1 ′ 491 (uracil), 2 ′ 391 (RA) and 2 ′ 553 (SAM) MM particles.

Original: Each water molecule is represented by three partial charges centered on each atom.
Revised: Each water molecule is represented by three partial charges centered on each atom.

Original: These datasets are challenging due to the large number of surrounding MM particles that polarize the QM system, the resulting electrostatic long-range interactions, as well as the size and flexibility of RA and SAM.
Revised: These datasets are challenging due to the large number of surrounding MM particles that polarize the QM system, the resulting electrostatic long-range interactions, as well as the size and flexibility of RA and SAM.

Original: A ∆ -ML approach, which learned a correction on top of a semi-empirical tight-binding method (DFTB (Elstner et al., 1998)), was used in previous work (B¨oselt et al., 2021; Hofstetter et al., 2022).
Revised: A ∆ -ML approach, which learned a correction on top of a semi-empirical tight-binding method (DFTB (Elstner et al., 1998)), was used in previous work (B¨oselt et al., 2021; Hofstetter et al., 2022).

Original: [Table]
Revised: [Table]

Original: No ∆ -learning scheme was used in this work.
Revised: No ∆ -learning scheme was used in this work.

Original: In Table 2, results for a 3 -layer AMP( 3 )-DP model are reported.
Revised: In Table 1, results for a 3 -layer AMP( 3 )-DP model are reported.

Original: The performance is compared for two pre-processing steps: While (unshuffled) uses the temporally sequential ordering used in the initial work B ¨ oselt et al.
Revised: The performance is compared for two pre-processing steps: While (unshuffled) uses the temporally sequential ordering used in the initial work B ¨ oselt et al.

Original: ; Hofstetter et al.
Revised: ; Hofstetter et al.

Original: (2022), (shuffled) refers to a shuffled (i.e. randomized) version of the same data.
Revised: (2022), (shuffled) refers to a shuffled (i.e. randomized) version of the same data.

Original: Comparing the same dataset (unshuffled), we observe in most cases larger errors compared to the previously reported ∆ - models but smaller errors than previously reported results without ∆ -learning (B ¨ oselt et al., 2021;
Revised: Comparing the same dataset (unshuffled), we observe in most cases larger errors compared to the previously reported ∆ -models but smaller errors than previously reported results without ∆ -learning. For uracil, (unshuffled) errors were reported without ∆ -learning: Using a HDNNP architecture, B ¨ oselt et al. (2021) reported MAEs of 2 . 9 / 6 .

Revised: 8 kJ mol − 1 for E QM , 6 . 38 / 12 . 6 / 12 .

Revised: 4 kJ mol − 1 ˚A − 1 for

Original:  Hofstetter et al., 2022).
Revised: F QM and 0 . 65 kJ mol − 1 ˚A − 1 for F MM respectively. With a GNN architecture, Hofstetter et al. (2022) reported MAEs of 7 . 3 kJ mol − 1 for E QM , 13 .

Revised: 52 kJ mol − 1 ˚A − 1 for F QM and 0 .

Revised: 58 kJ mol − 1 ˚A − 1 for F MM on the test-set.

Original: These results suggest that the proposed model canin principle successfully incorporate long-range polarization and electrostatic interactions.
Revised: These results suggest that the proposed model can successfully incorporate long-range polarization and electrostatic interactions.

Original: However, the relatively large drop in accuracy for the (unshuffled) dataset, which was previously observed and discussed in Hofstetter et al.
Revised: However, the relatively large drop in accuracy for the (unshuffled) dataset, which was previously observed and discussed in Hofstetter et al.

Original: (2022), indicates a limited capability to generalise to unseen data points.
Revised: (2022), indicates a limited capability to generalise.

Original: The difference between random splits or time splits might be important for commonly used benchmark datasets such as MD17 (Chmiela et al., 2017), as random splits tend to give a too optimistic performance assessment.
Revised: The difference between random splits or time splits might be important for commonly used benchmark datasets such as MD17 (Chmiela et al., 2017), as random splits tend to give a too optimistic performance assessment.

Original: This issue is already well known in biological activity prediction Sheridan (2013).
Revised: This issue is already well known in biological activity prediction Sheridan (2013).

Original: A single call of a 3 -layer AMP( 3 )-DP with a long-range cutoff of 50 ˚A (including QM energy, QM gradients, and MM gradients) for uracil, with 12 QM atoms and an average of 630 MM atoms, requires around 9 . 5 ms on a NVIDIA Titan V, averaged over a batch of size 40 .
Revised: Calculating QM energy, QM gradients [Table] and MM gradients for uracil ( 12 QM atoms and an average of 1490 MM atoms) using a 3 -layer AMP( 3 )-DP takes around 10 ms on a Nvidia Titan V. Additional results regarding computational costs are reported in the Appendix A.1.

Original: For an AMP( 2 )-DP model, this cost is reduced to 7 .

Original: 5 ms. Thus, a (QM)ML/MM simulation based on the AMP model could in principle remove the bottleneck of current QM/MM molecular dynamics (MD) simulations almost entirely.
Revised: Thus, a (QM)ML/MM simulation based on the AMP model could in principle remove the bottleneck of current QM/MM molecular dynamics (MD) simulations almost entirely.

Original: However, further improvements are required to bring the cost down to a level comparable to the calculation of classical force fields, which is on the order of 1 ms/step for a system with around 14 ′atoms (Eastman et al., 2017).
Revised: However, further improvements are required to bring the cost down to a level comparable to the calculation of classical force fields, which is on the order of 1 ms/step for a system with around 14 ′ 000 atoms (Eastman et al., 2017).

Original: Such improvements are clearly within reach, for instance through the use of the fast multipole method (Rokhlin, 1985).
Revised: Such improvements are clearly within reach, for instance through the use of the fast multipole method (Rokhlin, 1985).

Original: Nevertheless, it is important to mention that a low MAE does not guarantee a stable MD simulation (Stocker et al., 2022).
Revised: Nevertheless, it is important to mention that a low MAE does not guarantee a stable MD simulation (Stocker et al., 2022).

Original: In practice, a ∆ -learning scheme might still be required to obtain stable simulations over long timescales.
Revised: In practice, a ∆ -learning scheme might still be required to obtain stable simulations over long timescales.

Original: The robustness of the AMP( k )-DP model for the propagation of (QM)ML/MM MD simulations will therefore be investigated in the future.
Revised: The robustness of the AMP( k )-DP model for the propagation of (QM)ML/MM MD simulations will therefore be investigated in the future.

Original: [Table]
Revised: [Table]

Original:  In addition [Table] two long-range cutoffs, 30 ˚A and 50 ˚A, for the interactions between QM and MM particles are investigated (Table 3).
Revised: 0.13/0.13 SAM 6.0/6. In addition, two long-range cutoffs, 30 ˚A and 50 ˚A, for the interactions between QM and MM particles are investigated (Table 2).

Original: Notable performance differences are observed for the two investigated long-range cutoffs, which reaffirms the importance of long-range interactions.
Revised: Notable performance differences are observed for the two investigated long-range cutoffs, which reaffirms the importance of long-range interactions.

Original: In both cases, a smooth cutoff function is used, thus the differences cannot simply be explained by cutoff artifacts.
Revised: In both cases, a smooth cutoff function is used, thus the differences cannot simply be explained by cutoff artifacts.

Original: In this context, it is further important to mention that an increased long-range cutoff does not change the number of parameters and QM–QM interactions but only the number of QM–MM interactions.
Revised: In this context, it is further important to mention that an increased long-range cutoff does not change the number of parameters and QM–QM interactions but only the number of QM–MM interactions.

Original: A 50 ˚A cutoff covers all observed QM–MM interactions for all systems.
Revised: A 50 ˚A cutoff covers all observed QM–MM interactions for all systems.

Original: For a 30 ˚A cutoff, the same is only true for uracil.
Revised: For a 30 ˚A cutoff, the same is only true for uracil.

Original: Particularly for retinoic acid, for which the solvent-shell is elongated, the difference of the MAE for E QM between the two cutoffs is considerable.
Revised: Particularly for retinoic acid, for which the solvent-shell is elongated, the difference of the MAE for E QM between the two cutoffs is considerable.

Original: Hence, even if astandard message passing architecture with six layers and a cutoff of 5 ˚A, could accurately resolve longrange interactions, these interactions would still fall outside the perceptive radius.
Revised: Hence, even if a message passing architecture with six layers and a cutoff of 5 ˚A, could accurately resolve long-range interactions, these interactions would still fall outside the perceptive radius.

Original: However, given that there is little evidence that message passing models can faithfully resolve and incorporate such long-range interactions in the first place, this observation might be crucial forthe future development of ML potentials.
Revised: However, given that there is little evidence that message passing models can faithfully resolve and incorporate such long-range interactions in the first place, this observation might be crucial for future work on ML potentials.

Original: 4. QM9 D ATASET
Revised: 4. QM9 D ATASET

Original: Results for a 4 -layer AMP( 3 ) model with a cutoff of 5 ˚A for the 12 properties of the QM9 dataset are given in Table 4 and compared with existing models reported in the literature.
Revised: Results for a 4 -layer AMP( 3 ) model with a cutoff of 5 ˚ A for the 12 properties of the QM9 dataset are given in Table 3 and compared with existing models reported in the literature.

Original: Inference of a 4 -layer AMP( 3 ) model used here requires approximately 24 ms for a batch of 100 samples with a total of 1 ′ 833 atoms (NVIDIA Titan V).
Revised: Inference of a 4 -layer AMP( 3 ) model used here requires approximately 26 ms for a batch of 100 samples with a total of 1 ′ 833 atoms (NVIDIA Titan V).

Original: As can be seen in Table 4, both the computational cost and the reported results compare favourably with alternative models proposed in the literature.
Revised: As can be seen in Table 3, both the computational cost and the reported results compare favourably with alternative models proposed in the literature.

Original: For several molecular properties, considerable improvements are observed.
Revised: For several molecular properties, considerable improvements are observed.

Original: However, the error for targets related to the potential energy (i.e., U, U 0 , H and G) is higher than for some other models (Allegro,
Revised: However, the error for targets related to the potential energy (i.e., U, U 0 , H and G) is higher than for some other models (Allegro,

Original: PaiNN, PaxNet).
Revised: PaiNN, PaxNet).

Original: [Table]
Revised: [Table]

Original:  O UTLOOK
Revised:  O UTLOOK

Original: In this work
Revised: In this work

Original: [Table]
Revised: [Table]

Original: a modified message passing formalism that allows for the efficient incorporation of directional and long-range information was proposed.
Revised: a modified message passing formalism that allows for the efficient incorporation of directional and long-range information was proposed.

Original: For future work, an implementation [Table] which makes use of the fast multipole method, as well as the explicit inclusion of many-body interactions, as demonstrated by Klicpera et al.
Revised: For future work, an implementation, which makes use of the fast multipole method, as well as the explicit inclusion of many-body interactions, as demonstrated by Klicpera et al.

Original: (2020) and Batatia et al.
Revised: (2020) and Batatia et al.

Original: (2022b) [Table] and additional weights, as shown by Sch¨utt et al.
Revised: (2022b), and additional weights, as shown by Sch¨utt et al.

Original: (2021), could be promising avenues.
Revised: (2021), could be promising avenues.

Original: As a particular use case, we envision the application in (QM)ML/MM MD simulations, which could facilitate the study of large biomolecular systems and enzymatic reactions.
Revised: As a particular use case, we envision the application in (QM)ML/MM MD simulations, which could facilitate the study of large biomolecular systems and enzymatic reactions.

Original:  R EPRODUCIBILITY S TATEMENT
Revised:  R EPRODUCIBILITY S TATEMENT

Original: Code and model parameters used to produce the results in this work are made available at https://anonymous.4open.science/r/AMP-BC40/.
Revised: Code and model parameters used to produce the results in this work are made available at https://anonymous.4open.science/r/AMP-BC40/.

Original: All datasets used in this work are either already publicly available or made available under the same URL
Revised: All datasets used in this work are either already publicly available or made available under the same URL

Original: A A PPENDIX A. G ENERAL I MPLEMENTATION D ETAILS
Revised: A A PPENDIX A. C OMPUTATIONAL C OST

Revised: The modifications proposed in this work will result in additional computational costs.

Revised: However, it is important to note that the proposed changes will only increase the constant cost of each message but not the overall complexity.

Revised: The following two paragraphs show empirical running times in a pure QM and in a QM/MM setting.

Revised: All timings are reported based on a Nvidia Titan V.

Revised: A.1. C OMPUTATIONAL C OST : QM

Revised: Here, the computational cost for various orders and number of layers is reported for a single batch of the QM9 dataset containing 100 samples ( 1 ′ 833 ).

Revised: This task only includes a single forward pass.

Revised: Timings were averaged over 100 calls.

Revised: In addition to absolute costs (in brackets), relative costs with respect to the baseline GNN described in Eq.

Revised: 1 are reported in Table 4.

Revised: [Table]

Revised: A.1. C OMPUTATIONAL C OST : QM/MM

Revised: For QM/MM simulations, which generally include a large number of MM particles and a large number of repeated calls

Revised: [Table]

Revised: the cost of the ML potential will significantly contribute to the simulation cost.

Revised: In general, the computational cost will be dominated by the QM/MM interactions due to the larger number of MM particles.

Revised: Since the QM/MM interaction is based on a one-time unidirectional pairwise interaction, these costs will be proportional to N QM × N MM , i

Revised: [Table]

Revised: e., the number of MM particles times the number of QM particles.

Revised: The dominating N MM can be reduced further based on the fast multipole method Rokhlin (1985).

Revised: Here

Revised: [Table]

Revised: we report timings for two settings.

Revised: Setting 1 considers the computational cost without interactions with the MM particles (i

Revised: [Table]

Revised: e., the isolated molecule).

Revised: In setting 2, the computational cost due to the QM/MM interaction is investigated.

Revised: In both cases, the three layer AMP( 3 ) model used for the systems described in Section 4.3 is used.

Revised: In this paragraph, computational costs for the isolated molecule are reported.

Revised: Batches of sizewere used.

Revised: Timings were averaged over all batches using 100 runs and are reported per molecule.

Revised: A cutoff of 5 ˚Aand three layers were used.

Revised: Results are reported in Table A.1.2.

Revised: We find that for the

Revised: [Table]

Revised: two larger systems, the computational cost per edge saturates at roughly 2 µ s per edge as shown in

Revised: Figure 4.

Revised: For both molecules, the cost per edge is roughly 50% higher with AMP( 3 ) compared to

Revised: AMP( 2 ).

Revised: [Figure]

Revised: Finally, we report the total computational cost to predict energies, QM gradients, and MM-gradients for varying numbers of MM particles in Table A.1.2.

Revised: Results were averaged over batches of sizeusing 100 runs.

Revised: Reported are the costs per system.

Revised: [Table]

Revised: A. G ENERAL I MPLEMENTATION D ETAILS

Original: If not noted otherwise the following architecture and hyperparameters were used.
Revised: If not noted otherwise, the following architecture and hyperparameters were used.

Original: Three message passing layers were used except for the QM9 dataset where we used four message passing layers
Revised: Three message passing layers were used except for the QM9 dataset, where we used four message passing layers.

Original: The architecture looked as follows:
Revised: The architecture looked as follows:

Original: Generation of edge weights with [ Linear (16)] , which parametrized ϕ b .
Revised: Generation of edge weights with [ Linear (16)] , which parametrized ϕ b .

Original: As input, the onehot features of the two connecting atoms and Bessel function ( n = 20 ) expanded distance features were used.
Revised: As input, the onehot features of the two connecting atoms and Bessel function ( n = 20 ) expanded distance features were used.

Original: Embedding of node features: [ Linear (128)] 3.
Revised: Embedding of node features: [ Linear (128)] 3.

Original: Message layers with [ Linear (128) , Swish , Linear (128) , Swish ] for ϕ h and [ Linear (128) , Swish , Linear (128) , Swish , Linear(k+1) ] for the prediction of the coefficients
Revised: Message layers with [ Linear (128) , Swish , Linear (128) , Swish ] for ϕ h and [ Linear (128) , Swish , Linear (128) , Swish , Linear(k+1) ] for the prediction of the coefficients

Original: c ij , i.e., ϕ M ( k ) , ϕ M QM ( k ) and ϕ M MM ( k ) .
Revised: c ij , i.e., ϕ M ( k ) , ϕ M QM ( k ) and ϕ M MM ( k ) .

Original: The coefficients for all orders k were predicted with the same module.
Revised: The coefficients for all orders k were predicted with the same module.

Original: A readout layer [ Linear (128) , Swish , Linear (128) , Swish , Linear(1) ] .
Revised: A readout layer [ Linear (128) , Swish , Linear (128) , Swish , Linear(1) ] .

Original: Linear( n ) refers to a standard fully connected network layer with bias.
Revised: Linear( n ) refers to a standard fully connected network layer with bias.

Original: No bias was used for the last layer of the output.
Revised: No bias was used for the last layer of the output.

Original: For each message passing iteration, independent modules were used.
Revised: For each message passing iteration, independent modules were used.

Original: Swish was employed as the non-linearity (Ramachandran et al., 2017).
Revised: Swish was employed as the non-linearity (Ramachandran et al., 2017).

Original: Layer weights were initialized based on the method proposed by He et al.
Revised: Layer weights were initialized based on the method proposed by He et al.

Original: Coefficients c ij were scaled by 10 − 4 to support training stability during early epochs.
Revised: Coefficients c ij were scaled by 10 − 4 to support training stability during early epochs.

Original: Node features were initialized as onehot vectors of the element.
Revised: Node features were initialized as onehot vectors of the element.

Original: Norms of the multipoles were concatenated with the hidden feature.
Revised: Norms of the multipoles were concatenated with the hidden feature.

Original: Edges were added for all atom pairs within a distance of 5 ˚A.
Revised: Edges were added for all atom pairs within a distance of 5 ˚A.

Original: For interactions between QM and MM particles, an independent long-range cutoff of 50 ˚A was used.
Revised: For interactions between QM and MM particles, an independent long-range cutoff of 50 ˚ A was used.

Original: Radial weights were conditioned on the initial onehot features, i.e., ϕ b ( h 0 i , h 

Revised: Radial weights were conditioned on the initial onehot features, i.e., ϕ b ( h 0 i , h

Original: j , || r ij || ) .
Revised: j , || r ij || ) .

Original: Distances || r ij || were encoded with the enveloped Bessel functions proposed by Klicpera et al.
Revised: Distances || r ij || were encoded with the enveloped Bessel functions proposed by Klicpera et al.

Original: (2020) withfunctions and p = 5 for the envelope.
Revised: (2020) with 20 functions and p = 5 for the envelope.

Original: The frequencies were not optimized.
Revised: The frequencies were not optimized.

Original: A ResNet-like update of the hidden feature was used, i.e., h l +1 i =
Revised: A ResNet-like update of the hidden feature was used, i.e., h l +1 i =

Original: h li + ϕ h
Revised: h li + ϕ h

Original: ( h li , m [Equation]
Revised: ( h li , m i )

Original: (He et al., 2016).
Revised: (He et al., 2016).

Original: Model parameters were optimized using ADAM (Kingma & Ba, 2017) with default parameters ( β 1 = 0 .
Revised: Model parameters were optimized using ADAM (Kingma & Ba, 2017) with default parameters ( β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 999 , ϵ = 10 − 7 ) and an exponentially decaying learning rate ( 5 · 10 − 4 , 10 − 5 ).
Revised: 999 , ϵ = 10 − 7 ) and an exponentially decaying learning rate ( 5 · 10 − 4 , 10 − 5 ).

Original: Gradients were clipped by their global norm with a clip norm of 1 (Pascanu et al., 2012).
Revised: Gradients were clipped by their global norm with a clip norm of 1 (Pascanu et al., 2012).

Original: Mean-squared errors were optimized.
Revised: Mean-squared errors were optimized.

Original: If the model was jointly trained on energies and gradients, the following loss function was used
Revised: If the model was jointly trained on energies and gradients, the following loss function was used

Original: [Equation]
Revised: [Equation]

Original:  with the potential energy V and the force component F i,α for each Cartesian dimension α . λ = 0 .was used to balance the contributions of the energies and gradients to the loss.
Revised: (12) with the potential energy V and the force component F i,α for each Cartesian dimension α . λ = 0 .was used to balance the contributions of the energies and gradients to the loss.

Original: The loss term for the MM gradients was obtained in the same way but scaled by an additional factor 100 as proposed in (Hofstetter et al., 2022).
Revised: The loss term for the MM gradients was obtained in the same way but scaled by an additional factor 100 as proposed in (Hofstetter et al., 2022).

Original: Models were implemented with TensorFlow (2.6.2 and 2.9.1) (Abadi et al., 2015; Developers, 2021).
Revised: Models were implemented with TensorFlow (2.6.2 and 2.9.1) (Abadi et al., 2015; Developers, 2021).

Original: Single precision float32 was used.
Revised: Single precision float32 was used.

Original: A. W ATER D IMER
Revised: A. W ATER D IMER

Original: Calculations were performed with PSI4 (1.6) (Turney et al., 2012; Parrish et al., 2017; Smith et al., 2020) on a MP2/aug-cc-pVTZ
Revised: Calculations were performed with PSI4 (1.6) (Turney et al., 2012; Parrish et al., 2017; Smith et al., 2020) on a MP2/aug-cc-pVTZ

Original: level of theory using density fitted MP2 (Dunlap, 2000; Distasio JR.
Revised: level of theory using density fitted MP2 (Dunlap, 2000; Distasio JR.

Original: et al., 2007).
Revised: et al., 2007).

Original: The molecules were placed such that the two oxygen atoms were separated by 4 ˚A.
Revised: The molecules were placed such that the two oxygen atoms were separated by 4 ˚A.

Original: Structures were assigned to training and test sets in an alternating manner.
Revised: Structures were assigned to training and test sets in an alternating manner.

Original: Single layer models were used for all experiments.
Revised: Single layer models were used for all experiments.

Original: Models were trained for 10 ′ 000 steps using a batch size of five.
Revised: Models were trained for 10 ′ 000 steps using a batch size of five.

Original: 64dimensional features were used as node features and four edge weights.
Revised: 64dimensional features were used as node features and four edge weights.

Original: Since the whole system was simulated with MP2, no partial charge embedding was used.
Revised: Since the whole system was simulated with MP2, no partial charge embedding was used.

Original: Instead, the pairwise interaction for the model ’-DP’ was implemented as
Revised: Instead, the pairwise interaction for the model ’-DP’ was implemented as

Original: [Equation]
Revised: [Equation]

Original: for all long-range pairs, i.e., || r
Revised: for all long-range pairs, i.e., || r

Original: ij || ≥ r cut .
Revised: ij || ≥ r cut .

Original: Double precision float64 was used for this system.
Revised: Double precision float64 was used for this system.

Revised: A.3. W ATER D IMER : L EARNING C URVE

Revised: Work on equivariant and directional message passing models has shown that inclusion of directional information can improve data efficiency (Sch ¨ utt et al., 2021; Batzner et al., 2022; Batatia et al., 2022b).

Revised: Figure 5 illustrates the convergence of AMP models for the fully connected topology of the water dimer in Figure 2A.

Revised: [Figure]

Revised: In addition, shown in Figure 6, we also explored the training behaviour for the AMP( 2 ) model on the sparsely-connected topology shown in Figure 2B. Interestingly, there appear to be cases where training proceeds in a phase-transition like manner in contrast to the consistent convergence observed for the fully connected topology in Figure 2A.

Original: A. W ATER C LUSTERS
Revised: A. W ATER C LUSTERS

Original: The dataset with the water clusters was created based on the geometries presented in (Rakshit et al., 2019)
Revised: The dataset with the water clusters was created based on the geometries presented in (Rakshit et al., 2019)

Original: For each set of clusters ranging from 6 to 20 water molecules, up to 2 ′ 000 clusters were randomly sampled.
Revised: For each set of clusters ranging from 6 to 20 water molecules, up to 2 ′ 000 clusters were randomly sampled.

Original: For each cluster, one water molecule was randomly designated as the QM zone with the remaining molecules forming the MM zone.
Revised: For each cluster, one water molecule was randomly designated as the QM zone with the remaining molecules forming the MM zone.

Original: The long-range cutoff was set to 30 ˚A , including all
Revised: The long-range cutoff was set to 30 ˚A , including all

Original: QM–MM interactions.
Revised: QM–MM interactions.

Original: All accessible clusters were used, i.e., all known clusters within a 5 kcal/mol energy window above the putative minimum.
Revised: All accessible clusters were used, i.e., all known clusters within a 5 kcal/mol energy window above the putative minimum.

Original: All clusters of size n = 10 were used for training while the other clusters were used for testing.
Revised: All clusters of size n = 10 were used for training while the other clusters were used for testing.

Original: For the  GNN models, the energy was predicted for the QM particles.
Revised: For the PaiNN, SchNet, and GNN models, the energy was predicted for the QM particles.

Original: Furthermore, in addition to the existing atom types (oxygen-QM and  hydrogen-QM), two new types were introduced: oxygen-MM and hydrogen-MM.
Revised: Furthermore, in addition to the existing atom types (oxygen-QM and [Figure] hydrogen-QM), two new types were introduced: oxygen-MM and hydrogen-MM.

Revised: The implementation provided in the SchNetPack (Sch ¨ utt et al., 2019) was used for PaiNN and SchNet (Sch ¨ utt et al., 2021;

Revised: Sch ¨ utt et al., 2018).

Revised: For PaiNN, the size of the scalar representation ( 160 ) was increased to provide a comparable number of trainable parameters ( 1 M ).

Revised: For SchNet, the size of layers and the number of filters was set to 256 .

Revised: The same cutoff ( 5 ˚A), readout and radial basis functions were used as in the AMP model.

Original:  Partial charges of the MM zone were assigned based on MBIS monopoles (Verstraelen et al., 2016), i.e., − 0 .
Revised: Numerical results for all models are reported in Table A.4.1. Partial charges of the MM zone were assigned based on MBIS monopoles (Verstraelen et al., 2016), i.e., − 0 .

Original: 914197 e for oxygen and 0 .
Revised: 914197 e for oxygen and 0 .

Original: e for hydrogen.
Revised: e for hydrogen.

Original: Calculations were performed on a MP2/aug-cc-pVTZ level of theory (Møller & Plesset, 1934; Dunning, 1989; Kendall et al., 1992) using density fitted MP2 (Dunlap, 2000; Distasio JR.
Revised: Calculations were performed on a MP2/aug-cc-pVTZ level of theory (Møller & Plesset, 1934; Dunning, 1989; Kendall et al., 1992) using density fitted MP (Dunlap, 2000; Distasio JR.

Original: et al., 2007) as implemented in PSI4 (1.6) (Turney et al., 2012; Parrish et al., 2017; Smith et al., 2020).
Revised: et al., 2007) as implemented in PSI4 (1.6) (Turney et al., 2012; Parrish et al., 2017; Smith et al., 2020).

Original: Gradients and energies of the isolated QM zone were subtracted.
Revised: Gradients and energies of the isolated QM zone were subtracted.

Original: The models were trained over 50 ′ 000 steps, presenting each time a batch of 16 randomly drawn samples. 
Revised: The models were trained over 50 ′ 000 steps, presenting each time a batch of 16 randomly drawn samples. A.4. W ATER C LUSTERS : R ESULTS [Table] [Table]

Original: A. QM/MM S YSTEMS
Revised: A. QM/MM S YSTEMS

Original: The datasets were taken from B¨oselt et al.
Revised: The datasets were taken from B¨oselt et al.

Original: (2021) and randomly split into training/validation/test sets of 7 ′ 000 / 2 ′ 000 / 1 ′ 000 samples, respectively.
Revised: (2021) and randomly split into training/validation/test sets of 7 ′ 000 / 2 ′ 000 / 1 ′ 000 samples, respectively.

Original: Polarization due to the MM particles described in
Revised: Polarization due to the MM particles described in

Original: Eq.
Revised: Eq.

Original: 10 was only incorporated during the last message passing step.
Revised: 10 was only incorporated during the last message passing step.

Original: In addition, only eight distance weights were used for interactions between QM and MM particles while the default of 16 was used for edges within the graph, i.e., between QM particles.
Revised: In addition, only eight distance weights were used for interactions between QM and MM particles while the default of 16 was used for edges within the graph, i.e., between QM particles.

Original: The models were trained over 400 ′ 000 steps using batches of size 5 for uracil and batches of size 1 for retinoic acid and SAM/cysteine.
Revised: The models were trained over 400 ′ 000 steps using batches of size 5 for uracil and batches of size 1 for retinoic acid and SAM/cysteine.

Original: For the comparison between the long-range cutoffs, only 200 ′ 000 training steps were used.
Revised: For the comparison between the long-range cutoffs, only 200 ′ 000 training steps were used.

Original: A. QM9 D ATASET
Revised: A. QM9 D ATASET

Original: The dataset was taken from Ramakrishnan et al.
Revised: The dataset was taken from Ramakrishnan et al.

Original: An independent 4 -layer AMP( 3 ) model was trained for each property over 2 ′ 000 epochs with an exponentially decaying learning rate ( 5 · 10 − 4 , 5 · 10 − 6 ).
Revised: An independent 4 -layer AMP( 3 ) model was trained for each property over 2 ′ 000 epochs with an exponentially decaying learning rate ( 5 · 10 − 4 , 5 · 10 − 6 ).

Original: A 4 -layer AMP( 3 ) results in 1 ′ 

Revised: A 4 -layer AMP( 3 ) results in 1 ′

Original: ′ 793 trainable parameters.
Revised: ′ 793 trainable parameters.

Original: Properties in units of Eh were converted and trained in units of kcal mol − 1 .
Revised: Properties in units of Eh were converted and trained in units of kcal mol − 1 .

Original: Each property was predicted as the sum of atomic contributions except for the dipole moment µ .
Revised: Each property was predicted as the sum of atomic contributions except for the dipole moment µ .

Original: For all properties, except for the dipole moment µ , the mean of the training set was subtracted.
Revised: For all properties, except for the dipole moment µ , the mean of the training set was subtracted.

Original: In addition, the energy of the isolated atoms was subtracted for U 0 , U, H, and G. 3 ′ 054 molecules for which the geometry consistency checks failed were removed from the dataset Ramakrishnan et al.
Revised: In addition, the energy of the isolated atoms was subtracted for U 0 , U, H, and G. 3 ′ 054 molecules for which the geometry consistency checks failed were removed from the dataset Ramakrishnan et al.

Original: The remaining data points were randomly split into a training/validation/test sets ( 110 ′ 000 , 10 ′ 000 , 10 ′ 697 ) using a batch size of 100 .
Revised: The remaining data points were randomly split into a training/validation/test sets ( 110 ′ 000 , 10 ′ 000 , 10 ′ 697 ) using a batch size of 100 .

Original: As proposed by Veit et al.
Revised: As proposed by Veit et al.

Original: (2020), the molecular dipole moment µ was modeled as the sum of the contribution from atomic monopoles q i and atomic dipoles µ i
Revised: (2020), the molecular dipole moment µ was modeled as the sum of the contribution from atomic monopoles q i and atomic dipoles µ i

Original: [Equation]
Revised: [Equation]

Original: (14) with each atomic dipoleµ i being predicted according to Eq.
Revised: [Equation] with each atomic dipole

Original: The mean excess charge was subtracted from each q i to enforce charge conservation.