Original: Modular Flows: Differential Molecular Generation
Revised: Modular Flows: Differential Molecular Generation

Original: Abstract
Revised: Abstract

Original: Generating new molecules is fundamental to advancing critical applications suchas drug discovery and material synthesis.
Revised: Generating new molecules is fundamental to advancing critical applications suchas drug discovery and material synthesis.

Original: Flows can generate molecules effectivelyby inverting the encoding process, however, existing flow models either requireartifactual dequantization or specific node/edge orderings, lack desiderata such aspermutation invariance, or induce discrepancy between encoding and decodingsteps that necessitates post hoc validity correction.
Revised: Flows can generate molecules effectivelyby inverting the encoding process, however, existing flow models either requireartifactual dequantization or specific node/edge orderings, lack desiderata such aspermutation invariance, or induce discrepancy between encoding and decodingsteps that necessitates post hoc validity correction.

Original: We circumvent these issueswith novel continuous normalizing E(3)-equivariant flows, based on a system ofnode ODEs coupled as a graph PDE, that repeatedly reconcile locally towardglobally aligned densities.
Revised: We circumvent these issueswith novel continuous normalizing E(3)-equivariant flows, based on a system ofnode ODEs coupled as a graph PDE, that repeatedly reconcile locally towardglobally aligned densities.

Original: Our models can be cast as message passing temporalnetworks, and result in superlative performance on the tasks of density estimationand molecular generation.
Revised: Our models can be cast as message passing temporalnetworks, and result in superlative performance on the tasks of density estimationand molecular generation.

Original: In particular, our generated samples achieve state of theart on both the standard QM9 and ZINC250K benchmarks.
Revised: In particular, our generated samples achieve state of theart on both the standard QM9 and ZINC250K benchmarks.

Original:  Introduction
Revised:  Introduction

Original: Generative models have rapidly become ubiquitous in machine learning with advances from image
Revised: Generative models have rapidly become ubiquitous in machine learning with advances from image

Original: synthesis(Ramesh et al., 2022) to protein design (Ingraham et al.
Revised: synthesis(Ramesh et al., 2022) to protein design (Ingraham et al.

Original: Molecular generation (Stokes et al., 2020) hasalso received significant attention owing to its promisefor discovering new drugs and materials.
Revised: Molecular generation (Stokes et al., 2020) hasalso received significant attention owing to its promisefor discovering new drugs and materials.

Original: Searching forvalid molecules in prohibitively large discrete spaces is,however, challenging: estimates for drug-like structuresrange between 10 23 and 10 60 but only a tiny fractionon the order of 10 8 - has been synthesized (Polishchuket al., 2013;
Revised: Searching forvalid molecules in prohibitively large discrete spaces is,however, challenging: estimates for drug-like structuresrange between 10 23 and 10 60 but only a tiny fractionon the order of 10 8 - has been synthesized (Polishchuket al., 2013;

Original: Merz et al., 2020).
Revised: Merz et al., 2020).

Original: Thus, learning representations that exploit appropriate molecular inductivebiases (e.g., spatial correlations) becomes crucial.
Revised: Thus, learning representations that exploit appropriate molecular inductivebiases (e.g., spatial correlations) becomes crucial.

Original: Earlier models focused on generating sequences basedon the SMILES notation (Weininger, 1988) used in
Revised: Earlier models focused on generating sequences basedon the SMILES notation (Weininger, 1988) used in

Original: Chemistry to describe the molecular structure as strings.
Revised: Chemistry to describe the molecular structure as strings.

Original: However, they were supplanted by generative models that capture valuable spatial information such asbond strengths and dihedral angles, e.g., by embedding molecular graphs via some graph neural network(GNNs)
Revised: However, they were supplanted by generative models that capture valuable spatial information such asbond strengths and dihedral angles, e.g., by embedding molecular graphs via some graph neural network(GNNs)

Original: (Scarselli et al., 2009; Garg et al., 2020).
Revised: (Scarselli et al., 2009; Garg et al., 2020).

Original: Suchmodels primarily include variants of Generative Adversarial Networks (GANs), Variational Autoen- coders (VAEs), and Normalizing Flows (Dinh et al., 2014, 2016).
Revised: Suchmodels primarily include variants of Generative Adversarial Networks (GANs), Variational Autoen- coders (VAEs), and Normalizing Flows (Dinh et al., 2014, 2016).

Original: Besides known issues with training,
Revised: Besides known issues with training,

Original: GANs (Goodfellow et al., 2014; Maziarka et al., 2020) suffer from the well-documented problem ofmode collapse, thereby generating molecules that lack diversity.
Revised: GANs (Goodfellow et al., 2014; Maziarka et al., 2020) suffer from the well-documented problem ofmode collapse, thereby generating molecules that lack diversity.

Original: VAEs (Kingma and Welling, 2013;
Revised: VAEs (Kingma and Welling, 2013;

Original: Lim et al., 2018; Jin et al., 2018), on the other hand, are susceptible to a distributional shift betweenthe training data and the generated samples.
Revised: Lim et al., 2018; Jin et al., 2018), on the other hand, are susceptible to a distributional shift betweenthe training data and the generated samples.

Original: Moreover, optimizing for likelihood via a surrogatelower bound is likely insufficient to capture the complex dependencies inherent in molecules.
Revised: Moreover, optimizing for likelihood via a surrogatelower bound is likely insufficient to capture the complex dependencies inherent in molecules.

Original: Flows are especially appealing since, in principle, they enable estimating (and sampling from)complex data distributions using a sequence of invertible transformations on samples from a moretractable continuous distribution.
Revised: Flows are especially appealing since, in principle, they enable estimating (and sampling from)complex data distributions using a sequence of invertible transformations on samples from a moretractable continuous distribution.

Original: Molecules are discrete, so many flow models (Madhawa et al.
Revised: Molecules are discrete, so many flow models (Madhawa et al.

Original: ,2019; Honda et al., 2019; Shi et al., 2020) add noise during encoding and later apply a dequantizationprocedure.
Revised: ,2019; Honda et al., 2019; Shi et al., 2020) add noise during encoding and later apply a dequantizationprocedure.

Original: However, dequantization begets distortion and issues related to convergence (Luo et al.,2021).
Revised: However, dequantization begets distortion and issues related to convergence (Luo et al.,2021).

Original: Moroever, many methods segregate the generation of atoms from bonds, so the assembledstructure is often not a valid molecule and requires post hoc correction to ensure validity (Zangand Wang, 2020), effecting a discrepancy between the encoding and the decoded distributions.
Revised: Moroever, many methods segregate the generation of atoms from bonds, so the assembledstructure is often not a valid molecule and requires post hoc correction to ensure validity (Zangand Wang, 2020), effecting a discrepancy between the encoding and the decoded distributions.

Original: Permutation dependence is another undesirable artifact of these methods.
Revised: Permutation dependence is another undesirable artifact of these methods.

Original: Some alternatives havebeen explored to avoid dequantization, e.g., (Lippe and Gavves, 2021) encodes molecules in acontinuous latent space via variational inference and jointly optimizes a flow model for generation.
Revised: Some alternatives havebeen explored to avoid dequantization, e.g., (Lippe and Gavves, 2021) encodes molecules in acontinuous latent space via variational inference and jointly optimizes a flow model for generation.

Original: Discrete graph flows (Luo et al., 2021) also circumvent the many pitfalls of dequantization byresorting to discrete latent variables, and performing validity checks during the generative process.
Revised: Discrete graph flows (Luo et al., 2021) also circumvent the many pitfalls of dequantization byresorting to discrete latent variables, and performing validity checks during the generative process.

Original: However, discrete flows follow an autoregressive procedure that requires a specific ordering of nodesand edges during training.
Revised: However, discrete flows follow an autoregressive procedure that requires a specific ordering of nodesand edges during training.

Original: In general, one shot methods can generate much faster than discrete flows.
Revised: In general, one shot methods can generate much faster than discrete flows.

Original: We offer a different flow-based perspective tailored to molecules.
Revised: We offer a different flow-based perspective tailored to molecules.

Original: Specifically, we suggest coupledcontinuous normalizing E(3)-equivariant flows that bestow generative capabilities from neural partialdifferential equation (PDE) models on graphs.
Revised: Specifically, we suggest coupledcontinuous normalizing E(3)-equivariant flows that bestow generative capabilities from neural partialdifferential equation (PDE) models on graphs.

Original: Graph PDEs have been known to enable designingnew embedding methods such as variants of GNNs (Chamberlain et al., 2021), extending GNNsto continuous layers as Neural ODEs (Poli et al., 2019), and accommodating spatial information(Iakovlev et al., 2020).
Revised: Graph PDEs have been known to enable designingnew embedding methods such as variants of GNNs (Chamberlain et al., 2021), extending GNNsto continuous layers as Neural ODEs (Poli et al., 2019), and accommodating spatial information(Iakovlev et al., 2020).

Original: We instead seek to bring to the fore their efficacy and elegance as tools tohelp generate complex objects, such as molecules, viewed as outcomes resulting from an interplay ofco-adapting latent trajectories (i.e., underlying dynamics).
Revised: We instead seek to bring to the fore their efficacy and elegance as tools tohelp generate complex objects, such as molecules, viewed as outcomes resulting from an interplay ofco-adapting latent trajectories (i.e., underlying dynamics).

Original: Concretely, a flow is associated with eachnode of the graph, and these flows are conjoined as a joint ODE system conditioned on neighboringnodes.
Revised: Concretely, a flow is associated with eachnode of the graph, and these flows are conjoined as a joint ODE system conditioned on neighboringnodes.

Original: While these flows originate independently as samples from simple distributions, they adjustprogressively toward more complex joint distributions as they repeatedly interact with the neighboringflows.
Revised: While these flows originate independently as samples from simple distributions, they adjustprogressively toward more complex joint distributions as they repeatedly interact with the neighboringflows.

Original: We view molecules as samples generated from the globally aligned distributions obtained aftermany such local feedback iterations.
Revised: We view molecules as samples generated from the globally aligned distributions obtained aftermany such local feedback iterations.

Original: We call the proposed method Modular Flows ( ModFlow ) tounderscore that each node can be regarded as a module that coordinates with other modules.
Revised: We call the proposed method Modular Flows ( ModFlow ) tounderscore that each node can be regarded as a module that coordinates with other modules.

Original: Table 1summarizes the capabilities of ModFlow compared to some previous generative works.
Revised: Table 1summarizes the capabilities of ModFlow compared to some previous generative works.

Original: Contributions.
Revised: Contributions.

Original: We propose to learn continuous-time, flow based generative models, grounded ongraph PDEs, for generating molecules without any validity correction.
Revised: We propose to learn continuous-time, flow based generative models, grounded ongraph PDEs, for generating molecules without any validity correction.

Original: In particular,
Revised: In particular,

Original: [Table]
Revised: [Table]

Original: JT
Revised: JT

Original: [Table]
Revised: [Table]

Original: ✓ ✓ ✗ ✗
Revised: ✓ ✓ ✗ ✗

Original: [Table]
Revised: [Table]

Original: al.
Revised: al.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: ✗ ✗ ✗ ✗
Revised: ✗ ✗ ✗ ✗

Original: [Table]
Revised: [Table]

Original: et al.
Revised: et al.

Original: GraphAF ✗ ✗ ✓ ✗
Revised: GraphAF ✗ ✗ ✓ ✗

Original: Shi et al.
Revised: Shi et al.

Original: GraphDF ✗ ✗ ✓ ✗
Revised: GraphDF ✗ ✗ ✓ ✗

Original: Luo et al.
Revised: Luo et al.

Original: [Table]
Revised: [Table]

Original: ✓ ✗ ✓ ✗
Revised: ✓ ✗ ✓ ✗

Original: Zang and Wang (2020)
Revised: Zang and Wang (2020)

Original: GraphNVP ✓ ✗ ✓ ✗
Revised: GraphNVP ✓ ✗ ✓ ✗

Original: Madhawa et al.
Revised: Madhawa et al.

Original: [Table]
Revised: [Table]

Original: ✓ ✓ ✓ ✓ this work
Revised: ✓ ✓ ✓ ✓ this work

Original: • we propose ModFlow , a novel generative model based on coupled continuous normalizingE(3)-equivariant flows.
Revised: • we propose ModFlow , a novel generative model based on coupled continuous normalizingE(3)-equivariant flows.

Original: ModFlow encapsulates essential inductive bias using PDEs, anddefines multiple flows that interact locally toward a globally consistent joint density;
Revised: ModFlow encapsulates essential inductive bias using PDEs, anddefines multiple flows that interact locally toward a globally consistent joint density;

Original: • we encode permutation, translation, rotation, and reflection equivariance, with E(3) equivariant GNNs adapted to molecular generation, and can leverage 3D geometric information;
Revised: • we encode permutation, translation, rotation, and reflection equivariance, with E(3) equivariant GNNs adapted to molecular generation, and can leverage 3D geometric information;

Original: • ModFlow is end-to-end trainable, non-autoregressive, and obviates the need for any external
Revised: • ModFlow is end-to-end trainable, non-autoregressive, and obviates the need for any external

Original: validity checks or correction;
Revised: validity checks or correction;

Original: • empirically, ModFlow achieves state-of-the-art performance on both the standard QM9 (Ramakrishnan et al., 2014) and ZINC250K (Irwin et al., 2012) benchmarks.
Revised: • empirically, ModFlow achieves state-of-the-art performance on both the standard QM9 (Ramakrishnan et al., 2014) and ZINC250K (Irwin et al., 2012) benchmarks.

Original:  Related works
Revised:  Related works

Original: Generative models.
Revised: Generative models.

Original: Earlier attempts for molecule generation (Kusner et al., 2017; Dai et al., 2018)aimed at representing molecules as SMILES strings (Weininger, 1988) and developed sequencegeneration models.
Revised: Earlier attempts for molecule generation (Kusner et al., 2017; Dai et al., 2018)aimed at representing molecules as SMILES strings (Weininger, 1988) and developed sequencegeneration models.

Original: One of the challenges of these approaches is to learn complicated grammar rulesthat can generate syntactically valid sequences of molecules.
Revised: One of the challenges of these approaches is to learn complicated grammar rulesthat can generate syntactically valid sequences of molecules.

Original: Recently, representing molecules asgraphs has inspired new deep generative model for molecular generation (Segler et al., 2018; Samantaet al., 2018; Neil et al., 2018) ranging from VAEs (Jin et al., 2018; Kajino, 2019) to flows (Madhawaet al., 2019; Luo et al., 2021; Shi et al., 2020).
Revised: Recently, representing molecules asgraphs has inspired new deep generative model for molecular generation (Segler et al., 2018; Samantaet al., 2018; Neil et al., 2018) ranging from VAEs (Jin et al., 2018; Kajino, 2019) to flows (Madhawaet al., 2019; Luo et al., 2021; Shi et al., 2020).

Original: The core idea is to learn to encode molecular graphsinto a latent space, and subsequently decode samples from the latent space to generate new molecules(Atwood and Towsley, 2016; Xhonneux et al., 2020; You et al., 2018).
Revised: The core idea is to learn to encode molecular graphsinto a latent space, and subsequently decode samples from the latent space to generate new molecules(Atwood and Towsley, 2016; Xhonneux et al., 2020; You et al., 2018).

Original: Graph partial differential equations.
Revised: Graph partial differential equations.

Original: Graph PDEs is an emerging area that studies PDEs onstructured data encoded as graphs.
Revised: Graph PDEs is an emerging area that studies PDEs onstructured data encoded as graphs.

Original: For instance, one can define a PDE on graphs to track the evolutionof signals defined over the graph nodes under some dynamics.
Revised: For instance, one can define a PDE on graphs to track the evolutionof signals defined over the graph nodes under some dynamics.

Original: Graph PDEs have enabled, amongothers, design of new graph neural networks; see, e.g., works such as GRAND (Chamberlain et al.,2021),
Revised: Graph PDEs have enabled, amongothers, design of new graph neural networks; see, e.g., works such as GRAND (Chamberlain et al.,2021),

Original: GNODE (Poli et al., 2019), NeuralPDE (Iakovlev et al., 2020), Neural operator (Li et al.,2020) and PDE-GCN (Eliasof et al., 2021).
Revised: GNODE (Poli et al., 2019), NeuralPDE (Iakovlev et al., 2020), Neural operator (Li et al.,2020) and PDE-GCN (Eliasof et al., 2021).

Original: Different from all these works, we focus on using PDEsfor generative modeling of molecules (graph-structured objects).
Revised: Different from all these works, we focus on using PDEsfor generative modeling of molecules (graph-structured objects).

Original: Interestingly, ModFlow proposedin this work may be viewed as a new equivariant temporal graph network.
Revised: Interestingly, ModFlow proposedin this work may be viewed as a new equivariant temporal graph network.

Original: Validity oracles.
Revised: Validity oracles.

Original: A key challenge of molecular generative models is to be able to generate validmolecules, according to various criteria for molecular validity or feasibility.
Revised: A key challenge of molecular generative models is to be able to generate validmolecules, according to various criteria for molecular validity or feasibility.

Original: It is a common practiceto use external chemical software as rejection oracles to reduce or exclude invalid molecules, or dovalidity checks as part of autoregressive generation.
Revised: It is a common practiceto use external chemical software as rejection oracles to reduce or exclude invalid molecules, or dovalidity checks as part of autoregressive generation.

Original: (Luo et al., 2021; Shi et al., 2020; Popova et al.,
Revised: (Luo et al., 2021; Shi et al., 2020; Popova et al.,

Original: An important open question has been whether generative models can learn to achieve highgenerative validity intrinsically , i.e., without being aided by oracles or performing additional checks.
Revised: An important open question has been whether generative models can learn to achieve highgenerative validity intrinsically , i.e., without being aided by oracles or performing additional checks.

Original: ModFlow represents a major step forward toward that goal.
Revised: ModFlow represents a major step forward toward that goal.

Original:  Modular Flows
Revised:  Modular Flows

Original: We focus on unsupervised learning of an underlying graph density p ( G ) using a dataset D
Revised: We focus on unsupervised learning of an underlying graph density p ( G ) using a dataset D

Original: ={ G n } Nn =1 of observed molecular graphs G n .
Revised: ={ G n } Nn =1 of observed molecular graphs G n .

Original: We learn a generative flow model p θ ( G ) specified byflow parameters θ , and use it to sample novel high-probability molecules.
Revised: We learn a generative flow model p θ ( G ) specified byflow parameters θ , and use it to sample novel high-probability molecules.

Original: 3.1 Molecular Representation
Revised: 3.1 Molecular Representation

Original: Graph representation.
Revised: Graph representation.

Original: We represent the molecular graph G =
Revised: We represent the molecular graph G =

Original: ( V, E ) of size M as a tuple ofvertices V
Revised: ( V, E ) of size M as a tuple ofvertices V

Original: =
Revised: =

Original: ( v 1 , . .
Revised: ( v 1 , . .

Original: . , v M ) and edges E ⊂ V × V .
Revised: . , v M ) and edges E ⊂ V × V .

Original: Each vertex takes value from an alphabet onatoms: v ∈ A
Revised: Each vertex takes value from an alphabet onatoms: v ∈ A

Original: = { C , H , N , O , P , S , . . .
Revised: = { C , H , N , O , P , S , . . .

Original: while the edges e ∈ B = { 1 , 2 , 3 } abstract the type of bond(i.e.
Revised: while the edges e ∈ B = { 1 , 2 , 3 } abstract the type of bond(i.e.

Original: , single, double, or triple).
Revised: , single, double, or triple).

Original: We assume that, conditioned on the edges, the graph likelihoodfactorizes as a Categorical distribution over vertices given their latent representations:
Revised: We assume that, conditioned on the edges, the graph likelihoodfactorizes as a Categorical distribution over vertices given their latent representations:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: where z i = ( z i C , z
Revised: where z i = ( z i C , z

Original: i H , . . . )
Revised: i H , . . . )

Original: ∈ R |A| is a set of atom score parameters, where z ik ∈ R pertains to node iand type k ∈ A , and σ is the softmax function
Revised: ∈ R |A| is a set of atom score parameters, where z ik ∈ R pertains to node iand type k ∈ A , and σ is the softmax function

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original:  which turns the real-valued scores
Revised: ) which turns the real-valued scores

Original: z i of a node i into normalized probabilities.
Revised: z i of a node i into normalized probabilities.

Original: ModFlow alsosupports 3D molecular graphs that contain atomic coordinates and angles.
Revised: ModFlow alsosupports 3D molecular graphs that contain atomic coordinates and angles.

Original: Tree representations.
Revised: Tree representations.

Original: We can obtain an alternative representation for molecules based on decomposing a molecule into a tree-like structure, by contracting certain vertices into a single node(denoted as a cluster) such that the molecular graph G becomes acyclic.
Revised: We can obtain an alternative representation for molecules based on decomposing a molecule into a tree-like structure, by contracting certain vertices into a single node(denoted as a cluster) such that the molecular graph G becomes acyclic.

Original: Following the decomposition investigated by Jin et al.
Revised: Following the decomposition investigated by Jin et al.

Original: (2018), we restrict these clusters to ring-substructures presentin the molecular data, in addition to the atom alphabet.
Revised: (2018), we restrict these clusters to ring-substructures presentin the molecular data, in addition to the atom alphabet.

Original: Thus, we obtain an extended alphabet A tree =
Revised: Thus, we obtain an extended alphabet A tree =

Original: { C , H , N , . . .
Revised: { C , H , N , . . .

Original: , C 1 , C 2 , . . .
Revised: , C 1 , C 2 , . . .

Original: } , where each cluster label C r corresponds to the some ringsubstructure in the label vocabulary χ .
Revised: } , where each cluster label C r corresponds to the some ringsubstructure in the label vocabulary χ .

Original: We reduce the vocabulary to the 30 most commonly occurringsubstructures of A tree .
Revised: We reduce the vocabulary to the 30 most commonly occurringsubstructures of A tree .

Original: For further details, see Appendix A.1.
Revised: For further details, see Appendix A.1.

Original: 3.2 Differential modular flows
Revised: 3.2 Differential modular flows

Original: Normalizing flows (Kobyzev et al., 2021) provide a general recipe for constructing flexible probabilitydistributions, used in density estimation (Cramer et al., 2021;
Revised: Normalizing flows (Kobyzev et al., 2021) provide a general recipe for constructing flexible probabilitydistributions, used in density estimation (Cramer et al., 2021;

Original: Huang et al., 2018) and generativemodeling (Zhen et al., 2020; Zang and Wang, 2020).
Revised: Huang et al., 2018) and generativemodeling (Zhen et al., 2020; Zang and Wang, 2020).

Original: We propose to model the atom scores z i ( t ) as a
Revised: We propose to model the atom scores z i ( t ) as a

Original: Continuous-time Normalizing Flow (CNF) (Grathwohl et al., 2018) over time t ∈ R + .
Revised: Continuous-time Normalizing Flow (CNF) (Grathwohl et al., 2018) over time t ∈ R + .

Original: We assumethe initial scores at time t = 0 follow an uninformative Gaussian base distribution
Revised: We assumethe initial scores at time t = 0 follow an uninformative Gaussian base distribution

Original: z i (0) ∼ N (0 , I )for each node i .
Revised: z i (0) ∼ N (0 , I )for each node i .

Original: Node scores evolve in parallel over time by a differential equation,
Revised: Node scores evolve in parallel over time by a differential equation,

Original: [Equation]
Revised: [Equation]

Original: where N i = { z j : ( i, j ) ∈ E } is the set of neighbor scores at time t ,x i , x N i is the positionalinformation (2D/3D) of node and the neighbours, and θ are the parameters of the flow function f tobe learned.
Revised: where N i = { z j : ( i, j ) ∈ E } is the set of neighbor scores at time t , and θ are the parameters of theflow function f to be learned.

Original: By collecting all node differentials we obtain a modular joint, coupled ODE, which is our key contribution:,
Revised: By collecting all node differentials we obtain a modular joint, coupled ODE, which is our key contribution:,

Original: [Equation]
Revised: [Equation]

Original: The coupled system of ODEs is equivalent to a graph PDE (Iakovlev et al., 2020; Chamberlain et al.,2021), where the evolution of each node only depends on its immediate neighbors.
Revised: The coupled system of ODEs is equivalent to a graph PDE (Iakovlev et al., 2020; Chamberlain et al.,2021), where the evolution of each node only depends on its immediate neighbors.

Original: The flow induces corresponding density changes by the divergence of f , (Chen et al., 2018)
Revised: The flow induces corresponding density changes by the divergence of f , (Chen et al., 2018)

Original: [Equation]
Revised: [Equation]

Original: starting from the base distribution p 

Revised: starting from the base distribution p

Original: ( z i (0))
Revised: ( z i (0))

Original: =
Revised: =

Original: N ( z i (0) | 0 , I ) .
Revised: N ( z i (0) | 0 , I ) .

Original: The trace picks only the diagonalelements of the Jacobian ∂f ∂ z , which interprets the neighborhood input z N i term as a ‘control’ foreach node z i at each instant t .
Revised: The trace picks only the diagonalelements of the Jacobian ∂f ∂ z , which interprets the neighborhood input z N i term as a ‘control’ foreach node z i at each instant t .

Original: The above system is usually solved via an ODE solver where gradientsare computed via adjoint sensitivity method (Kolmogorov et al., 1962).
Revised: The above system is usually solved via an ODE solver where gradientsare computed via adjoint sensitivity method (Kolmogorov et al., 1962).

Original: This approach incurs alow memory cost, and explicitly controls numerical error.
Revised: This approach incurs alow memory cost, and explicitly controls numerical error.

Original: Notably, moving towards modular flowstranslates sparsity also to the adjoints.
Revised: Notably, moving towards modular flowstranslates sparsity also to the adjoints.

Original: Preposition 1: Modular adjoints are sparser than regular adjoints.
Revised: Preposition 1: Modular adjoints are sparser than regular adjoints.

Original: They can be computed by
Revised: They can be computed by

Original: [Equation]
Revised: [Equation]

Original: i ∪{ i } λ j ( t )

Original: ⊤ ∂f (cid:0) t, z i ( t ) , z N i ( t ) (cid:1) ∂

Original: z j , (7) where the partial derivatives ∂f∂ z =
Revised: where the partial derivatives ∂f∂ z =

Original: [ ∂f i ∂ z j ] ij are sparse (See Appendix A.5 for derivation).
Revised: [ ∂f i ∂ z j ] ij are sparse (See Appendix A.5 for derivation).

Original: 3.3 Equivariant local differential
Revised: 3.3 Equivariant local differential

Original: Our goal is to have a differential function f
Revised: Our goal is to have a differential function f

Original: that is a PDE operator used in Eq. 4, and that satisfies thenatural equivariances and invariances of molecules.
Revised: that is a PDE operator used in Eq. 4, and that satisfies thenatural equivariances and invariances of molecules.

Original: Specifically, the function must be (i) translationequivariant: translating the input results in an equivalent translation of the output; (ii) rotational(and reflection) equivariant: rotating the input results in an equivalent rotation of the output; (iii)permutation equivariant: permuting the input results in the same permutation of the output.
Revised: Specifically, the function must be (i) translationequivariant: translating the input results in an equivalent translation of the output; (ii) rotational (andreflection) invariant: rotating the input results in an equivalent rotation of the output; (iii) permutationinvariant: permuting the input results in the same permutation of the output.

Original: Therefore,we chose to use E(3)-Equivariant GNN (EGNN) (Satorras et al., 2021), which is translation, rotationand reflection equivariant (E(n)), and permutation equivariant with respect to an input set of points(See Appendix A.2 for details).
Revised: Therefore, we choseto use E(3)-Equivariant GNN (EGNN) (Satorras et al., 2021), which is translation, rotation andreflection equivariant (E(n)), and permutation equivariant with respect to an input set of points (See Appendix A.2 for details).

Original: The input to the EGNN are the node embeddings and the geometricinformation (polar coordinates (2D) and spherical polar coordinates (3D)).
Revised: The input to the EGNN are the node embeddings and the geometricinformation (polar coordinates (2D) and spherical polar coordinates (3D)).

Original: Interestingly, ModFlow canbe viewed as a message passing temporal graph network (Rossi et al., 2020) as shown next.
Revised: Interestingly, ModFlow canbe viewed as a message passing temporal graph network (Rossi et al., 2020) as shown next.

Original: Preposition 2: Modular Flows can be cast as Temporal Graph Networks (TGN).
Revised: Preposition 2: Modular Flows can be cast as Temporal Graph Networks (TGN).

Original: The operationsare listed in Table 2, where ModFlow is subjected to a single layer of EGNN.
Revised: The operationsare listed in Table 2, where ModFlow is subjected to a single layer of EGNN.

Original: (See Appendix A.3 formore details).
Revised: (See Appendix A.3 formore details).

Original: 3.4 Training objective
Revised: 3.4 Training objective

Original: Normalizing flows are predominantly trained to minimize KL divergence KL[ p data || p
Revised: Normalizing flows are predominantly trained to minimize KL divergence KL[ p data || p

Original: θ ] between theunknown data distribution p and the flow-generated distribution p θ .
Revised: θ ] between theunknown data distribution p and the flow-generated distribution p θ .

Original: This is equivalent to maximizingtheir cross-entropy E p data [log p θ ]
Revised: This is equivalent to maximizingtheir cross-entropy E p data [log p θ ]

Original: (Papamakarios et al., 2021).
Revised: (Papamakarios et al., 2021).

Original: However, this requires the observationsto live in the flow space, which is not true for us: the discrete graphs G and continuous atom scoresz ( t ) reside in different spaces.
Revised: However, this requires the observationsto live in the flow space, which is not true for us: the discrete graphs G and continuous atom scoresz ( t ) reside in different spaces.

Original: To apply normalizing flows, a mapping between the spaces is needed.
Revised: To apply normalizing flows, a mapping between the spaces is needed.

Original: [Figure]
Revised: [Figure]

Original: Earlier approaches use dequantisation to turn a graph G into a distribution of latent states
Revised: Earlier approaches use dequantisation to turn a graph G into a distribution of latent states

Original: [Figure]
Revised: [Figure]

Original: and argmaxto deterministically map latent states to graphs (Zang and Wang, 2020).
Revised: and argmaxto deterministically map latent states to graphs (Zang and Wang, 2020).

Original: We reduce the learning problem to maximizing the score cross-entropy E
Revised: We reduce the learning problem to maximizing the score cross-entropy E

Original: [Figure]
Revised: [Figure]

Original: z ( T ))
Revised: z ( T ))

Original: [log p θ ( z ( T ))] , where we turn the observed set of graphs { G n } into a set of scores { z n } by using one-hot encoding
Revised: [log p θ ( z ( T ))] , where we turn the observed set of graphs { G n } into a set of scores { z n } by using one-hot encoding

Original: [Equation]
Revised: [Equation]

Original: where onehot( G n ) is a matrix of size M ( n ) × |A
Revised: where onehot( G n ) is a matrix of size M ( n ) × |A

Original: ∫ | (i.e., rows equal to number of nodes of G n andcolumns equal to the number of possible node labels) such that G n ( i, k )
Revised: ∫ | (i.e., rows equal to number of nodes of G n andcolumns equal to the number of possible node labels) such that G n ( i, k )

Original: = 1 if v i = a k ∈ A ∫ , thatis if the vertex i is labeled with atom k , and 0 otherwise; 1 q is a vector with q entries each set to 1;
Revised: = 1 if v i = a k ∈ A ∫ , thatis if the vertex i is labeled with atom k , and 0 otherwise; 1 q is a vector with q entries each set to 1;

Original: A ∫ ∈ {A , A tree } ; and ϵ ∈ [0 , 1] is added to model the noise in estimating the posterior p ( z ( T ) | G )due to short-circuiting the inference process from G to z ( T ) skipping the intermediate dependencies,as shown in the plate diagram of Figure 3.
Revised: A ∫ ∈ {A , A tree } ; and ϵ ∈ [0 , 1] is added to model the noise in estimating the posterior p ( z ( T ) | G )due to short-circuiting the inference process from G to z ( T ) skipping the intermediate dependencies,as shown in the plate diagram of Figure 3.

Original: Effectively, we exploit the (non-reversible) composition of the argmax and softmax operations totransition from the continuous flow space to the discrete graph space, but skip this compositionaltogether in the reverse direction.
Revised: Effectively, we exploit the (non-reversible) composition of the argmax and softmax operations totransition from the continuous flow space to the discrete graph space, but skip this compositionaltogether in the reverse direction.

Original: Importantly, this short-circuiting allows ModFlow to keep theforward and backward flows between z (0) and z ( T ) completely aligned unlike previous approaches.
Revised: Importantly, this short-circuiting allows ModFlow to keep theforward and backward flows between z (0) and z ( T ) completely aligned unlike previous approaches.

Original: We thus maximize an objective over N training graphs,
Revised: We thus maximize an objective over N training graphs,

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which factorises over the size M ( n ) of the n ’th training molecule
Revised: which factorises over the size M ( n ) of the n ’th training molecule

Original: The encoding probability followsfrom the flow Eq. 6, where z (0) is the reverse solution from z n at time T following the flow fbackward in time until t = 0 .
Revised: The encoding probability followsfrom the flow Eq. 6, where z (0) is the reverse solution from z n at time T following the flow fbackward in time until t = 0 .

Original: In practice we solve ODE integrals using numerical solvers, such as
Revised: In practice we solve ODE integrals using numerical solvers, such as

Original: Runge-Kutta.
Revised: Runge-Kutta.

Original: We delegate this task to a general solver of the form ODESolve ( z , f θ , T ) , where mapf θ is applied for T steps starting on z .
Revised: We delegate this task to a general solver of the form ODESolve ( z , f θ , T ) , where mapf θ is applied for T steps starting on z .

Original: An optimizer optim is also required for updating θ .
Revised: An optimizer optim is also required for updating θ .

Original: Require: Dataset D , iterations n iter , batch size
Revised: Require: Dataset D , iterations n iter , batch size

Original: B , number of batches n B 1: Initialise parameters θ of ModFlow (EGNN) 2: for k = 1 , . .
Revised: B , number of batches n B 1: Initialise parameters θ of ModFlow (EGNN) 2: for k = 1 , . .

Original: . , n iter do 3: for b = 1 , . .
Revised: . , n iter do 3: for b = 1 , . .

Original: . , n B do 4:
Revised: . , n B do 4:

Original: Sample D b =
Revised: Sample D b =

Original: ( G 1 , . . .
Revised: ( G 1 , . . .

Original: , G B ) ∼ D 5:
Revised: , G B ) ∼ D 5:

Original: Init z b ( T ) to z b ( G b ; ϵ ) 6: z b (0) =
Revised: Init z b ( T ) to z b ( G b ; ϵ ) 6: z b (0) =

Original: ODESolve ( z b ( T ) , f θ , T ) 7:
Revised: ODESolve ( z b ( T ) , f θ , T ) 7:

Original: L b = log p θ ( G b ) 8: end for 9: θ ←− optim ( 1 n B (cid:80) n B b =1 L b ; θ ) 10: end for
Revised: L b = log p θ ( G b ) 8: end for 9: θ ←− optim ( 1 n B (cid:80) n B b =1 L b ; θ ) 10: end for

Original: Algorithm
Revised: Algorithm

Original: 1: z (0) ∼ N (0 , I ) 2: z ( T ) =
Revised: 1: z (0) ∼ N (0 , I ) 2: z ( T ) =

Original: ODESolve ( z (0) , f − 1 θ , T ) 3: Assign labels by argmax( σ ( z ( T )))
Revised: ODESolve ( z (0) , f − 1 θ , T ) 3: Assign labels by argmax( σ ( z ( T )))

Original: 3.5 Molecular generation
Revised: 3.5 Molecular generation

Original: We generate novel molecules by sampling an initial state z (0) ∼ N (0 , I ) based on structure,and running the modular flow forward in time until z ( T ) .
Revised: We generate novel molecules by sampling an initial state z (0) ∼ N (0 , I ) based on structure,and running the modular flow forward in time until z ( T ) .

Original: This procedure maps a tractable basedistribution p 0 to some more complex distribution p T .
Revised: This procedure maps a tractable basedistribution p 0 to some more complex distribution p T .

Original: We follow argmax to pick the most probablelabel assignment for each node (Zang and Wang, 2020).
Revised: We follow argmax to pick the most probablelabel assignment for each node (Zang and Wang, 2020).

Original: The pseudo algorithms are shown in
Revised: The pseudo algorithms are shown in

Original: Algorithms 1 and 2.
Revised: Algorithms 1 and 2.

Original:  Experiments
Revised:  Experiments

Original: We evaluated ModFlow models trained, variously, on (i) 2D coordinates (ii) 3D coordinates (iii)2D coordinates + Tree representation, (iv) 3D coordinates + Tree representation on the tasks ofdensity estimation and molecular generation.
Revised: We evaluated ModFlow models trained, variously, on (i) 2D coordinates (ii) 3D coordinates (iii)2D coordinates + Tree representation, (iv) 3D coordinates + Tree representation on the tasks ofdensity estimation and molecular generation.

Original: We show that ModFlow compares favorably to otherprominent flow and non-flow based molecular generative models including GraphDF (Luo etal.,2021), GraphNVP (Madhawa et al., 2019), MRNN(Popova et al., 2019), GraphAF
Revised: We show that ModFlow compares favorably to otherprominent flow-based molecular generative models including GraphDF (Luo et al., 2021), GraphNVP(Madhawa et al., 2019), GraphAF

Original: (Shi et al., 2020)etc.
Revised: (Shi et al., 2020) and MoFlow (Zang and Wang, 2020).

Original: Notably, ModFlow achieves state-of-the-art results without validity checks or post hoc correction.
Revised: Notably, ModFlow achieves state-of-the-art results without validity checks or post hoc correction.

Original: We also provide ablation results to underscore the relevance of geometric features and equivariancetoward this superlative empirical performance.
Revised: We alsoprovide ablation results to underscore the relevance of geometric features and equivariance towardthis superlative empirical performance.

Original: 4.1 Density Estimation
Revised: 4.1 Density Estimation

Original: [Figure]
Revised: [Figure]

Original: We now demonstrate the high ex-
Revised: We now demonstrate the high ex-

Original: [Figure]
Revised: [Figure]

Original: pressive power of Modular Flows( ModFlow ) on learning highly discontinuous patterns on 2D grid graphs.
Revised: pressive power of Modular Flows( ModFlow ) on learning highly discontinuous patterns on 2D grid graphs.

Original: We considered two variants of chessboard pattern as (i) 4 × 4 grid whereevery node has opposite value to itsneighbors (ii) 16 × 16 grid whereblocks of 4 × 4 nodes have uniformvalues, but opposite across blocks, and(iii) a 20 × 20 grid describing alternating stripes.
Revised: We considered two variants of chessboard pattern as (i) 4 × 4 grid whereevery node has opposite value to itsneighbors (ii) 16 × 16 grid whereblocks of 4 × 4 nodes have uniformvalues, but opposite across blocks, and(iii) a 20 × 20 grid describing alternating stripes.

Original: Figure 4 shows that wecan learn a neural differential f θ function that reproduces the patterns almost perfectly, indicating sufficient capacity to model complexpatterns.
Revised: Figure 4 shows that wecan learn a neural differential f θ function that reproduces the patterns almost perfectly, indicating sufficient capacity to model complexpatterns.

Original: Our model is able to transform the initial Gaussian into multi-modal and discontinuousdistributions.
Revised: Our model is able to transform the initial Gaussian into multi-modal and discontinuousdistributions.

Original: 4.2 Molecule Generation
Revised: 4.2 Molecule Generation

Original: Data.
Revised: Data.

Original: We train and evaluate the models on ZINC250k (Irwin et al., 2012) and QM9 (Ramakrishnanet al., 2014) datasets.
Revised: We train and evaluate the models on ZINC250k (Irwin et al., 2012) and QM9 (Ramakrishnanet al., 2014) datasets.

Original: The ZINC250k set contains 250,000 drug-like molecules up to 38 atoms.
Revised: The ZINC250k set contains 250,000 drug-like molecules up to 38 atoms.

Original: The
Revised: The

Original: QM9 set contains 134,000 stable small organic molecules consisting of CHONF atoms.
Revised: QM9 set contains 134,000 stable small organic molecules consisting of CHONF atoms.

Original: The moleculesare in kekulized form with hydrogens removed by the RDkit software (Landrum et al., 2013).
Revised: The moleculesare in kekulized form with hydrogens removed by the RDkit software (Landrum et al., 2013).

Original: Setup.
Revised: Setup.

Original: We adopt common quality metrics to evaluate molecular generation.
Revised: We adopt common quality metrics to evaluate molecular generation.

Original: (i)
Revised: (i)

Original: Validity is thefraction of molecules that satisfy chemical valency rule.
Revised: Validity is thefraction of molecules that do not violate chemical valency rule.

Original: In this work we strictly limit ourselves tocomparing all methods on validity without external correction procedures (ii) Uniqueness is thefraction of non-duplicate generations, and (iii) Novelty is the fraction not present in training data.
Revised: In this work we strictly limit ourselvesto comparing all methods on validity without external correction procedures (ii) Uniqueness is thefraction of non-duplicate generations, and (iii) Novelty is the fraction not present in training data.

Original: (iv)
Revised: (iv)

Original: Reconstruction is the fraction of molecules that can be reconstructed from their encoding.
Revised: Reconstruction is the fraction of molecules that can be reconstructed from their encoding.

Original: We trainthe model over 5 different seed initialization, and generate 50,000 molecular graphs for evaluation.
Revised: We trainthe model over 5 different seed initialization, and generate 50,000 molecular graphs for evaluation.

Original: We report the mean and the standard deviation of the results from different seeds.
Revised: We report the mean and the standard deviation of the results from different seeds.

Original: Implementation.
Revised: Implementation.

Original: The models are implemented in PyTorch (Paszke et al., 2019).
Revised: The models are implemented in PyTorch (Paszke et al., 2019).

Original: The EGNNmethod used only a single layer with embedding dimension of 32.
Revised: The EGNNmethod used only a single layer with embedding dimension of 32.

Original: The input concatenates timeand scalar vocabulary scores, per node.
Revised: The input concatenates time andscalar vocabulary scores, per node.

Original: We train with Adam optimizer (Kingma and Ba, 2014), for50-100 epochs with batch size 1000 and learning rate 0 .
Revised: We traing with Adam optimizer (Kingma and Ba, 2014), for 100epochs with batch size 1000 and learning rate 0 .

Original: For more details, see Appendix A.4.
Revised: For more details, see Appendix A.4.

Original: Results.
Revised: Results.

Original: Tables 3 and 4 show the results on QM9 and ZINC250K. ModFlow achieves state-of-theart results across all metrics.
Revised: Tables 3 and 4 show the results on QM9 and ZINC250K. ModFlow achieves state-of-theart results across all metrics.

Original: Notably, its reconstruction rate is 100% similar to other flow models;in addition, however, novelty (100%) and uniqueness scores ( 99%) are also very high.
Revised: Notably, its reconstruction rate is 100% similar to other flow models;in addition, however, novelty (100%) and uniqueness scores ( 99%) are also very high.

Original: Moreover,
Revised: Moreover,

Original: ModFlow surpassed all early methods on validity (95%-99%).
Revised: ModFlow surpassed all early methods on validity (95%-99%).

Original: These results underscore the promiseof ModFlow as an effective tool for graph density estimation and molecular generation.
Revised: These results underscore the promiseof ModFlow as an effective tool for graph density estimation and molecular generation.

Original: Additionalevaluation metrics, property optimization and comparison with properties distribution are shown in
Revised: Method [Table] [Table] ModFlow (2D-EGNN)

Original: [-Appendix A.6 and A.
-]
Revised: [-Appendix A.6 and A.
-][Table] 2 ± [Table] [Table]

Revised: ModFlow (3D-EGNN)

Original: [Table]
Revised: [Table]

Revised: 3 ±

Revised: [Table]

Revised: [Table]

Original: [Figure]
Revised: ModFlow (JT-2D-EGNN) [Table] [Table] [Table]

Revised: ModFlow (JT-3D-EGNN)

Revised: [Table]

Revised: [Figure]

Original: 4.3 Ablation Studies
Revised: 4.3 Ablation Studies

Original: We performed ablation experiments to gain further insights about ModFlow , as described next.
Revised: We performed ablation experiments to gain further insights about ModFlow , as described next.

Original: [Table]
Revised: [Table]

Original: E(3) Equivariant vs Not Equivariant.
Revised: E(3) Equivariant vs Not Equivariant.

Original: Molecules exhibit translational and rotational symmetries,so we conducted an ablation study to quantify the effect of incorporating these symmetries in ourmodel.
Revised: Molecules exhibit translational and rotational symmetries,so we conducted an ablation study to quantify the effect of incorporating these symmetries in ourmodel.

Original: We compare the results obtained via using EGNN to non-equivariant graph convolutionalnetworks (GCN).
Revised: We compare the results obtained via using EGNN to non-equivariant graph convolutionalnetworks (GCN).

Original: We used a 3-layer GCN with layer sizes 64-32-32.
Revised: We used a 3-layer GCN with layer sizes 64-32-32.

Original: Table 5 provides strong evidencein favor of modeling symmetries explicitly in Modular Flows
Revised: Table 5 provides strong evidencein favor of modeling symmetries explicitly in Modular Flows

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 2D vs 3D. Finally, we study whether including 3D coordinate information improves the model.
Revised: 2D vs 3D. Finally, we study whether including 3D coordinate information improves the model.

Original: The EGNN coupled differential function obtains either the 2D or 3D positions as polar coordinates,whereas 3D has an extra degree of freedom.
Revised: The EGNN coupled differential function obtains either the 2D or 3D positions as polar coordinates,whereas 3D has an extra degree of freedom.

Original: Table 6 shows that transitioning to 3D gives a consistent≈ 2% percentage unit performance improvement with respect to validity.
Revised: Table 6 shows that transitioning to 3D gives a consistent≈ 2% percentage unit performance improvement with respect to validity.

Original: [Table]
Revised: [Table]

Original:  Conclusion
Revised:  Conclusion

Original: We proposed ModFlow , a new generative flow model where multiple flows interact locally accordingto a coupled ODE [Table] resulting in accurate modeling of graph densities and high quality moleculargeneration without any validity checks or correction.
Revised: We proposed ModFlow , a new generative flow model where multiple flows interact locally accordingto a coupled ODE, resulting in accurate modeling of graph densities and high quality moleculargeneration without any validity checks or correction.

Original: Interesting avenues open up, including the design of (a) more nuanced mappings between discreteand continuous spaces [Table] and (b) extensions of modular flows to (semi-)supervised settings.
Revised: Interesting avenues open up, including the design of (a) more nuanced mappings between discreteand continuous spaces, and (b) extensions of modular flows to (semi-)supervised settings.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu-
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu-

Original: tions and scope?
Revised: tions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: a) Did you include the code, data, and instructions needed to reproduce the main experimentalresults
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental

Original:  (either in the supplemental material or as a URL)?
Revised: results (either in the supplemental material or as a URL)?

Original: [Yes] See the supplementary material.
Revised: [Yes] See the supplementary material.

Original: b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?

Original: Yes] See the Appendix
Revised: [Yes] See the Appendix.

Original: c ) Did you report error bars (e.g. with respect to the random seed after running experimentsmultiple times
Revised: (c) Did you report error bars (e g., with respect to the random seed after running experimentsmultiple times)?

Original: [Yes] See the results section.
Revised: [Yes] See the results section.

Original: d)

Original:  Did you include the total amount of compute and the type of resources used ( e .g. , type of GPUs,internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs,internal cluster, or cloud provider)?

Original: Yes] See the Appendix4.
Revised: [Yes] See the Appendix

Original: If you are using existing assets (e.g., code, data, models) or curating releasing new assets...a) If
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original:  your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: c) Did you include

Original: any

Original:  new assets either in the supplemental material or 
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: as

Original: a URL

Original: [Yes]
Revised: [Yes]

Original: d) Did you discuss whether and how consent was obtained from people whose data you’re using cu
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/cu-

Original: rating
Revised: rating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: informa
Revised: informa-

Original: tion or offensive content
Revised: tion or offensive content?

Original: [N/A
Revised: [N/A

Original: If

Original:  you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects.

Original: [Table] a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table] b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB)approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table] c) Did you include the estimated hourly wage paid to participants and the total amount spent on [Table] compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent onparticipant compensation?

Original: [N/A]
Revised: [N/A]