Original: Deep Surrogate Docking: Accelerating Automated Drug Discovery with Graph Neural Networks
Revised: Deep Surrogate Docking: Accelerating Automated Drug Discovery with Graph Neural Networks

Original: Author(s)
Revised: Author(s)

Original: Abstract
Revised: Abstract

Original: The process of screening molecules for desirable properties is a key step in severalapplications, ranging from drug discovery to material design.
Revised: The process of screening molecules for desirable properties is a key step in severalapplications, ranging from drug discovery to material design.

Original: During the processof drug discovery specifically, protein-ligand docking, or chemical docking , is astandard in-silico scoring technique that estimates the binding affinity of moleculeswith a specific protein target.
Revised: During the processof drug discovery specifically, protein-ligand docking, or chemical docking , is astandard in-silico scoring technique that estimates the binding affinity of moleculeswith a specific protein target.

Original: Recently, however, as the number of virtual molecules available to test has rapidly grown, these classical docking algorithms have createda significant computational bottleneck.
Revised: Recently, however, as the number of virtual molecules available to test has rapidly grown, these classical docking algorithms have createda significant computational bottleneck.

Original: We address this problem by introducing
Revised: We address this problem by introducing

Original: Deep Surrogate Docking (DSD) , a framework that applies deep learning-basedsurrogate modeling to accelerate the docking process substantially.
Revised: Deep Surrogate Docking (DSD) , a framework that applies deep learning-basedsurrogate modeling to accelerate the docking process substantially.

Original: DSD canbe interpreted as a formalism of several earlier surrogate prefiltering techniques.
Revised: DSD canbe interpreted as a formalism of several earlier surrogate prefiltering techniques.

Original: Specifically, we show that graph neural networks (GNNs) can serve as fast andaccurate estimators of classical docking algorithms.
Revised: Specifically, we show that graph neural networks (GNNs) can serve as fast andaccurate estimators of classical docking algorithms.

Original: Additionally, we introduce
Revised: Additionally, we introduce

Original: FiLMv2 , a novel GNN architecture which we show outperforms existing stateof-the-art GNN architectures, attaining more accurate and stable performance by allowing the model to filter out irrelevant information from data more efficiently.
Revised: FiLMv2 , a novel GNN architecture which we show outperforms existing stateof-the-art GNN architectures, attaining more accurate and stable performance by allowing the model to filter out irrelevant information from data more efficiently.

Original: Through extensive experimentation and analysis, we show that the DSD workflowcombined with the FiLMv2 architecture provides a 9.496x speedup in moleculescreening with a < 3% recall error rate on an example docking task.
Revised: Through extensive experimentation and analysis, we show that the DSD workflowcombined with the FiLMv2 architecture provides a 9.496x speedup in moleculescreening with a < 3% recall error rate on an example docking task.

Original: Our opensource code is available at [ hidden for anonymous review ].
Revised: Our opensource code is available at [ hidden for anonymous review ].

Original:  Introduction
Revised:  Introduction

Original: The discovery of new chemical compounds is critical to many scientific fields, such as the developmentof new medicines, environmental energy sources, and sustainable agriculture.
Revised: The discovery of new chemical compounds is critical to many scientific fields, such as the developmentof new medicines, environmental energy sources, and sustainable agriculture.

Original: Previously, discoveryof such molecules involved costly in-vitro synthesis and testing of several candidate compounds [42].
Revised: Previously, discoveryof such molecules involved costly in-vitro synthesis and testing of several candidate compounds [42].

Original: However, in recent years, advancement in computational hardware has led to in-silico screening ofnew chemical compounds.
Revised: However, in recent years, advancement in computational hardware has led to in-silico screening ofnew chemical compounds.

Original: Concretely, representations of chemicals stored in virtual libraries arescreened for desirability using computational scoring functions and top candidates are selected forin-vitro synthesis, testing and analysis [19; 28; 42].One of the primary applications of this process isnew drug discovery.
Revised: Concretely, representations of chemicals stored in virtual libraries arescreened for desirability using computational scoring functions and top candidates are selected forin-vitro synthesis, testing and analysis [19; 28; 42].One of the primary applications of this process isnew drug discovery.

Original: A particularly popular method for in-silico screening of potential compoundsfor new drug desirability is protein-ligand docking .
Revised: A particularly popular method for in-silico screening of potential compoundsfor new drug desirability is protein-ligand docking .

Original: At a high level, docking software analyzes theconformation and orientation (known together as pose ) of a candidate molecule into the bindingsite of a particular target [19].
Revised: At a high level, docking software analyzes theconformation and orientation (known together as pose ) of a candidate molecule into the bindingsite of a particular target [19].

Original: By simulating the pose of the candidate molecule several times, thealgorithm assigns the candidate a docking score , which represents the estimated binding affinity of themolecule to the target [42].
Revised: By simulating the pose of the candidate molecule several times, thealgorithm assigns the candidate a docking score , which represents the estimated binding affinity of themolecule to the target [42].

Original: Top docking candidates 1 (those with the most negative docking scores)
Revised: Top docking candidates 1 (those with the most negative docking scores)

Original: Submitted to NeurIPS 2021 AI for
Revised: Submitted to NeurIPS 2021 AI for

Original: Workshop.
Revised: Workshop.

Original: are then selected for wet-lab synthesis and testing [42].
Revised: are then selected for wet-lab synthesis and testing [42].

Original: Recently, [28] demonstrated the viabilityof this docking-enabled drug discovery at scale.
Revised: Recently, [28] demonstrated the viabilityof this docking-enabled drug discovery at scale.

Original: Specifically, they demonstrate docking a virtuallibrary of over 170 million molecules against two target receptors, and, by selecting only the top44 and 549 chemicals, respectively, yielded effective new potential drugs that target each of thesereceptors.
Revised: Specifically, they demonstrate docking a virtuallibrary of over 170 million molecules against two target receptors, and, by selecting only the top44 and 549 chemicals, respectively, yielded effective new potential drugs that target each of thesereceptors.

Original: However, this docking process is computationally expensive, and generally CPU bound[46; 42].
Revised: However, this docking process is computationally expensive, and generally CPU bound[46; 42].

Original: Indeed, the approach used by [28] to dock 138 million molecule against a single targetrequired 43,563 CPU core hours to compute.
Revised: Indeed, the approach used by [28] to dock 138 million molecule against a single targetrequired 43,563 CPU core hours to compute.

Original: In order to solve this problem, recent research has focused on using deep learning to acceleratethe docking process [19].
Revised: In order to solve this problem, recent research has focused on using deep learning to acceleratethe docking process [19].

Original: Several methods attempt to circumvent the classical docking algorithmaltogether, either by using generative models to produce de-novo candidates with desirable properties [3; 49; 17; 35; 51], or by using surrogate models to predict docking scores [19; 9].
Revised: Several methods attempt to circumvent the classical docking algorithmaltogether, either by using generative models to produce de-novo candidates with desirable properties [3; 49; 17; 35; 51], or by using surrogate models to predict docking scores [19; 9].

Original: Recently,graph neural networks (GNNs) have attracted increasing attention from this field, as moleculescan be concisely represented as graphs [27; 51; 19].
Revised: Recently,graph neural networks (GNNs) have attracted increasing attention from this field, as moleculescan be concisely represented as graphs [27; 51; 19].

Original: GNNs are generalizations of common deeplearning architectures, such as CNNs and attention networks, and are able to directly operate on graphstructured data such as molecules [50].
Revised: GNNs are generalizations of common deeplearning architectures, such as CNNs and attention networks, and are able to directly operate on graphstructured data such as molecules [50].

Original: While current methods show significant promise, they sufferfrom issues such as candidate drug synthesizability issues [3] and model inaccuracy [19].
Revised: While current methods show significant promise, they sufferfrom issues such as candidate drug synthesizability issues [3] and model inaccuracy [19].

Original: In this work, we attempt to address the issues with current deep-learning enabled drug discoveryworkflows.
Revised: In this work, we attempt to address the issues with current deep-learning enabled drug discoveryworkflows.

Original: We do so by (i) proposing a novel Deep Surrogate Docking (DSD) workflow, whichsimplifies and improves the hit-rate of current methods by combining ML-based surrogate dockingwith classical docking, (ii) proposing a novel GNN architecture, FiLMv2 , which we show outperformsexisting state of the art GNN architectures in an example DSD workflow, and (iii) providing additionalauxiliary improvements to the surrogate training process that increase the overall hit rate of the model.
Revised: We do so by (i) proposing a novel Deep Surrogate Docking (DSD) workflow, whichsimplifies and improves the hit-rate of current methods by combining ML-based surrogate dockingwith classical docking, (ii) proposing a novel GNN architecture, FiLMv2 , which we show outperformsexisting state of the art GNN architectures in an example DSD workflow, and (iii) providing additionalauxiliary improvements to the surrogate training process that increase the overall hit rate of the model.

Original: Theoretical discussion and extensive experimentation demonstrates the viability of both DSD
Revised: Theoretical discussion and extensive experimentation demonstrates the viability of both DSD

Original: andthe FiLMv2 architecture in accurately accelerating the throughput of the chemical docking task.
Revised: andthe FiLMv2 architecture in accurately accelerating the throughput of the chemical docking task.

Original: Specifically, we demonstrate a 9.496x computational speedup with < 3% error rate compared toclassical docking on the ZINC virtual library when targeting the D 4 dopamine receptor.
Revised: Specifically, we demonstrate a 9.496x computational speedup with < 3% error rate compared toclassical docking on the ZINC virtual library when targeting the D 4 dopamine receptor.

Original: Section 2.1 describes the DSD workflow.
Revised: Section 2.1 describes the DSD workflow.

Original: Concretely, a small randomly sampled subset of the virtualmolecule library is run through a classical docking algorithm to generate labels (i.e. ground truthdocking scores), which are in turn used to train the surrogate model to predict docking scores givena molecule.
Revised: Concretely, a small randomly sampled subset of the virtualmolecule library is run through a classical docking algorithm to generate labels (i.e. ground truthdocking scores), which are in turn used to train the surrogate model to predict docking scores givena molecule.

Original: The trained surrogate model is then used to perform inference on the full moleculelibrary, and the top scoring molecule candidates are run through the classical docking algorithm.
Revised: The trained surrogate model is then used to perform inference on the full moleculelibrary, and the top scoring molecule candidates are run through the classical docking algorithm.

Original: Thenovelties of the DSD formalism with respect to earlier pre-filtering methods include the introductionof refined evaluation metrics and training strategies.
Revised: Thenovelties of the DSD formalism with respect to earlier pre-filtering methods include the introductionof refined evaluation metrics and training strategies.

Original: Our analysis framework indicates our workflowis nearly 10x as fast as the traditional-only docking method with less than 0.1% error of detecting tophits.
Revised: Our analysis framework indicates our workflowis nearly 10x as fast as the traditional-only docking method with less than 0.1% error of detecting tophits.

Original: Section 2.2 describes the FiLMv2 architecture, a simple iterative improvement to the existing
Revised: Section 2.2 describes the FiLMv2 architecture, a simple iterative improvement to the existing

Original: FiLM architecture which improves model performance in our application.
Revised: FiLM architecture which improves model performance in our application.

Original: Specifically, the affinetransformation used in the FiLM architecture is added to the node representation after , rather thanbefore, the non-linearity.
Revised: Specifically, the affinetransformation used in the FiLM architecture is added to the node representation after , rather thanbefore, the non-linearity.

Original: This increases the expressiveness of the model by preventing collapse toa single affine transformation.
Revised: This increases the expressiveness of the model by preventing collapse toa single affine transformation.

Original: We experimentally demonstrate the improvement of the FiLMv2architecture compared to other GNN architectures in the DSD surrogate model task.
Revised: We experimentally demonstrate the improvement of the FiLMv2architecture compared to other GNN architectures in the DSD surrogate model task.

Original:  Methods
Revised:  Methods

Original: Here, we present the key methods of our paper, including the details of our surrogate model workflow,the FiLMv2 architecture, the Exponentially Weighted Mean Squared Error Loss and Virtual Node.
Revised: Here, we present the key methods of our paper, including the details of our surrogate model workflow,the FiLMv2 architecture, the Exponentially Weighted Mean Squared Error Loss and Virtual Node.

Original: 2.1 Deep Surrogate Docking
Revised: 2.1 Deep Surrogate Docking

Original: At a high level, surrogate models aim to emulate, and in many cases, replace [13] [31], classicaldocking simulation software described in Sections 1 and 4.
Revised: At a high level, surrogate models aim to emulate, and in many cases, replace [13] [31], classicaldocking simulation software described in Sections 1 and 4.

Original: These classical docking methods aregenerally computationally intensive and limited to CPU [28; 42].
Revised: These classical docking methods aregenerally computationally intensive and limited to CPU [28; 42].

Original: Thus, surrogate models can enablean exciting opportunity to extend docking over to the ever-increasing size [38; 42] of virtual libraries.
Revised: Thus, surrogate models can enablean exciting opportunity to extend docking over to the ever-increasing size [38; 42] of virtual libraries.

Original: However, previous analysis [42; 13] has shown that predicting exact docking scores as a regressiontask can be infeasible.
Revised: However, previous analysis [42; 13] has shown that predicting exact docking scores as a regressiontask can be infeasible.

Original: In order to address this issue, we propose the Deep Surrogate Docking (DSD)framework.
Revised: In order to address this issue, we propose the Deep Surrogate Docking (DSD)framework.

Original: In this workflow, surrogate models are trained to select top candidate molecules which are then docked using a classical algorithm in order to determine the exact docking score and poseinformation as standard docking workflow.
Revised: In this workflow, surrogate models are trained to select top candidate molecules which are then docked using a classical algorithm in order to determine the exact docking score and poseinformation as standard docking workflow.

Original: Existing prefiltering techniques such as those proposedin [39] and [54] can be interpreted as special cases of our DSD framework.
Revised: Existing prefiltering techniques such as those proposedin [39] and [54] can be interpreted as special cases of our DSD framework.

Original: Specifically, we definea molecule as a graph m =
Revised: Specifically, we definea molecule as a graph m =

Original: ( V, E ) where V is a set of vertices and E a set of edges.
Revised: ( V, E ) where V is a set of vertices and E a set of edges.

Original: We define agiven virtual library of molecules as M = { m 1 , m 2 , . . .
Revised: We define agiven virtual library of molecules as M = { m 1 , m 2 , . . .

Original: , m N } with | M | = N .
Revised: , m N } with | M | = N .

Original: We then define aclassical docking algorithm as a function
Revised: We then define aclassical docking algorithm as a function

Original: f D : ( V, E ) → R which maps a given molecule to a scalardocking score 2 .
Revised: f D : ( V, E ) → R which maps a given molecule to a scalardocking score 2 .

Original: Similarly, we can define a surrogate docking model as a function f S : ( V, E ) → R ,which attempts to estimate the output of f D .
Revised: Similarly, we can define a surrogate docking model as a function f S : ( V, E ) → R ,which attempts to estimate the output of f D .

Original: We implement this surrogate model as a novel GNNarchitecture described in Section 2.2.
Revised: We implement this surrogate model as a novel GNNarchitecture described in Section 2.2.

Original: In our workflow, we first randomly sample a (typically small) subset of M : D ⊆ M , with | D | =
Revised: In our workflow, we first randomly sample a (typically small) subset of M : D ⊆ M , with | D | =

Original: ρNdetermined by hyperparameter ρ (See Section 3.2 for more details).
Revised: ρNdetermined by hyperparameter ρ (See Section 3.2 for more details).

Original: We then run this subset Dthrough the docking algorithm in order to generate labels (docking scores) for each m ∈ D : Y ={ y i =
Revised: We then run this subset Dthrough the docking algorithm in order to generate labels (docking scores) for each m ∈ D : Y ={ y i =

Original: f D ( m i ) } | D | i =1 , where ∀ i m i ∈
Revised: f D ( m i ) } | D | i =1 , where ∀ i m i ∈

Original: D .
Revised: D .

Original: We can then define the training set used for training f S as
Revised: We can then define the training set used for training f S as

Original: T = { ( m ( i ) , y ( i ) ) } | D | i =1 , where ∀
Revised: T = { ( m ( i ) , y ( i ) ) } | D | i =1 , where ∀

Original: i y i ∈ Y .
Revised: i y i ∈ Y .

Original: This train set can optionally further be split into typicaltrain-val-test sets for the purposes of model evaluation (See 3.1 for details on our data splits).
Revised: This train set can optionally further be split into typicaltrain-val-test sets for the purposes of model evaluation (See 3.1 for details on our data splits).

Original: Weemphasize here that in chemical docking, molecules with the lowest docking scores correspond to themost desirable , and we thus use lowest and best scores interchangeably.
Revised: Weemphasize here that in chemical docking, molecules with the lowest docking scores correspond to themost desirable , and we thus use lowest and best scores interchangeably.

Original: This thus coverts the problemof training f S into a fully supervised machine learning task.
Revised: This thus coverts the problemof training f S into a fully supervised machine learning task.

Original: f S is then trained on T using typicalgradient descent methods (See Section 3 and Appendix for more details on our training methods).
Revised: f S is then trained on T using typicalgradient descent methods (See Section 3 and Appendix for more details on our training methods).

Original: Next, given a fully trained surrogate model f S , we can perform inference on the full virtual library
Revised: Next, given a fully trained surrogate model f S , we can perform inference on the full virtual library

Original: M in order to generate estimated docking scores y S ∈ Y S =
Revised: M in order to generate estimated docking scores y S ∈ Y S =

Original: { f S ( m i ) } Ni =1 , where ∀ i m i ∈ M .
Revised: { f S ( m i ) } Ni =1 , where ∀ i m i ∈ M .

Original: Wethen define the set P ⊂ M selecting the top | P | = σN molecules with the lowest estimated dockingscores.
Revised: Wethen define the set P ⊂ M selecting the top | P | = σN molecules with the lowest estimated dockingscores.

Original: The Data Reduction Factor σ ≪ 1 is the hyperparamter that determines the fraction of mostpromising (top scoring) molecules that are docked by f S .
Revised: The Data Reduction Factor σ ≪ 1 is the hyperparamter that determines the fraction of mostpromising (top scoring) molecules that are docked by f S .

Original: These | P | molecules are then input to f Din order to determine the exact docking score and pose information as standard docking workflow:
Revised: These | P | molecules are then input to f Din order to determine the exact docking score and pose information as standard docking workflow:

Original: Y P = { f D ( m i ) } | P | i =1 , where ∀ i m i ∈ P .
Revised: Y P = { f D ( m i ) } | P | i =1 , where ∀ i m i ∈ P .

Original: We now define, for the purposes of evaluation, the Hit Threshold , ζ , a third hyperparameter whichdetermines the fraction of molecules which should be recalled from the model.
Revised: We now define, for the purposes of evaluation, the Hit Threshold , ζ , a third hyperparameter whichdetermines the fraction of molecules which should be recalled from the model.

Original: For example, whenζ = 0 .
Revised: For example, whenζ = 0 .

Original: 001 , we are interested in correctly selecting the top 0.1% of molecules in a given virtual library.
Revised: 001 , we are interested in correctly selecting the top 0.1% of molecules in a given virtual library.

Original: Our analysis in Section 3 uses ζ = 0 .
Revised: Our analysis in Section 3 uses ζ = 0 .

Original: 001 and ζ = 0 .
Revised: 001 and ζ = 0 .

Original: 01 , which is consistent with typical practices instandard docking [42; 28].
Revised: 01 , which is consistent with typical practices instandard docking [42; 28].

Original: One benefit of such a workflow is that it easily allows introduction of inductive bias into f S so as toproperly rank only top performing molecule candidates.
Revised: One benefit of such a workflow is that it easily allows introduction of inductive bias into f S so as toproperly rank only top performing molecule candidates.

Original: Typically, only molecules on the order of thetop 0.1% or less of the total virtual library are selected for additional domain-specific analysis andpossible synthesis [28; 42; 46].
Revised: Typically, only molecules on the order of thetop 0.1% or less of the total virtual library are selected for additional domain-specific analysis andpossible synthesis [28; 42; 46].

Original: Thus, biases that aid the model in classifying these top molecules,perhaps at the expense of total performance across all docking scores, is useful to the task.
Revised: Thus, biases that aid the model in classifying these top molecules,perhaps at the expense of total performance across all docking scores, is useful to the task.

Original: Oneexample of such an inductive bias is the exponentially weighted MSE loss described in Section2.3.
Revised: Oneexample of such an inductive bias is the exponentially weighted MSE loss described in Section2.3.

Original: Another benefit of this model is the ability to customize the runtime performance vs. accuracytradeoff.
Revised: Another benefit of this model is the ability to customize the runtime performance vs. accuracytradeoff.

Original: Specifically, as hyperparameter σ is varied, the total error rate with respect to traditionaldocking and time required for the full DSD is intuitively traded off.
Revised: Specifically, as hyperparameter σ is varied, the total error rate with respect to traditionaldocking and time required for the full DSD is intuitively traded off.

Original: Section 3.2 shows that theintuitive notion that σ and model accuracy are positively correlated, holds empirically.
Revised: Section 3.2 shows that theintuitive notion that σ and model accuracy are positively correlated, holds empirically.

Original: 2.2 FiLMv2 Architecture
Revised: 2.2 FiLMv2 Architecture

Original: Our proposed model to identify molecules with the lowest docking scores is based on a new GNNarchitecture called FiLMv2.
Revised: Our proposed model to identify molecules with the lowest docking scores is based on a new GNNarchitecture called FiLMv2.

Original: FiLMv2 is a variant of the GNN-FiLM architecture [4], which hasbeen shown to outperform other state-of-the-art architectures, like Graph Convolutional Networks(GCN)
Revised: FiLMv2 is a variant of the GNN-FiLM architecture [4], which hasbeen shown to outperform other state-of-the-art architectures, like Graph Convolutional Networks(GCN)

Original: [24] and Graph Attention Networks (GAT) [43], on regression tasks on molecular graphs.
Revised: [24] and Graph Attention Networks (GAT) [43], on regression tasks on molecular graphs.

Original: The idea behind the original GNN-FiLM model is to perform feature-wise linear modulation of amessage’s source representation, h tu , using a function of the target representation, h tv :
Revised: The idea behind the original GNN-FiLM model is to perform feature-wise linear modulation of amessage’s source representation, h tu , using a function of the target representation, h tv :

Original: [Equation]
Revised: [Equation]

Original: where W α , W β , W γ are learned matrices, ⊙ is the Hadamard element-wise product, N v denotes theneighbors of the target node v , and σ and l are ReLU nonlinearities.
Revised: where W α , W β , W γ are learned matrices, ⊙ is the Hadamard element-wise product, N v denotes theneighbors of the target node v , and σ and l are ReLU nonlinearities.

Original: The GNN-FiLM architecture of
Revised: The GNN-FiLM architecture of

Original: Eq.
Revised: Eq.

Original: 1 allows to build dynamic filters based on the target node representation to decide which featuresof the source representation should be discarded or selected in the updated state of the target node.
Revised: 1 allows to build dynamic filters based on the target node representation to decide which featuresof the source representation should be discarded or selected in the updated state of the target node.

Original: The possibility to apply element-wise modulation of a source’s representation based on the target’srepresentation is absent in GCN and GAT architectures, where the target representation is eitherused as an additional incoming message (GCN) or used to modulate the entire source representation(GAT).
Revised: The possibility to apply element-wise modulation of a source’s representation based on the target’srepresentation is absent in GCN and GAT architectures, where the target representation is eitherused as an additional incoming message (GCN) or used to modulate the entire source representation(GAT).

Original: Our FiLMv2 architecture introduces a simple modification to the original GNN-FiLM Eq.
Revised: Our FiLMv2 architecture introduces a simple modification to the original GNN-FiLM Eq.

Original: 1,consisting in applying a ReLU nonlinearity to the target representations on both the multiplicativeand additive terms:
Revised: 1,consisting in applying a ReLU nonlinearity to the target representations on both the multiplicativeand additive terms:

Original: [Equation]
Revised: [Equation]

Original: The addition of the ReLU nonlinearities enables to perform a more effective selection of the mostrelevant information from the source representation.
Revised: The addition of the ReLU nonlinearities enables to perform a more effective selection of the mostrelevant information from the source representation.

Original: Indeed, target features that are mapped bythe linear transformation to any negative number are zeroed out by the ReLU activation functionand hence can effectively filter out the corresponding entries of the source representation via theelement-wise products.
Revised: Indeed, target features that are mapped bythe linear transformation to any negative number are zeroed out by the ReLU activation functionand hence can effectively filter out the corresponding entries of the source representation via theelement-wise products.

Original: This filtering mechanism is considerably more accurate and stable thanthe original GNN-FiLM, where a fine-tuned calibration of the linear transformation’s parametersis necessary in order to produce an output close to zero.
Revised: This filtering mechanism is considerably more accurate and stable thanthe original GNN-FiLM, where a fine-tuned calibration of the linear transformation’s parametersis necessary in order to produce an output close to zero.

Original: Moreover, without the activation function,for each different target representation a different set of parameter values must be learnt in order tozero-out a particular entry of the source representation, which is highly inefficient and unstable tochanges of the target representation.
Revised: Moreover, without the activation function,for each different target representation a different set of parameter values must be learnt in order tozero-out a particular entry of the source representation, which is highly inefficient and unstable tochanges of the target representation.

Original: These insights are confirmed by a series of tests performed toassess the effectiveness of various combinations of nonlinearities.
Revised: These insights are confirmed by a series of tests performed toassess the effectiveness of various combinations of nonlinearities.

Original: In particular, we observe that (see
Revised: In particular, we observe that (see

Original: Appendix for details of the experiments): (i) the nonlinearity σ in Eq.
Revised: Appendix for details of the experiments): (i) the nonlinearity σ in Eq.

Original: 1 can be removed as it doesnot improve the model’s performance; (ii) adding a nonlinearity to the source term ( W α h tu ) doesnot further improve the model performance; (iii) using a tanh instead of ReLU nonlinearity doesnot bring any improvement with respect to the original GNN-FiLMv1.
Revised: 1 can be removed as it doesnot improve the model’s performance; (ii) adding a nonlinearity to the source term ( W α h tu ) doesnot further improve the model performance; (iii) using a tanh instead of ReLU nonlinearity doesnot bring any improvement with respect to the original GNN-FiLMv1.

Original: The last result in particularcrucially demonstrates that only a nonlinearity that can map to zero a large range of input values caneffectively act as a filter, and for this reason the tanh function does not work well as a filter because itsasymptotic values are − 1 and 1 .
Revised: The last result in particularcrucially demonstrates that only a nonlinearity that can map to zero a large range of input values caneffectively act as a filter, and for this reason the tanh function does not work well as a filter because itsasymptotic values are − 1 and 1 .

Original: However, it is important to note that the above analysis of FiLMv2 islargely empirically driven, and further work is needed to assess whether the improvements of FiLMv2compared to FiLM hold more generally.
Revised: However, it is important to note that the above analysis of FiLMv2 islargely empirically driven, and further work is needed to assess whether the improvements of FiLMv2compared to FiLM hold more generally.

Original: 2.3 Exponentially Weighted Mean Squared Error Loss (W-MSE)
Revised: 2.3 Exponentially Weighted Mean Squared Error Loss (W-MSE)

Original: As described in Section 2.1, the goal of the surrogate model in the DSD framework is to selecttop molecule candidates which are then run through a classical docking algorithm.
Revised: As described in Section 2.1, the goal of the surrogate model in the DSD framework is to selecttop molecule candidates which are then run through a classical docking algorithm.

Original: Thus, correctclassification of top scoring data are far more important than correct classification of comparativelyworse scoring data.
Revised: Thus, correctclassification of top scoring data are far more important than correct classification of comparativelyworse scoring data.

Original: Additionally, perhaps due to the presence of inductive biases in their algorithmdesign, current docking methods tend to produce much noisier scores for comparatively lower scoringmolecules than higher scoring ones [28].
Revised: Additionally, perhaps due to the presence of inductive biases in their algorithmdesign, current docking methods tend to produce much noisier scores for comparatively lower scoringmolecules than higher scoring ones [28].

Original: In order to address these two points, we introduce a weightedmean squared error (W-MSE) loss for use during training.
Revised: In order to address these two points, we introduce a weightedmean squared error (W-MSE) loss for use during training.

Original: This incentivizes our model to properlyscore the training samples with good docking scores.
Revised: This incentivizes our model to properlyscore the training samples with good docking scores.

Original: Additionally, the model is penalized less forbad predictions of bad scoring chemicals, as these labels may be comparatively noisier.
Revised: Additionally, the model is penalized less forbad predictions of bad scoring chemicals, as these labels may be comparatively noisier.

Original: Here, theloss of a single example is weighted by a negative exponential function of the label (docking score).
Revised: Here, theloss of a single example is weighted by a negative exponential function of the label (docking score).

Original: Thus, labels associated with better docking scores (smaller scores) give a higher contribution to theoverall loss.
Revised: Thus, labels associated with better docking scores (smaller scores) give a higher contribution to theoverall loss.

Original: Explicitly, for a scalar valued model predicted score
Revised: Explicitly, for a scalar valued model predicted score

Original: z i for molecule m i , associatedlabel y i , mean squared error (MSE) loss function l : R → R , and exponential coefficient α , we have:loss =
Revised: z i for molecule m i , associatedlabel y i , mean squared error (MSE) loss function l : R → R , and exponential coefficient α , we have:loss =

Original: (cid:80) i e − αy i · l ( z i , y i )
Revised: (cid:80) i e − αy i · l ( z i , y i )

Original: Given that the most desirable (i.e. "best scoring") ligands correspond tothose with the most negative scores, α hyperparameter in the range [0 , ∞ ) , with α = 0 correspondingto the case where no weighing occurs.
Revised: Given that the most desirable (i.e. "best scoring") ligands correspond tothose with the most negative scores, α hyperparameter in the range [0 , ∞ ) , with α = 0 correspondingto the case where no weighing occurs.

Original: In practice, we find that α = 0 .
Revised: In practice, we find that α = 0 .

Original: 8 works well and thus use itfor all models unless otherewise noted, though the effect of this hyperparameter is explored in moredetail in the Appendix.
Revised: 8 works well and thus use itfor all models unless otherewise noted, though the effect of this hyperparameter is explored in moredetail in the Appendix.

Original: 2.4 Virtual Node
Revised: 2.4 Virtual Node

Original: First introduced by [14], a so called virtual node is a single node added to a graph with a connection toevery other node in the given graph.
Revised: First introduced by [14], a so called virtual node is a single node added to a graph with a connection toevery other node in the given graph.

Original: The edge connections of the virtual node are given a special edge
Revised: The edge connections of the virtual node are given a special edge

Original: [Figure]
Revised: [Figure]

Original: type to differentiate them from other nodes in the graph.
Revised: type to differentiate them from other nodes in the graph.

Original: It has been shown that the use of a virtualnode improves performance by facilitating long distance connections between faraway nodes duringthe propogation phase of a GNN architecture[14].
Revised: It has been shown that the use of a virtualnode improves performance by facilitating long distance connections between faraway nodes duringthe propogation phase of a GNN architecture[14].

Original: We introduce this feature to this application, addinga virtual node to each graph as a preprocessing step.
Revised: We introduce this feature to this application, addinga virtual node to each graph as a preprocessing step.

Original: Empirically we find that performance improvesacross all architectures tested.
Revised: Empirically we find that performance improvesacross all architectures tested.

Original: Further, the use of virtual nodes reduces reliance on computationallycomplex global pooling operations, thus reducing both train time and costly hyperaparamter tuning.
Revised: Further, the use of virtual nodes reduces reliance on computationallycomplex global pooling operations, thus reducing both train time and costly hyperaparamter tuning.

Original: The effects of the virtual node on this experiement are explored in ablation (Section 3.4).
Revised: The effects of the virtual node on this experiement are explored in ablation (Section 3.4).

Original:  Experiments
Revised:  Experiments

Original: We now describe and justify our experimental setup, defining the dataset, models, and metrics.
Revised: We now describe and justify our experimental setup, defining the dataset, models, and metrics.

Original: Weapply DSD to docking a virtual library of 128 million molecules with a single receptor target and wecompare results of the FiLMv2 architecture with other state-of-the-art GNN models on a small heldout test dataset, demonstrating the scalability and performance of the DSD framework.
Revised: Weapply DSD to docking a virtual library of 128 million molecules with a single receptor target and wecompare results of the FiLMv2 architecture with other state-of-the-art GNN models on a small heldout test dataset, demonstrating the scalability and performance of the DSD framework.

Original: Finally, weablate the important design elements of the work.
Revised: Finally, weablate the important design elements of the work.

Original: 3.1 Experimental Details
Revised: 3.1 Experimental Details

Original: Datasets For experimentation, we use the ZINC dataset [38] of 128 million virtual molecules withdocking scores against the single D 4 dopamine receptor target, generated using the method describedby [28].
Revised: Datasets For experimentation, we use the ZINC dataset [38] of 128 million virtual molecules withdocking scores against the single D 4 dopamine receptor target, generated using the method describedby [28].

Original: Though to the best of our knowledge, no standard benchmark datasets yet exist for chemicaldocking, the ZINC dataset is one of the largest in size, diverse in molecule scaffold representation,and contains other optimizations not found in earlier virtual libraries [28; 38].
Revised: Though to the best of our knowledge, no standard benchmark datasets yet exist for chemicaldocking, the ZINC dataset is one of the largest in size, diverse in molecule scaffold representation,and contains other optimizations not found in earlier virtual libraries [28; 38].

Original: Additional detailsabout the dataset description and preprocessing can be found in the Appendix.
Revised: Additional detailsabout the dataset description and preprocessing can be found in the Appendix.

Original: For the majorityof our experiments, we randomly sample 500,000 molecules with ground truth labels from thefull dataset, corresponding to ρ = 0 .
Revised: For the majorityof our experiments, we randomly sample 500,000 molecules with ground truth labels from thefull dataset, corresponding to ρ = 0 .

Original: 00313 (see Section 2.1), which are then split into 80-10-10train-validation-test partitions.
Revised: 00313 (see Section 2.1), which are then split into 80-10-10train-validation-test partitions.

Original: Models are tuned using the validation set and final results (in Section3.2) are reported on the held out test partition.
Revised: Models are tuned using the validation set and final results (in Section3.2) are reported on the held out test partition.

Original: In Section 3.2 we explicitly vary the ρ parameter inorder to demonstrate its effect on the overall framework.
Revised: In Section 3.2 we explicitly vary the ρ parameter inorder to demonstrate its effect on the overall framework.

Original: In Section 3.3, labels for the full ZINCdataset are generated and used to evaluate model inference over the entire virtual library.
Revised: In Section 3.3, labels for the full ZINCdataset are generated and used to evaluate model inference over the entire virtual library.

Original: Models We experiment implementing the surrogate model f S with four GNN convolution operations to use as a backbone for our architecture: the graph isomorphism operator (GIN) [53], graphattention convolution operation (GATv2) [5], the feature-wise linear modulation operation (FiLM)
Revised: Models We experiment implementing the surrogate model f S with four GNN convolution operations to use as a backbone for our architecture: the graph isomorphism operator (GIN) [53], graphattention convolution operation (GATv2) [5], the feature-wise linear modulation operation (FiLM)

Original: [4],and our novel FiLMv2 operation, which is discussed in detail in Section 2.2.
Revised: [4],and our novel FiLMv2 operation, which is discussed in detail in Section 2.2.

Original: Our full architectureconsists of a backbone containing a hyperparameterized number of these graph operations, followedby a global mean pool layer and a MLP mapping the pooled features to a single regression score.
Revised: Our full architectureconsists of a backbone containing a hyperparameterized number of these graph operations, followedby a global mean pool layer and a MLP mapping the pooled features to a single regression score.

Original: Additionally, a single dropout layer and layer normalization layer are added between each graphconvolution operation.
Revised: Additionally, a single dropout layer and layer normalization layer are added between each graphconvolution operation.

Original: Each of the four models were trained using the Adam optimizer in order tominimize the weighted mean squared error (W-MSE) loss presented in Section 2.3 and tuned usingstandard hyperparameter tuning practices.
Revised: Each of the four models were trained using the Adam optimizer in order tominimize the weighted mean squared error (W-MSE) loss presented in Section 2.3 and tuned usingstandard hyperparameter tuning practices.

Original: Full details regarding this process can be found in the
Revised: Full details regarding this process can be found in the

Original: Appendix.
Revised: Appendix.

Original: Metrics Given that the overall task of fitting a model with respect to the DSD framework is novelto the best of our knowledge, careful consideration was given to metrics used to evaluate f S .
Revised: Metrics Given that the overall task of fitting a model with respect to the DSD framework is novelto the best of our knowledge, careful consideration was given to metrics used to evaluate f S .

Original: Forthe purposes of evaluation, we can apply a hard threshold to both the ground truth scores and modelpredictions in order to convert the regression task into a binary classification problem.
Revised: Forthe purposes of evaluation, we can apply a hard threshold to both the ground truth scores and modelpredictions in order to convert the regression task into a binary classification problem.

Original: That is,following the logic in Section 2.1, we can label the top ζ fraction of molecules scored by the classicaldocking algorithm f D as positive and otherwise negative.
Revised: That is,following the logic in Section 2.1, we can label the top ζ fraction of molecules scored by the classicaldocking algorithm f D as positive and otherwise negative.

Original: Similarly, we can threshold the top σfraction of molecules scored by the surrogate model f S as positive and otherwise negative.
Revised: Similarly, we can threshold the top σfraction of molecules scored by the surrogate model f S as positive and otherwise negative.

Original: Thus, wecan treat these labels as y true and y pred for the purposes of evaluating f S and use traditional binaryclassification metrics, like precision and recall.
Revised: Thus, wecan treat these labels as y true and y pred for the purposes of evaluating f S and use traditional binaryclassification metrics, like precision and recall.

Original: However, being functions of ζ and σ , the traditionalmetrics vary when different values of the two thresholds are selected and thus cannot offer a fair andcomprehensive performance evaluation of the models.
Revised: However, being functions of ζ and σ , the traditionalmetrics vary when different values of the two thresholds are selected and thus cannot offer a fair andcomprehensive performance evaluation of the models.

Original: Thus, we introduce the RES Score metric tomeasure the overall performance of the models for all combinations of threshold values, the AURTC ζmetric to measure the performance for a fixed ζ , and provide the parametrized Recall values R σ,ζ forspecific values of ζ and σ .
Revised: Thus, we introduce the RES Score metric tomeasure the overall performance of the models for all combinations of threshold values, the AURTC ζmetric to measure the performance for a fixed ζ , and provide the parametrized Recall values R σ,ζ forspecific values of ζ and σ .

Original: The next three paragraphs describe the RES Score, AURTC ζ , and R σ,ζ ,respectively.
Revised: The next three paragraphs describe the RES Score, AURTC ζ , and R σ,ζ ,respectively.

Original: Figure 1 visualizes the relationship between the three metrics.
Revised: Figure 1 visualizes the relationship between the three metrics.

Original: We also provide othertypical classification metrics (e.g. AUROC, F-1, etc) for specific values of ζ and σ in the Appendix.
Revised: We also provide othertypical classification metrics (e.g. AUROC, F-1, etc) for specific values of ζ and σ in the Appendix.

Original: Regression Enrichment Surfaces and the RES Score Regression Enrichment Surfaces (RES)have been introduced as a tool to mitigate some of the issues discussed above [8].
Revised: Regression Enrichment Surfaces and the RES Score Regression Enrichment Surfaces (RES)have been introduced as a tool to mitigate some of the issues discussed above [8].

Original: RES can beconceptualized as a 3D surface parameterized by σ and ζ .
Revised: RES can beconceptualized as a 3D surface parameterized by σ and ζ .

Original: For each σ , ζ ∈ (0 , 1) , the value of RES isthe recall of y true and y pred , that is the fraction of all top-scoring molecules that were identified bythe model.
Revised: For each σ , ζ ∈ (0 , 1) , the value of RES isthe recall of y true and y pred , that is the fraction of all top-scoring molecules that were identified bythe model.

Original: RES can be visualized using a heatmap, such as in Figure 2, where axes are log scaled inorder to highlight performance where σ and ζ are close to zero.
Revised: RES can be visualized using a heatmap, such as in Figure 2, where axes are log scaled inorder to highlight performance where σ and ζ are close to zero.

Original: Additionally, a RES score can becalculated by taking the volume under the RES surface with log-scaled axes.
Revised: Additionally, a RES score can becalculated by taking the volume under the RES surface with log-scaled axes.

Original: This scalar summarizesthe quality of the RES in a single number.
Revised: This scalar summarizesthe quality of the RES in a single number.

Original: Area Under Recall Threshold Curve (AURTC ζ ) We define the Recall Threshold Curve (RTC) asa single dimensional slice of RES obtained by fixing a ζ value and expressing the recall as a functionof σ .
Revised: Area Under Recall Threshold Curve (AURTC ζ ) We define the Recall Threshold Curve (RTC) asa single dimensional slice of RES obtained by fixing a ζ value and expressing the recall as a functionof σ .

Original: We define the area under this curve as AURTC ζ , which allows assessment of performance atparticular values of ζ .
Revised: We define the area under this curve as AURTC ζ , which allows assessment of performance atparticular values of ζ .

Original: We report AURTC ζ for all of our experiments at ζ = 0 .
Revised: We report AURTC ζ for all of our experiments at ζ = 0 .

Original: 01 and ζ = 0 .
Revised: 01 and ζ = 0 .

Original: Parameterized Recall (R σ,ζ ) Finally, we can follow the logic above and once again reduce thedimensionality of the AURTC ζ to a single point by fixing σ and obtaining the corresponding recall.
Revised: Parameterized Recall (R σ,ζ ) Finally, we can follow the logic above and once again reduce thedimensionality of the AURTC ζ to a single point by fixing σ and obtaining the corresponding recall.

Original: We thus define parameterized recall, R σ,ζ , which reports a single recall value for a given σ and ζ .
Revised: We thus define parameterized recall, R σ,ζ , which reports a single recall value for a given σ and ζ .

Original: For all experiments, we report R 0 .
Revised: For all experiments, we report R 0 .

Original: 01 and R 0 .
Revised: 01 and R 0 .

Original: 001 , which we believe represent reasonable typical 254 parameters for most docking tasks.
Revised: 001 , which we believe represent reasonable typical 254 parameters for most docking tasks.

Original: 3.2 Main Results
Revised: 3.2 Main Results

Original: All four models were trained and hyperparameters tuned according to the details in Section 3.1.
Revised: All four models were trained and hyperparameters tuned according to the details in Section 3.1.

Original: The performance on a held out test partition for each model is shown in Table 1.
Revised: The performance on a held out test partition for each model is shown in Table 1.

Original: We note that ournovel FiLMv2 architecture outperforms all other models in all but one of the selected metrics, whereperfomance is equivalent to the original FiLM architecture.
Revised: We note that ournovel FiLMv2 architecture outperforms all other models in all but one of the selected metrics, whereperfomance is equivalent to the original FiLM architecture.

Original: Additionally, we find that AURTC ζimproves as ζ decreases (see Table 1 and Figure 1) demonstrating the model’s success at accuratelyrecalling molecules with the best scoring labels.
Revised: Additionally, we find that AURTC ζimproves as ζ decreases (see Table 1 and Figure 1) demonstrating the model’s success at accuratelyrecalling molecules with the best scoring labels.

Original: Figure 2 shows the associated RES plots for FiLMand FiLMv2, as well as a heatmap showing the their difference.
Revised: Figure 2 shows the associated RES plots for FiLMand FiLMv2, as well as a heatmap showing the their difference.

Original: We note from this Figure that
Revised: We note from this Figure that

Original: FiLMv2 additionally shows performance improvements in regions where both σ and ζ are very small.
Revised: FiLMv2 additionally shows performance improvements in regions where both σ and ζ are very small.

Original: As discussed previously, this is especially useful to the overall task and is indicative of FiLMv2’sability to adapt to the W-MSE loss function.
Revised: As discussed previously, this is especially useful to the overall task and is indicative of FiLMv2’sability to adapt to the W-MSE loss function.

Original: Details about the final hyperparameters of each model,total number of model parameters, etc. can be found in the Appendix.
Revised: Details about the final hyperparameters of each model,total number of model parameters, etc. can be found in the Appendix.

Original: While the majority of our experiments focus on a 500,000 molecule train-val-test dataset (i.e. ρ =0 . 00313 ), we vary the size of the dataset D in order to observe its effect of performance.
Revised: While the majority of our experiments focus on a 500,000 molecule train-val-test dataset (i.e. ρ =0 . 00313 ), we vary the size of the dataset D in order to observe its effect of performance.

Original: Thus,hyperparameter tuning on FiLMv2 architecture was conducted for several datset sizes.
Revised: Thus,hyperparameter tuning on FiLMv2 architecture was conducted for several datset sizes.

Original: Table 2 inthe Appendix shows the intuitive result that for each size tested, RES score and model size increasemonotonically with the size of D .
Revised: Table 2 inthe Appendix shows the intuitive result that for each size tested, RES score and model size increasemonotonically with the size of D .

Original: [Table]
Revised: [Table]

Original: 3.3 Inference on 128 million virtual molecules
Revised: 3.3 Inference on 128 million virtual molecules

Original: Here, we demonstrate the runtime and performance benefits of the DSD framework at scale byperforming inference of the same 128 million ZINC dataset docked by [28].
Revised: Here, we demonstrate the runtime and performance benefits of the DSD framework at scale byperforming inference of the same 128 million ZINC dataset docked by [28].

Original: To evaluate performance,the entire dataset (including the training samples) were given predictions by the final FiLMv2 model.
Revised: To evaluate performance,the entire dataset (including the training samples) were given predictions by the final FiLMv2 model.

Original: The results of this inference is shown in Table 1 as FiLMv2 (128M inf.) and demonstrates that theresults of evaluation on a small held out test set generalize well to the entire virtual library.
Revised: The results of this inference is shown in Table 1 as FiLMv2 (128M inf.) and demonstrates that theresults of evaluation on a small held out test set generalize well to the entire virtual library.

Original: Weevaluate inference speed (i.e. screening throughput ) compared to traditional docking.
Revised: Weevaluate inference speed (i.e. screening throughput ) compared to traditional docking.

Original: We define t D tobe the time required to dock this dataset using the traditional method in [28] and t DSD = t inf + σt Dto be the total time required for inference using our DSD framework.
Revised: We define t D tobe the time required to dock this dataset using the traditional method in [28] and t DSD = t inf + σt Dto be the total time required for inference using our DSD framework.

Original: Here t inf is the time requiredto perform model inference on the full dataset and σt D time required to dock the σN top candidates.
Revised: Here t inf is the time requiredto perform model inference on the full dataset and σt D time required to dock the σN top candidates.

Original: For the given virtual library docked against the D 4 receptor, t inf was reported as 1,728 hours on1,500 CPU cores.
Revised: For the given virtual library docked against the D 4 receptor, t inf was reported as 1,728 hours on1,500 CPU cores.

Original: t inf for FiLMv2 architecture on a single A100 GPU is 9.167 hours.
Revised: t inf for FiLMv2 architecture on a single A100 GPU is 9.167 hours.

Original: Thus, forσ = 0 .
Revised: Thus, forσ = 0 .

Original: 1 the DSD method provides a t D t inf + σt D = 1 , 728 9 .
Revised: 1 the DSD method provides a t D t inf + σt D = 1 , 728 9 .

Original: 167+0 . 1 · 1 , 728 = 9.496x speedup, while recalling97.7% of the top 0.1% of molecules.
Revised: 167+0 . 1 · 1 , 728 = 9.496x speedup, while recalling97.7% of the top 0.1% of molecules.

Original: We note that this speed performance discussion is limited toinference time (known as docking throughput in the literature [42]), and does not include the one-timecost of generating a typically small labeled subset of the virtual library and training the model.
Revised: We note that this speed performance discussion is limited toinference time (known as docking throughput in the literature [42]), and does not include the one-timecost of generating a typically small labeled subset of the virtual library and training the model.

Original: [Figure]
Revised: [Figure]

Original: 3.4 Ablation Studies
Revised: 3.4 Ablation Studies

Original: We ablate two major components of our model, the W-MSE loss presented in Section 2.3 and thevirtual node presented in 2.4.
Revised: We ablate two major components of our model, the W-MSE loss presented in Section 2.3 and thevirtual node presented in 2.4.

Original: We thus train a FiLMv2 model with the same hyperparameters as theanalogous FiLMv2 model in Section 3.2, but trained with vanilla MSE loss.
Revised: We thus train a FiLMv2 model with the same hyperparameters as theanalogous FiLMv2 model in Section 3.2, but trained with vanilla MSE loss.

Original: We also train another
Revised: We also train another

Original: FiLMv2 model without use of a virtual node in both training and inference.
Revised: FiLMv2 model without use of a virtual node in both training and inference.

Original: Table 1 shows thatboth features result in improvement of the model across all reported metrics.
Revised: Table 1 shows thatboth features result in improvement of the model across all reported metrics.

Original: W-MSE is especiallyeffective in metrics that deal with the ability of the model to recall top scoring molecules, such as
Revised: W-MSE is especiallyeffective in metrics that deal with the ability of the model to recall top scoring molecules, such as

Original: R 0 .
Revised: R 0 .

Original: 01 and R 0 .
Revised: 01 and R 0 .

Original: 001 , improving performance by 0.05 and 0.023, respectively.
Revised: 001 , improving performance by 0.05 and 0.023, respectively.

Original: This is consistentwith our earlier discussions of W-MSE.
Revised: This is consistentwith our earlier discussions of W-MSE.

Original: The virtual node improves performance across metrics, forexample increasing the RES Score and AUROC 0 .
Revised: The virtual node improves performance across metrics, forexample increasing the RES Score and AUROC 0 .

Original: 01 by 0.013 and 0.006, respectively.
Revised: 01 by 0.013 and 0.006, respectively.

Original: However, itis important to note that the use of the virtual node can substantially increase training time, as a virtual node with edge connections to every other node increases the density of each graph.
Revised: However, itis important to note that the use of the virtual node can substantially increase training time, as a virtual node with edge connections to every other node increases the density of each graph.

Original: Thus,we additionally experiment with increasing model size when training on data without virtual nodeaugmentation and find that, at a model size such that overall training time is similar to our final
Revised: Thus,we additionally experiment with increasing model size when training on data without virtual nodeaugmentation and find that, at a model size such that overall training time is similar to our final

Original: FiLMv2 model (with virtual node), the effect of the virtual node is less pronounced when comparedto a FiLMv2 model of same size but trained with the virtual node augmentation.
Revised: FiLMv2 model (with virtual node), the effect of the virtual node is less pronounced when comparedto a FiLMv2 model of same size but trained with the virtual node augmentation.

Original: However, the overallperformance of such models are worse than our final model.
Revised: However, the overallperformance of such models are worse than our final model.

Original: This substantiates our claim that use ofvirtual node both improves overall performance and allows use of smaller model sizes.
Revised: This substantiates our claim that use ofvirtual node both improves overall performance and allows use of smaller model sizes.

Original:  Related Work
Revised:  Related Work

Original: There are generally two high level approaches to automated drug discovery, de-novo design ofmolecules with desirable properties [17; 3; 35; 41; 49] and screening based approaches [42].
Revised: There are generally two high level approaches to automated drug discovery, de-novo design ofmolecules with desirable properties [17; 3; 35; 41; 49] and screening based approaches [42].

Original: Whilede-novo drug design, wherein generative models directly produce novel molecules with usefulproperties, shows significant promise, current approaches suffer from issues such as lack of moleculesynthesizability, algorithmic instability, reduced molecule sizes, etc.
Revised: Whilede-novo drug design, wherein generative models directly produce novel molecules with usefulproperties, shows significant promise, current approaches suffer from issues such as lack of moleculesynthesizability, algorithmic instability, reduced molecule sizes, etc.

Original: Indeed, even de-novogenerated drugs may require or benefit from virtual screening, albeit at a comparatively smallerscale.
Revised: Indeed, even de-novogenerated drugs may require or benefit from virtual screening, albeit at a comparatively smallerscale.

Original: Conversely, screening based approaches attempt to use a docking algorithm in order to scoremolecules from a generally large virtual library of compounds for desirability by evaluating itsbinding affinity with a particular target (also called receptor , or ligand )
Revised: Conversely, screening based approaches attempt to use a docking algorithm in order to scoremolecules from a generally large virtual library of compounds for desirability by evaluating itsbinding affinity with a particular target (also called receptor , or ligand )

Original: Recently, substantialprogress has been made to generate larger and more robust virtual libraries.
Revised: Recently, substantialprogress has been made to generate larger and more robust virtual libraries.

Original: Notably, [28] demonstratesuccessfully docking on a virtual library of 170 million molecules, and ultimately discovering anew class of inhibitors for the AMPC receptor.
Revised: Notably, [28] demonstratesuccessfully docking on a virtual library of 170 million molecules, and ultimately discovering anew class of inhibitors for the AMPC receptor.

Original: Crucially, they also show that hit rates of moleculesdecrease monotonically with docking score.
Revised: Crucially, they also show that hit rates of moleculesdecrease monotonically with docking score.

Original: This serves as one of the justifications for biasing our lossfunction to emphasize prediction of the best scoring molecules (See Section 2.3).
Revised: This serves as one of the justifications for biasing our lossfunction to emphasize prediction of the best scoring molecules (See Section 2.3).

Original: Recently, substantialwork has focused on development of machine learning based surrogate models to accelerate docking.
Revised: Recently, substantialwork has focused on development of machine learning based surrogate models to accelerate docking.

Original: Some approaches focus on traditional feature based methods [9; 20], while others apply languagemodels directly on SMILES strings [47; 18] or GNNs on graph representations of molecules [27; 19].
Revised: Some approaches focus on traditional feature based methods [9; 20], while others apply languagemodels directly on SMILES strings [47; 18] or GNNs on graph representations of molecules [27; 19].

Original: Of these GNN based approaches, most use trained models to infer docking scores directly (e.g. [27]).
Revised: Of these GNN based approaches, most use trained models to infer docking scores directly (e.g. [27]).

Original: Some existing screening methods [39; 54; 9] do use a surrogate prefiltering approach, though theylack several of the key training and evaluation strategies proposed in our DSD workflow.
Revised: Some existing screening methods [39; 54; 9] do use a surrogate prefiltering approach, though theylack several of the key training and evaluation strategies proposed in our DSD workflow.

Original: Graph neuralnetworks (GNNs) are generalizations of neural network architectures such as convolution that operateon non-euclidean data (i.e. graphs) [50].
Revised: Graph neuralnetworks (GNNs) are generalizations of neural network architectures such as convolution that operateon non-euclidean data (i.e. graphs) [50].

Original: At a high level, GNNs operate through message passing ,wherein the state of each node is iteratively updated using a function of the state of its neighbornodes.
Revised: At a high level, GNNs operate through message passing ,wherein the state of each node is iteratively updated using a function of the state of its neighbornodes.

Original: Recently, substantial progress has been made in designing GNN general architectures thatare analogous to common DL architectures such as convolution [4; 24; 7; 16; 29; 36; 10], VAEs[23; 33], attention [43; 5; 1; 40; 6; 22; 32; 45], and others [44; 15; 21; 12].
Revised: Recently, substantial progress has been made in designing GNN general architectures thatare analogous to common DL architectures such as convolution [4; 24; 7; 16; 29; 36; 10], VAEs[23; 33], attention [43; 5; 1; 40; 6; 22; 32; 45], and others [44; 15; 21; 12].

Original: Additionally, substantialwork has been made in creating GNN architectures suitable for specific applications such as formodeling quantum interactions [37] and molecular property prediction [25; 52], though we generallyavoid these models in our experimentation as they may inadvertently introduce implicit biases to ourframework.
Revised: Additionally, substantialwork has been made in creating GNN architectures suitable for specific applications such as formodeling quantum interactions [37] and molecular property prediction [25; 52], though we generallyavoid these models in our experimentation as they may inadvertently introduce implicit biases to ourframework.

Original:  Conclusions
Revised:  Conclusions

Original: In this work, we present Deep Surrogate Docking (DSD), a workflow which accelerates traditionalchemical docking through the use of surrogate models.
Revised: In this work, we present Deep Surrogate Docking (DSD), a workflow which accelerates traditionalchemical docking through the use of surrogate models.

Original: We also demonstrate the viability of DSD inthe drug discovery domain by using it to dock 128 million molecules in the ZINC dataset against the
Revised: We also demonstrate the viability of DSD inthe drug discovery domain by using it to dock 128 million molecules in the ZINC dataset against the

Original: D 4 dopamine receptor.
Revised: D 4 dopamine receptor.

Original: We show that DSD substantially accelerates docking throughput with minimialerror.
Revised: We show that DSD substantially accelerates docking throughput with minimialerror.

Original: For example, in one such setup, we are able to speedup a standard docking algorithm by 9.496x,while recalling > 97% of top 0.1% molecules.
Revised: For example, in one such setup, we are able to speedup a standard docking algorithm by 9.496x,while recalling > 97% of top 0.1% molecules.

Original: Moreover, we show that the DSD workflow allowsscientists flexibility to tradeoff inference time and hit accuracy by varying a single data reductionparameter σ .
Revised: Moreover, we show that the DSD workflow allowsscientists flexibility to tradeoff inference time and hit accuracy by varying a single data reductionparameter σ .

Original: We also present a novel GNN architecture, FiLMv2, which we show outperformsother models from the literature in the example docking task above.
Revised: We also present a novel GNN architecture, FiLMv2, which we show outperformsother models from the literature in the example docking task above.

Original: FiLMv2 is implemented as asimple modification to the existing Feature-wise linear modulation (FiLM) architecture and attainsconsiderably more accurate and stable performance by allowing the model to more easily filter outirrelevant information from data.
Revised: FiLMv2 is implemented as asimple modification to the existing Feature-wise linear modulation (FiLM) architecture and attainsconsiderably more accurate and stable performance by allowing the model to more easily filter outirrelevant information from data.

Original: This work demonstrates the viability of the DSD workflow with asingle large scale example.
Revised: This work demonstrates the viability of the DSD workflow with asingle large scale example.

Original: We leave the analysis of generalizability to other virtual molecule librariesand receptors to future work.Appendix should i
Revised: We leave the analysis of generalizability to other virtual molecule librariesand receptors to future work.

Original: remove

Original: explicit appendix title
Revised: AppendixA Additional Details Regarding Chemical Docking for Drug Discovery

Original: A Additional Details Regarding Chemical Docking for Drug Discovery

Original: Here, we discuss and clarify details about classical docking algorithms which are relevant to thiswork.
Revised: Here, we discuss and clarify details about classical docking algorithms which are relevant to thiswork.

Original: Overview of Classical Docking Algorithms At a high level, docking algorithms are functionsthat (1) sample and (2) score molecules against a ligand/receptor.
Revised: Overview of Classical Docking Algorithms At a high level, docking algorithms are functionsthat (1) sample and (2) score molecules against a ligand/receptor.

Original: This scoring function estimatesthe binding affinity of a candidate molecule against the receptor.
Revised: This scoring function estimatesthe binding affinity of a candidate molecule against the receptor.

Original: As discussed in the main text, theyare thus often used to screen molecules (i.e. potential drug candidates) against a specific ligand.
Revised: As discussed in the main text, theyare thus often used to screen molecules (i.e. potential drug candidates) against a specific ligand.

Original: The binding affinity calculation is acheived by simulating the interaction of a given molecule withthe ligand at several orientations (that is, poses ) [46].
Revised: The binding affinity calculation is acheived by simulating the interaction of a given molecule withthe ligand at several orientations (that is, poses ) [46].

Original: However, most docking algorithms onlysucceed in ranking the relative binding affinities of a set of molecules and either do not aim to, orare not successful in, predicting the true binding affinity of the molecule-lingand pair[46].
Revised: However, most docking algorithms onlysucceed in ranking the relative binding affinities of a set of molecules and either do not aim to, orare not successful in, predicting the true binding affinity of the molecule-lingand pair[46].

Original: It isalso important to note that recently [28] showed that the hit-rate , or quality of docking prediction,decreases monotonically with the docking score.
Revised: It isalso important to note that recently [28] showed that the hit-rate , or quality of docking prediction,decreases monotonically with the docking score.

Original: Thus, many modern screening workflows usedocking as a prefilter, and then use domain specific heuristics to further narrow down the set ofmolecules for in-vitro synthesis [28; 42; 9].
Revised: Thus, many modern screening workflows usedocking as a prefilter, and then use domain specific heuristics to further narrow down the set ofmolecules for in-vitro synthesis [28; 42; 9].

Original: This further motivates the use of surrogate models asdrop-in prefilters in the automated drug discovery pipeline.
Revised: This further motivates the use of surrogate models asdrop-in prefilters in the automated drug discovery pipeline.

Original: Pose Estimations As discussed above, pose estimations are calculated by docking algorithms,and are thus output by most docking software [46].
Revised: Pose Estimations As discussed above, pose estimations are calculated by docking algorithms,and are thus output by most docking software [46].

Original: However, for simplicity, we define the dockingfunction f D in the main text as only outputting a docking score.
Revised: However, for simplicity, we define the dockingfunction f D in the main text as only outputting a docking score.

Original: In practice, both the score and posecan be recovered from f D .
Revised: In practice, both the score and posecan be recovered from f D .

Original: However, the pose is not relevant to the DSD workflow.
Revised: However, the pose is not relevant to the DSD workflow.

Original: Lack of Parallelizability Due to the sequencial nature of the docking process, there is currently nogood way to parallelize the docking of a single molecule, and thus docking algorithms are generally
Revised: Lack of Parallelizability Due to the sequencial nature of the docking process, there is currently nogood way to parallelize the docking of a single molecule, and thus docking algorithms are generally

Original: CPU bound [28; 46].
Revised: CPU bound [28; 46].

Original: In our opinion, this further motivates the need for accurate surrogate models inorder to accelerate the automated drug discovery process.
Revised: In our opinion, this further motivates the need for accurate surrogate models inorder to accelerate the automated drug discovery process.

Original:  B Experimental Details
Revised:  B Experimental Details

Original: Here, we detail our preprocessing and training setup, hyperparameter tuning strategy, provide detailsabout the hardware system used, and provide additional metrics for the performance of the top modelsfor each of the four tested architectures listed in Table 1 one of the main text.
Revised: Here, we detail our preprocessing and training setup, hyperparameter tuning strategy, provide detailsabout the hardware system used, and provide additional metrics for the performance of the top modelsfor each of the four tested architectures listed in Table 1 one of the main text.

Original: Preprocessing Details ZINC data is provided in Simplified Molecular Input Line Entry System , or
Revised: Preprocessing Details ZINC data is provided in Simplified Molecular Input Line Entry System , or

Original: SMILES , format, a common string representation of molecules [48].
Revised: SMILES , format, a common string representation of molecules [48].

Original: We use standard techniques [19]to convert each SMILES string into a (two-dimensional) graph representation for use by a GNN.
Revised: We use standard techniques [19]to convert each SMILES string into a (two-dimensional) graph representation for use by a GNN.

Original: Specifically, we create a graph representation of each molecule where the node represents a singleatom in a molecule and contains nine features: atomic number, chirality, degree, formal charge,number of associated hydrogen bonds, number of radical electrons, hybridization, whether it isaromatic, and whether it is contained in a ring.
Revised: Specifically, we create a graph representation of each molecule where the node represents a singleatom in a molecule and contains nine features: atomic number, chirality, degree, formal charge,number of associated hydrogen bonds, number of radical electrons, hybridization, whether it isaromatic, and whether it is contained in a ring.

Original: Edges represent the bond between atoms in a moleculeand are encoded with three features: the bond type, stereochemical information, and whether it isconjugated.
Revised: Edges represent the bond between atoms in a moleculeand are encoded with three features: the bond type, stereochemical information, and whether it isconjugated.

Original: More information about the technical details of these features, as well as their in-silicorepresentations can be found in the RDKit documentation [26], which was the tool used to performthis preprocessing.
Revised: More information about the technical details of these features, as well as their in-silicorepresentations can be found in the RDKit documentation [26], which was the tool used to performthis preprocessing.

Original: Additionally, as described in Section 2.4 of the main text, a virtual node witha distinct virtual edge type to every other node in the graph was added to each molecule duringpreprocessing.
Revised: Additionally, as described in Section 2.4 of the main text, a virtual node witha distinct virtual edge type to every other node in the graph was added to each molecule duringpreprocessing.

Original: Molecules with NaN docking values where dropped, as the docking algorithm usedby [28] represents either noisy or poor performing molecules with NaNs.
Revised: Molecules with NaN docking values where dropped, as the docking algorithm usedby [28] represents either noisy or poor performing molecules with NaNs.

Original: Finally, all docking labelswere standardized by the mean and standard deviation of the training set label distribution.
Revised: Finally, all docking labelswere standardized by the mean and standard deviation of the training set label distribution.

Original: Model Training and Hyperparameter Tuning
Revised: Model Training and Hyperparameter Tuning

Original: As described in Section 3 of the main text, weexperiment with implementing four GNN convolution operations to use as a backbone for ourarchitecture: the graph isomorphism operator (GIN)[53], graph attention convolution operation(GATv2)
Revised: As described in Section 3 of the main text, weexperiment with implementing four GNN convolution operations to use as a backbone for ourarchitecture: the graph isomorphism operator (GIN)[53], graph attention convolution operation(GATv2)

Original: [5], the feature-wise linear modulation operation (FiLM) [4], and our novel FiLMv2operation, which is discussed in detail in Section 2.2 of the main text.
Revised: [5], the feature-wise linear modulation operation (FiLM) [4], and our novel FiLMv2 operation, which is discussed in detail in Section 2.2 of the main text.

Original: For each graph operation, westack a hyperparameterized number of layers of said operation in order to create the backbone of themodel.
Revised: For each graph operation, westack a hyperparameterized number of layers of said operation in order to create the backbone of themodel.

Original: A layer normalization layer, dropout layer, and ReLU non-linearity are applied between each of these layers.
Revised: A layer normalization layer, dropout layer, and ReLU non-linearity are applied between eachof these layers.

Original: After this backbone, a global mean pool aggregates the node representations and asingle MLP layer maps this aggregated representation to a single regression score.
Revised: After this backbone, a global mean pool aggregates the node representations and asingle MLP layer maps this aggregated representation to a single regression score.

Original: In order to simplifythe hyperparameter tuning process, a single hidden dimension size and dropout amount is chosen foreach of the layers.
Revised: In order to simplifythe hyperparameter tuning process, a single hidden dimension size and dropout amount is chosen foreach of the layers.

Original: Thus, the hyperparameters are the size of this hidden layer, the number of layersin the backbone, and the amount of dropout.
Revised: Thus, the hyperparameters are the size of this hidden layer, the number of layersin the backbone, and the amount of dropout.

Original: For all models, hyperparameter tuning was run, viagrid search, on hidden layer sizes of { 2 , 4 , 8 , 16 , 32 , 64 , 128 , 256 } , number of layers of { 1 , 2 , 4 , 5 , 6 } ,and dropout values of { 0 . 0 , 0 . 1 , 0 . 15 , 0 . 2 , 0 . 3 } .
Revised: For all models, hyperparameter tuning was run, viagrid search, on hidden layer sizes of { 2 , 4 , 8 , 16 , 32 , 64 , 128 , 256 } , number of layers of { 1 , 2 , 4 , 5 , 6 } ,and dropout values of { 0 . 0 , 0 . 1 , 0 . 15 , 0 . 2 , 0 . 3 } .

Original: For hidden layer sizes, additional tuning was done inmultiples of four at a local neighborhood around the initial hidden layer size chosen.
Revised: For hidden layer sizes, additional tuning was done inmultiples of four at a local neighborhood around the initial hidden layer size chosen.

Original: For example, ifthe best performing hidden layer size in the initial grid search was 64, additional testing was donefor { 52 , 56 , 60 , 64 , 68 , 72 , 76 , 80 } .
Revised: For example, ifthe best performing hidden layer size in the initial grid search was 64, additional testing was donefor { 52 , 56 , 60 , 64 , 68 , 72 , 76 , 80 } .

Original: Each model was trained for 1,000 epochs, and the epoch wherevalidation loss was best was selected as the final model.
Revised: Each model was trained for 1,000 epochs, and the epoch wherevalidation loss was best was selected as the final model.

Original: All models were trained with the Adamoptimizer with a learning rate of 0.001 and a batch size of 512.
Revised: All models were trained with the Adamoptimizer with a learning rate of 0.001 and a batch size of 512.

Original: The final hyperparameter values forthe best model for each of the four architectures (corresponding to the first four rows of Table 1 in themain text)
Revised: The final hyperparameter values forthe best model for each of the four architectures (corresponding to the first four rows of Table 1 in themain text)

Original: are shown in Table 2.
Revised: are shown in Table 2.

Original: Hardware Details Training was conducted using an internal cluster of 8 NVIDIA DGX A

Revised: Hardware Details Training was conducted using an internal cluster of 8 NVIDIA DGX A

Original: GPUs (40GB memory), using Pytorch’s built-in distributed data parallel (DDP) tool.
Revised: GPUs (40GB memory), using Pytorch’s built-in distributed data parallel (DDP) tool.

Original: Inference wasconducted using a single NVIDIA DGX A100 GPU (40GB memory).
Revised: Inference wasconducted using a single NVIDIA DGX A100 GPU (40GB memory).

Original: Total computation required totrain the best performing model is 29.421 GPU hours.
Revised: Total computation required totrain the best performing model is 29.421 GPU hours.

Original: Total computation used to perform inferenceon the total dataset of 128 million molecules is 9.167 GPU hours.
Revised: Total computation used to perform inferenceon the total dataset of 128 million molecules is 9.167 GPU hours.

Original: Additional Metrics for Final Models As discussed in Section 3.1 of the main text, it is possibleto convert the DSD task to a classification problem by fixing the parameters σ and ζ .
Revised: Additional Metrics for Final Models As discussed in Section 3.1 of the main text, it is possibleto convert the DSD task to a classification problem by fixing the parameters σ and ζ .

Original: Table 2 alsoreports the AUROC and F-1 scores for each of the models for σ = 0 .
Revised: Table 2 alsoreports the AUROC and F-1 scores for each of the models for σ = 0 .

Original: 1 and ζ = 0 .
Revised: 1 and ζ = 0 .

Original: Note that as
Revised: Note that as

Original: AUROC by nature varies the classification threshold (i.e. σ ), a fixed σ is not used to calculate the
Revised: AUROC by nature varies the classification threshold (i.e. σ ), a fixed σ is not used to calculate the

Original: AUROC.
Revised: AUROC.

Original: [Table]
Revised: [Table]

Original: Error Bars As part of our experimentation, we retrained the best performing models for all fourarchitectures 5 times, varying the random seeds used each time.
Revised: Error Bars As part of our experimentation, we retrained the best performing models for all fourarchitectures 5 times, varying the random seeds used each time.

Original: For all models, the range of W-MSEwas < 0 . 0002 and the range of the RES Score was < 0 . 0004 .
Revised: For all models, the range of W-MSEwas < 0 . 0002 and the range of the RES Score was < 0 . 0004 .

Original: Thus, we conclude that the resultsobtained in this work are not significantly affected by stochasticity in the training process.
Revised: Thus, we conclude that the resultsobtained in this work are not significantly affected by stochasticity in the training process.

Original: C Additional Details on the Effects of Dataset Size
Revised: C Additional Details on the Effects of Dataset Size

Original: Section 3.3 describes the phenomenon that as the dataset size used for training is increased, sodoes the RES Score and number of parameters of the best tested model.
Revised: Section 3.3 describes the phenomenon that as the dataset size used for training is increased, sodoes the RES Score and number of parameters of the best tested model.

Original: In our experiments,we performed grid search on the FiLMv2 model, the best performing architecture from Section3.2.
Revised: In our experiments,we performed grid search on the FiLMv2 model, the best performing architecture from Section3.2.

Original: Specifically, we used the same assumptions for model architecture as in B and tested hiddendimension sizes of { 4 , 8 , 16 , 32 , 64 , 128 } , number of convolutional layers of { 1 , 2 , 4 , 5 , 6 } , anddropout of { 0 . 1 , 0 . 15 , 0 . 2 } .
Revised: Specifically, we used the same assumptions for model architecture as in B and tested hiddendimension sizes of { 4 , 8 , 16 , 32 , 64 , 128 } , number of convolutional layers of { 1 , 2 , 4 , 5 , 6 } , anddropout of { 0 . 1 , 0 . 15 , 0 . 2 } .

Original: After finding the optimal model associated with the combination thehyperparameters above, we performed a smaller search around the chosen hidden dimension in orderto further tune this hyperparameter.
Revised: After finding the optimal model associated with the combination thehyperparameters above, we performed a smaller search around the chosen hidden dimension in orderto further tune this hyperparameter.

Original: Each model was trained for 1,000 epochs, and the epoch wherevalidation loss was best was selected as the final model.
Revised: Each model was trained for 1,000 epochs, and the epoch wherevalidation loss was best was selected as the final model.

Original: Table 3 shows the RES score and numberof parameters for the final model chosen for testing after hyperparameter tuning.
Revised: Table 3 shows the RES score and numberof parameters for the final model chosen for testing after hyperparameter tuning.

Original: Please note thatthe dataset size refers to the total dataset size and, as described in 3.1, this number is further brokendown into a 80-10-10, train-val-test split.
Revised: Please note that the dataset size refers to the total dataset size and, as described in 3.1, this number is further brokendown into a 80-10-10, train-val-test split.

Original: For example, the first row of Table 3 has a dataset of 5,000, of which 5 , 000 · 0 . 8 = 4 , 000 examples were used for training.
Revised: For example, the first row of Table 3 has a dataset of5,000, of which 5 , 000 · 0 . 8 = 4 , 000 examples were used for training.

Original: Table 4 shows the chosenhyperparameters for each of these models.
Revised: Table 4 shows the chosenhyperparameters for each of these models.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D Additional FiLMv2 Experiments
Revised: D Additional FiLMv2 Experiments

Original: Section 2.2 of the main text describes the architecture of our novel FiLMv2 convolution in detail.
Revised: Section 2.2 of the main text describes the architecture of our novel FiLMv2 convolution in detail.

Original: There, three main empirical claims are used for discussion:
Revised: There, three main empirical claims are used for discussion:

Original: The nonlinearity sigma in Eq.
Revised: The nonlinearity sigma in Eq.

Original: 1 of the main text can be removed asit does not improve the
Revised: 1 of the main text can be removed as

Revised: it does not

Revised: improve

Original:  model’s performance.
Revised: the model’s performance.

Original: Adding a nonlinearity to the source term ( ( W α h tu ) ) does not further improve the modelperformance.
Revised: Adding a nonlinearity to the source term ( ( W α h tu ) ) does not further improve the model performance.

Original: Using a tanh instead of ReLU nonlinearity does not bring any improvement with respect tothe original FiLMv1.
Revised: Using a tanh instead of ReLU nonlinearity does not bring any improvement with respect to the original FiLMv1.

Original: The experiments conducted that led to these conclusions are detailed below for completeness.
Revised: The experiments conducted that led to these conclusions are detailed below for completeness.

Original: Wedefine FiLMv2 (tanh) as the FiLMv2 model defined in Section 2.2 of the main text, but with allinstances of ReLU non-linearity replaced with tanh non-linearity.
Revised: Wedefine FiLMv2 (tanh) as the FiLMv2 model defined in Section 2.2 of the main text, but with allinstances of ReLU non-linearity replaced with tanh non-linearity.

Original: We also define FiLMv2 (sourceact.)
Revised: We also define FiLMv2 (sourceact.)

Original: as the FiLMv2 model defined in Section 2.2 of the main text, but with an additional ReLUnon-linearity added to source term ( ( W α h tu ) ).
Revised: as the FiLMv2 model defined in Section 2.2 of the main text, but with an additional ReLUnon-linearity added to source term ( ( W α h tu ) ).

Original: FiLM and FiLMv2 are as defined in the main text.
Revised: FiLM and FiLMv2 are as defined in the main text.

Original: Table 5 shows the results of testing each of these models on the 500k train-val-test set used fortraining the main models.
Revised: Table 5 shows the results of testing each of these models on the 500k train-val-test set used fortraining the main models.

Original: Each model was created by using one of the four convolution operationsdescribed above (FiLM, FiLMv2, FiLMv2 (tanh), FiLMv2 (source act.)) and building a backbone offour layers with a hidden dimension size of 64.
Revised: Each model was created by using one of the four convolution operationsdescribed above (FiLM, FiLMv2, FiLMv2 (tanh), FiLMv2 (source act.)) and building a backbone offour layers with a hidden dimension size of 64.

Original: All other details, such as dropout and layer norm,global pooling, and MLP, etc. are the same as in B. Thus, all models have the same number ofparameters, and are trained with the same seed on the same dataset.
Revised: All other details, such as dropout and layer norm,global pooling, and MLP, etc. are the same as in B. Thus, all models have the same number ofparameters, and are trained with the same seed on the same dataset.

Original: [Table]
Revised: [Table]

Original: E W-MSE Loss and the alpha Hyperparameter
Revised: E W-MSE Loss and the alpha Hyperparameter

Original: Section 2.3 in the main text describes the weighted mean square error (W-MSE) used in order toincrease the recall of top scoring molecules.
Revised: Section 2.3 in the main text describes the weighted mean square error (W-MSE) used in order toincrease the recall of top scoring molecules.

Original: Here, we show the results for training the top performing
Revised: Here, we show the results for training the top performing

Original: FiLMv2 model from Section 3.2 of the paper at several values of α , the hyperparameter responsiblefor the degree to which each molecule is weighted (See Section 2.3 for more details).
Revised: FiLMv2 model from Section 3.2 of the paper at several values of α , the hyperparameter responsiblefor the degree to which each molecule is weighted (See Section 2.3 for more details).

Original: Table 6 showsthe results of such experiments.
Revised: Table 6 showsthe results of such experiments.

Original: Since varying α changes the value of the overall loss for a constantdataset and model, the absolute W-MSE loss when varying α may not necessarily be indicative ofdownstream performance.
Revised: Since varying α changes the value of the overall loss for a constantdataset and model, the absolute W-MSE loss when varying α may not necessarily be indicative ofdownstream performance.

Original: Thus, Table 6 also shows performance of each model for all the metricsused for evaluation in Table 1 of the main text.
Revised: Thus, Table 6 also shows performance of each model for all the metricsused for evaluation in Table 1 of the main text.

Original: We see that α = 0 .
Revised: We see that α = 0 .

Original: 8 matches or beats performance onall
Revised: 8 matches or beats performance onall

Original: but one of these metrics and thus use α = 0 .
Revised: but one of these metrics and thus use α = 0 .

Original: 8 for all other experiments.
Revised: 8 for all other experiments.

Original: [Table]
Revised: [Table]

Original: F Tools
Revised: F Tools

Original: Software Tools Substantial portions of this project were implemented in the Pytorch [34] and
Revised: Software Tools Substantial portions of this project were implemented in the Pytorch [34] and

Original: Pytorch Geometric [11] libraries, both of which are open source (BSD and MiT licenses, respectively)software tools.
Revised: Pytorch Geometric [11] libraries, both of which are open source (BSD and MiT licenses, respectively)software tools.

Original: Each tool was utilized with adherence to their licensing conditions.
Revised: Each tool was utilized with adherence to their licensing conditions.

Original: We used Weights& Biases [2], which is free for academic use (MiT license) for experiment tracking and hyperparamtersweeps.
Revised: We used Weights& Biases [2], which is free for academic use (MiT license) for experiment tracking and hyperparamtersweeps.

Original: Additionally, several other Python libraries were used for various auxiliary tasks ranging frompreprocessing to utility tasks.
Revised: Additionally, several other Python libraries were used for various auxiliary tasks ranging frompreprocessing to utility tasks.

Original: A full list of these tools can be found in our code’s requirements.txtfile.
Revised: A full list of these tools can be found in our code’s requirements.txtfile.

Original: Again, each tool was used according to their respective library’s licensing conditions.
Revised: Again, each tool was used according to their respective library’s licensing conditions.

Original: Data As discussed in the main text, data from the ZINC [38] dataset was used.
Revised: Data As discussed in the main text, data from the ZINC [38] dataset was used.

Original: ZINC is a free andpublicly available dataset, and was used according to their licensing policies.
Revised: ZINC is a free andpublicly available dataset, and was used according to their licensing policies.

Original: Additionally, labels forthis data was provided by the docking conducted by [28], and was used in this project with permissionfrom the authors.
Revised: Additionally, labels forthis data was provided by the docking conducted by [28], and was used in this project with permissionfrom the authors.