Combination therapies have shown to be more effective than single drugs in multiple diseases such as HIV and tuberculosis [26, 29] . Synergistic combinations can improve both potency and efficacy, either achieving stronger therapeutic effect and/or decreasing the dosage thereby reducing side-effects. In the times of the current public health crisis, finding a successful combination of approved molecules have an additional benefit over designing a de-novo molecule: time to clinical adoption. Approved drugs are typically commercially available and have well studied safety profiles. Taken in aggregate, these considerations motivate us to explore combination therapies for SARS-CoV-2 antivirals.
Since exploring the space of combinations via high-throughput screening is prohibitively expensive as it involves combinatorial search, in-silico screening based on machine learning is an appealing alternative. In fact, a number of such methods have been reported in the literature [23, 27] . These techniques have been shown effective when the model was provided with large amounts of training data capturing synergy of various combinations. Unfortunately, this requirement prevents us from utilizing these techniques for many diseases where such data is not available. Therefore, it is crucial to reduce data dependence to make combination algorithm applicable in multiple therapeutic contexts.
In this paper, we present a novel algorithm for finding combinations that achieves this goal. Our main hypothesis is that by explicitly modeling interaction between compounds and the biological targets, we can significantly decrease dependence on combination training data. The proposed model has two components. The first component models drug-target interactions (DTI) predicting which targets are inhibited by a compound. This component is trained on individual compounds since DTI information is readily available for multiple targets across multiple diseases. Our second component focuses on modeling target-disease association. Explicitly incorporating this information enables the model to capture the relation between targets that results in synergistic activity.
We further expand this basic architecture to refine our biological modeling. The first such expansion concerns target identification. Since our knowledge of disease relevant targets is often incomplete which may negatively impact model's predictive power, we augment the target set with additional latent targets learned from single compound and combination data. Our second extension focuses on multi-disease training. Since screening data for combinations is typically sparse, we exploit additional combination data from related diseases for better generalization. We develop our model using single agent and drug combinations of multiple diseases. The model incorporates known biological targets relevant to SARS-CoV-2 [9] , as well as the corresponding drug-target activity data collected from ChEMBL [7] . Our model is evaluated on a SARS-CoV-2 combination screen from Bobrowski et al. [2] and achieves 0.777 AUC in terms of drug synergy prediction. Moreover, we demonstrate that incorporating known COVID targets and multi-disease training yields 50% relative increase in model accuracy. Finally, we show biological interpretation of the latent targets learned by the model.
Existing approach on drug synergy prediction can be roughly divided into two categories:
• Supervised learning: In this approach, the model is trained on combination data generated from high-throughput screens. For example, Preuer et al. [23] have trained a deep neural network on a large-scale oncology screen [22] (23K training examples) to predict anti-cancer drug synergy. Xia et al. [27] and Sidorov et al. [25] have trained deep neural networks to predict anti-cancer drug synergy on a larger dataset compiled by NCI-ALMANAC [12] , which contains around 300K training examples across 40 different cell lines. • Biological networks: Another category of drug synergy models builds on biological networks, assuming that drugs with complementary mechanism of actions are more likely to be synergistic. In particular, Cheng et al. [3] , Zhou et al. [30] proposed to model synergy using distance measures on drug-target interaction and protein-protein interaction networks.
The major challenge of supervised approaches is the lack of combination data. For many diseases such as SARS-CoV-2 and tuberculosis, the size of drug combination data is very limited (less than 200) [2, 29] . Deep learning methods are prone to over-fitting in this low-resource scenario. Moreover, the number of possible pair-wise combinations grows quadratically with the number of drugs. In fact, the current largest combination screen for cancer [12] only covers around 100 different drugs. This significantly limits the ability of trained models to generalize to new drugs outside of the training set. On the other hand, while network-driven methods have a wider coverage over the chemical space, they cannot make predictions on new compounds outside of the biological network (i.e., no edge between this compound and the targets) since the model is not parametric.
We propose a new method that combines the merit of both approaches while addressing their limitations. As drug interaction is often characterized by their biological targets, we train our model to predict both drug-target interaction and drug synergy. This enables us to make predictions on new compounds even if their drug-target interaction is unknown. This also addresses the data scarcity challenge since there are large amounts of drug-target interaction publicly available.
In this section, we describe our model architecture for drug combinations. A drug combination is called synergistic if its antiviral effect is greater than the sum of the individual effects. Synergy arises from various types of drug interaction. For example, two drugs can be synergistic when they interact with different sets of biological targets or pathways. Indeed, most of the anti-HIV drug combinations, such as Dolutegravir and Lamivudine, are drugs with different mechanisms of actions (i.e., interacting with different biological targets). To account for this inductive bias, it is crucial to model the interaction between drugs and biological targets in our model architecture.
Motivated by these observations, we propose to decompose our model into two parts: a drug-target interaction (DTI) module φ and a target-disease association module f . The DTI module predicts the biological targets activated by a given compound. The target-disease association module decides the importance of a biological target to the disease. The vocabulary of biological targets are determined based on its relevance to the disease treatment given by expert knowledge. To introduce our method, we first describe how these two modules are used to predict antiviral activity of single compounds and then extend it to drug combinations.
Each drug is represented as a graph G, whose nodes represent atoms and edges represent bonds. Each node/edge is labeled with its atom/bond type. To predict the antiviral effect of a single compound A, our model needs to accomplish two tasks: 1) it has to identify the interaction between A and a list of biological targets V = {t 1 , · · · , t m }; 2) decide the relevance of each target t i to the disease.
We parametrize the DTI module φ as a graph convolutional network (GCN) [5, 8] . The GCN translates a molecular graph G A into a continuous vector through directed message passing operations [28] , which associate hidden vectors h v with each node v and updates these vectors by passing messages h uv over edges (u, v). The output of φ is a vector z A representing the biological targets activated by drug A:
where σ(·) is a sigmoid function and MLP is a two-layer feed-forward network. Each element z A,k represents the probability of drug A being active to target t k . Each target t k is associated with a drug-target interaction dataset
We will train this module on the DTI dataset of all biological targets in our vocabulary.
We parametrize the target-disease association module f as a simple linear layer (w, b) due to its interpretability. As shown in Figure 1 , our model predicts its antiviral activity of a single drug A as:
Synergy are often quantified under Bliss independence assumption [1] : suppose the individual antiviral effect of drugs A, B are P A , P B . The expected effect of combination (A, B) is given as
Following this definition, we introduce a new Bliss layer to predict the synergistic effect of a drug combination (A, B) . Given two drugs and their predicted DTI vectors z A , z B , the Bliss layer computes the DTI vector z AB as
where stands for element-wise multiplication. With this aggregation function, a drug combination will benefit most from complementary targets. If only one drug is active to target t i (e.g., z A,i = 1, z B,i = 0), the combination (A, B) is still active to t i (z AB,i = 1). In other words, the set of active targets for (A, B) is the union of active targets of the two drugs.
As shown in Figure 2, for a drug combination (A, B) , our model predicts its antiviral activity as:
Following Bliss independence model, we predict the synergy score of a combination as p AB − e AB , where e AB = p A + p B − p A p B . Intuitively, a combination is more likely to be synergistic if they have complementary targets with high target-disease association score. 
Let be a cross-entropy loss. Our training loss L = λ DTI DTI + λ S S + C consists of three components:
• DTI Loss: The drug-target interaction loss DTI enforces the predicted DTI vector z A to be biologically accurate. The corresponding training data is the union of DTI datasets D DTI = k D k . For each drug and its DTI label (X i , y i ) ∈ D DTI , the DTI module φ is trained to minimize:
• Single Compound Loss: Our second training data D S = {(X 1 , y 1 ), · · · , (X n , y n )} contains molecules and their labeled antiviral activity (active/inactive). Both modules φ, f are trained to 
We further introduce two extensions to the basic architecture described above that lead to notable performance improvement in practice:
Latent Targets In order to learn synergy, it is important to incorporate all the relevant biological targets into our model. However, this is challenging for two reasons: First, many biological targets do not have drug-target interaction data and thus cannot be incorporated in our model. Second, the current biological understanding of a disease may be incomplete. For instance, Riva et al. [24] reported around 50 new biological targets that contributes to anti-SARS-CoV-2 activity, but they are not reported in the previous work by Gordon et al. [9] .
To this end, we propose to create additional latent targets that can be learned indirectly from single compound and combination data. Specifically, we allow the dimension of z A to be greater than the total number of considered targets in V. The first m entries in z A corresponds to the real biological targets and the other entries are latent targets. As we will show in the experiments, it is possible for us to identify new biological targets related to given diseases.
Another challenge is the scarcity of drug combination data. To our best knowledge, there are only around 180 drug combinations with labeled synergy against SARS-CoV-2.
To address the low-resource challenge, we utilize drug synergy data from other viral diseases such as HIV. In particular, we augment our model with additional HIV biological targets as well as HIV single-drug and drug combination data. The HIV related datasets will be elaborated in Section 4.
The DTI module φ now outputs a DTI vector z d A for each disease d. φ is shared across two diseases and trained to learn drug-target interaction for all diseases. Since each disease operates on different targets, we create a target-disease association module f D for each disease. Let d DTI , d S , d C be the losses for each disease d ∈ {SARS-CoV-2, HIV} computed from its own dataset. Our multi-disease training loss becomes
4 Experiments SARS-CoV-2 Data For SARS-CoV-2 infection, we consider three types of biological targets in our target vocabulary V = {t 1 , · · · , t m }:
• Viral proteases: Replication of SARS-CoV-2 virus requires the processing of two polyproteins by two virally encoded proteases: chymotrypsin-like protease (3CLpro) and papain-like protease (PLpro). Inhibitors that block either protease could inhibit viral replication. We have compiled 3CLpro enzymatic activity [16] and PLpro inhibition [4] data made public by NCATS and ReframeDB.
• Viral entry proteins: SARS-CoV-2 cell entry depends on angiotensin converting enzyme 2 (ACE2) [11] . Inhibiting ACE2 enzyme or the interaction between SARS-CoV-2 and ACE2 could block viral entry. To this end, we utilize ACE2 enzymatic activity [17] and Spike-ACE2 protein-protein interaction [20] from NCATS.
• Host proteins: Gordon et al. [9] identified 335 human proteins physically associated with SARS-CoV-2 viral proteins. Inhibitors for these proteins may also hinder viral replication. Among these proteins, we selected 31 proteins that have sufficient amount of drug-target interaction data in the ChEMBL database (i.e., both positive and negative interactions).
The above drug-target interaction data contains around 20K compounds in total. Our training data for SARS-CoV-2 utilizes another two assays:
• Single-agent Activity: We use the NCATS CPE assay in VeroE6 cells [18] , which contains around 10K compounds and 320 hits with EC50 ≤ 10µM.
• Drug Combination: NCATS performed two combination assays in VeroE6 cells, which contain 160 two-drug combinations [2, 19] . Riva et al. [24] also analyzed synergy between Remdesivir and 20 active compounds identified from their high-throughput screen.
HIV Data The training data for HIV comes from the following assays:
• Drug-target Interaction: Existing anti-HIV drugs mainly target viral proteins (HIV-1 protease, integrase and reverse transcriptase) or host proteins involved in viral entry (CCR5, CXCR4 and CD4). We compiled DTI data for these six targets from ChEMBL.
• Single-agent Activity: NCI conducted an anti-HIV assay [21] with 35K compounds, among which 309 compounds are active (EC50 ≤ 1µM).
• Drug Combination: Tan et al. [26] conducted high-throughput screen for HIV drug combinations. The dataset contains 114 two-drug combinations.
Evaluation Protocol Since our goal is to predict synergy against SARS-CoV-2, our validation and test set only consist of SARS-CoV-2 combinations. All the drug-target interaction, single-drug activity and HIV data are used for training only. Our validation set contains 20 combinations from Riva et al. [24] and test set contains 72 combinations from Bobrowski et al. [2] . The training set contains 90 SARS-CoV-2 combinations from [19] , where we remove combinations that appear in both the training and test set.
Hyperparameters For DTI module φ, we adopt default hyperparameters from Yang et al. [28] , with hidden dimension 300 and three message passing iterations. We set the dimension of DTI vector |z| = 100, with 42 real biological targets (SARS-CoV-2 and HIV) and 58 latent targets, so that the number of real and latent targets are roughly equal. We set λ 1 = 10, λ 2 = 0.1 for our final model. [24] . The blue circles represent latent targets learned by our best model and the red circles represent new biological targets. We draw an edge between a latent target and a biological target if their matching AUROC is higher than 0.7.
To show the effectiveness of different components, we compare with the following baselines:
• A GCN trained only on SARS-CoV-2 single-agent and combination data (λ DTI = 0, |z| = 100).
• +DTI: A GCN trained only on SARS-CoV-2 single-agent, combination as well as drug-target interaction data (λ DTI = 10, |z| = 100). • +MultiD: A GCN trained on both SARS-CoV-2 and HIV data (single-agent + combination), but without drug-target interaction data (λ DTI = 0, |z| = 100). • +All,-latent: A GCN trained on both SARS-CoV-2 and HIV data (single-agent + combination + drug-target interaction), but the latent targets are removed (λ DTI = 10, |z| = 42). • +All: A GCN trained on both SARS-CoV-2 and HIV data (single-agent + combination + drug-target interaction), but the latent targets are removed (λ DTI = 10, |z| = 100).
Our results are shown in Figure 3 . As expected, the GCN baseline performs poorly, with 0.537±0.075 AUC. Adding drug-target interaction data (+DTI) improves the AUC to 0.658 ± 0.079. Adding HIV data (+MultiD) improves the AUC to 0.706 ± 0.088. Our final model, trained with both HIV and drug-target interaction (+All), achieves the best AUC of 0.777 ± 0.066. This validates the advantage of adding drug-target interaction data and multi-disease training. Note that if we remove the latent targets (+All,-latent), the performance decreases to 0.718 ± 0.021. This also shows the importance of using latent targets to complement missing biological information.
While the latent targets are trained to represent biological targets, we found that they are predictive of certain SARS-CoV-2 biological targets that are not provided during training. Therefore, we propose to interpret their biological meaning by the following procedure:
1. Suppose we have a list of real SARS-CoV-2 targets C = {c 1 , · · · , c n } that are not provided to the model during training (i.e., V ∩ C = ∅). Let D cj be the drug-target interaction dataset for c j . 2. Compute DTI vectors z X1 , · · · , z Xn for each compound X i ∈ D cj . For each latent target k, compute the AUROC between DTI labels y 1 , · · · , y n and predicted DTI scores z X1,k , · · · , z Xn,k . 3. Match latent target k to target c j if the AUROC is higher than threshold τ = 0.7.
The target set C contains 34 biological targets reported in Riva et al. [24] that are correlated with SARS-CoV-2 antiviral activity and have drug-target interaction data in the ChEMBL database. These targets are not provided to the model during training. As shown in Figure 3 , among the 58 latent targets, 27 of them are matched to the antiviral targets in Riva et al. [24] using the above procedure. This shows that our model is capable of capturing additional SARS-CoV-2 targets through single-agent and combination data.
In this paper, we present a new algorithm for predicting SARS-CoV-2 drug synergy. The model consists of two components: a drug-target interaction module and target-disease association module. Our method requires much less combination data since we explicitly model the drug-target interaction in our model architecture. The proposed method is general and applicable to many other diseases.