## 2 Materials and Methods
### BERT Model for SMILES  
Recently in the field of NLP, advanced text analysis has become increasingly important both for academia and industry. This is in fact enabled due to the emergence of powerful language models (LMs) such as transformer-based architecture, namely BERT introduced by [PERSON] et al. [70], and generative pretraining (GPT) introduced by [PERSON] et al. [71].  
Ever since its rise and discovery by Google researchers in 2018, BERT has been considered a relatively powerful and widely used LM for various NLP tasks. The performance of BERT towards a specific task involves two essential mechanisms, which are pretraining and fine-tuning [70]. During pretraining, the model is trained on unlabeled data over different pretraining algorithms. In fine-tuning, the BERT model is first initialized with the pretrained parameters, and all of the parameters are fine-tuned using labeled data from the downstream tasks.  
BERT improves upon standard transformers by removing the unidirectionality constraint by using the masked language modeling (MLM) pretraining objective. In our study, the MLM randomly masks certain tokens within the input sequence, specifically the SMILES strings representing antimalarial drugs and potential targets, and the objective is to predict the original id of the vocabulary of the masked token only based on its context from the SMILES strings. Unlike left-to-right LM pretraining, the MLM objective enables the representation to fuse the left and the right contexts, which allows NLP researchers to pretrain a deep bidirectional transformer. In addition to the MLM, BERT uses a next sentence prediction (NSP) task that jointly pretrains text-pair representations [70]. However, in the pharmaceutical research field, NSP is ignored since, for example, our study involves  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Hyperparameters** & **Descriptions** \\ \hline R-GCN & \# of layers & One RGCNConv \\  & Training technique & 10-fold cross-validation \\  & Learning rate & 0.00195 \\  & Optimizer & Adam \\  & Dropout & 0.645 \\  & Forward module \(A_{F}\) & ReLU \\  & Predictive module \(A_{F}\) & Softmax \\ \hline \hline \end{tabular}  
* Conv—convolution. \(A_{F}\)—activation function. \(\#\)—number.  
\end{table}
Table 4: Hyperparameters and training strategy in this study.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**RGCN \({}^{\mathbf{a}}\) Input** & **\# of Features** & **Evaluation** & **Single** & **Double** \\
**Features** & **Evaluation** & **RGCNconv.** & **RGCNconv.** \\  & & & **Layer** & **Layer** \\ \hline BERT and Mordred combined & 6829 & Accuracy & **0.9958** & 0.9931 \\  & & Sensitivity & **0.9968** & 0.9904 \\  & & Specificity & 0.9951 & 0.9951 \\  & & MCC \({}^{b}\) & **0.9915** & 0.9858 \\  & & AUROC \({}^{c}\) & 0.9947 & **0.9965** \\  & & AUPRC \({}^{d}\) & **0.9958** & 0.9950 \\ \hline \hline \end{tabular}  
* Relational graph convolution network. \([^{e}]\) [PERSON]’s correlation coefficient. \([^{f}]\) Area under the receiver operating characteristics. \([^{f}]\) Area under the precision–recall curve. The RGCN’s high scores, as denoted by bold values, are indicative of the layer’s architecture. \(\#\)—number.  
\end{table}
Table 5: Experimental performance comparison and effect of different \(\#\) of layers on RGCN.  
only single sequences of antimalarialal drugs and targets encoded as SMILES strings during model pretraining.  
In this study, we adopted the SmilesTokenizer module from DeepChem [51] inherited from the BertTokenizer class in transformers to generate embedding features as shown in Figure 2 that were later used by our proposed GNN model (i.e., RGCN) as node features after standardize and preprocessing. This runs a WordPiece tokenization algorithm over antimalarial drugs and targets SMILES strings using the tokenization SMILES regex developed by [PERSON] et al. [52].