# ABPCaps: A Novel Capsule Network-Based Method for the Prediction of Antibacterial Peptides
## 2 Materials and Methods
### Development of ABPCaps
\[\text{Squash }(\text{S})=\frac{\|\text{S}\|^{2}}{1+\|\text{S}\|^{2}}\frac{\text{S}}{ \|\text{S}\|} \tag{7}\]  
This function compresses the length of the vector to a value between 0 and 1, while maintaining the direction of the vector. If the magnitude of the input vector is relatively large, the norm of the output vector will approach unity; conversely, if the magnitude of the input vector is very small, the norm of the output vector will approach zero. The primary capsule layer outputs are the compressed scaled vectors. In a binary classification task, the class capsule layer consists of two capsules-a positive capsule and a negative capsule-each represented by a 16-dimension vector. Each capsule represents a different class of input sequence. The computational process between the primary capsule layer and the class capsule layer is illustrated in Supplementary Figure S1. To obtain the prediction vectors, \(\hat{u}_{j|i}\) the outputs of the primary capsule layer, \(u_{i}\), are first multiplied by the corresponding weight matrix, \(W_{i,j}\).\(S_{j}\), given below, is the weighted sum of all \(\hat{u}_{j|i}\) computed using coupling coefficients, \(c_{i,j}\), determined by the dynamic routing process. \[S_{j}=\sum_{i=1}^{L}c_{i,j}\hat{u}_{j|i} \tag{8}\]  
These coefficients describe the probabilities of primary capsules to be coupled to the class capsules activated by specific input sequences. \(L\) is the total number of primary capsules. The process of dynamic routing, as shown in Supplementary Materials--Algorithm S1, is a key feature of capsule networks. The log prior probability between primary capsule \(i\) and class capsule \(j\), denoted by \(b_{i,j}\), was computed to determine the coupling coefficients \(c_{i,j}\). These coefficients were computed using a softmax function on \(b_{i,j}\) that ensures that the sum of the coupling coefficients of class capsule \(i\) to all primary capsules is equal to one. The number of iterations \(r\) is a hyperparameter that must be set prior to model training. Two output vectors \(v_{j}\), each representing one of the two classification categories, ABP or non-ABP, were generated by the class capsule layer during dynamic routing. The elements of \(v_{j}\) encode the features, and their length, determined by the L2 norm (\(\|\cdot\|_{2}\)), denotes the probability distribution (\(p_{j}\)) of the two classes. In the last step of the network, the length of these output vectors was calculated and shown below. \[f_{t}=\sigma\Big{(}W_{f}[h_{t-1},x_{t}]+b_{f}\Big{)} \tag{1}\]0  
To train the model, this study used the margin loss introduced by [PERSON] et al. (2011), which considered the classification category and an indicator function. \[f_{t}=\sigma\Big{(}W_{f}[h_{t-1},x_{t}]+b_{f}\Big{)} \tag{1}\]1  
\(c\) and \(I\) represent the classification category and the indicator function, respectively. Specifically, \(l_{c}=1\) if the current sample belongs to class \(c\); otherwise, \(l_{c}=0\). The \(\text{m}^{+},\text{m}^{-}\), and \(\lambda\) are hyperparameters in this function, with suggested values of 0.9, 0.1, and 0.5, respectively. The Keras framework ([[https://keras.io/](https://keras.io/)]([https://keras.io/](https://keras.io/)) (accessed on 27 March 2015)) and the TensorFlow ([PERSON], 2017) library backend were used to build the model in this study. To avoid overfitting, an early termination strategy was implemented during training. This strategy was the termination of the training process if there was no improvement in accuracy within several epochs. The Adam ([PERSON] and [PERSON], 2014) optimizer was used with learning rate and batch size set to 0.001 and 128, respectively. This study applied the training process with 4\(\times\)Nvidia 2080 Ti graphics processing units (GPUs), which significantly reduced the computational time.