# ABPCaps: A Novel Capsule Network-Based Method for the Prediction of Antibacterial Peptides
## 2 Materials and Methods
### Development of ABPCaps  
In various fields such as natural language processing, computer vision, and bioinformatics, deep learning algorithms such as CNNs and recurrent neural networks (RNNs) have become ubiquitous. Although CNNs have been successful in many tasks, they are still limited by pooling invariance and the inability to understand spatial relationships between features. Capsule networks, which have shown excellent performance on many datasets [43; 44; 45], have been developed to overcome these limitations. In this study, we attempted to develop a novel model that integrates 1D convolution (Conv1D), LSTM, and capsule layers to improve the feature extraction process. Conv1D was used to extract local features, which were then passed to the LSTM layer responsible for capturing long range dependencies. The high-level features extracted by the LSTM layer were then passed to the capsule layer. An overview of the proposed model is shown in Figure 1.  
A one-dimensional protein sequence with 20 types of amino acids was first converted into two-dimensional data using the popular binary one-hot encoding method to input peptides into the network. The transformation converted the input protein sequence into a \(k\times 20\) matrix, where \(k\) is the length of the input sequence and each row represents an amino acid. Due to the variable length of peptides, this matrix was converted to a zero-padded numerical matrix of size \(150\times 20\) to fit the data set. The protein sequence was ready for input to the network after encoding. The first layer of the network was Conv1D, which had 64 filters, and the step was set to 1. The activation function of the rectified linear units (ReLU) was chosen for this layer. The local features extracted by Conv1D were passed to the RNN layer, which can do long-range dependency capture. However, RNNs suffered from slope disappearance, which would affect performance. Therefore, in this study, we used an LSTM layer, which could avoid the gradient vanishing problem and had shown superior performance compared to other RNN variants such as gated recurrent units (GRU) [46]. Each LSTM unit consists of a forget gate, an input gate, and an output gate, which were calculated using the following equations:  
\[f_{t}=\sigma\Big{(}W_{f}[h_{t-1},x_{t}]+b_{f}\Big{)} \tag{1}\] \[i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i})\] (2) \[\widetilde{C}_{t}=\tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C})\] (3) \[C_{t}=f_{t}*C_{t-1}+i_{t}*\tilde{C}_{t}\] (4) \[o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o})\] (5) \[h_{t}=o_{t}*\tanh(C_{t}) \tag{6}\]  
\(f_{t}\), \(i_{t}\), \(o_{t}\), denote the forget gate, input gate, and output gate, respectively. The forget gate and input gate are responsible for updating the cell state. In this study, using GRU instead of LSTM was tried, but the performance of GRU was not satisfactory. A capsule, consisting of a set of neurons in the form of a vector, is a fundamental component of capsule networks. In contrast to traditional neural networks, capsule networks use vectors as both the input and the output units. Each capsule represents a pattern, with each dimension representing unique characteristics. In this study, the primary capsule layer used a conv1D for further extraction of relevant features, with the resultant outputs being transformed into multiple 8-dimensional vectors. According to the concept of capsule networks, the length of a vector refers to the probability of the represented pattern. A non  
Figure 1: The architecture of ABPCaps. The proposed model, ABPCaps, is a fusion of Conv1D, LSTM, and a capsule network. The input sequences are encoded using the one-hot encoding method, and the Conv1D and LSTM layers are used to extract crucial features. The hybrid features are passed to the capsule layer, which encapsulates the relationships between different features and provides interpretable results for downstream tasks. linear function called squash is applied to these 8-dimensional vectors and is given below.