G ENERATING A NTIMICROBIAL P EPTIDES FROM L ATENT S ECONDARY S TRUCTURE S PACE
Anonymous authors
Paper under double-blind review
A BSTRACT
Antimicrobial peptides (AMPs) have shown promising results in broad-spectrum antibiotics and resistant infection treatments, making it a magnet for the ﬁeld of drug discovery.
Recently, an increasing number of researchers have been introducing deep generative models to AMP design.
However, few studies consider secondary structure information during the generation, though it has shown crucial inﬂuence on antimicrobial activity in all AMP mechanism theories.
This paper proposes LSSAMP which uses the multi-scale VQ-VAE to learn the positional latent spaces modeling the secondary structure.
By Sampling in the latent secondary structure space, we can generate peptides with ideal amino acids and secondary structures simultaneously.
Experimental results show that our LSSAMP can generate peptides with multiple ideal physical attributes and a high probability of being predicted as AMPs by public AMP prediction models
I NTRODUCTION
Developing neural networks for drug discovery has attracted increasing attention recently.
It can facilitate the discovery of potential therapies and reduce the time and cost of drug development (Stokes et al., 2020).
Plenty of works have been done to employ deep generative models in searching for drug-like molecules with desired properties and achieved great success (Jin et al., 2018; Shi et al., 2019; Schwalbe-Koda & G ´ omez-Bombarelli, 2020; Xie et al., 2020).
However, these works mainly focus on small molecules, and more complicated biochemicals, such as proteins, are still rarely explored.
Antimicrobial peptides (AMPs), deﬁned as short proteins of less than 50 amino acids with potent antimicrobial activity, are an emerging category of therapeutic agents.
AMPs exist widely in the natural immune system for all species and kill bacteria in a physical way (Aronica et al., 2021; Cardoso et al., 2020).
They attach to the bacterial membrane and insert into the membrane to form pores, which leads to the death of bacteria by allowing cytoplasmic leakage.
This mechanism makes them more promising for handling extensively drug-resistant bacteria than traditional antibiotics (Mahlapuu et al., 2016).
However, the theoretical chemical space of peptides is enormous and the sequence number grows exponentially as the length increases.
Thus, it is challenging to search for valid peptides with antimicrobial properties from such a huge sequence space.
Several factors can affect the antimicrobial activity of peptides (Boman, 2003).
Amino acids with positive charges are more likely to bind with bacterial membrane as most bacterial surfaces are anionic, while those with high hydrophobicities tend to move from the solution environment to the bacterial membrane.
However, the mechanisms of antibacterial peptides need not only a reasonable sequence but also an appropriate structure.
For example, by forming the helix structure, a peptide can gather the hydrophobic amino acids on one side and hydrophilic ones on the other.
This ability, named amphipathy, helps it insert into the membrane and maintain a stable hole with other peptide molecules in the membrane, as shown in Figure 1.
The hole will drain the cytoplasm and ﬁnally kill the bacteria.
This mechanism of killing the bacteria is called ‘barrel stave’.
Amphipathy plays an important role in deciding the antibacterial activity of peptides and is closely related to the secondary structure of the peptide (Aronica et al., 2021).
According to the antimicrobial mechanism, a new AMP should meet the following criteria.
C1 :
It possesses several ideal physical attributes (e.g. positive charge and high hydrophobicity).
C2 :
It
[Figure]
has appropriate secondary structures (e.g. alpha-helix).
C3 :
It differs from existing AMPs to some extent.
The existing works mainly focus on sequential features of amino acids and ignore the secondary structure.
The traditional methods replace subsequences with patterns from the pattern database in a given template (Porto et al., 2018).
Inspired by success in deep neural networks, many researchers apply neural generative models to AMP discovery.
They often use the physical attributes as the extra input to control the generation phase (Das et al., 2018; Van Oort et al., 2021), or train classiﬁers on each attribute to ﬁlter the peptides after the generation (Capecchi et al., 2021; Das et al., 2021).
The former ones usually generate peptides that have low correlation with the input attributes and the ﬁlter phase of the latter ones make the sampling inefﬁcient.
As described above, the antimicrobial activity is determined by both the amino acid composition and secondary structure of the peptide.
Thus, we propose LSSAMP to generate antimicrobial peptides from the latent semantic and structure space.
Taking the peptide sequence as the time series, we assign a latent variable on each position.
Since it is computationally intractable to sum continuous latent variables over all positions, we employ the vector quantized-variational autoencoder (VQ-VAE) (van den Oord et al., 2017) to learn the discrete distribution for each position and further design a multi-scale codebooks strategy to capture different local patterns to ﬁt various length ranges for amino acid and structure sequences.
During the generation process, LSSAMP will sample a backbone from the secondary structure latent space and generate the amino acid sequence simultaneously.
We evaluate LSSAMP and several baselines through physical properties that are closely related to the antibacterial mechanism.
Besides, we use some public AMP prediction models to predict generated sequences being AMPs as previous works did (Das et al., 2020; Van Oort et al., 2021).
To conclude, our contributions are as follows:
• We propose LSSAMP , a generative model which samples peptides from the latent secondary structure space to control the peptide properties.
• We develop a multi-scale VQ-VAE to learn positional latent spaces from different aspects and model semantic sequences and structural sequences in the same space.
• Experimental results show that LSSAMP can generate peptides with multiple ideal features such as positive charge, better hydrophobicity, and better amphipathicity.
The results of public AMP classiﬁers also verify that our model can generate peptides with high AMP probability.
R ELATED WORK
Antimicrobial Peptides Generation Traditional methods for AMP design can be divided into three approaches (Torres & de la Fuente-Nunez, 2019): (i)
The pattern recognition algorithms build a sequential pattern database from existing AMPs, and then pick a template peptide and replace local sequence with patterns (Loose et al., 2006; Porto et al., 2018).
(ii)
The genetic algorithms analyze the AMP database and design some antibiotic activity functions (Maccari et al., 2013).
(iii)The
[Figure]
molecular modeling and molecular dynamics methods build 3D models of peptides and analyze activity (Matyus et al., 2007; Bolintineanu & Kaznessis, 2011).
Deep generative models take a rapid growth in recent years.
Dean & Walper (2020) encodes the peptide into the latent space and interpolates across a predictive vector between a known AMP and its scrambled version to generate novel peptides.
The PepCVAE (Das et al., 2018) and CLaSS (Das et al., 2021) employ the variational auto-encoder model to generate sequences.
The AMPGAN (Van Oort et al., 2021) uses the generative adversarial network to generate new peptide sequences with conditions.
To our knowledge, this is the ﬁrst study to take secondary structure information into consideration during the generative phase, which is conducive to effectively generate well-structured sequences with desired properties .
Sequence Generation via VQ-VAE
The variational auto-encoders (VAEs) were ﬁrst proposed by Kingma & Welling (2014) for image generation, and then widely applied to sequence generation tasks such as language modeling (Bowman et al., 2016), paraphrase generation (Gupta et al., 2018), machine translation (Bao et al., 2019) and so on.
Instead of mapping the input to a continuous latent space in VAE, the vector quantized-variational autoencoder (VQ-VAE) (van den Oord et al., 2017) learns the codebook to obtain a discrete latent representation.
It can avoid issues of posterior collapse while has comparable performance with VAEs.
Based on it, Razavi et al.
(2019) uses a multi-scale hierarchical organization to capture global and local features for image generation.
Bao et al.
(2021) learns implicit categorical information of target words with VQ-VAE and models the categorical sequence with conditional random ﬁelds in non-autoregressive machine translation.
In this paper, we employ the multi-scale vector quantized technique to obtain the discrete representation for each position of the peptide.
M ETHOD
Given a peptide sequence x = { a 1 , a 2 , · · · , a L } , where a belongs to the 20 common amino acids and L is the sequence length, the corresponding secondary structure can be denoted as y =
{ y 1 , y 2 , · · · , y L } .
Following the deﬁnition in Kabsch & Sander (1983), there are 8 secondary structure types, including one unknown label, so y i ∈
{ H, B, E, G, I, T, S, −} 1 .
We ﬁrst employ
VQ-VAE for the sequence reconstruction task to learn the sequential latent space (Section 3.1).
Then, we enforce the latent space to model the structure information by the secondary structure task (Section 3.2).
Besides, we design the multi-scale codebooks to capture different local patterns (Sec- tion 3.3).
Finally, we describe the training and inference phase in Section 3.4.
The overview of our model is shown in Figure 2.
3. M ODELING P EPTIDE S EQUENCES
For sequential information, we embed the input peptide x = { a 1 , a 2 , · · · , a L } to the latent space via the encoder and use the generator to reconstruct x .
We assume that each a i is determined by a latent variable z i , and the input sequences x = a 1: L will be assigned to a latent sequence
z = z 1: L .
Since it is computationally intractable to sum continuous latent variables over the sequence, we use VQ-VAE (van den Oord et al., 2017) to lookup the discrete embedding vector z q = { z q ( a 1 ) , · · · , z q ( a L ) } for each position by vector quantization.
Speciﬁcally, the encoder output z e ( a
i ) ∈
R d will be replaced by the codebook embedding z q ( a i ) ∈
R d via a nearest neighbors lookup from the codebook B ∈ R K × d :
[Equation]
=
[Equation]
k , and k = argmin
[Equation]
[Equation]
Here, K is the size of the codebook and d is the dimension of the codebook entry e .
Then, the generator will take z q ( a i ) as its input and reconstruct x .
The training objective L r is deﬁned as:
[Equation]
Here, sg ( · ) is the stop gradient operator, which becomes 0 at the backward pass.
β is the commit coefﬁcient to control the codebook loss.
3. M ODELING S ECONDARY S TRUCTURES
In order to model the categorical information of the secondary structure, we deﬁne an 8-category sequence labeling task on the latent space, which takes x as the input and the structure label sequence y as the target.
Similar with sequence reconstruction, we use the same encoder to get z e ( a i ) and employ VQ-VAE to obtain discrete representation.
Then, z (cid:48) q ( a i ) is fed to a separate classiﬁer for the secondary structure prediction:
[Equation]
Peptide sequences and structures have distinctive local features, which are often utilized in traditional design algorithms.
The patterns of amino acids are often used for template-based design and feature-based recognition.
For structure motifs such as α -helix with at least 3.6 consecutive amino acids, they will determine the position of amino acids in the 3D space and affect the function of peptides.
However, the structure motifs are often much longer than sequence patterns.
Therefore, we establish codebooks of multiple scales to learn latent spaces for different local patterns.
3. M ULTI - SCALE VQ-VAE
For the encoder output h i =
z (0) e ( a i ) , we ﬁrst use N pattern selectors F ( n ) to extract local patterns from different scales, which will get z ( n ) e ( a i ) =
F ( n ) ( h i ) .
Then, we establish the codebook for each pattern, and use Eqn. 1 to look up the nearest codebook embedding z ( n ) q ( a i ) .
We share the codebooks between the sequence reconstruction and secondary structure prediction to capture common features and relationships between the amino acid and its structure.
Speciﬁcally, we make N r codebook for the sequence information and N s for the structure.
The z q r ( a i ) is the concatenation of N r codebook embeddings and is fed to the sequence generator:
[Equation]
Based on Eqn. 2, the reconstruction training objective for multi-scale VQ-VAE can be denoted as:
[Equation] [Equation]
The loss of secondary structure prediction task L s can be formulated in the same way from Eqn. 3.
And the total training loss is:
[Equation]
where the γ is the weight of the secondary structure prediction.
3. T RAINING AND I NFERENCE
We discuss the training and sampling process in this section.
Since the AMP dataset D amp is very small, we use two extra datasets D r
and D s to pre-train the reconstruction and prediction task.
The whole training process is described in Algorithm 1.
Training To enable the generator to generate valid peptide sequences, we ﬁrst pre-train our model on a large protein dataset D r for the sequence construction task.
Then, we jointly train the sequence reconstruction and the structure prediction on a peptide dataset D s with secondary structure information.
This phase maps the positional latent space to an entangled sequential and structural distribution.
Finally, we ﬁnetune our model on the small AMP dataset D amp to capture the characteristics of AMPs.
Following Kaiser et al.
(2018), we use Exponential Moving Average (EMA) to update the embedding vectors in the codebooks.
Speciﬁcally, we keep a count c k measuring the number of times that the embedding vector e k is chosen as the nearest neighbor of z e ( a i ) via Eqn. 1.
Thus, the counts are updated with a sort of momentum:
c k ← λc k + (1 − λ ) (cid:80)
i
I
[ z q ( a i ) =
e k ] , with the embedding e k being updated as: e k ← λe k + (1 − λ ) (cid:80) i
I [ z q ( a i )=
e k ] z e ( a i ) c k .
Here, λ is the decay parameter.
Prior Model The prior distribution over the codebook is a categorical distribution and can be made auto-regressive by the extra prior model.
In order to model the dependency between z 1: L , we train
Transformer-based language models on the embedding entries.
We extract the index sequences generated by Eqn. 1 for each codebook n and then train M prior n on them.
Inference We sample several index sequences from the prior models for each codebook n , and then lookup the codebook to get the embedding vector z ( n ) q .
Finally, z ( n ) q is fed to the generator and classiﬁer to generate the sequence with its secondary structure.
We also try to control the secondary structure by existing AMP structure patterns to further improve the generation quality.
E XPERIMENT 4. E XPERIMENT S ETUP
Dataset The Universal Protein Resource (UniProt) 2 is a comprehensive protein dataset.
We download reviewed protein sequences (550k) with the limitation of 100 in length as D r (57k examples).
Then we use ProSPr 3 (Billings et al., 2019) to predict the secondary structure for D r and ﬁlter some low-quality examples.
Therefore, we get D s with 46k examples, which have both sequence and secondary structure information.
For antimicrobial peptide dataset, we download from Antimicrobial
Peptide Database (APD3) 4 (Wang et al., 2016) and ﬁlter repeated ones to get 3222 AMPs as D amp .
We randomly extract 3,000 examples as validation and 3,000 as test on D r and D s .
For D amp , the size of validation and test is both 100.
Following Veltri et al.
(2018), we establish a decoy set with negative examples that do not have antimicrobial activities to examine the evaluation metric.
It ﬁrst removes peptide sequences with anti-microbe activity from Uniprot, and then removes sequences with length < 10 and sequences with > 40 sequence identity with AMP sequences, resulting in 2021 non-AMP sequences.
Baseline
We implement several baselines for our experiment.
Traditional methods usually randomly substitute several amino acids on the existing AMPs and conducting biological experiments on them.
Thus, we use the Random replacement with probability p as the baseline.
Following Dean & Walper (2020), we use VAE to embed the peptides into the latent space and sample latent variable z from the standard Gaussian distribution p ∼ N (0 , 1) .
For a fair comparison, we use the same Transformer architecture as our model LSSAMP and train on the Uniprot D r and APD dataset D amp .
AMPGAN is proposed by Van Oort et al.
(2021), which uses a BiCGAN architecture with convolution layers.
It consists of three components: the generator, discriminator, and encoder.
The generator and discriminator share the same encoder.
It is trained on 49k false negative sequences from UniProt and 7k positive AMP sequences.
PepCVAE (Das et al., 2018) is a VAE-based generative model with a structured variable c to control the attribute of the sequence.
Since the authors did not release their code, we use the model architecture from Hu et al.
and modify the reproduced code 5 for
AMPs.
The original paper use 93k sequences from UniProt and 7960/6948 positive/negative AMPs for training.
For comparison, we use our UniProt dataset D r and ADP dataset D amp to train it.
MLPeptide (Capecchi et al., 2021) is RNN-based generator.
It is ﬁrst trained on 3580 AMPs and then transfer against speciﬁc bacteria.
Implementation Details There are three main modules for LSSAMP .
The encoder and decoder are based on 2-layer Transformer (Vaswani et al., 2017) with d model = 128 and head = 8 .
The size of FFN projection is d ffn = 512 and all drouput rate are 0 .
For the classiﬁer, we use the same
CNN block as Billings et al.
with 32 input channels and a dilation scale of [1 , 2 , 4 , 8 , 10] .
For multi-scale codebooks, we ﬁrst apply CNN as F ( n ) to extract features.
We set n = 4 and kernel width ranging in [1 , 2 , 4 , 8] .
The features will be padded to the same length as the input sequence.
Then, we use 4 codebooks with K = 8 and d = 128 .
The reconstruction and prediction share the same codebooks, which means N r =
N s = 4 .
The commit coefﬁcient is set to β = 0 .
We use PyTorch for implementation and train on the single Tesla-V100-32GB.
We optimize the parameter with Adam Optimizer (Kingma & Ba, 2015).
During pre-training for sequence construction on D r , we limit the max length to 100 and set the maximum tokens in a batch bz as 30,000, learning rate lr as 0.01 with 8,000 warmup steps, and decoy weight for EMA as λ = 0 . 8 .
For secondary structure prediction on D s , the max length is limited to 32, bz = 10 , 000 , lr = 0 .
003 , λ = 0 .
95 , and the prediction loss coefﬁcient γ = 1 .
Finally, we transfer to D amp with the same hyperparameters except the lr = 0 .
4. E VALUATION M ETRIC
To evaluate the antimicrobial activity of the generated peptide S = { a 1 , a 2 , · · · , a L } , we use several physical features according to the mechanism of AMPs.
Besides, we also apply some open-source
AMP predictive models to estimate the probability of the generated peptides being AMPs.
4.2. P HYSICAL A TTRIBUTES
Charge The bacterial membrane usually takes the negative charge, so peptides with the positive charge prefer to bind with the bacteria.
The whole net charge of the peptide sequence S is deﬁned as the sum of the charge of all its amino acids C ( a i ) at pH 7.4, which is C ( S ) =
(cid:80) a i ∈ S C ( a i )
Hydrophobicity The hydrophobicity reﬂects the tendency to bind lipids on the bacterial membrane.
It is more likely for the peptide with a high hydrophobicity to move from the solution environment to the bacterial membrane.
We use the hydrophobicity scale H ( a i ) in Eisenberg et al.
(1984) to calculate the hydrophobicity of a sequence, which is H ( S ) =
(cid:80) a i ∈ S H ( a i )
Amphipathicity / Hydrophobic Momentum
The amphipathicity measures the ability of the peptide to bind water and lipid at the same time, which is a deﬁnitive feature of antimicrobial peptides(Hancock & Rozek, 2002).
It can be quantiﬁed by the hydrophobic momentum uH ( S, θ ) , deﬁned by Eisenberg et al.
The hydrophobic momentum is determined by the hydrophobicity H ( a i ) of each amino acid a i , along with the angle θ between amino acids.
The angle can be estimated by the secondary structure.
For the α -helix structure, θ is 100 ◦ and for β -sheet, θ is 180 ◦ .
[Equation]
For each peptide, we calculate the above properties to measure its antimicrobial activity.
For comparison, we draw the distribution on APD and decoy dataset and choose a range for each property 6 .
We use the percentage of peptides in the range for each attribute to leverage the generation performance and use Combination to measure the percentage of those that satisfy three conditions at the same time.
The detailed information is attached in the appendix.
[Table]
AMP C LASSIFIERS
Following previous works (Das et al., 2020; Van Oort et al., 2021), we also use several open-source
AMP prediction tools to predict the AMP probability of generated sequence.
Since these prediction tools are trained and report results in different AMP datasets, we ﬁrst use them to predict sequences in APD and decoy dataset as a reference of their performance.
Thomas et al.
(2010) trained on the AMP database of 3782 sequences with random forest ( RF ), discriminant respectively.
AMP Scanner v2 9 (Veltri et al., 2018), short as Scanner , is a CNN-&LSTM-based deep neural network trained on 1778 AMPs picked from APD.
AMPMIC 10 (Witten & Witten, 2019) trained a CNN-based regression model on 6760 unique sequences and 51345 MIC measurement to predict MIC values.
IAMPE 11 (Kavousi et al., 2020) is a model based on Xtreme Gradient Boosting.
It achieves the highest correct prediction rate on a a set of ten more recent AMPs (Aronica et al., 2021).
ampPEP 12 (Lawrence et al., 2021) is a random forest based model which is trained on
AMP sequences.
It has the best performance across multiple data sets (Aronica et al., 2021).
4. R ESULTS AND A NALYSIS
We generate 5000 sequences for each baseline and ﬁlter the repeated ones.
During the generation, we add some structural restrictions based on the antimicrobial mechanism on the secondary structure to improve the performance.
Speciﬁcally, we reject peptides with more than 30% coil structure (‘-’), which can hardly fold in the solution environment and insert into the bacterial membrane.
Besides, we limit the minimum length of a helix structure (‘H’) to 4 based on physical rules.
We name our model with structural control as LSSAMP and the model without extra conditions as LSSAMP w/o condition .
We discuss the structure conditions and give some generated peptides in the appendix.
[Table]
Physical Attributes As listed in Table 1, LSSAMP outperforms on the combination percentage by a large margin (5.56%), which indicates that our model can generate sequences satisfying multiple properties at the same time.
If we do not control the structure, the combination percentage is similar to APD, which implies our model learns the latent distribution of APD.
For the speciﬁc attribute,
LSSAMP tends to generate peptides with higher hydrophobicity, while AMP-GAN and MLPeptide have more positive sequences.
Compared with other models, PepCVAE tends to generate redundant sequences, which results in a signiﬁcant decline in the number of unique sequences.
Also, we can ﬁnd that by control the secondary structure, all metrics can be improved.
This veriﬁes that secondary structure information has a great inﬂuence on peptide properties and it is beneﬁcial to take it into consideration during the peptide generation.
AMP Prediction
The results of prediction tools are shown in Table 2.
By the comparison of
LSSAMP and LSSAMP w/o condition, we can draw the same conclusion as above that controlling the secondary structure can further improve the generation performance.
For AMPMIC and
IAMPE, PepCVAE has a signiﬁcant advantage over other methods, but it has a poor performance in other classiﬁers.
LSSAMP performs better than MLPeptide in SVM and amPEP but does not perform as well as it in other classiﬁers.
On the classiﬁer with the largest performance gap IAMPE, we can ﬁnd that the results of LSSAMP and LSSAMP w/o condition are similar, which implies that different secondary structures do not affect the results of IAMPE.
However, in real scenarios, the structure is very important for the antimicrobial mechanism, which makes the classiﬁer less reliable.
[Table] [Table]
capture local features with window sizes of
[Figure]
Ablation Study We conduct the ablation study for our LSSAMP and show the results in Table
[Figure]
PPL is the perplexity of generated sequences that can measure ﬂuency.
Loss is the model loss on the validation set.
AA Acc. is the reconstruction accuracy of amino acids sequences and SS
Acc.
is the prediction accuracy of the secondary structure.
We can ﬁnd that without the ﬁrst pretraining phase on D r for sequence reconstruction, the model can hardly generate valid sequences.
The second phase to train the model on the large-scale secondary structure dataset D s will affect the prediction performance on the target AMP dataset.
If we remove multiple sub-codebooks and use a single codebook with the same size, the performance will decline a little.
Codebook Number We explore the effect of different numbers of codebooks on generation performance.
From Table 4, we ﬁnd that a single small codebook can hardly learn enough information to reconstruct the sequences.
The PPL, loss, and secondary structure accuracy become better with the increase of codebook items.
However, the reconstruction accuracy achieves the best performance when the codebook number is 3.
This may be due to the relatively short local patterns for the amino acid sequence, which makes the window of size 8 too long for it.
Visualization We plot the distribution of amino acids, charge, sequence length, hydrophobicity, and hydrophobic momentum for APD, Decoy, and our models in Figure 3.
Without condition, the distribution of LSSAMP is similar to APD, which indicates that LSSAMP successfully learns the latent space of APD.
However, if we control the secondary structure, it is more likely to generate sequences with a longer length and more positive charges.
For hydrophobicity and hydrophobic momentum, the distribution of generated sequences is more concentrated.
C ONCLUSION
In this paper, we propose a multi-scale VQ-VAE to generate peptides.
Motivated by the important role of structure in the antimicrobial mechanism, LSSAMP learns the latent spaces for each position with both sequential and structural features.
Our model shows excellent performance on physical attributes related to antimicrobial activities and has a high probability to be predicted as AMPs by public classiﬁers.
As the structural feature determines the function of macromolecules, LSSAMP has great potential for macromolecules design, including proteins and RNAs.
A A PPENDIX A. A TTRIBUTE D ISTRIBUTION
To determine the threshold of net charge, hydrophobicity, and hydrophobic moment
[Figure]
we analyze the distribution of sequences in APD and decoy in Figure 4.
For charge
[Figure]
we follow the rule summarized by specialists and choose sequences whose net charge is +2 to +10.
For two characters left, we plot the histogram and compare the proportion in every bin.
If the proportion of APD is larger than that in decoy, we add the bin into the acceptation range of the evaluation metric.
The ﬁnal ranges are
C ∈ [2 , 10] , H ∈ [0 . 25 , ∞ ]
[Figure]
and uH ∈ [0 . 5 , 0 .
75] ∪ [1 . 75 , ∞ ]
[Figure]
A. G ENERATED AMP S
We show 10 peptides generated by LSSAMP in Table 5, and build 3D models for four of them by
PEPFold 3 (Shen et al., 2014) and draw the picture by PyMOL (Schr ¨ odinger, LLC, 2015c;a;b) in
Figure 5.
We can ﬁnd that all these peptides have long helix structures, which make them more likely to have the antimicrobial ability.
At the same time, although the model predicts a long continuous helix structure for peptide ID = 4 and ID = 9 , in fact, they have a small coil structure between two helix structures.
It indicates that our model tends to predict a long continuous secondary structure instead of several discontinuous small fragments
[Table]
A. S TRUCTURE
[Table]
ONDITION
As described above, controlling the secondary structure can affect the properties of generated peptides.
Thus we limit the percentage of the coil structure with different ratio and calculate the physical attributes of generated peptides.
The results are shown in Figure 6.
We can ﬁnd that with the decrease of number of coil structures, the percentage of positive peptides keep growing.
However, for
[Figure]
[Table]
hydrophobicity and hydrophobic moment, the percentage drop after 0.3.
Therefore, we limit the length of the coil structure to 30% in our main experiments.
[Figure]
A. F ILTER S TRATEGY
Different from the existing work, we control the secondary structures during the generation phase instead of ﬁltering sequences after the generation.
But does this ﬁltering mechanism make sense for other methods as the post-process?
To answer this question, we use the same rules to ﬁlter the generated peptides of comparison partners.
The results are shown in Table 6.
Comparing Table
[Figure]
and 6, we can ﬁnd that all results are improved by limited sequences to the alpha-helical structures.
It implies that by controlling the structure, the physical attributes can be improved, which further veriﬁes the importance of secondary structures.
However, the sequence number have signiﬁcantly declined, indicating that this generate-then-ﬁlter pipeline is inefﬁcient.
A. D ISTRIBUTION ON A MINO A CIDS
To illustrate the distribution of amino acids in generated peptides, we use the tSNE plot shown in
Figure 7.
We convert use the vector with each dimension representing the probability of a certain amino acid to represent the peptide.
Then we use tSNE to convert the high-dimensional vector to 2D and visualize them.
From Figure 7, we can obverse that there is a large overlap between LSSAMP w/o condition and APD, which indicates that our model has captured the global distribution of APD instead of collapsing to a local mode.
Furthermore, LSSAMP covers APD and has some outliers.
It implies that with the secondary structure condition, our model can not only learn the existing AMP space but also explore more possible space.
A. N OVELTY
To measure the novelty of generated peptides, we deﬁne three evaluation metrics:
Uniqueness,
Diversity, and Similarity.
Uniqueness is the percentage of the unique peptides.
Diversity measures the similarity among generated peptides.
We calculate the Levenshtein distance between every two sequences and normalize it by the sequence length.
Then we average the normalized distance to get the mean as its diversity.
The higher the diversity, the more dissimilar the generated peptides are.
Similarity is the similarity between the generated peptides and the training AMP set (APD).
For each generated sequence, we ﬁnd a peptide from the training set which has the smallest Levenshtein distance with it and normalize the distance by its length.
And we calculate the average length as the similarity.
From Table 7, we obverse that VAE has the highest diversity and lowest similarity.
However, in
Table 1 and 2, VAE performs the worst, which implies we can not measure the generation quality based on the novelty alone.
Besides, the limitation on secondary structures will cause a signiﬁcant decline in diversity.
However, it does not result in the increase of the number of redundant peptides since the uniqueness does not increase.
It implies that the restrictions make the model capture similar local patterns but not generate the exact same sequence.
[Table] [Table] A. R EPRODUCTION We describe the architecture of our model and the hyperparameters used in the experiments in detail in Section 4.1.
We run the model for several times and calculate the mean and variance.
In Table 8, we add the mean and the standard deviation of each setting in Table 4.
We will also release our code after the anonymous period.