Generating Antimicrobial Peptides from Latent Secondary Structure Space

1 INTRODUCTION . Developing neural networks for drug discovery has attracted increasing attention recently . It can facilitate the discovery of potential therapies and reduce the time and cost of drug development ( Stokes et al. , 2020 ) . Plenty of works have been done to employ deep generative models in searching for drug-like molecules with desired properties and achieved great success ( Jin et al. , 2018 ; Shi et al. , 2019 ; Schwalbe-Koda & Gómez-Bombarelli , 2020 ; Xie et al. , 2020 ) . However , these works mainly focus on small molecules , and more complicated biochemicals , such as proteins , are still rarely explored . Antimicrobial peptides ( AMPs ) , defined as short proteins of less than 50 amino acids with potent antimicrobial activity , are an emerging category of therapeutic agents . AMPs exist widely in the natural immune system for all species and kill bacteria in a physical way ( Aronica et al. , 2021 ; Cardoso et al. , 2020 ) . They attach to the bacterial membrane and insert into the membrane to form pores , which leads to the death of bacteria by allowing cytoplasmic leakage . This mechanism makes them more promising for handling extensively drug-resistant bacteria than traditional antibiotics ( Mahlapuu et al. , 2016 ) . However , the theoretical chemical space of peptides is enormous and the sequence number grows exponentially as the length increases . Thus , it is challenging to search for valid peptides with antimicrobial properties from such a huge sequence space . Several factors can affect the antimicrobial activity of peptides ( Boman , 2003 ) . Amino acids with positive charges are more likely to bind with bacterial membrane as most bacterial surfaces are anionic , while those with high hydrophobicities tend to move from the solution environment to the bacterial membrane . However , the mechanisms of antibacterial peptides need not only a reasonable sequence but also an appropriate structure . For example , by forming the helix structure , a peptide can gather the hydrophobic amino acids on one side and hydrophilic ones on the other . This ability , named amphipathy , helps it insert into the membrane and maintain a stable hole with other peptide molecules in the membrane , as shown in Figure 1 . The hole will drain the cytoplasm and finally kill the bacteria . This mechanism of killing the bacteria is called ‘ barrel stave ’ . Amphipathy plays an important role in deciding the antibacterial activity of peptides and is closely related to the secondary structure of the peptide ( Aronica et al. , 2021 ) . According to the antimicrobial mechanism , a new AMP should meet the following criteria . C1 : It possesses several ideal physical attributes ( e.g . positive charge and high hydrophobicity ) . C2 : It has appropriate secondary structures ( e.g . alpha-helix ) . C3 : It differs from existing AMPs to some extent . The existing works mainly focus on sequential features of amino acids and ignore the secondary structure . The traditional methods replace subsequences with patterns from the pattern database in a given template ( Porto et al. , 2018 ) . Inspired by success in deep neural networks , many researchers apply neural generative models to AMP discovery . They often use the physical attributes as the extra input to control the generation phase ( Das et al. , 2018 ; Van Oort et al. , 2021 ) , or train classifiers on each attribute to filter the peptides after the generation ( Capecchi et al. , 2021 ; Das et al. , 2021 ) . The former ones usually generate peptides that have low correlation with the input attributes and the filter phase of the latter ones make the sampling inefficient . As described above , the antimicrobial activity is determined by both the amino acid composition and secondary structure of the peptide . Thus , we propose LSSAMP to generate antimicrobial peptides from the latent semantic and structure space . Taking the peptide sequence as the time series , we assign a latent variable on each position . Since it is computationally intractable to sum continuous latent variables over all positions , we employ the vector quantized-variational autoencoder ( VQ-VAE ) ( van den Oord et al. , 2017 ) to learn the discrete distribution for each position and further design a multi-scale codebooks strategy to capture different local patterns to fit various length ranges for amino acid and structure sequences . During the generation process , LSSAMP will sample a backbone from the secondary structure latent space and generate the amino acid sequence simultaneously . We evaluate LSSAMP and several baselines through physical properties that are closely related to the antibacterial mechanism . Besides , we use some public AMP prediction models to predict generated sequences being AMPs as previous works did ( Das et al. , 2020 ; Van Oort et al. , 2021 ) . To conclude , our contributions are as follows : • We propose LSSAMP , a generative model which samples peptides from the latent secondary structure space to control the peptide properties . • We develop a multi-scale VQ-VAE to learn positional latent spaces from different aspects and model semantic sequences and structural sequences in the same space . • Experimental results show that LSSAMP can generate peptides with multiple ideal features such as positive charge , better hydrophobicity , and better amphipathicity . The results of public AMP classifiers also verify that our model can generate peptides with high AMP probability . 2 RELATED WORK . Antimicrobial Peptides Generation Traditional methods for AMP design can be divided into three approaches ( Torres & de la Fuente-Nunez , 2019 ) : ( i ) The pattern recognition algorithms build a sequential pattern database from existing AMPs , and then pick a template peptide and replace local sequence with patterns ( Loose et al. , 2006 ; Porto et al. , 2018 ) . ( ii ) The genetic algorithms analyze the AMP database and design some antibiotic activity functions ( Maccari et al. , 2013 ) . ( iii ) The molecular modeling and molecular dynamics methods build 3D models of peptides and analyze activity ( Matyus et al. , 2007 ; Bolintineanu & Kaznessis , 2011 ) . Deep generative models take a rapid growth in recent years . Dean & Walper ( 2020 ) encodes the peptide into the latent space and interpolates across a predictive vector between a known AMP and its scrambled version to generate novel peptides . The PepCVAE ( Das et al. , 2018 ) and CLaSS ( Das et al. , 2021 ) employ the variational auto-encoder model to generate sequences . The AMPGAN ( Van Oort et al. , 2021 ) uses the generative adversarial network to generate new peptide sequences with conditions . To our knowledge , this is the first study to take secondary structure information into consideration during the generative phase , which is conducive to effectively generate well-structured sequences with desired properties . Sequence Generation via VQ-VAE The variational auto-encoders ( VAEs ) were first proposed by Kingma & Welling ( 2014 ) for image generation , and then widely applied to sequence generation tasks such as language modeling ( Bowman et al. , 2016 ) , paraphrase generation ( Gupta et al. , 2018 ) , machine translation ( Bao et al. , 2019 ) and so on . Instead of mapping the input to a continuous latent space in VAE , the vector quantized-variational autoencoder ( VQ-VAE ) ( van den Oord et al. , 2017 ) learns the codebook to obtain a discrete latent representation . It can avoid issues of posterior collapse while has comparable performance with VAEs . Based on it , Razavi et al . ( 2019 ) uses a multi-scale hierarchical organization to capture global and local features for image generation . Bao et al . ( 2021 ) learns implicit categorical information of target words with VQ-VAE and models the categorical sequence with conditional random fields in non-autoregressive machine translation . In this paper , we employ the multi-scale vector quantized technique to obtain the discrete representation for each position of the peptide . 3 METHOD . Given a peptide sequence x = { a1 , a2 , · · · , aL } , where a belongs to the 20 common amino acids and L is the sequence length , the corresponding secondary structure can be denoted as y = { y1 , y2 , · · · , yL } . Following the definition in Kabsch & Sander ( 1983 ) , there are 8 secondary structure types , including one unknown label , so yi ∈ { H , B , E , G , I , T , S , − } 1 . We first employ VQ-VAE for the sequence reconstruction task to learn the sequential latent space ( Section 3.1 ) . Then , we enforce the latent space to model the structure information by the secondary structure task ( Section 3.2 ) . Besides , we design the multi-scale codebooks to capture different local patterns ( Sec- 1H , G , I denote the alpha , 3-10 , and pi helix . E , T are the strand and turn . The others are coil structures . tion 3.3 ) . Finally , we describe the training and inference phase in Section 3.4 . The overview of our model is shown in Figure 2 . 3.1 MODELING PEPTIDE SEQUENCES . For sequential information , we embed the input peptide x = { a1 , a2 , · · · , aL } to the latent space via the encoder and use the generator to reconstruct x . We assume that each ai is determined by a latent variable zi , and the input sequences x = a1 : L will be assigned to a latent sequence z = z1 : L. Since it is computationally intractable to sum continuous latent variables over the sequence , we use VQ-VAE ( van den Oord et al. , 2017 ) to lookup the discrete embedding vector zq = { zq ( a1 ) , · · · , zq ( aL ) } for each position by vector quantization . Specifically , the encoder output ze ( ai ) ∈ Rd will be replaced by the codebook embedding zq ( ai ) ∈ Rd via a nearest neighbors lookup from the codebook B ∈ RK×d : zq ( ai ) = ek , and k = argminj∈ { 1 , ··· , K } ‖ze ( ai ) − ej‖2 . ( 1 ) Here , K is the size of the codebook and d is the dimension of the codebook entry e. Then , the generator will take zq ( ai ) as its input and reconstruct x . The training objective Lr is defined as : Lr = log p ( ai|zq ( ai ) ) + ‖sg [ ze ( ai ) ] − zq ( ai ) ‖22 + β ‖ze ( ai ) − sg [ zq ( ai ) ] ‖ 2 2 . ( 2 ) Here , sg ( · ) is the stop gradient operator , which becomes 0 at the backward pass . β is the commit coefficient to control the codebook loss . 3.2 MODELING SECONDARY STRUCTURES . In order to model the categorical information of the secondary structure , we define an 8-category sequence labeling task on the latent space , which takes x as the input and the structure label sequence y as the target . Similar with sequence reconstruction , we use the same encoder to get ze ( ai ) and employ VQ-VAE to obtain discrete representation . Then , z′q ( ai ) is fed to a separate classifier for the secondary structure prediction : Ls = log p ( yi|z′q ( ai ) ) + ∥∥sg [ ze ( ai ) ] − z′q ( ai ) ∥∥22 + β ∥∥ze ( ai ) − sg [ z′q ( ai ) ] ∥∥22 . ( 3 ) Peptide sequences and structures have distinctive local features , which are often utilized in traditional design algorithms . The patterns of amino acids are often used for template-based design and feature-based recognition . For structure motifs such as α-helix with at least 3.6 consecutive amino acids , they will determine the position of amino acids in the 3D space and affect the function of peptides . However , the structure motifs are often much longer than sequence patterns . Therefore , we establish codebooks of multiple scales to learn latent spaces for different local patterns .

Summary: The paper proposes a multi-scale VQ-VAE model to generate antimicrobial peptides (AMPs). The model both train on the amino-acid (aa) sequences and on secondary structure. The author evaluate the model by comparing attributes of the generated sequences with real AMPs. The authors also perform an ablation study of their model, an extensive benchmark of existing generative models that have been used for peptides.   The main contributions of the authors are: - Developing a generative model that accounts for both the aa sequences as well as the secondary structure of the AMPs.  - A large set of experiments to validate their approach, comprising the study of attributes of the generated sequences, benchmark of existing methods, evaluation of the generated sequences with existing predictors and an ablation study.