# A Bi-LSTM Based Ensemble Algorithm for Prediction of Protein Secondary Structure
## 5 Conclusions  
In this paper, we introduce a Bi-LSTM-based ensemble method for the prediction of protein secondary structure. Experimental results show that our method is often better than other well-known methods. The method is available as a prediction sever at [[http://ipv.math.sci.zstu.edu.cn/proteinPrediction.jsp](http://ipv.math.sci.zstu.edu.cn/proteinPrediction.jsp)]([http://ipv.math.sci.zstu.edu.cn/proteinPrediction.jsp](http://ipv.math.sci.zstu.edu.cn/proteinPrediction.jsp)) and the example code is also available at this website. Our algorithm can also be applied to other fields of bioinformatics, such as the prediction of the relative solvent accessibility and disordered regions (DISO), thus expanding its potential. Additionally, we currently predict three states of protein secondary structure. We can similarly extend it to predict eight states of protein secondary structure, which will be our future work.  
[PERSON] et al. (2018) used PSSM and HMM as input features and trained many distinct models based on Bi-LSTM and ResNets. They put all features into each model and selected the nine best models after  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{8}{c}{**Accuracy Measures**} \\ \cline{2-9}  & **SOV (\%)** & **SOV\({}_{\text{H}}\) (\%)** & **SOV\({}_{\text{E}}\) (\%)** & **SOV\({}_{\text{C}}\) (\%)** & **C\({}_{\text{H}}\)** & **C\({}_{\text{E}}\)** & **C\({}_{\text{C}}\)** \\ \hline ensemble & 80.60 & 84.50 & 82.60 & 75.45 & 0.8058 & 0.7854 & 0.6977 \\ SPIDER3 & 78.10 & 83.20 & 79.20 & 72.50 & 0.7978 & 0.7126 & 0.6721 \\ JPred4 & 72.20 & 76.80 & 75.10 & 64.60 & 0.7324 & 0.6796 & 0.6043 \\ RaptorX & 75.40 & 80.80 & 82.30 & 71.20 & 0.7839 & 0.7598 & 0.6757 \\ \hline \hline \end{tabular}
\end{table}
Table 4: SOV scores and MCC results by four different methods for 203 proteins from Critical Appraisals Skills Programme (CASP203).  
Figure 5: Q\({}_{3}\) accuracy of ensemble model and other three methods for 203 proteins from Critical Appraisals Skills Programme (CASP203).  
reviewing their performance on a validation set to ensemble the final model. Unlike their work, our ensemble model lets one type of features be put into one of the five sub-models to fully explore the properties of the protein sequence. Additionally, the ensemble model and the five sub-models can be trained at the same time.  
Although we obtain satisfying prediction results for most protein sequences, the performance of the ensemble model is degenerated for extremely long protein sequences, for example, when the number of residues in a protein sequence exceeds 1000. More powerful architecture with an attention mechanism, such as neural Turing machines [34], may be suitable for solving this problem. In addition, combining profiles or structural similarity with machine learning methods would be a referable strategy for further improving the performance.  
Conceptualization, H.H., S.X. and Z.L.; methodology, H.H. and Z.L.; software, S.X.; validation, H.H., S.X. and Z.L.; formal analysis, A.E.; writing--original draft preparation, H.H. and S.X.; writing--review and editing, Z.L. and A.E.; funding acquisition, Z.L. and H.H.  
This work was supported by National Natural Science Foundation of China under Grant No. 11671009, Zhejiang Provincial Natural Science Foundation of China under Grant No. LZ19A010002 and LY19F010014.  
We would like to thank National Natural Science Foundation and Zhejiang Provincial Natural Science Foundation of China for the research support.  
The authors declare no conflict of interest.