Design Proteins Using Large Language Models: Enhancements and
  Comparative Analyses
Authors: name: Kamyar Zeinalipour, name: Neda Jamshidi, name: Monica Bianchini, name: Marco Maggini, name: Marco Gori

Pre-trained LLMs have demonstrated substantial capabilities across a range of
conventional natural language processing (NLP) tasks, such as summarization and
entity recognition. In this paper, we explore the application of LLMs in the
generation of high-quality protein sequences. Specifically, we adopt a suite of
pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and
gemma-7B4, to produce valid protein sequences. All of these models are publicly
available.5 Unlike previous work in this field, our approach utilizes a
relatively small dataset comprising 42,000 distinct human protein sequences. We
retrain these models to process protein-related data, ensuring the generation
of biologically feasible protein structures. Our findings demonstrate that even
with limited data, the adapted models exhibit efficiency comparable to
established protein-focused models such as ProGen varieties, ProtGPT2, and
ProLLaMA, which were trained on millions of protein sequences. To validate and
quantify the performance of our models, we conduct comparative analyses
employing standard metrics such as pLDDT, RMSD, TM-score, and REU. Furthermore,
we commit to making the trained versions of all four models publicly available,
fostering greater transparency and collaboration in the field of computational
biology.