HIV_PR_resist model

Table of Contents

• Summary
• Model Description
• Intended Uses & Limitations
• How to Use
• Training Data
• Training Procedure
  • Preprocessing
  • Training
• Evaluation Results
• BibTeX Entry and Citation Info

Summary

The HIV-BERT-Protease-Resistance model was trained as a refinement of the HIV-BERT model (insert link) and serves to better predict whether an HIV protease sequence will be resistant to certain protease inhibitors. HIV-BERT is a model refined from the ProtBert-BFD model to better fulfill HIV-centric tasks. This model was then trained using HIV protease sequences from the Stanford HIV Genotype-Phenotype Database, allowing even more precise prediction protease inhibitor resistance than the HIV-BERT model can provide.

Model Description

The HIV-BERT-Protease-Resistance model is intended to predict the likelihood that an HIV protease sequence will be resistant to protease inhibitors. The protease gene is responsible for cleaving viral proteins into their active states, and as such is an ideal target for antiretroviral therapy. Annotation programs designed to predict and identify protease resistance using known mutations already exist, however with varied results. The HIV-BERT-Protease-Resistance model is designed to provide an alternative, NLP-based mechanism for predicting resistance mutations when provided with an HIV protease sequence.

Intended Uses & Limitations

This tool can be used as a predictor of protease resistance mutations within an HIV genomic sequence. It should not be considered a clinical diagnostic tool.

How to use

Prediction example of protease sequences

Training Data

This model was trained using the damlab/HIV-PI dataset using the 0th fold. The dataset consists of 1959 sequences (approximately 99 tokens each) extracted from the Stanford HIV Genotype-Phenotype Database.

Training Procedure

Preprocessing

As with the rostlab/Prot-bert-bfd model, the rare amino acids U, Z, O, and B were converted to X and spaces were added between each amino acid. All strings were concatenated and chunked into 256 token chunks for training. A random 20% of chunks were held for validation.

Training

The damlab/HIV-BERT model was used as the initial weights for an AutoModelforClassificiation. The model was trained with a learning rate of 1E-5, 50K warm-up steps, and a cosine_with_restarts learning rate schedule and continued until 3 consecutive epochs did not improve the loss on the held-out dataset. As this is a multiple classification task (a protein can be resistant to multiple drugs) the loss was calculated as the Binary Cross Entropy for each category. The BCE was weighted by the inverse of the class ratio to balance the weight across the class imbalance.

Evaluation Results

Need to add

BibTeX Entry and Citation Info

[More Information Needed]