HIV_V3_coreceptor model

Summary
Model Description
Intended Uses & Limitations
How to Use
Training Data
Training Procedure
- Preprocessing
- Training
Evaluation Results
BibTeX Entry and Citation Info

Summary

The HIV-BERT-Coreceptor model was trained as a refinement of the HIV-BERT model and serves to better predict HIV V3 coreceptor tropism. HIV-BERT is a model refined from the ProtBert-BFD model to better fulfill HIV-centric tasks. This model was then trained using HIV V3 sequences from the Los Alamos HIV Sequence Database, allowing even more precise prediction of V3 coreceptor tropism than the HIV-BERT model can provide.

Model Description

The HIV-BERT-Coreceptor model is intended to predict the Co-receptor tropism of HIV from a segment of the envelope protein. These envelope proteins encapsulate the virus and interact with the host cell through the human CD4 receptor. HIV then requires the interaction of one, of two, co-receptors: CCR5 or CXCR4. The availability of these co-receptors on different cell types allows the virus to invade different areas of the body and evade antiretroviral therapy. The 3rd variable loop of the envelope protein, the V3 loop, is responsible for this interaction. Given a V3 loop sequence, the HIV-BERT-Coreceptor model will predict the likelihood of binding to each of these co-receptors.

Intended Uses & Limitations

This tool can be used as a predictor of HIV tropism from the Env-V3 loop. It can recognize both R5, X4, and dual tropic viruses natively. It should not be considered a clinical diagnostic tool.

This tool was trained using the Los Alamos HIV sequence dataset. Due to the sampling nature of this database, it is predominantly composed of subtype B sequences from North America and Europe with only minor contributions of Subtype C, A, and D. Currently, there was no effort made to balance the performance across these classes. As such, one should consider refinement with additional sequences to perform well on non-B sequences.

How to use

Need to add

Training Data

This model was trained using the damlab/HIV_V3_coreceptor dataset using the 0th fold. The dataset consists of 2935 V3 sequences (approximately 35 tokens each) extracted from the Los Alamos HIV Sequence database.

Training Procedure

Preprocessing

As with the rostlab/Prot-bert-bfd model, the rare amino acids U, Z, O, and B were converted to X and spaces were added between each amino acid. All strings were concatenated and chunked into 256 token chunks for training. A random 20% of chunks were held for validation.

Training

The damlab/HIV-BERT model was used as the initial weights for an AutoModelforClassificiation. The model was trained with a learning rate of 1E-5, 50K warm-up steps, and a cosine_with_restarts learning rate schedule and continued until 3 consecutive epochs did not improve the loss on the held-out dataset. As this is a multiple classification task (a protein can bind to CCR5, CXCR4, neither, or both) the loss was calculated as the Binary Cross Entropy for each category. The BCE was weighted by the inverse of the class ratio to balance the weight across the class imbalance.

Evaluation Results