|
--- |
|
license: mit |
|
datasets: |
|
- COGNANO/VHHCorpus-2M |
|
library_name: transformers |
|
tags: |
|
- biology |
|
- protein |
|
- antibody |
|
- VHH |
|
--- |
|
|
|
## VHHBERT |
|
|
|
VHHBERT is a RoBERTa-based model pre-trained on two million VHH sequences in [VHHCorpus-2M](https://huggingface.co/datasets/COGNANO/VHHCorpus-2M). |
|
VHHBERT has the same model parameters as RoBERTa<sub>BASE</sub>, except that it used positional embeddings with a length of 185 to cover the maximum sequence length of 179 in VHHCorpus-2M. |
|
Further details on VHHBERT are described in our paper "[A SARS-CoV-2 Interaction Dataset and VHH Sequence Corpus for Antibody Language Models](https://arxiv.org/abs/2405.18749).” |
|
|
|
## Usage |
|
|
|
The model and tokenizer can be loaded using the `transformers` library. |
|
|
|
```python |
|
from transformers import BertTokenizer, RobertaModel |
|
tokenizer = BertTokenizer.from_pretrained("COGNANO/VHHBERT") |
|
model = RobertaModel.from_pretrained("COGNANO/VHHBERT") |
|
``` |
|
|
|
## Links |
|
|
|
- Pre-training Corpus: https://huggingface.co/datasets/COGNANO/VHHCorpus-2M |
|
- Code: https://github.com/cognano/AVIDa-SARS-CoV-2 |
|
- Paper: https://arxiv.org/abs/2405.18749 |
|
|
|
## Citation |
|
|
|
If you use VHHBERT in your research, please cite the following paper. |
|
|
|
```bibtex |
|
@inproceedings{tsuruta2024sars, |
|
title={A {SARS}-{C}o{V}-2 Interaction Dataset and {VHH} Sequence Corpus for Antibody Language Models}, |
|
author={Hirofumi Tsuruta and Hiroyuki Yamazaki and Ryota Maeda and Ryotaro Tamura and Akihiro Imura}, |
|
booktitle={Advances in Neural Information Processing Systems 37}, |
|
year={2024} |
|
} |
|
``` |
