File size: 3,752 Bytes
9ae8e1c 7756bc9 503222f 7756bc9 503222f 7756bc9 503222f 7756bc9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
---
license: apache-2.0
---
---
license: apache-2.0
---
# ProteinForceGPT: Generative strategies for modeling, design and analysis of protein mechanics
### Load model
This model is an autoregressive transformer model in GPT-style, trained to analyze and predict the mechanical properties of a large number of protein sequences.
The pretraining task is defined as "Sequence<...>" where ... is an amino acid sequence.
Mechanics-related forward and inverse tasks are:
```raw
CalculateForce<GEECDCGSPSNP..>,
CalculateEnergy<GEECDCGSPSNP..>
CalculateForceEnergy<GEECDCGSPSNP...>
CalculateForceHistory<GEECDCGSPSNP...>
GenerateForce<0.262>
GenerateForce<0.220>
GenerateForceEnergy<0.262,0.220>
GenerateForceHistory<0.004,0.034,0.125,0.142,0.159,0.102,0.079,0.073,0.131,0.105,0.071,0.058,0.072,0.060,0.049,0.114,0.122,0.108,0.173,0.192,0.208,0.153,0.212,0.222,0.244>
```
Load pretrained model:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
pretrained_model_name='lamm-mit/ProteinForceGPT'
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
model_name = pretrained_model_name
model = AutoModelForCausalLM.from_pretrained(
model_name,
trust_remote_code=True
).to(device)
model.config.use_cache = False
```
Sample inference using the "Sequence<...>" task, where here, the model will simply autocomplete the sequence starting with "AIIAA":
```python
prompt = "Sequence<GEECDC"
generated = torch.tensor(tokenizer.encode(prompt, add_special_tokens = False)) .unsqueeze(0).to(device)
print(generated.shape, generated)
sample_outputs = model.generate(
inputs=generated,
eos_token_id =tokenizer.eos_token_id,
do_sample=True,
top_k=500,
max_length = 300,
top_p=0.9,
num_return_sequences=1,
temperature=1,
).to(device)
for i, sample_output in enumerate(sample_outputs):
print("{}: {}\n\n".format(i, tokenizer.decode(sample_output, skip_special_tokens=True)))
```
Sample inference using the "CalculateForce<...>" task, where here, the model will calculate the maximum unfolding force of a given sequence:
```python
prompt = "'CalculateForce<GEECDCGSPSNPCCDAATCKLRPGAQCADGLCCDQCRFKKKRTICRIARGDFPDDRCTGQSADCPRWN>"
generated = torch.tensor(tokenizer.encode(prompt, add_special_tokens = False)) .unsqueeze(0).to(device)
sample_outputs = model.generate(
inputs=generated,
eos_token_id =tokenizer.eos_token_id,
do_sample=True,
top_k=500,
max_length = 300,
top_p=0.9,
num_return_sequences=3,
temperature=1,
).to(device)
for i, sample_output in enumerate(sample_outputs):
print("{}: {}\n\n".format(i, tokenizer.decode(sample_output, skip_special_tokens=True)))
```
Output:
```raw
0: CalculateForce<GEECDCGSPSNPCCDAATCKLRPGAQCADGLCCDQCRFKKKRTICRIARGDFPDDRCTGQSADCPRWN> [0.262]```
```
## Citation
To cite this work:
```
@article{GhafarollahiBuehler_2024,
title = {ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learning },
author = {A. Ghafarollahi, M.J. Buehler},
journal = {},
year = {2024},
volume = {},
pages = {},
url = {}
}
``` |