ProteinSage-650

ProteinSage-650M is a protein language model checkpoint for extracting sequence representations from protein sequences.

Usage

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "ProteinSage650"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).eval()

seq = "MILMCQHFSGQFSKYFLAVSSDFCHFVFPIILVSHVNFKQMKRKGFALWNDRAVPFTQGIFTTVMILLQYLHGTGM"
inputs = tokenizer(seq, add_special_tokens=True, return_tensors="pt")

with torch.inference_mode():
    outputs = model(**inputs, output_hidden_states=True, output_attentions=True, return_dict=True)

hidden_states = outputs.hidden_states
attentions = outputs.attentions

Use trust_remote_code=True because this checkpoint includes custom ProteinSage model and tokenizer code.

Notes

• The checkpoint was verified with transformers and PyTorch in the proteinsage environment.
• It is intended primarily for protein sequence representation extraction.
• During loading, output_layer.weight may be newly initialized, so masked language modeling predictions may require downstream training or a matching head checkpoint.

License

CC BY-NC 4.0.