ESMCapsid-S

Built with ESM.

ESMCapsid-S is a fine-tuned derivative of Synthyra/ESMplusplus_large for the candidate-screening stage of the ESMCapsid workflow. The repository includes the model weights, tokenizer, configuration, and custom Transformers implementation required for loading the model.

About

ESM++ Large provides a Transformers-compatible implementation of ESMC 600M. ESMCapsid-S was initialized from that checkpoint and fine-tuned as a sequence-classification derivative. In the released ESMCapsid pipeline, the model is used as an encoder: Layer 16 representations are mean-pooled and passed to the auxiliary normal and hard-negative classifier heads.

Property	Value
Base model	Synthyra/ESMplusplus_large
Model type	Fine-tuned ESM++ / ESMC 600M derivative
Transformer layers	36
Hidden size	1,152
Attention heads	18
ESMCapsid representation	Layer 16, attention-mask mean pooling

Usage

Installation

pip install torch transformers tokenizers

Load the model and extract Layer 16 embeddings

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "Shuofang127/ESMCapsid-S"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
).eval()

batch = tokenizer(
    ["MSTNPKPQRKTKRNT", "MKTIIALSYIFCLVFA"],
    padding=True,
    return_tensors="pt",
)

with torch.inference_mode():
    output = model(**batch, output_hidden_states=True)

# hidden_states[0] is the input embedding; encoder Layer 16 is index 17.
layer16 = output.hidden_states[1 + 16]
mask = batch["attention_mask"].unsqueeze(-1).to(layer16.dtype)
embedding = (layer16 * mask).sum(dim=1) / mask.sum(dim=1).clamp_min(1)

print(embedding.shape)  # (batch_size, 1152)

For offline loading, replace model_id with the local model directory and set local_files_only=True in both from_pretrained calls.

Auxiliary classifier heads

The released classifier package is available at heads/two_stage_layer16_hardneg/. The final decision rule is:

normal_score >= 0.99 AND hardneg_score >= 0.9675

Use the thresholds in heads/two_stage_layer16_hardneg/two_stage_config.json as the authoritative release settings. The generic sequence-classification logits from the root model are not a replacement for the two-head decision rule.

Important

• This model is intended for protein-sequence representation and the ESMCapsid screening workflow.
• Predictions do not establish infectivity, host range, taxonomy, structure, or biological function.
• Keep the released layer, pooling method, normalization files, and thresholds together.
• This repository uses custom code and requires trust_remote_code=True. Review the code before loading it.
• The packaged modeling_esm_plusplus.py is the runtime implementation for this release; features added later to the base repository are not automatically supported here.

Citation

• Base model: Synthyra/ESMplusplus_large, DOI 10.57967/hf/3726.
• Hallee et al. FastPLMs: Bringing ESMC and ESM3 to Transformers. DOI 10.5281/zenodo.16466575.
• Liu, S., Xia, S., and Wang, H. Capsid-specialized protein language models reveal higher-order viral architecture from sequence. bioRxiv (2026). DOI 10.64898/2026.09.06.749605.

License

ESMCapsid-S is a derivative of ESM++ Large and ESMC 600M. Review the repository's LICENSE and NOTICE.txt before use or redistribution.