InterSeek 35M B128-E10
InterSeek predicts whether two proteins interact from their amino-acid sequences. This variant fine-tunes every layer of Meta's pretrained ESM-2 35M backbone and a CLS interaction head using an effective batch of 128 pairs for 10 epochs. Training completed 12,750 updates on 17 November 2026.
The released weights are the best validation checkpoint at update 6,375, after epoch 5. The final epoch scored lower on validation. The original batch-16, 3-epoch model remains available at appliedCL/interseek-35m.
Validation results
| Checkpoint | Pairs | AUROC | Average precision | MCC |
|---|---|---|---|---|
| Released: epoch 5, update 6,375 | 59,260 | 0.6543 | 0.6593 | 0.2262 |
| Final: epoch 10, update 12,750 | 59,260 | 0.6476 | 0.6542 | 0.2172 |
These are development validation results, used to select the checkpoint.
Held-out human Intra2 and independent D-SCRIPT yeast tests have not yet been
evaluated for this variant. The original model's published test results do not
apply to these weights. Exact metrics and all 22 validation passes are included
in selection.json and validation-history.json.
The original B16-E3 model achieved validation AUROC 0.6532296301704599, compared with 0.654300940078951 here. This single-seed comparison changes batch size and training duration and does not establish a reliable generalisation improvement.
Run inference
Requires Python 3.10 or later and the included InterSeek wheel. Install a CUDA-enabled PyTorch build for NVIDIA GPUs.
pip install huggingface_hub
hf download appliedCL/interseek-35m-b128-e10 --local-dir interseek-35m-b128-e10
cd interseek-35m-b128-e10
pip install -r requirements-inference.txt ./interseek-0.3.0-py3-none-any.whl
python predict.py --data example_pairs.csv --predictions predictions.csv --device cpu
Use --device cuda:0 for an NVIDIA GPU or --device mps for Apple Silicon.
Input CSVs require sequence_a and sequence_b; pair_id is optional.
The example sequences are illustrative and have no claimed interaction label.
Output preserves row order and includes pair_id, label (empty for unlabelled
input), and a score between 0 and 1. No pretrained-weight download is required
after downloading this release.
The validation-MCC threshold is 0.634100615978241. Scores have not been established as calibrated probabilities of physical binding.
model.safetensors contains the complete pair predictor and requires the
included implementation. It is not a standard Transformers classification
checkpoint. esm2/ separately contains the fine-tuned backbone:
from transformers import EsmForMaskedLM
backbone = EsmForMaskedLM.from_pretrained(
"appliedCL/interseek-35m-b128-e10", subfolder="esm2"
)
The backbone alone does not produce interaction scores.
Training and data
Both protein sequences are tokenised jointly as [CLS] A [EOS] B [EOS] with
up to 1,024 tokens and two window views. The CLS head produces view logits,
which are pooled using log-mean-exp. Inference averages the two input-order
logits before applying sigmoid, giving a symmetric interaction score.
Training uses binary cross-entropy plus masked amino-acid prediction weighted by 0.1, with mask probability 0.15. Settings: microbatch 4, accumulation 32, seed 17, AdamW, encoder/head learning rates 2e-5/1e-4, weight decay 0.01 and 100 warm-up updates. Hardware: NVIDIA H100 PCIe, BF16; PyTorch 2.8.0+cu128 and Transformers 5.12.1. Checkpoint selection uses validation AUROC; threshold selection uses validation MCC.
Training uses 163,192 human HIPPIE-derived protein pairs from the published Bernett gold-standard Intra1 split, with 81,596 positive and 81,596 negative labels. Validation uses 59,260 Intra0 pairs, also balanced. Sequences come from the authors' corrected Figshare version 4. The development split fingerprints exactly match the original B16-E3 experiment. Human Intra2 (52,048 pairs) and D-SCRIPT yeast (55,000 pairs) are held out.
Intended use and limitations
Use for protein-interaction research, candidate ranking and experimental follow-up. A score does not establish physical binding or causation. Sampled negatives, dataset biases, possible ESM-2 pretraining exposure and bounded sequence windows limit conclusions. Local audits check protein identifiers and exact sequences; no new all-against-all homology or pretraining-exposure audit was run. Additional seeds, matched baselines, held-out tests and experimental validation remain needed.
Release contents
model.safetensors: complete selected pair model, with checkpoint tensors unchanged.esm2/: fine-tuned ESM-2 backbone;interaction_head.safetensors: separate head.config.json: settings, source identity, checkpoint identity and threshold.predict.py, InterSeek wheel andrequirements-inference.txt: matching inference package.selection.json,validation-history.json,training-provenance.json: exact run records.study.jsonanddata-audit.json: pinned dataset provenance and audit.inference-verification.json: checkpoint/export equivalence and inference checks.SHA256SUMS: file-integrity checks.
The release contains inference weights, without optimiser state or training data. See source and experiment report. InterSeek and the upstream ESM-2 weights use the MIT license; both licenses are included.
- Downloads last month
- 4
Model tree for appliedCL/interseek
Base model
facebook/esm2_t12_35M_UR50D