iona-peptide-embedder-400m

iona-peptide-embedder-400m maps a peptide sequence (with modifications) and a precursor charge to a 2560-dimensional unit vector in the spectrum embedding space of iona-contrastive-400m. A spectrum and a candidate peptide can then be compared directly by cosine similarity, for spectrum-to-peptide retrieval or for scoring database-search candidates.

Model details

• Architecture: a small transformer (4 layers, hidden size 256, 8 heads; 4.77M parameters). Each residue is embedded from its amino acid, position and the precursor charge, plus Fourier features of its modification mass. The outputs are mean- and max-pooled and projected by an MLP.
• Input: peptides of up to 64 residues and a precursor charge (clamped to 0–7).
• Output: a 2560-dimensional, L2-normalized embedding.

Usage

import sys

path = "path/to/iona-peptide-embedder-400m"  # a local copy of this repository
sys.path.insert(0, path)
from peptide_embedder import PeptideEmbedder

peptides = PeptideEmbedder.from_pretrained(path).eval()
p = peptides.embed(["PEPTIDEK", "AC[57.0215]M[15.9949]K"], charges=[2, 2])  # (2, 2560)

# Embed spectra with iona-contrastive-400m (see its model card), then:
# scores = spectrum_embeddings @ p.T

Write modifications as a bracketed mass after the residue (C[57.0215], M[15.9949], N[0.9840]), and an N-terminal modification as a leading bracketed mass ([42.0106]PEPTIDE).

Training

The model was trained as a student to regress the frozen iona-contrastive-400m embeddings of each peptide's spectra (MSE between unit vectors). Training used the experimental spectra in the training split of a peptide-labeled retrieval dataset of experimental and consensus spectra, excluding peptides in a dataset of replicate spectra grouped by peptide: 3 epochs, learning rate 1e-3, seed 0. The released checkpoint is the best of 3 seeds by validation cross-modal Hit@1 (0.946).

Evaluation

Every experimental test spectrum of the retrieval dataset (25,848) was ranked against every test peptide–charge pair (9,771 candidates), with spectra embedded by iona-contrastive-400m.

Hit@1	Hit@5	MRR
0.9229	0.9489	0.9354

Across 3 seeds, Hit@1 ranged from 0.9226 to 0.9229. The same recipe with the 50m contrastive model as teacher reaches 0.898, so the student's quality tracks its teacher's.

Limitations

• The embeddings are only meaningful alongside spectra embedded by iona-contrastive-400m.
• Training and evaluation use splits of the same dataset. Retrieval quality drops substantially on low-resolution ion-trap MS2 spectra, which weren't in training.