VenusX: Unlocking Fine-Grained Functional Understanding of Proteins
Authors: Yang Tan
Wenrui Gou
Bozitao Zhong
Huiqun Yu
Liang Hong
Bingxin Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: protein substructure prediction
protein function prediction
molecule representation learning
pre-trained protein language model
fine-grained protein annotation
Primary area: datasets and benchmarks
TL;DR: We present VenusX, the first large-scale benchmark for fine-grained protein understanding, featuring over 878k annotations across 17 tasks for residues, fragments, and domains.

Abstract: Deep learning models have driven significant progress in predicting protein function and interactions at the protein level. While these advancements have been invaluable for many biological applications such as enzyme engineering and function annotation, a more detailed perspective is essential for understanding protein functional mechanisms and evaluating the biological knowledge captured by models. This study introduces VenusX, the first benchmark designed to assess protein representation learning with a focus on fine-grained intra-protein functional understanding. VenusX comprises three major task categories across six types of annotations, including residue-level binary classification, fragment-level multi-class classification, and pairwise functional similarity scoring for identifying critical active sites, binding sites, conserved sites, motifs, domains, and epitopes. The benchmark features over 878,000 samples curated from major open-source databases such as InterPro, BioLiP, and SAbDab. By providing mixed-family and cross-family splits at three sequence identity thresholds, our benchmark enables a comprehensive assessment of model performance on both in-distribution and out-of-distribution scenarios. For baseline evaluation, we assess a diverse set of popular and open-source models, including pre-trained protein language models, sequence-structure hybrids, structure-based methods, and alignment-based techniques. Their performance is reported across all benchmark datasets and evaluation settings using multiple metrics, offering a thorough comparison and a strong foundation for future research. Our code (https://github.com/ai4protein/VenusX), data (https://huggingface.co/collections/AI4Protein/venusx-dataset), and a leaderboard (https://ai4protein.github.io/venusx/) are provided as open-source resources.

Published as a conference paper at ICLR 2026
VENUSX: UNLOCKING FINE-GRAINED FUNCTIONAL UNDERSTANDING OF PROTEINS
Yang Tan 122,4 Wenrui Gou
Bozitao Zhong'
Huiqun Yu? Liang Hong 1.1
Bingxin Zhou' f
1 Shanghai Jiao Tong University.
2 Shanghai Innovation Institute.
3 East China University of Science and Technology.
Shanghai Matwings Technology Co., Ltd.
ABSTRACT
Deep learning models have driven significant progress in predicting protein fune- tion and interactions at the protein level. While these advancements have been invaluable for many biological applications such enzyme engineering and func- tion annotation, a more detailed perspective is essential for understanding pro- tein functional mechanisms and evaluating the biological knowledge captured by models. This study introduces VENUSX, the first benchmark designed to assess protein representation learning with focus on fine- -raigrai intra-proteis func- tional understanding. VENUSX comprises three major task categories across six types of annotations, including residue-level binary classification, fragment-level multi-class classification, and pairwise functional similarity scoring for identifying critical active sites, binding sites, conserved sites, motifs, domains, and epitopes. The benchmark features over 878, 000 samples curated from major open-source databases such as InterPro, BioLiP, and SAbDab. By providing mixed-famil and cross-family splits three sequence identity thresholds, our benchmark enables comprehensive assessment of model performance on both in-distribution and out- of-distribution scenarios. For baseline evaluation, we assess diverse of popular and open-source models, including pre-trained protein language models, sequence- structure hybrids, siructure-base methods, and alignment- performance is reported across all benchmark datasets and evaluation settings using multiple metrics, offering thorough comparison and strong foundation for future research. Our code, data, and a leaderboard are provided as open-source resource
techniques. Their
INTRODUCTION
Deep learning has significantly advanced the analysis of large-scale protein data, enabling efficient solutions key inference tasks across sequence, structure, and function (Zhou et al., 2024a). No- table successes include structure prediction (Jumper et al., 2222 Abramson al. (024), sequence engineering (Lu et al.) 2022] Zhou et al. 2024b Tan et al. 2025b), and functional annotation (Yu et al. 2023 Wang et al. 2026)) The rapid progress in this field is supported not only by the models' scientific and practical value, but also by the availability of high-quality benchmarks that define clear learning objectives and ensure fair, reproducible evaluation.
wide range of datasets and evaluation protocols have been developed to facilitate model training and assessment, especially those centered large-scale protein sequence and structure data (Orengo ltt al. 1997; Varadi et af.] [022] Consortium] (025). While some benchmarks include functional annotations, they predominantly target protein-level properties, where the goal is assign a single label to an entire protein or protein pair. Some representative tasks include function annotation (Xu
al. 2022 Tan al. 2024b YE al. 2025
et al. 2025), protein protein-protein protein interaction prediction
'Equal contribution first authors.
Corresponding authors (bingxin.zhol @ww...dcusuuu hongl3liang (eoinn@@stt.chiee sjtu.ccu..c)..
Cooe VenusX Github), dataset (VenuxX Huggingface). and leaderboard VenusX Wehsite

---

Published as a conference paper at ICLR 2026
(a) source data collection
InterPro
HERE antbody
SAbDab
NI
postive
domain
active sites
epitope
HTH rost
A G
Sinding
label
motit
evolutioanry pressure
bindingssite
BioLiP
(b) benchmark task category
(c) processing and split strategies
reskdue-level binary classification
fragment-level multi-lcass classification
prrtens
Raggarts
S0COCCOOOOOCC. Panii wwppsseeiiiinnnnnnooo Hwm.in CENDAAC
O0OOOOCOOCOOOO
Villogg chistrrig
Presi www.soaesssssoooooooooooo
0,,00 3 CO
HTH 1MM
mix-family split
croes-family split
pairwise functional similarity scoring
Figure I: Overview of the VENUSX benchmark. (a) Six types of functional annotations collected
from InterPro, BioLiP, and SAbDab (Section[] (b) Three benchmark task categories: residue-level
and fragment-level classification, and pairwise similarity scoring (Sections 3.1 3.3 (c) Sequence
identity- clustering and mix-family and cross-family data split strategies (Sections 3.4).
(Pan et al. 2010; Szklarczyk et al. 2019 Jankauskait et al., 2019 Liu et al. 2025), and protein
fitness estimation (Gray et al. 2018 Riesselman et al. [018]] Notin et al.) 20224] Zhang et al.) 20025).
Despite the overwhelming focus protein-level benchmarks, biological functions are often governed by specific subregions within proteins rather than the entire molecule. Global labels can obscure mechanistic details and may even lead models to rely on biologically implausible features for prediction. This increases the risk of overfitting to noise, reduces interpretability. and compromises accuracy in tasks where local features are critical, such as function annotation (Cagiada et al. 2023 Lee et al. 2007) and paratope design (Attique et al., 2023). As a result, there is a growing demand for benchmarks that support supervision and evaluation at fine-grained resolution. It is essential not only for advancing functional understanding but also for systematically assessing how well learned representations capture biologically meaningful signals beyond sequence similarity.
We address this gap by introducing VENUSX, the first large-scale and biologically grounded bench- mark for fine-grained protein understanding. VENUSX spans multiple subprotein devels-including residues, motif', fragments, and domains-ann and is designed to evaluate model performance across three task categories: (1) residue-level binary classification, which assesses whether individual amino acids contribute critically to protein function, such as catalysis. ligand binding, evolutionary constraint, or domain boundaries: (2) fragment-level multi-class classification, which identifies functional subregions within protein and assigns them to specific biological roles; and (3) pairwise functional similarity scoring, which matches functionally similar proteins or substructures without requiring explicit function labels.
The raw residue-level annotations are sourced from three high-quality databases: InterPro (Paysan Lafosse et al.) 2023), BioLiP (Yang et al. 2012), and SAbDab (Dunbar et al.) 2014). We curate over 878, 000 high-confidence samples and form diverse tasks across three categories of fine-grained functional prediction. To enable comprehensive evaluation of model fitness, robustness, and general- izability, we consider both label distribution and input similarity at the fragment and protein levels and define multiple evaluation setups with different partitioning strategies for training and testing.
2

---

Published as a conference paper at ICLR 2026
We benchmark broad spectrum of popular protein representative models to assess their effectiveness
on VENUSX. These include pre-trained protein language models (Rives et al. 2021| Elnaggar et al.
2021 2023 Heinzinger et al. 2023 Lin et al. 2023: Hamamsy et al. (022). J. sequence-struccte
hybrid models (Su et al. 2023 Tan et al. 2025c), inverse folding models (Yng et al. 2023 Hsu
al. 2022), structure- maucture-based geometric networks (Jing et al. (02T), and traditional alignment- based
methods (Altschul et al. 1990, Zhang & Skolnick 2005; Van Kempen et al., (024). We observe
substantial variation in performance across annotation types, sequence identity thresholds, and task formulations. These findings reveal that strong performance on conventional global protein-level tasks does not necessarily translate to fine- ane-grained functional understanding. The results further suggest that many current models rely heavily on global or distributional cues, rather than capturing precise, localized biological signals. These limitations highlight the need for future model designs that are better aligned with the demands of fineigrained benchmarks, which emphasize robustness, generalization across protein familics, and biological interpretability.
In summary, while prior benchmarks have focused protein-level properties, VENUSX establishes the first comprehensive. multi-level benchmark for fine-grained protein understanding by introducing suite of tasks the residue, fragment, and domain levels. It offers biologically meaningful evaluation dimensions for future protein models and enables systematic assessment of their ability to capture true biological knowledge. All datasets, task definitions, evaluation protocols, and baseline leaderboards are publicly available to support followup research in the future.
DATA COLLECTION AND CURATION
We collect residue- and fragment-level annotations from InterPro (Paysan-Lafosse et al. 2023), BioLiP Yangg et al., 2012), and SAbDab (Dunbar et al., 2014), followed by thorough cleaning, redundancy removal, identity-based -baeed clustering, and alignment with structure and annotations.
2.1 INTERPRO: FUNCTIONAL ANNOTATIONS ACROSS DIVERSE PROTEIN FAMILIES
We use InterPro (Paysan- Lafosse et al. 2023) to collect residue- entries from various annotation categories, such as active sites (Act), binding sites (Bind), conserved sites (Evo), functional motifs, and domains, detailed definitions for each of these annotation types are provided in Appendix Section A.1 For each category, metadata are retrieved from InterPro Website in json format, including InterPro family identifiers, associated Gene Ontology (GO) terms, and annotated residue positions. The raw data of UniProt identifiers and their corresponding functional annotations are downloaded by VenusFactory (Tan et al., 2025a). Canonical protein sequences are obtained from UniProt [Consortium, (2025), and predicted structures are retrieved from the AlphaFold Protein Structure Database (Varadi et al. 2022), retaining only entries with available structural models.
To ensure non-redundancy, we remove additional entries with identical annotated fragments. If protein contains multiple distinct fragments annotated with the same function, the annotations are consolidated into a single entry. These fragments, along with their aligned sequence-structure pairs, form the basis for the various downstream benchmark tasks.
2.2 BIOLIP: EXPERIMENTALLY DERIVED LIGAND BINDING SITES
We incorporate additional residue-level binding site annotations from BioLiP (Yang et al. 2012). curated resource of pigtei interactions derived from experimentally resolved complexes in the Protein Data Bank (PDB) (Burley et al.) 2019). Binding residues are identified using distance-based criterion. A residue is labeled as part of the binding site if any of its atoms lie within the sum of the Van der Waals radii of the interacting atom pair plus a 0.5 5 Å empirical margin. This approach captures steric interactions and enables robust identification of physiologically relevant binding interfaces.
For each entry, we extracted the complete receptor sequence, its PDB chain identifier, the annotated binding site residues, and the ligand identity specified by its Chemical Component Dictionary (CCD) code. Sequence and coordinate information were parsed directly from the corresponding PDB files, and ligand annotations were stringently curated to retain only biologically relevant molecules. This
https: htwp://wwwww/... ebi ac. wtw.:eebb..c..u./eerrroo..
3

---

Published as a conference paper at ICLR 2026
Table Summary of the residue-level classification tasks, For sequence length, number of positive residues, and proportion of positives, averages are shown with standard deviations in parentheses.
Target
Deseription
Proteins
Seq. Len.
# Positive
% Positive
Cross
Mix
Act
active sites (InterPro)
667
482.5 (349.0)
16.7 (7.0)
4.6 (2.8)
BindI
binding sites (InterPro)
8, 959
486.5 (399.8)
24.5 (20.9)
7.9 (7.3)
BindB
hinding sites (BioLiP)
115, 505
348.7 (272.9)
9.4 (9.6)
4.1 (5.7)
Evo
evolutionary pressure (InterPro)
59, 948
366.9 (200.8)
23.4 (13.1)
10.4 (9.2)
Motif
motif (InterPro)
10, 271
595.2 (388.5)
78.0 (73.4)
20.2 (22.8)
Dom
domain (InterPro)
595, 454
537.5 (373.2)
169.2 (117:2)
40.3 (26.1)
Epi
epitope (SAbDab)
5,370
374.9 (201.3)
24.8 (10.5)
10.6 (9.2)
curated dataset augments the InterPro-derived entries with high-resolution protein-ligand binding-site pairs, providing biologically meaningful benchmark for training and evaluating deep learning models on protein-ligand complexes.
2.3 SABDAB: ANTIBODY-INDEPENDEER EPITOPE PREDICTION
This benchmark introduces task for seuibody-independdee epitope prediction, well-established and critical challenge in computational biology that is pivotal for accelerating the design of next- generation therapeutic antibodies and vaccines (Cia et al. 2023 Zeng al. 2023 Carroll al. 2024). The task aims to identify an antigen's 's intrinsically bindable or druggable regions, which serve high-potential targets for therapeutic antibodies. This approach is motivated by the observation that functional antibodies often target specific, high-affinity regions rather than the entire exposed protein surface (Hastie al., [021I. To construct dataset for this task, we extract epitope annotations from antibody-antigee complexes curated in SAbDab (Dunbar et al. (0014,, restricting our analysis to entries where the antigen protein. Metadata are sourced from curated tsv files from SAbDab Website including antibody heavy and light chains, as well as the associated antigen chains. Structures are parsed using BioPython (Cock et al. 2009). We only retain standard residues with defined C. coordinates. An antigen residue is considered part of the epitope if the Euclidean distance between its C. atom and any antibody C. atom is less than 10 Á. This geometric criterion captures both continuous epitopes (adjacent in sequence) and conformational epitopes (spatially clustered but sequence sequence-distant).
We treat antigens with identified epitope residues from this procedure as valid entries. For each of them, extract the complete amino acid sequence, the epitope residue indices (start from 0), and associated structure files. Entries with non-protein antigens, missing chain information, or structural inconsistencies excluded. The resulting dataset offers structre-deriver epitope labels aligned with sequence data to support targeted development and evaluation of immune-related protein models.
BENCHMARK TASKS
3.1 RESIDUE-LEVEI BINARY CLASSIFICATION
Task Description The first category of tasks focuses on identifying functionally important residues within a protein. Each task framed as a binary classification problem at the residue level, where positions labeled either funetionally relevant (positive) or irrelevant (negative). Related functions involve catalysis, binding, or other biological processes (Section 2 Table summarrzee all 7 subtasks. Unlike conventional protein-level function prediction, these tasks assess whether models can detect critical residues independent of global function or family context. They encourage models to go beyond coarse representations and capture residue-level signals that may support model analysis in terms of interpretability or explanability. In practice, accurate residue-level predictions can aid enzyme engineering, mutational effect analysis, and active-site redesign.
https://opig. stati. statts....... http://s..........////////////aa
4

---

Published as a conference paper at ICLR 2026
Table 2: Summary of the 5 fragment-level multi-class classification tasks.
Table Summary of the five pairwise similarity scoring tasks (in millions).
Protein
Fragment
Target
# Fragments
Seq. Len.
# Class
Target
Positive
Negative
Positive
# Negative
Act
9, 767
18.7 (7.0)
132
Bindl
562
26.6 (21.7)
1.3
45.4
46.4
Bindl
3.5
36.6
5.0
50.8
Evo
66, 916
25.5 (13.3)
740
Evo
7.7
1, 789.1
10.0
2, 2,2.9
Motif
13, 244
80.23 (73.8)
454
Motif
2.4
50.3
81.7
Dom
656, 669
171.3 (117.4)
13, 459
Dom
217.7
177, 06-1.
346.0
215,260.
Labeling Strategy For these residue-level classification tasks, adopt specific labeling approach. If a protein contains multiple. spatially distinct sites annotated with the same function (e.g., several separate binding sites), all residues belonging any of these sites labeled as positive (1), while all other residues are labeled as negative (0).
Evaluation Metrics These tasks exhibit strong class imbalance, as most residues are non ffnffunctiinall (see "% Positive" in Table T To better assess model performance the minority class, prioritize metrics that emphasize positive predictions. Specifically, we report class-specific precision, recall, and F1 score for the positive class, along with the under the precision-rects curve (AUPR).
3.2 FRAGMENT-LEVEL MULTI-CLAS, CLASSIFICATION
Task Description The second category task evaluates a model's s ability to classify pre-identified functional regions. Unlike remote homology detection (Rao et al. 2019), which typically uses full-length protein as input, this task provides only the annotated fragment sequence. The objective is assign this fragment to its corresponding InterPro family (see Table 2 This reflects a practical two- stage inference pipeline that first locates functionally relevant regions, then assigns them to functional families. Since proteins often contain multiple functional fragments, the task also supports multi-labe (InterPro family) annotations and encourages models to capture compositional functionality. It also bridges residue-level predictions with interpretable biological labels grounded in existing ontologies. Crucially, inputs for this task are continuous sequence motifs (e.g.. signature sequences defined by InterPro). We do not artificially concatenate non-consece residues. Consequently. tasks defined solely by 3D spatial proximity without conserved 1D) motifs (e.g., BioLiP binding sites, SAbDab epitopes) are excluded from this category and handled exclusively residue-level tasks.
Labeling Strategy The label for each functional fragment is its corresponding InterPro family ID, as curated from the source database. Since a single protein region can occasionally be assigned to multiple overlapping InterPro families, the task is formulated multi-label classification problem, though most instances have single label.
Evaluation Metrics These tasks assign each functional fragment to its corresponding InterPro family, which may include hundreds tens of thousands of distinct classes (see "# Class" in Table 2]. We report accuracy (ACC), macro-averaged precision, recall, F1 score, and Matthews correlation coefficient (MCC). We take ACC and macro-F1 as the primary metrics to reflect both overall correctness and class-balanced performance.
3.3 PAIRWISE FUNCTIONAL SIMILARITY SCORING
Task Description This third category of tasks evaluates how well learned representations can capture functional similarity in zero-shot, unsupervised setting. Given a pair of inputs (either proteins or fragments), the goal is for model to produce similarity score reflecting their functional relatedness. This task provides a retrieval-style evaluation of representation quality, particularly for identifying subtle but biologically relevant similarities. It has important practical value in applications such as enzyme mining, remote homolog detection, and functional clustering in metagenomic datasets. full dataset summary is provided in Table 3
5

---

Published as a conference paper at ICLR 2026
Labeling Strategy The ground truth for this task is defined by InterPro family membership. A pair of proteins or fragments is considered a positive sample both members are annotated with the same InterPro family ID. Conversely, pair is considered negative sample if they belong to different InterPro families.
Evaluation Metries Pairwise similarity scoring tasks are evaluated using the area under the ROC curve (AUC). We use cosine similarity between protein representations as the similarity score for embedding- methods. For alignment-baset methods that explicitly compute sequence or structure alignments, we employ the negative logarithm of the E-value (e-g., for FOLDSEEK (Van Kempen al. 2024) and BLAST (Altschul et al. 1990)) or the bi-directional average TM-score (e.g., for TM-ALIGN (Zhang Skolnick 2005)).
3.4 PARTITIONING PROTOCOL FOR TRAINING AND EVALUATION
Classification Tasks We evaluate both in-distribution and out-if-disttibutiin prediction perfor- mance. To this end, we construct mix-family and cross-family data splits for both residue-level and fragment-level tasks. (1) Mix-family splits assess in-distribution generalization by randomly partitioning proteins (or fragments) into training, validation, and test sets in an 8:1:1 ratio, without considering family assignments. (2) Cross-family splits evaluate out-of -ittr--diittibutionn generalization by assigning entire InterPro families to training, validation, and test sets in the same 8:1:1 ratio. For both strategies, first apply MMseqs22 clustering (Steinegger & Söding 2017) at 50%, 70%, and 90% sequence identity thresholds to reduce redundancy before splitting. Note that, due to limitations in available data and family annotations across the original source databases, mix-family splits on pro- teins are applied to datasets from all three sources, In contrast, fragment- level splits and cross-family splits based on family identity are applied only InterPro-sourcee datasets. Detailed availability and train/ wwl.iaaaiiinntestttt set statistics are provided in Tables 9 in Appendix Section A.5
Similarity Scoring Tasks As the third task category of pairwise functional similarity scoring does not involve model supervision, we do not perform data partitioning. Instead, following a similar strategy to (Tan al. 2024c), uniformly subsample complete dataset for evaluation. This random subsampling is necessary due to the combinatorially large number of possible protein pairs in the similarity scoring task (for instance, see Table B) Specifically, for both pairing tasks on fragments and proteins, we randomly sample 10, 000 positive pairs (i.e., proteins from the same InterPro family) and 10, 000 negative pairs (i.e., proteins from different InterPro families) as one evaluation dataset. We repeat the procedure using three different random seeds, and the final performance scores are averaged across the three repetitions.
of positive and negative pairs from the
3.5 NAMING PROTOCOL FOR BENCHMARK DATASETS
VENUSX includes total of 56 datasets. For clarity, each dataset is named by VENUSE category! [ [split], where each part denotes the task category, prediction target, and data split strategy.
Following the construction options introduced in Sections 3.1 3.4
[category] refers to the task category, with three choices of Res, Frag, and Pair to represent Residue-level tasks, fragment-level tasks, and pairwise scoring tasks.
[target] represents the 7 cases of targets, including Act (active sites), BindI (binding sites from InterPro), BindB (binding sites from BioLiP), Evc (evolutionary pressure), Motif (func- tional motif), Dom (functional domain), and Epi (epitope sites).
[sp] it] denotes the partitioning strategies, with x for cross-family and M for mix-family splits, P and F indicating protein or fragment, and the final number representing the clustering threshold.
For instance: (1) VENUUXX_Reee Res__eeee______ee_ee is a residue-level binary classification task that predicts binding sites (from BioLiP), using cross-family split. (2) VENUSX_FFrs X___Frr__ct_____ 9F90 denotes a fragment-level multi-class classification task targeting active sites, with a mix-family split on fragment entries clustered 90% identity. This benchmark is designed for community expansion, and we welcome contributions. Upon acceptance, a public repository will be made available for contributions via Pull Request. The primary data requirement format including seq_full full

---

Published as a conference paper at ICLR 2026
Table 4: Residue-level classification performance across datasets and data splits. "MFS0" and "MP50" refer to mixed-family splits with 50% sequence identity filtering applied to fragments and proteins, respectively. Top-1, Top-2, and Top-3 results for each target dataset are highlighted, respectively. Models are grouped by input modality. AUPR scores for each task are reported, and detailed results the Appendix Section
are provided Tables 19 21
Target Split
Sequence-only
Sequence-Structuug
Structure-only
ESM2-130
ESM2-r33
PROTBERT
ANKII-BASE
SAPRO1-35M
SAPROI-650M
PROTSSN
GVP-GNN
MF50
0.852
0.764
0.688
0.745
0.465
0.523
Act
MP50
0,932
0.955
0.895
0,960
0.905
0.945
0.917
0.898
0.143
0.131
0.114
0.185
0.156
0.101
MF50
0.912
0.904
0.857
0.907
0.807
0,838
0.801
0.611
Bindl
MP50
0.963
0.971
0.926
0.970
0.927
0.960
0.907
0,883
Cross
0,133
0.159
0.112
0.145
0.230
0.182
0.182
0,040
MF50
0,862
0.899
0.771
0.895
0.724
0.734
0.715
0.342
Evo
MP50
0.897
0.926
0.803
0.932
0.775
0.912
0.895
0,792
Cross
0.235
0.262
0.243
0.275
0.272
0.274
0.227
0.101
MF50
0,855
0.779
0.267
0,802
0.716
0.661
Motif
MP50
0.850
0.857
0.796
0.870
0.784
0,841
0.765
0.736
Cross
0.433
0.456
0.348
0.394
0,408
0,441
0.390
0.329
MF50
0.634
0.606
0.591
0.673
0.574
0.642
0,560
Dom
MP50
0.645
0.657
0.592
0.665
0.584
0.640
0.557
Cross
0.470
0.506
0.508
0.449
0.525
0.564
0,468
MP50
0.409
0.446
0.340
0.421
0.574
0.388
0.301
BindP
MP70
0.410
0.487
0.330
0.389
0.337
MP90
0.496
0.535
0.466
0.527
0.354
0.411
0.374
MP50
0.186
0.174
0.169
0.182
0.195
0.118
Epi
MP70
0.1841
0.202
0.177
0.215
0.194
0.237
0.200
0,139
MP90
0.277
0.290
0.266
0.270
0.256
0.308
0.274
0.196
field for the full-length sequence and a label field for the corresponding residue-level annotation array, with detailed instructions provided in the supplementary materials.
4
EXPERIMENTS
4.1
EXPERIMENTAL SETUP
Model Setup For both residue- and fragment-level classification tasks, pretrained sequence-based models (e.g., ESM2 (Lin et al. (023), PROTBERT (EInaggar et al. 202T)) and sequence-structure models (SAPROT (Su al. 2023), PROTSSN (Tan al. (2025c]) are used frozen feature extractors. This protocol isolates the intrinsic quality of pre-trained representations from fine-tuning confounders and ensures computational accessibility this large-scale benchmark. In contrast, the structure- based GVP-GNN (Ting et al.) (021) is trained from scratch with all parameters updated, this approach aligns with standard evaluation protocols for geometric GNNs (Jamasb et al. (2024) and ensures fairness in computational scale. For residue-level tasks, the encoders output embeddings of each residue, which are passed through two linear layers with ReLU activation and dropout. For fragment- level classification, mean pooling is applied to obtain fragment representations for InterPro family prediction. pair-level similarity evaluation, full-length sequences or fragments are encoded, mean- pooled, and compared via similarity metrics to assess family-level relationships. Parameter statistics for all models are provided Table 13 Appendix Section
Training Setup For all tasks, full-length protein sequences are truncated to a maximum of 1022 residues. For sequence-structuse models, 3D structure files were truncated in parallel include only the atoms of the retained residues, ensuring perfect capped 128 residues for Act, Bindl, Evo, and Motif, and at 512 residues for Dom. All models are trained with fixed random seed of 3407 to ensure reproducibility. Optimization is performed using AdamW (Loshchilov et al. 2017) with a learning rate of 0.001 and an effective batch size of 128 via gradient accumulation. Training proceeds for up 100 epochs, with early stopping triggered validation performance does not improve for 10 epochs. For residue-level and fragment- classification tasks, AUPR and accuracy the validation set are used early stopping criteria, respectively. All experiments are conducted on 16 NVIDIA RTX 4090D GPUs and 192 Intel(R) Xeon(R) Gold 6248R CPUs with 2 TB of memory for 45 days.
mapping. Fragments are
7

---

Published as a conference paper at ICLR 2026
Table 5: Fragment-level classification performance across InterPro datasets and data splits under 50% sequence identity. 1op-1. Top-2, and Top-3 results for each metric are highlighted, respectively. Detailed results are provided Table 22 of Appendix Section
Target
Metric
Sequence-only
Sequence-Structure
Structure-only
ESM2-130
ESM2-135
PROTBERT
ANKH-BASE
SAPROT-35M
SAPROT-630M
PROTSSN
GVP-GNN
Act
Macro-F1
ACC
0.647
0.819
0884
0.609
0.736
0.647
0.824
0,807
0,928
0.825
0.928
0.764
0.897
0,906
0.907
BindI
Macro-F1
ACC
0.937
0,913
0934
0.790
0.927
0.900
0.718
0.976
0.986
0.957
0,972
0.884
0.972
Evo
ACC
0.853
0.841
0.828
0. 0.866
0.939
0.950
0,915
0.14
Macro-F1
0.667
0.669
0.627
0.716
0.849
0.863
0.793
0.757
Motil
ACC
0,884
0.884
0.901
0,901
0.927
0.914
0.807
Macro-F1
0.457
0,542
0.452
0.499
0.504
0.552
0.556
0,370
Table 6: AUC (%) of baseline models on InterPro family alignment under two evaluation settings: F50 (fragment-level inputs with 50% sequence identity filtering) and P50 (full fell-sequence inputs with 50% identity). Models are grouped by modality. Cell colors indicate ranking: Top-1, To Top-4. Standard deviation over three folds is shown in parentheses.
Top-2,
Model Information
Act
Bindl
Evo
Motif
Dom
Name
Version
F50
P50
PS0
P50
F50
P50
F50
PS0
P50
P50
Allgnment-bused Methods
FOLDSERK
3DI-AA
3Di
96 1m.3)
96 (0.0
90.5ya.2
92. 2g
92. G
80.1ga2)
80,6
88. 4,az1
88.3.0
99.0g.. 11
90.0 11
74.7
71.8
64.7
61.9jo.))
TM-ALIGN
maan
91 91.6
90.1go.3
67.7.o.11
76.0
BLAST
9/..2)
7l.7
32.4pz1
51.4gam
54.0ka.)
49.9
56.2
Sequence-only Encoder Methods
130
69.4,a.5
69. 20.2
77.6
65.5
52.4
$7.5
84.3
68.2
78.0waz)
77.4gam
50.2
70.0
73.0gu
62.8pa))
19.3,0.5
59.0ka.a)
92. 92.1.0.3)
66.1
62.2
66.10
136
65.8
72.9
7133g3)
67.6
63.9,a.1)
82.1.a.m
90.1
70.0
66.5ma.)
66.7.am
ESM-1B
07.6,0.2)
73.8
84
69.8(a
57.0ka.)
58.4
87.2.00)
58.4
89.2
74.7
PROTBERT
bid
71.4g..)
68.7
84.9
66.8pm
84.6
84.2
85.1 85.1
68. 68.2
85.3
77.9
Sequence-only Encuder- Decnder Methods
PROFTS ANKH
xL ALunire550
91 8)..1)
78.1
98.5
77.4
71.0ga2
85.6
98. 2
67.6)o.3)
98 08.5go
$5.1
base
69 6,0.5)
NJ.4 W0.4
91.8pm
63.9)a-
98.3
86.7
69 60.77p..
47.6
88.0
Soquence- structure
SAPROT
35M AP2
95 81.8
74.0 74.6...3
68.2
94.3
g8.1
71.9(a2)
71.1g.1)
61. 61.9ka.)
62.6çam)
82.7
83.8
85.3
98.9
68.6,o.2
68.3
96.0ro.1)
91.7.o.1
76.1.02:
78.8
PROYSSN
K20 h512
79. 79.1
64.8)o.)
88 48.4
61.2(a.)
60.9ka44
86:2g..1
72.4
64.0)o.2)
82.9ku.)
60-400.0
ESM-IF
96.5
70.2:02:
95.0ga.)
65.6ça1))
61.3kaa)
90.6.0.4
80.4m.33
68.0)o.2)
97.1
70.5.022
MIEST
66.9ho.5
65.9ha.:
(6.1
50.2a.
61.3ka.3
S0.3ra.21
50.2/0.4)
68.3,0.4)
78.6ra.)
06.77a.0
TM-VRC
swiss, large AAZfold
93.6...3
80 80.9
98.0
82.4,00)
67.4/a2)
96.2
90 00:4.0.0
71.
98.2.0.)
50.9
PROSTTS
90.8,o.3)
80.7
98.0
79.2pum)
65.6ka.)
98.2gg.
98.5
69 (/.88
98.5pa43
79.3
4.2 EVALUATED METHODS
The summary of all baseline models, including architecture type, version, task scope, parameter count, and implementation source, is presented in Table T3 Appendix.
Classification Baselines For the residue-level and fragment-level classification tasks, we evaluate a set of pretrained models spanning diverse architectures. These include sequence-based language models such as ESM2 (Lin et al., 2023), PROTBERT (Elnaggar et al., 2021 and ANKH (Elnaggar et al.
hybrid models such as SAPROT (Su et al. 2023) and PROTSSN (Tan et al. 2025c); and a structure-only geometric network, GVP-GNN (Jing ct al. 2021).
2023);
Similarity Scoring Baselines For the pair-wise similarity evaluation task, we extract mean embed-
dings from pretrained language models, including ESM2 (Lin et al. 2023) PROTBERT (Elnaggar
et al.
2021). ESM-18 (Rives et al. 2021), PROTTS (Elnaggar et al. 2021). I), ANKH (Elnaggar et al.
2023
TM-VEC (Hamamsy al. 22024. and two inverse folding models, MIS-ST (Yang et al. (023)
and ESM-IF1 (Hsu et al.) 2022). For traditional alignment- based methods, we include BLAST
Altschul et al. 1990) (sequence alignment), TM-ALIGN (Zhang & Skolnick 2005) (structure align-
8

---

Published as a conference paper at ICLR 2026
ment), and FOLDSEEK (Van Kempen et al. 2024.. which supports both structure-only (3Di) and joint sequence-structure (3Di-AA) comparisons.
4.3 RESIDUE-LEVEL BINARY CALSSIFICATION
Table 4 lists AUPR on seven residue-annotation benchmarks under mixed-family (in-distribution) and cross-family -out-of-distribution) splits. Some experiments are omitted due to missing structural inputs or prohibitive computational costs. Key observations are:
Language models perform strongly on in-distribution splits. ANKH-BIR HIKH-BASE attains the high- est AUPR on 7/15 InterPro Mix tasks, while ESM2-T33 dominates the BindP and Epi mixes, confirming that sequence sufficient when test proteins remain close to the training set.
Sequence--tructture models generalize better for unseen families. SAPROT-650M achieves the best or second-bes performance across all InterPro Cross splits, and shows notable advantage in domain-level classification (e.g., +5.6% AUPR over PROTBERT).
Cross Cross-familyy residue prediction remains highly challenging. On Act and Bindl the best AUPR plummets by 70-80% in Cross, whereas Dom drops by < 10%, suggesting that catalytic and binding residues are harder to extrapolate than domain-wide patterns.
Dataset properties strongly affect difficulty. Mix splits from InterPro relatively well- structured and easier to predict. In contrast, Epi remains extremely difficult-no no model achieves an AUPR above 0.3 across all sequence identity levels.
4.4 FRAGMENT-LEVEL MULTI-CLASS CLASSIFICATION
Table reports ACC and Macro-F1 on four InterPro targets (Act, Bindl, Evo, Motif) with 50% sequence-identity filtering.
Sequence-structure models are consistently superior. SAPROT-650M or PROTSSN ranks first on 7/8 metrics; e.g. SAPROT-650M outperforms the strongest protein language model (ESM2-T33) on Act by +11.4% ACC and +22% Macro-F1.
Structure-only models show strong task-specific performance. GVP-GNN matches alignment- aware models on Act and Bindl (Macro-F1 0.906 / 0.884) but lags on Motif, indicating pure structure is task-dependent.
Sequence models are more sensitive to class imbalance. While ACC exceeds 80%, their Macro- F1 15-20%% lower; sequence sequence-structure structure models cut this gap to 100, showing better robustness to skewed label distributions.
4.5 PAIRWISE FUNCTIONAL SIMILARITY SCORING
Table reports AUC (%) for family-level alignment under two low-identity conditions-FS50 (frag- ment inputs clustered at 50% identity) and P50 (protein inputs at 50% identity). Because exhaustive structural alignment (e.g., TM-ALIGN) is prohibitively expensive at this scale, several entries are left blank; baseline models description and full command lines are given in Appendix Section B.2
Structure- based aligners remain the gold standard. FOLDSEEK delivers near-pertie near-perfect perfor- mance, topping 3/8 settings and peaking 99.0% AUC on Evo _50. TM-ALIGN is likewise competitive when evaluated, whereas sequence-only BLAST trails by > 40% on every task, underscoring the advantage of structural information.
Large decoder models close much of the gap. PROTT5 attains 98. 33.5% on Bindl [50 and 98. on Motif_FSS outperforming all pure sequence encoders (ESM2, ESM- 1B, PROTBERT) by 7-20% and surpassing "TM-ALIGN three fragment settings. Ankh shows similar strength full-sequence inputs (Act_ 90.4% vs. 69-74% for vanilla encoders).
Sequence-structuue hybrids are highly alignment-aware. TM-VEC reaches top-2 ranking in 5 of 10 cells (e.g.. 99,4% on Motif P50: 98.2% Dom (omm__50),,___) while SAPROT-6500 650M attains 98. 1% on Bindl P50. These results indicate that injecting structural inductive bias enables LM embeddings to rival specialised aligners fraction of the computational cost.

---

Published as a conference paper at ICLR 2026
5 RELATED WORK
Protein-wise Tasks A variety of benchmarks have been developed to support machine learning on protein sequence and structure data. Early efforts such as TAPE (Rao al. 2019) and ProteinNet (ATQuraishi 2019) focused sequence-level tasks including secondary structure prediction, contact prediction, and remote homology classification. More recently, benchmarks like PEER (Xu et al. PETA (Tan et al.) 2024b), VenusFactory (Tan et al. 2025a), and ProteinGLUPF (Capel et al. 2022) introduced multi-task evaluations for protein sequence understanding. emphasizing sequence- Khurana et al.) 2018 Envision (Gray et al. 2018), DeepSequence (Riesselman et al., 2018), and ProteinGiym (Notin
2022).
level predictions across diverse annotation types (Almagro Armenteros et al. 2huraKhurana
al. 2024) advanced lcrge-scale evaluation for fitness prediction under zero-shot or supervised mode, while FLIP (Dallago et al.) (2021) curated different split strategies (e.g., one-vs-rest, which trains models on single mutations and tests on the rest of the high-order mutations) to cover various scenarios. ProteinShake (Kucera et al., 2023) standardized structural datasets and task formulations across graph. point cloud, and voxel- based representations for protein structures. ProteinBench (YE et al. 2025) evaluates various methods for different tasks such as protein structure prediction, sequence design, structure design, requence-structurg design, and molecular dynamics.
Protein-pair Tasks Protein- pair modeling tasks encompass a broad spectrum of physical and functional interactions. PEER (Xu et al. (022) includes interaction classification in human (Pan al. 2010) and yeast (Guo et al. 2008) PPI networks, as well as affinity regression using SKEMPI (Jankauskaite et al., (019). Structural datasets such as MaSIF (Gainza et al.) (2220) and DIPS-pul ppus (Morehead et al. 2023) provide high-quality annotations of protein-protein interfaces, enabling al.
geometric modeling of interaction surfaces. The recent HDPL pocketome (Moine-France 2024) expands this scope by offering pocket-centric structural data related to PPIs and PPI-related ligand binding sites. Functional networks like STRING (Szklarczyk al. 2019) support large-scale classification of biological associations. In the and the Ligand Binding Affinity (LBA) tasks in Atom3D (Townshend al. 2020) provide affinity labels derived from co-crystal structures. While these resources facilitate pairwise prediction and interaction modeling, they generally lack residue-level supervision, limiting their utility in evaluating fine-grained functional inference.
domain, PDBbind (Wang et al.. (005)
DISCUSSION AND CONCLUSION
This work introduces VENUSX, a large-scale benchmark for assessing protein representation learning at fine-grained residue and fragment resolutions. By targeting precise localized signals rather than coarse global labels, VENUSX enables rigorous evaluation of mechanistic understanding.
Our extended experiments demonstrate that traditional aligmment methods (BLAST) cannot effectively transfer fine-grained functional labels (AUPR 0.04, Appendix ( validating the necessity of deep representation learning. We further show that siguence-structure models significantly outperform sequence-one baselines in low-identity settings (e.g.. MP30), suggesting that structural priors are critical for generalization when sequence homology is weak (Appendix D.4]. However, current models perform poorly on epitope prediction, revealing limitations in reasoning about conformational and
features. Cross-task correlations confirm that these fine-grained capabilities are distinct (Appendix(C)
Reliability was verified through stability analysis across random seeds and ablation studies predicted structure fidelity (Appendix D.II. VENUSX provides the community with a biologically grounded benchmark emphasizing rigorous out-of-distribution evaluation, serving resource for advancing functional protein modeling.
REPRODUCIBILITY AND ETHICS STATEMENT
Reproducibility Statement To ensure reproducibility. all code, data, and a leaderboard are made publicly available as open-source resources We detail our data curation in Section 2 task definitions
*Code VenusX Github), dataset Venuux Huggingface) and leaderboard (VenuxX Wehsite).
10

---

Published as a conference paper at ICLR 2026
and splitting protocols in Section B and experimental setup with hyperparameters in Section 4.1 and
the Appendix (Section B
Ethics Statement Our benchmark constructed exclusively from public, anonymized protein databases and involves no human subjects. We adhere to the Code of Ethics and acknowledge that our data may reflect historical biases from these sources, a point further discussed in our dataset analysis.
The Use of Large Language Models (LLM) In the preparation of this manuscript, we utilized LLMs, specifically Gemini 2.5 Pro and ChatGPT 4o, as writing assistants in the preparation of this manuscript. To refine the text, prompted the models with role-playing instructions, such as "Act as a professional scientific editor...". The use these tools was strictly limited to improving grammar, clarity, and overall readability. All scientific ideas, experimental results, and conclusions were conceived and formulated exclusively by the authors. All text polished or modified by the LLM was subsequently reviewed and edited by the authors ensure that the original scientific meaning was accurately preserved.
ACKNOWLEDGEMENTS
This work was supported by the grants from the AI for Science Program by Shanghai Municipal Commission of Economy and Informatization 2005-GZL-RG22--BBBX-X22 Foundation of China (62302291), and the National Key Research and Development Program of China (2024YFA091764)) 1603)).
National Science
REFERENCES
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature, pp. 1-3, 2024.
José Juan Almagro Armenteros, Casper Kaae Sønderby, Søren Kaae Sønderby, Henrik Nielsen, and Ole Winther. DeepLoc: prediction of protein subcellular localization using deep learn- ing. Bioinformatics, 33(21):3387-3395 2017. URL http::s//aaaadmmc..oom oup. com/
/333185
Mohammed AlQuraishi. ProteinNet: standardized data set for machine learning of protein structure. BMC Bioinformatics, 20:1-10, 2019.
Stephen F Altschul, Warren Gish, Webb Miller, Eugene W Myers, and David J Lipman. Basic local alignment search tool. Journal of Molecular Biology, 215(():403-410) 1990.
Muhammad Attique, Tamim Alkhalifah, Fahad Alturise, and Yaser Daanial Khan. Deepbce: evalua- tion of deep learning models for identification of immunogenic b-cell epitopes. Computational Biology and Chemistry, 107:747 2023.
Stephen K Burley, Helen M Berman, Charmi Bhikadiya, Chunxiao Bi, Li Chen, Luigi Di Costanzo, Cole Christie, Ken Dalenberg. Jose Duarte, Shuchismita Dutta, et al. RCSB Protein Data Bank: biological macromolecular structures enabling research and education in fundamental biology, biomedicine, biotechnology and energy, Nucleic Acids Research, 47(D)) D474, 2019, URL
Matteo Cagiada, Sandro Bottaro, Søren I Lindemose, Signe M Schenstrøm, Amelie Stein, Rasmus Hartmann-Petersen, and Kresten Lindorff-Larsen. Discovering functionally important sites in proteins. Nature Communications, 14(1):4175, 2023.
Henriette Capel, Robin Weiler, Maurits Dijkstra, Reinier Vleugels, Peter Bloem, and K Anton Feenstra. ProteinGLUUE multi-task benchmark suite for self-supervised protein modeling. Scientific Reports, 12(1):16047, 2022.
Moshe Carroll, Ezra Rosenbaum, and R Viswanathan. Computational methods to predict conforma- tional b-cell epitopes. Biomolecules, 14(8):983, 2024.

---

Published as a conference paper at ICLR 2026
Gabriel Cia, Fabrizio Pucci, and Marianne Rooman. Critical review of conformational b-cell epitope prediction methods. Briefings in Bioinformatics, 2.bbbbbb, 2023.
Peter JA Cock, Tiago Antao, Jeffrey T Chang, Brad A Chapman, Cymon J Cox, Andrew Dalke, Iddo Friedberg, Thomas Hamelryck, Frank Kauff, Bartek Wilczynski, et al. BioPython: freely available python tools for computational molecular biology and bioinformatics. Bioinformatics, 25(11):
1422, 2009.
UniProt Consortium. UniProt: the universal protein knowledgebase in 2025. Nucleic Acids Research, 53(D1):D609-D611, 2025. htRp:ttts:///at /accdemm C. oup. coom haa/ http://ataleeeee////e////
D6C9/7902999
Christian Dallago, Jody Mou, Kadina E Johnston, Bruce Wittmann, Nick Bhattacharya, Samuel Goldman, Ali Madani, and Kevin K Yang. FLIP: Benchmark tasks in fitness landscape inference for proteins. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Truck (Round 2), 2021.
James Dunbar, Konrad Krawezyk, Jinwoo Leem, Terry Baker, Angelika Fuchs, Guy Georges, Jiye Shi, and Charlotte M Deane. SAbDab: the structural antibody database. Nucleic Acids Research, ((DD)):D140-DDI 146, 2014.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: Toward understanding the language of life through supervidd learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(10)4))))))) 1112712777 2021.
Ahmed Elnaggar, Hazem Essam, Wafaa Salah- Ealah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Char- lotte Rochereau, and Burkhard Rost. Ankh: Optimized protein language model unlocks general- purpose modelling. MXiv:2301.00668 2023.
Pablo Gainza, Freyr Sverrisson, Frederico Monti, Emanuele Rodola, Davide Boscaini, Michael M Bronstein, and Bruno Corrcia. Deciphering interaction fingerprints from protein molccular surfaces using geometric deep learning. Nature Methods, 17(2):184-192 2020.
Vanessa E Gray, Ronald Hause, Jens Luebeck, Jay Shendure, and Douglas M Fowler. Quantitative missense variant effect prediction using large-scale mutagenesis data. Cell Systems, 6(1):116-124 2018.
Yanzhi Guo, Lezheng Yu, Zhining Wen, and Menglong Li. Using support vector machine combined with auto covariance to predict protein-protein interactions from protein sequences. Nucleic Acids Research, 360)):3025-3030, 2008.
Tymor Hamamsy, James T Morton, Robert Blackwell, Daniel Berenberg. Nicholas Carriero, Vladimir Gligorijevic, Charlie EM Strauss, Julia Koehler Leman, Kyunghyun Cho, and Richard Bon- neau. Protein remote homology detection and structural alignment using deep learning. Nature Biotechnology, 42(6))975 2024.
Kathryn M Hastie, Haoyang Li, Daniel Bedinger, Sharon L Schendel, Moses Dennison, Kan Li, Vamseedhar Rayaprolu, Xiaoying Yu, Colin Mann, Michelle Zandonatti, et al. Defining variant- resistant epitopes targeted by sars-cov-2 antibodies: A global consortium study. Science, 374 (6566):472-478, 2021.
Michael Heinzinger, Konstantin Weissenow, Joaquin Gomez Sanchez, Adrian Henkel, Martin Steineg- ger, and Burkhard Rost. ProstT5: Bilingual language model for protein sequence and structure. bioRxiv, pp. 2023-07, 2023.
Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures. In International Conference on Machine Learning, pp. 8946-8970. PMLR, 2022.
Arian Rokkum Jamasb, Alex Morehead, Chaitanya K. Joshi, Zuobai Zhang, Kieran Didi, Simon V Mathis, Charles Harris, Jian Tang, Jianlin Cheng, Pietro Lio, and Tom Leon Blundell. Evaluating representation learning the protein structure universe. In The Twelfth International Confer- ence on Learning Representations, 2024. URL bttps:://penzeet
net/ forum?idd
3TYuRV=dK3
12

---

Published as a conference paper at ICLR 2026
Justina Jankauskaité, Brian Jiménez- Jiménez-Garefa, Justas Dapkünas, Juan Gemández-Recio, and lain H Moal. Skempi 2.0: an updated benchmark of changes in protein-preee protein binding energy, kinetics and thermodynamies upon mutation. Bioinformatics, 35(3):462-44) 42022246622000 2019.
Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. In International Conference on Learning Representations, 2021.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Židek, Anna Potapenko, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583-
2021.
Sameer Khurana, Reda Rawi, Khalid Kunji, Gwo- Yu Chuang, Halima Bensmail, and Raghvendra Mall. DeepSol: a deep learning framework for sequence-based protein solubility prediction. Bioinformatics,
URL 34(15):6005-661333 03 2018. ISSN 1367-4803. doi: 10. 00.100%/hoonfformaticcshyyy cttt////
Tim Kucera, Carlos Oliver, Dexiong Chen, and Karsten Borgwardt. ProteinShake: Building datasets and benchmarks for deep learning on protein structures. In Thirty-seventh Confer- Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL Tttt:::://penzeview..et net/ 2tgpaaeeeewwiiaatooooooo2222222
https:
David Lee, Oliver Redfern, and Christine Orengo. Predicting protein function from sequence and structure. Nature Reviews Molecular Cell Biology, 8(12):95-10055, 2007.
Song Li, Yang Tan, Song Ke, Liang Hong, and Bingxin Zhou. Immunogenicity prediction with dual attention enables vaccine target selection. In The Thirleenth International Conference Learning Representations, 2025. URL /tpphtppss//ppeeeeieeww /ownrevieeeeeeeeeeee
net/ forum wttee//oormmmmdttmii.i.... Wid=:Whww,,iiii
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, al. Evolutionary--saaee prediction of atomic-level protein structure with a language model. Science, 379(6637): 120-3111::12211333: 123-1130, 2023.
Chen Liu, Mingchen Li, Yang Tan, Wenrui Gou, Guisheng Fan, and Bingxin Zhou. Sequence-only prediction of binding affinity changes: a robust and interpretable model for antibody engineering. Bioinformatics, 4 b188)bb 2025.
Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam. arXiv: 1711.50010000000 5, 2017.
Hongyuan Lu, Daniel Diaz, Natalie J Czarnecki, Congzhi Zhu, Wantae Kim, Raghav Shroff, Daniel Acosta, Bradley Alexander, Hannah O Cole, Yan Zhang, et al. Machine learning-aided engineering of hydrolases for PET depolymerization. Nature, 604((907):662 6667, 2022.
Alexandra Mline-FFranel Fabien Mareuil, Michael Nilges, Constantin Bogdan Ciambur, and Olivier Sperandio. comprehensive dataset of moteinn-roteein interactions and ligand binding pockets for advancing drug discovery. Scientific Data, 11(1):002 2024.
Alex Morehead, Chen Chen, Ada Sedova, and Jianlin Cheng. DIPS-plus: The enhanced database of interacting protein structures for interface prediction. Scientific Data, 10(1):509, 2023.
Pascal Notin, Aaron Kollasch, Daniel Ritter, Lood Van Nickerk, Steffanie Paul, Han Spinner, Nathan Rollins, Ada Shaw, Rose Orenbuch, Ruben Weitzman, et al. ProteinGym: large-scale benchmarks for protein fitness prediction and design. In Advances in Neural Information Processing Systems, volume 36, 2024.
CA Orengo, AD Michie, S Jones, DT Jones, MB Swindells, and JM Thortoo CATH a hierarchic classification of protein domain structures. Structure, 588):1093-1109, 1997. ISSN 0969-2126. doi: https://doi.org/I1
1ttp//0.100 100.011660 0/800699-122699777
Xiao Yong Pan, Ya-Nan Zhang, and Hong-Bi Bin Shen. Large-scale prediction of human protein- protein interactions from amino acid sequence based on latent topic features. Journal of Proteome Research, 49900,19,,, 2010.
13

---

Published as a conference paper at ICLR 2026
Typhaine Paysan-Lafosse Matthias Blum, Sara Chuguransky, Tiago Grego, Beatriz Lázaro Pinto, Gustavo A Salazar, Maxwell L Bileschi, Peer Bork, Alan Bridge, Lucy Colwell, et al. InterPro in 2022. Nucleic Acids Research, SL0DI):D488-D4A 2023. URL htttps://ccadmmic... oup
com ht////ar/aaa http:///aaratcceee/////////
8/ 68144444
Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Peter Chen, John Canny, Picter Abbeel, and Yun Song. Evaluating protein transfer learning with TAPE. Advances in Neural Information Processing Systems, 32, 2019. URL
cC/ cctp:/cccpppppppppa http://ppprrrllleepppeee//////// http//99pper//11/hahhhh/
Adam J Riesselman, John B Ingraham, and Debora S Marks. Deep generative models of genetic variation capture the effects mutations. Nature Methods, 15(10):816-22) 2018.
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, Lawrence Zitmick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning 250 million protein sequences. Proceedings of the National Academy of Sciences, (18(15)e20113391 18, 2021.
Martin Steinegger and Johannes Söding. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature Biotechnology, 35(11):1066-102 2017.
Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. SaProt: protein language modeling with structure-awar vocabulary. In The Twelfth International Conference on Learning Representations, 2023.
Damian Szklarczyk, Annika L. Gable, David Lyon, Alexander Junge, Stefan Wyder, Jaime Huerta- Cepas, Milan Simonovic, Nadezhda T Doncheva, John H Morris, Peer Bork, al. STRING v1 protein-protein association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets. Nucleic Acids Research, 470DI):D077-D613 2019. URL htttps://acadent oup. httpp/////////aaa..aa/////////// 9147779776611177766
Yang Tan, Mingchen Li, Bingxin Zhou, Bozitao Zhong, Lirong Zheng, Pan Tan, Ziyi Zhou, Huiqun Yu, Guisheng Fan, and Liang Hong. Simple, efficient, and scalable structure-aware adapter boosts protein language models. Journal of Chemical Information and Modeling, 2024a.
Yang Tan, Mingehen Li, Ziyi Zhou, Pan Tan, Huiqun Yu, Guisheng Fan, and Liang Hong. PETA: evaluating the impact of protein transfer learning with sub-word tokenization on downstream applications. Journal of Cheminformatiis, 16 16(1):92, 2024b. URL https::/ https:///jcheinn
biomedcentral. cwm wtt.c//tttttcee..ce. 1tt.::ccceeee...111111111131 13322 01/111///3311112 024 0088444--0--333
Yang Tan, Lirong Zheng, Bozitao Zhong, Liang Hong, and Bingxin Zhou. Protein representation learning with sequence information embedding: Does it always lead to a better performance? In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 233-239. IEEE, 2024c.
Yang Tan, Chen Liu, Jingyuan Gao, Wu Banghao, Mingchen Li, Ruilin Wang, I Lingrong Zhang, Huiqun Yu, Guisheng Fan, Liang Hong, al. VenusFactory: An integrated system for protein engineering with data retrieval and language model fine-tuning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pp. 230 241, 2025a.
Yang Tan, Ruilin Wang, Banghao Wu, Liang Hong, and Bingxin Zhou. From high-throughput evaluation to wet-lab studies: advancing mutation effect prediction with retrieval-enhanced model. Bioinformatics, 41, 07 2025b. ISSN 1367-4811. httphh//s/// http://///o..../1111//39333 wloinnormttics.///o btt.ic/////at./gg///
Yang Tan, Bingxin Zhou, Lirong Zheng, Guisheng Fan, and Liang Hong. Semantical and geometrical protein encoding toward enhanced bioactivity and thermostability. eLife, 13:RP98033, may 2025c. ISSN 2050-084X. doi: 10.7554/cLife.99e 98033. pRLpiptsss//oo..... 70111.0..5555ee 98633
14

---

Published as a conference paper at ICLR 2026
Raphael JL Townshend, Martin Vögele, Patricia Suriana, Alexander Derry, Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Bowen Jing, Brandon Anderson, Stephan Eismann, et al. Atom3d: Tasks molecules in three dimensions. arX1v:2012.04035, 2020.
Michel Van Kempen, Stephanie S Kim, Charlotte Tumescheit, Milot Mirdita, Jeongjae Lee, Cameron LM Gilchrist, Johannes Söding, and Martin Steinegger. Fast and accurate protein structure search with Foldseek. Nature Biotechnology. 42(2):243-244 2024.
Mihaly Varadi, Stephen Anyango, Mandar Deshpande, Sreenath Nair, Cindy Natassia, Galabina Yordanova, David Yuan, Oana Stroe, Gemma Wood, Agata Laydon, et al. Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic Acids Research, 50(D)) 2022. URL https://
academic.oup com/ hawr//aa////// httpa//ateee//////3/////////////
Renxiao Wang, Xueliang Fang, Yipin Lu, Chao-Yie Yie Yang, and Shaomeng Wang. The PDBbind database: methodologies and updates. Journal of Medicinal Chemistry, (1(1)):411-41111 2005.
Zhiyu Wang, Bingxin Zhou, Weishu Zhao, Yang Tan, Jing Wang. Pietro Lio, and Liang Hong. Fast and interpretable protein substructure alignment via optimal transport. In The Fourteenth International Conference on Learning Representations, 2026. URL URL littpss//operreviees
n Eorum/idu wiw.mmiiidddlllieiinn
Minghao Xu, Zuobai Zhang, Jiarui Lu, Zhaocheng Zhu, Yangtian Zhang, Chang Ma, Runcheng Liu, and Jian Tang. PEER: comprehensive and multi-task benchmark for protein sequence understanding. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Truck, 2022.
hhtttps::/pppeeeeeetee http::ee///fooo 2id= zJou
Jianyi Yang, Ambrish Roy, and Yang Zhang, BioLiP: semi-manually curated database for bio- logically relevant ligand protein interactions. Nucleic Acids Research, (1(D1):D1096-DI1 2012.
Kevin K Yang, Niccolò Zanichelli, and Hugh Yeh. Masked inverse folding with sequence transfer for protein representation learning. Protein Engineering, Design and Selection, 36:gzad015, 2023.
Fei YE, Zaixiang Zheng, Dongyu Xue, Yuning Shen, Lihao Wang, Yiming Ma, Yan Wang. Xinyou Wang, Xiangxin Zhou, and Quanquan Gu. ProteinBench: holistic evaluation of protein founda- tion models. In The Thirteenth International Conference on Learning Representations, 2025. URL
htttps:/opeereview/ net forum?iddi Bks
Tianhao Yu, Haiyang Cui, Jianan Canal Li, Yunan Luo, Guangde Jiang, and Huimin Zhao. Enzyme function prediction using contrastive learning. Science, 379(6699): 1358-13633 2023.
Xincheng Zeng, Ganggang Bai, Chuance Sun, and Buyong Ma. Recent progress in antibody epitope prediction. Antibodies, 12(3):52, 2023.
Liang Zhang, Hua Pang, Chenghao Zhang, Song Li, Yang Tan, Fan Jiang, Mingchen Li, Yuanxi Yu, Ziyi Zhou, Banghao Wu, et al. Venusmuthub: a systematic evaluation of protein mutation effect predictors on small-scale experimental data. Acta Pharmaceutica Sinica B, 2025.
Yang Zhang and Jeffrey Skolnick. TM-align: protein structure alignment algorithm based on the tm-score. Nucleic Acids Reseurch, 33(7):2302-2309 2005.
Bingxin Zhou, Yang Tan, Yutong Hu, Lirong Zheng, Bozitao Zhong, and Liang Hong. Protein engineering in the deep learning era. mLife, 3(4):477-491, 2024a.
Bingxin Zhou, Lirong Zheng, Banghao Wu, Kai Yi, Bozitao Zhong, Yang Tan, Qian Liu, Pietro Liò, and Liang Hong. A conditional protein diffusion model generates artificial programmable endonuclease sequences with enhanced activity. Cell Discovery, 10(1):95, 2024b.
15

---

Published as a conference paper at ICLR 2026
A DATASET
A.1 DEFINITIONS OF INTERPRO ANNOTATION TYPES
To ensure clarity for the benchmark tasks, we provide the definitions for the functional site annotation types sourced from the InterPro database:
Domain: Domains are distinct functional, structural or sequence units that may exist in a variety of biological contexts. A match to an InterPro entry of this type indicates the presence of domain. Common examples of protein domains are the PH domain, Immunoglobulin domain or the classical C2H2 zinc finger.
Motif: short, conserved sequence of amino acids that is associated with specific function. Motifs are typically shorter than domains and may not fold into stable structure on their own. They often serve as signatures for a particular protein family or function.
A short sequence that contains one or more conserved residues, which allow the protein bind to a ligand and earry out a catalytic activity.
Active Site:
Binding Site: short sequence that contains one or more conserved residues, which form a protein interaction site.
Conserved site: A short sequence that contains structural or functional importance.
conserved residues, typically implying
A.2 SEQUENCE LENGTH DISTRIBUTION
Table 7: Summary of fragment and full-sequence length statistics for each InterPro category.
Target
Type
#Samples
Min
Max
Mean
Median
Fragment Protcin
9,767
39
16.54
14.00
Act
9,667
2,753
482.51
379.00
Fragment 10.562
100
20.75
19.00
Bindl
Protein
8,959
45
2,631
486.54
415.00
Fragment 66,916
149
20.93
19.00
Evo
Protein
59,948
16
2,896
365.8 888888
305.00
Fragment
653,259
600
150.54
124,00
Dom
Protein
595,443
16
2,988
537. .41
444.00
Fragment 13,137
228
$7.02
62.00
Motif
Protcin
10,271
30
2,699
595.15
558.00
Figures to and 7 the length distributions of annotated protein fragments (top) and their corresponding full-length protein sequences (bottom) across five InterPro categories. To facilitate visualization, outliers were excluded: domain fragments longer than 600 residues, motif fragments exceeding 230 residues, and full-length proteins longer than 3000 residues.
Several trends can be observed:
At the fragment level, Act, BindI, and Evo sites exhibit relatively short lengths, typically under 50 residues, with Act sites showing clear multimodal peaks due to specific catalytic motifs. Dom and Motif fragments show broader distributions, with domain fragments exhibiting long tail.
In contrast, full-length proteins follow typical long-tailed distribution across all categories, with most proteins under 1000 residues but a small number extending beyond 2000. The distributions are highly skewed, especially in the Dom and Evo datasets, reflecting the diversity of protein sizes.
These distributions motivate the design of separate fragment- and full-sequence benchmarks, as the input length significantly impacts model performance and scalability.
16

---

Published as a conference paper at ICLR 2026
Length Distribution of Fragments and Full Sequences (Clustered Fragment Similarity)
Active
Binding
Evolutionary
Domain
Motif
250
Length
1000 196 2030
1:0
1983 1469 20di Pids
13:0 Longth
2012
10:30 Longth
2003
19: (A3y 1a0 Add Length
Length
Longe?
Length
Longen
Length
Longth
Longth
Lengen
Lengin
Length
Length
Length
12-
780
550
1909
3090
1099 1559 3000 2059
1000
1008
1000
2000 2003
1908
5X
1250 190 Longth
Longth
Longt)
Longth
Longth
Figure 2: Sequence length distribution of the VENUSX InterPro benchmark under different levels of sequence similarity, clustered by fragment.
Table 8: Statistics of InterPro type label frequency across five categories, computed separately on annotated fragments and full sequences.
Target
Type
#Types
Min
Max
Mean
Median
Fragment
132
1
1176
73.99
37.50
Act
Protein
132
1172
73.23
37.50
Fragment
76
2
2293
138.97
53,50
BindI
Protein
76
2
2279
117.88
53,00
Fragment
740
1418
90.43
36.00
Evo
Protein
740
1074
81.01
31.50
Fragment
12,529
12002
52.14
6.00
Dom
Protein
12,580
8109
47.33
6.00
Fragment
440
2222
29.86
3.00
Motif
Protein
454
1247
22.62
3.00
A.3 INTERPRO LABEL DISTRIBUTION
Based Table Figures and 6 analyze the distribution of residue-leve labels across InterPro categories. Due to the extreme imbalance class sizes, we apply a log-scale transformation when visualizing the number of annotations per InterPro type.
17

---

Published as a conference paper at ICLR 2026
Length Distribution of Fragments and Full Sequences (Clustered by Full-sequence Similarity)
Active
Binding
Evolutionary Pressure
Domain
Motif
Length
Length
1000 196 2030
1d0 1983 1953 3033 PiDs
2012
12: Longth
10:0 1430 180 Add
Length
Longeh
Longth
Length
Length
Longe7
Length
Longth
Longth
Length
Lorgin
Length
Length
Length
730
15
500 100 1050 1000 3000
500 1963 1959 3003 2999
1865
1008
1000
1250
5000
50 1905
1968 2900 Longth
1100
Longth
Longe)
Longth
Longth
Figure 3: Sequence length distribution of the VENUSX InterPro benchmark under different levels of sequence similarity, clustered by full sequences.
Severe long-tail distribution across datasets. All five datasets exhibit substantial class imbalance. For example, the domain task has over 12,000 InterPro types, but median of only 6 annotated proteins per type at both the fragment and full-sequence level. Similarly, motif labels have a median count of 3, emphasizing the prevalence of rare classes.
Skewed distribution dominated by few frequent families. In most datasets, small number of InterPro types contribute a disproportionate number of samples. For instance, the top 5 types the binding site fragments account for over 5,000 samples, while many other types appear fewer than 10 times.
Fragments exhibit slightly denser annotation than full sequences. Across all datasets, the mean and median counts per InterPro type are consistently higher at the fragment level than for full sequences. This suggests fragments are more focused on annotated functional regions, whereas full sequences dilute sparse labels across longer chains.
These trends reinforce the importance of using macro-averaged metrics and highlight the difficulty of for tasks such as domain
learning under Mlas--mbalanced. fine-grained label and motif classification.
A.4 DISTRIBUTION OF FUNCTIONAL DOMAIN START POSITIONS
Figures 7 and 8 show the distribution of relative start positions of protein functional domains. For an underlying protein, relative start position describes the normalized start location of its functional
18

---

Published as a conference paper at ICLR 2026
Binding (BioLip) (90%)
Binding (Biolip) (70%)
Binding site (Biolip) (50%)
25-
Count (M
Length
Epitope (SAbDab) (90%)
Epitope (SAbDab) (70%)
Epitope (SAbDab) (50%)
6.3
0.2
02-
Count 1k)
42
0.2
03
01
0.5
300
130
1297 Length
1000
1500
130 1000
2250
1997
902
750
1003
3150
Length
Length
Figure 4: Sequence length distribution of BioLip and SAbDab benchmarks under different levels of sequence similarity, clustered by full sequences.
region divided by the full sequence length, with the range between 0 (start of the sequence) and 1 (end of the sequence). Distribution of starting positions reveals that different functional site types exhibit strong and distinct positional preferences along the sequence (e.g.. active sites tend to appear around the middle of the sequence, while motifs favor the termini). This positional bias may introduce misleading signals for models trained on full-length sequences. When the positional distribution of functional domains in the training data is skewed toward one end of the sequence, models may overfit to these positional cues, limiting their ability to generalize to more diverse or unbiased scenarios. This underscores the need for our fine-grained benchmark to ensure that models learn true functional patterns rather than relying on location-bart based shortcuts.
A.5 DATASET NUMERICAL SPLIT DETAIL
Table Residue-level: Number of train/validdaiinn/ess examples under mixed-family (Mix50) and cross-family splits. "Fragment" and "Protein" refer to clustering at the fragment and full-sequence levels, respectively.
wwl./aaaaldaiaeeeeec (Mix50)
Train/ Vallddtiion Test (Cross)
Target
Duta Source
Irragment
Protein
Family
Protein
Act
InterPro (Paysan-I_afone Lafosse
2075
1-88/186/186
2.92//66/367
104/14/14
1,701/8001086
Bindl
InterPro (P55506- Laloss 0t
2025
1.640/205/205
2,366/296/296
60/8/8
7,729/551/679
BindB:
Biol.P
2012
19,412/2.42672 42162222226/244277
Evo
InteePro Pavsan-
et
2023
10,383/1.29811,3 298
13,552/1,604/1.06
542/74/74
48,437/5 445/6.006
Motif
InterPro
P2VSin--aaosse 21
2023
2.008/251/2511
2.720/340/34)
362/46/46
7,799/ .045/1.427
Paysan Lalosse
94,459/10.66/10.5622
113,607/1,,21//4.000
10.065/1.55/1.26/
477.149/56.60066..00
Epi
SAhDah
Duithar e: 2014
828/103/104
Pre-filtering and Clustering. For all InterPro-based datasets, we apply a pre-filterigg step to remove sequences lacking predicted structures from the AlphaFold Protein Structure Database (Varadi et al. 2022). ensuring structural consistency for downstream evaluations. Following this, we perform sequence identity clustering using MMseqs2 (Steinegger Söding (017)) under varying identity thresholds (50%. 70%, and 90%) construct non-redundant splits at both the fragment and full-sequence levels. Clustering is conducted with a coverage mode of (query coverage), and a minimum coverage of 0.8.
19

---

Published as a conference paper at ICLR 2026
Distribution of InterPro Label Frequencies (Clustered Fragment Similarity)
Active Site
Binding
Evelutionary Pressure
Domain
Motif
107
12
197
321
107
127
101
107
12"
123
209
187
127
107
161
Raggeri
12
121
107
107
107
10
122
70%
124
100
131
100
137
50%
10:
192
107
197
10
102
101 10" 10.
33
101
Fragment
Full
Figure 5: InterPro label distribution the VENUSX InterPro benchmark under different levels of sequence similarity, clustered by fragment.
Table 10: Residue-level: Number of train/validationntess examples under mixed-famil at 70% and 90% sequence identity. "Fragment" and "Protein" refer clustering the fragment and protein levels.
# Traain/Validutionn/IN Test (Mix70)
Truin/ wwwp/aaaaaiaillscccc Test (Mix90)
Target
Data Source
Fragment
Fragment
Act
InterPro
Payyan Lafosse et
2023
1,724/340/341
5,378/672/673
5,269/6598559
1,279/110/100
Bindl
InterPro
PSySe arose
20245
3,016/377/37
4,432/554/555
6,30666336644
6,428/033/804
BindB
BioLiP
142/3,142
Evo
InterPho
[Sysan- Lalose
2024
17,4592. 182/2.83
42,860/5 355/5,359
Motif
InterPro
L 3r05g
2024
3.539/42/443
-L6244578/5799
S. 5.172/68446885
6, 6,71L58839/8110
Dom
InterPro
160.201/20.22520..266
215,652/26.955 9152//26957/26695
263,870/32, 0.3.870052.88/12..96
348, 445.9444:436688443..66
Epi
SAbDab
999/124/125
1,524/190V191
Residue-level Split Settings. Tables and 10 summarize the number of train/validation//est ex-
amples under different split strategies. Table 9 reports counts under the mixed- mixed-family (Mix50) and fromily splits, where fragment-level and full full-sequence clustering are applied separately. Table further breaks down the mixed-family splits at 70% and 90% sequence identity thresholds.
20

---

Published as a conference paper at ICLR 2026
Distribution InterPro Label Frequencies (Clustered by Full-sequence Similarity)
Active Site
Binding
Evolutionary Pressure
Domain
Motif
121
197
12
127
107
124
9000 124
13
185
209
162
127
107
100
101
122
121
70%
10"
70N
12
102
100
107
10-
121
100
107
127
121
10
121
125
107
027
322
101 10" 10
33
Fragment
Full
Figure 6: InterPro label distribution the VENUSX InterPro benchmark under different levels of sequence similarity, clustered by full sequences.
Table 11: Fragment-level: Number of train/validationntest examples under mixed-family at 50%, 70%, and 90% sequence identity clustering at the fragment level.
Train/ Siiniaaaaaaainneesss
Target
MF50
MF90
Act
1.545/191/193
2.777/352/358
5,344/670/670
BindI
2.558/294/287
4,075/51
6,487/830/826
Evo
12,880/1,5966 16661
21,140/2,604/2.55
38.736/4,99 91044,844
93.736/4.91044.66
Motif
3,083/362/372
5.123/630/589
7.516/949/9228
Dom
10/31
300,661/37,533/77,,
InterPro-base datasets support all three types of splits, while BioLiP and SAbDab only provide mull-sequence annotations and thus are limited to full-protein splits. Dom and conserved datasets are the largest in scale, enabling more comprehensive evaluations across clustering thresholds.
21

---

Published as a conference paper at ICLR 2026
Distribution Functional Domain Start Postiions (Clustered
Active
Sinding
Exclut Pressure
Domain
Nodi
Activo Site
Brnding Site
Exdiut cnary Pressure
Oumain
M507
SC
32
Figure 7: Protein functional domain start position density the VENUSX InterPro benchmark under different levels of sequence similarity, clustered by fragment.
Table 12: Detailed number information of the unsupervised pair similarity evaluation task. "# Protein- P/N" and "# Frag-P/N" denote the total number of positive and negative pairs sampled by Protein sequences fragments within InterPro families. "# Protein/Frag-pooe denotes whether the structures of protein sequences or fragments are available.
Data Source
Protein-I
Prottein-N N
Frag-F
FFrag-N
Protei--pdb Frag-pal
Act
Payaand Lafosse ot ml. 2016
1,314,757
45,405
1,331,729
46,360.512
BindI
InterPro
Payaus 21083 dee 20163
3,550,288
36,577,073
4.957.073
50.815.568
Evo
InterPro
7,710,941
1.789.140, 140,437
9.990.111
2,228,851,
InterPho
7024
2,4152212
50.326.373
5,962.485
81,732,661
Dom
IntetPro
Poyst5 L10055 ct dl. 2024
217.681.629
177.064.753.702
346.047.386
215.260.712.060
V
Fragment-level Split Settings. Table II presents the number of train/validation/tss examples across five InterPro targets under mixed-famil family splits with increasing sequence identity thresholds (50%, 70%, and 90%) applied at the fragment level. As expected, raising the identity threshold increases the number of retained fragments, approximately doubling the dataset size from MF50 to
22

---

Published as a conference paper at ICLR 2026
Distribttion
(Clistered
Similar
Active Site
Binding Evolutionayy Peessure
Domale
Mod!
30
M
Active Site
Biiding Si6
Evoucoonry Prossure
Domals
M507
64
Vall
434
Figure 8: Protein functional domain start position density the VENUSX InterPro benchmark under different levels of sequence similarity, clustered by full-sequence.
MF90. For instance, Act expands from 1.5k to 5.3k training fragments, while Dom scales from 105k to over 300k. This progression supports finer-grained control over redundancy and task difficulty, enabling evaluation across spectrum of local similarity conditions. The setting facilitates analysis of model robustness to fragment diversity and homologous signal dilution.
reports the number of positive and negative pairs for unsupervised similarity evaluation. Positive pairs share the same InterPro family, while negatives are drawn from different families. All tasks exhibit a strong imbalance, especially in large-scale domains (e.g., 177 billion negative pairs). Structural coverage remains high across both full sequences and fragments, enabling comprehensive evaluation under both sequence- and structure-based settings.
Pair-level Statistics. Table 12
23

---

Published as a conference paper at ICLR 2026
Table 13: Summary of baseline models by input modality. "Task" indicates evaluation scope: "All" denotes all three tasks, "Sup." refers to supervised classification tasks only, and "Pair" to the pairwise functional similarity scoring task. We report model type, version, parameters, embedding size, and implementation source (via Hugging Face, GitHub, or Conda).
Input
Model
Version
Task
# Params
Truin. Params
Embed. Dim
Implementation
(30
All
150M
410K
640
HF: ESM2-300
ESM2
1.6M
136
Pair.
3,000M
2,560
HP: ESML-11
ESM-18
Rives al 2021
(33
Pair.
662M
1,280
HP: ESM 16
Soguence-Oal
PROTBeg
PROTTS
Elaapaare 2024
uniref50
420M
1.0M
HP:
ANKH
paperon at F1 7074
basc
All
450M
591K
768
HF: Ankhi
TM-Vit
2014
swiss large
Pair.
3,054M
512
Luirnan: 1 M.
PROSTTS
AAZfold
BLAST
Alsseil drall 1950
Pair
Conda: BLAST
SAPROT Su et al. 2023
650M_PDB PDB
35MLAF2
AlI
AlI
650M
35M
231K
1.6M
1,280
480
HF: SaPror AF2
Soguence-SSrruttn
ESM-IFI 1TB BI BUE
PROTSSN
hS12
Pair.
148M
512
ERP USS. 1F7
MIF-ST
Tusg 0E 2025
Pair.
613M
256
Oithub: MIF ST
FOLDSEUX
2014
3Di-AA
GVP
Jing
3-layers
Sup.
Structure-Only
Foi DSEEK
Van Kenaenn ef
2014
3D1
Pair.
Conder Faloseckk
TM-AI 1GN
ZTinn Skoinic
A00
meza
Pair.
Condi: 1M-augo
B BASELINES
B. DEEP LEARNING MODELS
Sequence-Only. Sequence-only baselines include both encoder-only and encoder-decoder archi- tectures. Encoder-only models such ESM2 (130, 133, (36) (Lin et 2023), ESM- 1B (Rives) ct al. 2021), and PROTBERT (Elnaggar et al. 202T) are pretrained protein language models using masked language modeling on large sequence corpora. ANKH (Elnaggar al, 2023) and PROTT5 (Elnaggar et al., 20221) in contrast, adopt encoder decoder architectures, enabling bidirectional con- textualization and autoregressive decoding. While TM-VEC (Hamamsy et al.) 2024) and PROSTTS (Heinzinger et al. 2023) only require sequence inputs, both incorporate structural inductive signals during training: TM-VEC is trained to regress TM-scores, and PROSTT5 is fine-tuned to translate Foldseck-derived structural tokens.
Sequence-Structtue. Sequence struccure models combine sequence and structural information in diverse ways. SAPROT (Su et al. 2023) fuses amino acid tokens with Foldseek-derives structural tokens and is trained using multi-modal masked language modeling. PROTSSN (Tan et al. (025c)
integrates ESM2 (Lin et al. 2023) embeddings with geometric graph neural networks, enabling
joint scquence-structuue representation learning. Both ESM-IFI (Hsu et al. (002)) and MIF-ST (Yang al.] 2023) are inverse folding models: ESSMIIF1 is pretrained on large-scale backbone recovery, while MIF-ST uses structure-conditione geometric networks initialized from large protein transformers.
Structure-Onie Only. Structure-only baselines rely purely 3D geometrie inputs. GVP-GNT (Jing et al. 2021 is a non-pretrained geometric deep learning model that uses residue type and atomic coordinate features for message passing.
B.2 ALIGNMENT-BASEE METHODS
Foldseek. We employ FOLDSEEK (Van Kempen et al.) 2024) to evaluate structural similarity be- tween query and proteinss under two alignment modes: 3Di-only (-ali (-alignment-tyye
0) and 3Di+AA (-aligrmen (-alignment-tyype 2). To maximize sensitivity, we activate exhaustive pairwise com- parison via 0.0 to allow all sequence identity levels. We retain up to 100,000 alignments per query using -max-seqs 100000, and parallelize computation across available CPU threads. The output aligmment scores are used to compute similarity for unsupervised levrl evaluation (e.g.. AUC). FOLDSEEK achieves state-of-the-an tradeoffs between alignment speed and accuracy for large-scale protein structure comparison.
set high -e threshold of 1,000, and use -min-seq- id
24

---

Published as a conference paper at ICLR 2026
Correlation Model Rankings (Rasidue-Level Tasks, AUPR, MP50)
Correlation Model Rankings
(Pragment-Lavel Tasks, Macro-F1, MF50)
1.000
AAt
1,000
3.00
3.50
6.5
Bindi 0.033
1.000
1.25
Dlndi
0.747
1.000
0.25
0.905
0.786
1,000
-0
Eee
0.766
0.778
1.000
Noor
1.000
-1.00
0.571
1.000
-000
Task
Task
(a) Residue-Land Levell Correlation (AUPR, MP50)
(b) Pragment-Lever Correlation (Macro-F1, MF50)
Figure 9: Spearman's rank correlation (p) of model performance across tasks. (a) Correlation of AUPR scores for Residue-Level tasks on the MP50 split. We observe strong correlations between Act, BindI, and Evo tasks. (b) Correlation Macro-FI scores for Fragment- tasks on the MF50 split. We find striking zero correlation between Act and Motif tasks.
We adopt BLAST Altschuul et al. 1990) classical sequence-based alignment baseline. Our setup disables low-complexity masking (-seg no) to preserve short functional regions and sets permissive E-value threshold (-evalue 1 1000000) to retain weak similarities. Word size is reduced to 2 (-word_ste size 2) to improve alignment sensitivity, and a large hit buffer 100000000) ensures comprehensive coverage. The output is recorded in tabular format (- (-out fmt 6), including sequence IDs, identity, alignment length, mismatches, gaps, and bit scores. BLAST is used baseline for fragment-level and full-sequence pair similarity evaluation.
BLASTp.
(-max_aargt__see
TM-align. To benchmark structure-only alignment, we apply TM-ALIGN (Zhang & Skolnick 20055 on PDB-format query and reference proteins via the default command-lime interface (TMa gn query. ppb ref ref.páb) TM-ALIGN returns key statistics, including RMSD, sequence identity, aligned length, and two TM-scores (normalized by query and reference, respectively). We record the average TM-score between the two directions the final similarity metric for evaluation. TM-ALIGN widely regarded as reliable tool for structure- based homology assessment, though its quadratic computational complexity limits scalability large benchmarks.
C CROSS-TASK CORRELATION ANALYSIS
We performed cross-task correlation analysis to determine if model strengths are task-specific or general. We computed the Spearman's rank correlation (p) the model performance scores (AUPR for residue tasks [MP50 split], Macro-F1 for fragment tasks [MF50 split]). The results are visualized in the heatmaps in Figure
Residue- Level Analysis (Figure 91): We observe strong, positive correlations across the Act (Active Site), BindI (Binding Site), and Evo (Evolutionary Pressure) tasks (p ranging from 0.79 0.90). This is intuitive and confirms the validity of our tasks, as active and binding sites are, by definition, under high evolutionary pressure and functionally related. Models that excel identifying one typically excel the others. The Motif task shows weaker correlation, suggesting that identifying these (often longer) fragments a partially distinct capability.
Fragment-Level Analysis (Figure 9p): A similar pattern emerges for fragment classification, where Act, Bindl, and Evo tasks are highly correlated (p z 0.75 - 0.78). However, we find a striking result: there is effectively zero correlation (p -0.01) between the Act and Motif tasks. This empirically demonstrates that the model features required to classify a general "Motif" are completely different from those required to classify specifie "Active Site motif."
25

---

Published as a conference paper at ICLR 2026
Table 14: Ablation study on structure quality. (a) Performance comparison (AUPR) of SaProt models on the Epitope (Epi) task using experimental PDB structures versus AlphaFold2 (AF2) predicted structures. Both settings use the MP50 split. (b) Distribution of pLDDT confidence scores for the AlphaFold structures used in VENUSX, indicating high structural reliability.
(a) Model Performance
(b) AlphaFold pLDDT Statistics
Model
PDB
AF2
Level
Mean
Min
Max
25%
50 %
SaProt 35M
081820
0. 0.1721
Protein
91.01
42.00
98.71
89. 10
92.70
SaProt 650M
0.1948
0.1889
Residue
91.40
19.44
99.00
89.75
94.88
Table 15: AUPR comparison between GVP-GNN (from scratch) and GearNet (pre-trained) on the Epitope task.
Model
Res_Epi Ep0__P50
MP50
Res_ Res_Epi_ll 1Pi_
Res_Epi_ (pi_MP90
GVP-GNN
0.118
0.139
0.196
GearNet
0.116
0.129
0.167
D ABLATION STUDIES
D.1 IMPACT OF STRUCTURE SOURCE (PREDICTED VS. EXPERIMENTAL)
To address concerns regarding the use of predicted structures (AlphaFold DB) instead of experimental ones (PDB), we performed an ablation study on the Epitope (Epi) task. We evaluated the structure- aware SaProt models on the subset of the MP50 split where experimental PDB structures were available, and compared this to the standard setting using AlphaFold structures.
Results: Table 14 aa) presents the performance comparison. We observe that the AUPR scores are highly consistent between the settings. For instance, SaProt-650M achieves an AUPR 0.1948 on PDB structures and O. 0.188% 1889 on AlphaFold structures. This minimal performance difference is attributable to the high confidence of the predicted structures used VENUSX. As shown in Table b), the mean pLDDT score is >91 at both the protein and residue levels, with median of 94.88 for residues. This indicates that the predicted structures are of sufficient quality to serve as a reliable proxy for experimental structures, justifying their use to scale up the benchmark.
D.2 EVALUATION OF PRE-TRE PRE-TRAINED STRUCTURE-ONE ONLY MODELS
To assess the impact of structural pre-training, we compared GearNet (pre-trained) against GVP-GNN (trained from scratch) the Residue-level Epitope task (Res_Epp _pi). As shown in Table 15 GearNet performs comparably to GVP-GNN across all splits (e.g.. 0.116 vs. 0.118 on MP50). This indicates that for this specific lask relying on fine-grained surface properties, general structural pre-training objectives do not immediately outperform supervised geometric learning.
D.3 TRAINING STABILITY AND RANDOM SEED SENSITIVITY
To verify that our benchmark results are robust to random initialization and not artifacts a specific seed, we analyzed the stability of model performance.
Methodology: We re-trained six representative baseline models (ESM2, ProtBert, ProtT5, Ankh) on two of the most challenging datasets: Res_Act___ MP3_ and Res Bindl MP30. Each model was trained independently using three different random seeds: 42, 3407 (our default), and 12345.
Results: Table 16 summarizes the Mean Recall and Standard Deviation (Std) for each model. We observe:
Low Variance: The standard deviations are consistently low across all models and tasks, typically ranging from 0.003 to 0.014.
26

---

Published as a conference paper at ICLR 2026
Table 16: Stability analysis of model performance (Recall) across three different random seeds (42, 3407, 12345) on the challenging MP30 split. We report the Mean + Standard Deviation. The consistently low standard deviation (< 0.015) across diverse models and tasks confirms that the benchmark rankings are robust to random initialization.
Res Les__Att_uu MP30
Res BindI _M330
Model
Mean
Std
Mean
Std
ESM2 t30
0.765
0.011
0.818
0.003
ESM2 t33
0.785
0.004
0.826
0.012
ProtBert
0.587
0.007
0.668
0.014
ProtT5 xl 1_niree50
0.767
0.006
0.883
0.010
Ankh base
0.798
0.010
0.895
0.008
Ankh large
0.776
0.010
0.842
0.008
Table 17: Performance comparison (Recall) on Active Site (Act) and Evolutionary Pressure (Evo)
tasks.
Model
Res Act MP30
Res Act MP50
Res Evo MP30
Res Evo MP50
ESM2 133 (Seq-only)
0.787
0.870
0.703
0.861
SaProt 650M (Seq-Struct)
0.792
0.850
0.714
0.866
Stable Rankings: The relative performance ranking of different models remains consistent across seeds. For example, Ankh base consistently outperforms ProtBert by a significant margin both tasks, regardless of the seed.
These results confirm that the performance differences observed in VENUSX, are driven by the quality of the representations and the model architectures, rather than hyperparameter noise or initialization luck.
D.4 ANALYSIS OF ROBUSTNESS TO Low SEQUENCE IDENTITY
Sequence-structio hybrid models (e.g., SaProt) exhibit superior generalization cross-family (OOD) splits compared to sequence-only models. To investigate the mechanism behind this, we analyzed model performance under decreasing levels of sequence identity, isolating the effect of "low homology" which characterizes remote generalization.
Hypothesis: We hypothesized that sequence identity drops, the predictive signal from sequence alone weakens. However, since structural folds are often conserved even when sequences diverge, structure-aware models should be more robust in low-identity regimes.
We evaluated models on two mix-family splits with different difficulty levels: MP50 (50% identity) and the significantly harder MP30 (30% identity). Table 17 summarizes the Recall scores.
Results:
Moderate Homology (MP50): On the MP50 splt, the sequence-only ESM2 model performs comparably or slightly better than SaProt (e.g., Recall of 0.870 vs. 0.850 Act). This suggests that when sequence homology sufficient, sequence features alone are highly effective.
Low Homology (MP30): When the sequence identity threshold is tightened to 30%, the advantage shifts. SaProt consistently outperforms ESM2 both tasks, achieving higher Recall of 0.792 vs. 0.787 on Act, and 0.714 vs. 0.703 on Evo.
These findings clarify why sequence- -trruccurr models excel in cross-family tasks. While sequence features dominate in high-homology settings, structural information provides critical signal when sequence similarity fades, enabling better generalization remote homologs.
Conclusion:
27

---

Published as a conference paper at ICLR 2026
Table 18: Performance the BLAST baseline on residue-level active site classification (Act). We report the specific AUPR scores for the top-3 runs out of 10 independent trials. Even the best-case results (~0.005 are drastically lower than deep learning models (>x0.5), confirming that alignment insufficient for this task.
Res MFF0
Res Les_Act_Moa AF7_
P-Identity Threshold
Run
Run 2
Run 3
Run 1
Run 2
Run
100 (Exact Match)
0.0407
0.0410
0.0417
0.0471
0.0480
0.0492
90 (High Identity)
0.0435
0.0431
0.0439
0.0497
0.0528
0.0520
E NON-DEEP LEARNING BASELINE PERFORMANCE
To assess whether fine-gran fine-grained functional residues be accurately identified solety through sequence homology transfer, we evaluated a traditional alignment-bane baseline for the residue-level active site classification task (Act). This baseline utilizes NCBI BLAST+ transfer functional annotations from the training set to the test set.
E.I METHODOLOGY
Query Generation: We implemented a stratified sampling strategy to construct the query database. For each experimental run, randomly select 100 distinct InterPro IDs from the training set. For each selected ID, we randomly sampled 100 sequences (or all available sequences if the family size was smaller). From these sampled sequences, we extracted all continuous sub-sequences annotated as positive (label=1) to serve as functional motif queries.
Search Strategy: These fragments were compiled into a query database and searched against the full-length protein sequences in the test set. We utilized the blas blastp algorithm with the task=ble astp-sshort option, which is optimized for aligning short sequences that might otherwise be missed by standard parameters due to low complexity or short length.
Scoring Mechanism: For every residue in a test sequence covered by significant BLAST hit, we assigned a prediction score equal to the percentage identity (p miident/ / 100.0) that alignment. If residue covered by multiple overlapping hits, the maximum score was retained to represent the highest confidence match. Residues not covered by any alignment were assigned score of O.
To minimize sampling variance and estimate an upper bound for alignment-based performance, we independently repeated this procedure 10 times using different random subsets of queries (different IDs and sequences). We report the specific AUPR scores of the top-3 performing runs.
Robust Evaluation:
E.2 RESULTS AND ANALYSIS
Table 18 summarizes the performance on the mixed-family splits (MF50 and MF70) under strict (pident=100) and relaxed (pident=90) filtering thresholds.
As shown in Table 18 the alignment-baas method yields extremely low AUPR scores. Even in the best-performing runs, the AUPR remains approximately 0.04-0.05 (e.g., 0.0528 for the best run on Act_MF70). This poor performance stands in stark contrast to deep learning models (e-g., HSM2, SaProt), which consistently achieve AUPR scores exceeding 0.85 on the same tasks (see Table 4
This significant performance disparity highlights the limitations of simple homology-based label transfer for fine-grained residue-level prediction. While BLAST is effective for global homology detection, it struggles to precisely map functional residues when the local sequence motif exhibits variability degenerate patterns, even within homologous families. These results empirically validate the necessity of VENUSX, and the representation learning approach, which can capture complex, non-linear functional signals that alignment methods miss.
28

---

Published as a conference paper at ICLR 2026
Table 19: Detailed residue-level classification performance across BindB and Epi datasets and data splits. "MP50", "MP70", and "MP90" refer to mixed-family splits with 50%, 70%, and 90% sequence identity filtering applied at the full-sequence level. Metries reported include AUPR, Precision, Recall, Fl scores for negative and positive classes, and Macro-F1.
BindB
Epi
Metric
Model
MP50
MP70
MP90
MP50
MP70
MP90
ESM2 130
0.408
0,465
0.496
0.186
0.184
0.277
ESM2 t33
0.446
0.494
0.535
0.174
0.200
0.290
AUPR
ProtBert
0.340
0.410
0,466
0.169
0.177
0.266
Ankh
0.421
0.487
0.527
0.167
0.215
0.270
ESM2 t30
0.598
0.637
0.674
1.0
0.384
0.545
ESM2 133
0.605
0.646
0.675
0.512
precision
ProtBert
0,547
0.619
0.706
1.0
0.432
0,534
Ankh
0.634
0.660
0.677
0.0
1.0
0.571
ESM2 130
0.289
0.317
0.316
0.001
0.043
0.091
ESM2 t33
0.329
0.356
0,386
0.0
0.003
0.139
Recall
ProtBert
0.238
0.264
0.257
0.001
0.005
0.072
Ankh
0.260
0.335
0.357
0.0
0.008
0.054
ESM2 t30
0.987
0.986
0,986
0.958
0.960
0.968
ESM2 133
0.987
0.987
0.987
0.958
0.961
0.968
F1-Negative
ProtBert
0.986
0.986
0.986
0.958
0.961
0.968
Ankh
0.987
0.987
0.987
0.958
0.961
0.968
ESM2 130
0.390
0.423
0.430
0.002
0.077
0.156
ESM2 133
0.427
0.459
0.491
0.006
0.218
Fl-Positive
ProtBert
0.332
0.370
0.377
0.002
0.010
0.126
Ankh
0.369
0.444
0.483
0.0
0.002
0.098
ESM2 130
0.689
0.705
0.708
0.480
0.518
0.562
ESM2 133
0.707
0.723
0.739
0.479
0.484
0.593
Macro-FI
ProtBert
0.659
0.678
0.682
0.480
0.486
0.547
Ankh
0.678
0.715
0.735
0.479
0.489
0.533
F DETAILED EXPERIMENTAL RESULTS
Here, we provide detailed experimental results for two tasks: Residue-Level Binary Classification and -eggmentt-Leell Multi- Classs Classification. We report all evaluation metries recorded during the experiments to offer comprehensive assessment model performance across different aspects. Please refer Tables 19 22 for the complete results.
Model scores on the mix-family tasks are relatively high. To further demonstrate the benchmark's remaining headroom, Table 23 presents under stricter "MP30" split (30% sequence identity). The results reveal substantial performance variance across different models, and overall performance generally declines, with the best model achieving an average Recall of 0.73 across the five test sets.
results from more challenging experiment conducted
To evaluate the impact of data on model performance, with the sequence similarity of the training data in Act and Evo tasks. For each task, we filtered the datasets using sequence identity thresholds of 90%. 70%. 50%. and 30%. Subsets with lower sequence identity thresholds contain fewer training and testing samples (see the statistics in Tables 9 T0). As shown in Table (4,, model generalization performance decreases progressively as both the sequence similarity threshold and the dataset size are reduced.
analyzed how model accuracy varies
29

---

Published as a conference paper at ICLR 2026
Table 20: Detailed residue-level classification performance across Act. BindI, and Evo datasets and data splits. "MF50" and "MPS0" refer to mixed-family splits with 50% sequence identity filtering applied the fragment and full-sequence levels, respectively. Metrics reported include AUPR, Precision, Recall, Fl scores for negative and positive classes, and Macro-F1.
Act
BindI
Evo
Metric
Model
MF50
MP50
Cross
MF50
MP50
Cross
MF50
MP50
Cross
ESM2 t30
0.855
0.932
0.143
0.912
0,963
0.133
0.862
0.897
0.235
ESM2 133
0.852
0.955
0.143
0.904
0.971
0.159
0.899
0.926
0.262
ProtBert
0.764
0.895
0.131
0.857
0.926
0.111
0.771
0.803
0.243
Ankh base
0.873
0.895
0.166
0.907
0.970
0.145
0.895
0.932
0.275
AUPR
GVP-GNN
0.523
0.896
0.101
0.611
0.883
0.040
0.342
0.792
0.101
SaProt 35M
0.688
0,90S
0.114
0.807
0.927
0.230
0.724
0.772
0,272
SaProt 650M
0.745
0,945
0.185
0.838
0.960
0.182
0.734
0.912
0,274
ProtSSN
0.465
0.917
0.156
0.801
0.907
0.095
0.715
0.895
0.227
ESM2 130
0.826
0,851
0.278
0.859
0.915
0.525
0,816
0.879
0,374
ESM2 133
0.845
0.851
0.126
0.869
0.905
0.581
0.856
0.856
0,403
ProtBert
0.791
0.833
0.131
0.855
0.897
0.416
0.805
0.803
0.482
Ankh base
0.862
0.873
0.190
0.849
0.919
0.437
0.882
0.932
0.387
Precision
GVP-GNN
0.735
0.824
0,019
0.730
0.874
0.0
0.810
0.781
0.176
SaProt 35M
0.818
0.879
0.132
0.813
0.902
0.634
0.819
0.841
0.382
SaProt 650M
0.812
0.845
0.241
0.827
0.900
0.661
0.809
0.828
0.456
ProtSSN
0.523
0.835
0.241
0.818
0.887
0.379
0.790
0.815
0.452
ESM2 130
0,676
0.793
0,060
0.859
0.897
0.078
0.783
0.750
0.097
ESM2 133
0.682
0.848
0.031
0.830
0.924
0.108
0.806
0.755
0.122
ProtBert
0.565
0.750
0.020
0.694
0.839
0.048
0.610
0.597
0.009
Ankh base
0.700
0.864
0.025
0.866
0.922
0.086
0.735
0.744
0.169
Recall
GVP-GNN
0.362
0.798
0.001
0.519
0.788
0.091
0.718
0.035
SaProt 35M
0.408
0.733
0.036
0.705
0.822
0.135
0.520
0.649
0.172
SaProt 650M
0.511
0.850
0.072
0.768
0.918
0.135
0.554
0,700
0.111
ProtSSN
0.209
0.801
0.014
0.705
0.788
0.029
0.507
0.852
0.034
ESM2 130
0.992
0.995
0.967
0.993
0.996
0.975
0.988
0.990
0.957
ESM2 133
0.993
0.996
0.964
0.992
0.996
0.976
0.990
0.992
0.957
ProtBert
0.991
0.994
0.967
0,989
0.994
0.974
0.984
0.986
0.960
Ankh base
0.993
0.996
0.968
0.992
0.996
0.974
0.989
0.992
0.955
Fl-Negative
GVP-GNN 0.989
0.995
0.969
0.982
0.993
0.975
0.973
0.986
0.955
SaProt 35M
0,990
0.995
0.964
0.988
0.994
0.976
0.982
0.985
0.955
SaProt 650M
0.986
0.996
0.965
0.989
0.996
0.977
0.983
0.991
0.959
ProtSSN
0.988
0.995
0.969
0.988
0.993
0.975
0.982
0.991
0.960
FSM2 t30
0.744
0.821
0.098
0.859
0.906
0.136
0.799
0.810
0.154
ESM2 t33
0.755
0.850
0.050
0.849
0.915
0.181
0.831
0.858
0.187
ProtBert
0.659
0.789
0.035
0.766
0.867
0.086
0.694
0.701
0.017
Ankh base
0.773
0.869
0.045
0.857
0.920
0.144
0.802
0.858
0.235
F1-Positive
GVP-GNN
0.485
0.810
0.002
0.607
0.829
0.0
0.164
0.748
0.058
SaProt 35M
0.544
0,800
0.056
0.755
0.860
0.223
0.636
0.689
0.238
SaProt 650M
0.627
0.848
0.110
0.796
0.909
0..24
0.658
0.851
0.178
ProtSSN
0.329
0.818
0.026
0.757
0.839
0.053
0.618
0.833
0.062
ESM2 130
0.868
0.908
0.533
0.926
0.951
0.556
0.894
0.900
0.555
ESM2 133
0.874
0,923
0.507
0.921
0.955
0.579
0.910
0.925
0.572
ProtBert
0.825
0.892
0.501
0.878
0,931
0.530
0.839
0.843
0.489
Ankh base
0.883
0.933
0.507
0.925
0.958
0.559
0.896
0.925
0.595
Macro-F1
GVP-GNN
0.736
0.903
0.485
0.795
0.911
0.488
0.569
0.867
0.506
SaProt 35M
0.767
0.897
0.510
0.871
0.927
0.599
0.809
0.837
0.596
SaProt 650M
0.808
0.922
0.538
0.893
0.953
0.600
0.820
0.921
0.568
ProiSSN
0.658
0.906
0.498
0.873
0.911
0.514
0.800
0.912
0.511
30

---

Published as a conference paper at ICLR 2026
Table 21: Detailed residue-level classification performance across Motif and Dom datasets and data splits. "MF50" and "MP50" refer to mixed-family splits with 50% sequence identity filtering applied at the fragment and full-sequence levels, respectively. Metries reported include AUPR, Precision, Recall, Fl scores for negative and positive classes, and Macro-F1.
Motif
Dom
Metric
Model
MF50
MP50
Cross
MF50
MP50
Cross
ESM2 t30
0.855
0.850
0.433
0.634
0.634
0.470
ESM2 t33
0.874
0.857
0,456
0.666
0.657
0.506
ProtBert
0.779
0.796
0.348
0.591
0.592
0,508
Ankh base
0.884
0.870
0.394
0.673
0.665
0.449
AUPR
GVP-GNN
0.661
0.736
0.329
0.560
0.557
0.468
SaProt 35M
0.767
0.784
0.408
0.574
0.584
0.525
SaProt 650M
0.802
0.841
0.441
0.642
0.640
0.564
ProtSSN
0.716
0.765
0.390
ESM2
0.824
0.802
0.510
0.648
0.644
0.496
ESM2 t33
0.851
0.795
0.566
0.661
0.634
0.530
ProtBert
0.784
0.793
0.472
0.636
0.596
0.588
Ankh base
0.846
0.817
0.499
0.674
0.646
0.494
Precision
GVP-GNN
0.748
0.756
0.329
0.591
0.557
0,519
SaProt 35M
0.821
0.783
0,485
0.632
0.615
0.548
SaProt 650M
0.841
0.818
0.504
0.635
0.656
0.572
ProtSSN
0.772
0.775
0.390
ESM2 t30
0.775
0.731
0.432
0.433
0.423
0.360
ESM2 t33
0.748
0,861
0.384
0.467
0.478
0.367
ProtBert
0.678
0.592
0.231
0.353
0.420
0.138
Ankh base
0.789
0,831
0.303
0.467
0.490
0.280
Recall
GVP-GNN
0.525
0.669
0.453
0.344
0.309
0.087
SaProt 35M
0.582
0.954
0.411
0.322
0.840
0.349
SaProt 650M
0.615
0.960
0.350
0.472
0.414
0.444
ProtSSN
0.550
0.676
0.365
ESM2 t30
0.972
0.961
0.946
0.839
0.849
0.738
ESM2 133
0.972
0.962
0.951
0.844
0.848
0.752
ProtBert
0.963
0.952
0.945
0.834
0.837
0.779
Ankh base
0.974
0.963
0.946
0.847
0.853
0.748
Fl-Negative
GVP-GNN
0.954
0.953
0.924
0.836
0.837
0.774
SaProt 35M
0.961
0.954
0.944
0.832
0,840
0.761
SaProt 650M
0.964
0.960
0.946
0.837
0.850
0.765
ProtSSN
0.956
0.954
0.944
ESM2 t30
0.799
0.765
0.467
0.519
0.510
0.417
ESM2 t33
0.796
0.774
0.457
0.547
0.545
0.433
ProtBert
0.727
0.681
0.310
0.454
0.494
0.223
Ankh base
0.817
0.779
0.377
0.552
0.557
0.357
F1-Positive
GVP-GNN
0.618
0.710
0.399
0.435
0.408
0.149
SaProt 35M
0.681
0.709
0.445
0.427
0.462
0.427
SaProt 650M
0.710
0.754
0.414
0.542
0.508
0.500
ProtSSN
0.642
0.720
0.412
ESM2 t30
0.885
0.863
0.707
0.679
0.680
0.578
ESM2 t33
0.884
0.868
0.704
0.696
0.697
0.593
ProtBert
0.845
0.816
0.628
0.644
0.665
0.501
Ankh base
0.895
0.871
0.662
0.700
0.745
0.552
Macro-F1
GVP-GNN
0.786
0.831
0.661
0.636
0.623
0.462
SaProt 35M
0.821
0.832
0.695
0.629
0.651
0.594
SaProt 650M
0.837
0.857
0.680
0.689
0.679
0.632
ProtSSN
0.799
0.837
0.678
31

---

Published as a conference paper at ICLR 2026
Table 22: Detailed fragment-level classification results on "MF50" split across Act, BindI, Evo, and Motif datasets. "MF50" refers mixed-family splits with 50% sequence identity filtering applied the fragment level. Metrics reported include Accuracy, Precision, Recall, Macro-F1, and Matthews Correlation Coefficient (MCC).
Metric
Model
Act
BindI
Evo
Motif
ESM2 t30
0.819
0.937
0.853
0.884
ESM2 t33
0.814
0.934
0.841
0.906
ProtBert
0.736
0.927
0.828
0.884
Ankh base
0.824
0.920
0.866
0.901
Accuracy
GVP-GNN
0.907
0.972
0.914
0.807
SaProt 35M
0.928
0.976
0.939
0.901
SaProt 650M
0.928
0.986
0.950
0.927
ProtSSN
0.891
0.972
0.915
0.914
ESM2 t30
0.659
0.834
0.681
0.458
ESM2 t33
0.603
0.755
0.682
0.547
ProtBert
0.618
0.838
0.644
0.455
Ankh base
0.661
0.733
0.727
0.508
Precision
GVP-GNN
0.826
0.901
0.763
0.387
SaProt 35M
0.810
0.943
0.857
0.509
SaProt 650M
0.830
0.968
0.868
0.546
ProtSSN
0.773
0.940
0.804
0.564
ESM2 t30
0.670
0.819
0.684
0.461
ESM2 t33
0.634
0.775
0.682
0.543
ProtBert
0.636
0.794
0.646
0.458
Ankh base
0.665
0.732
0.729
0.501
Recall
GVP-GNN
0.833
0.882
0.768
0.371
SaProt 35M
0,823
0.929
0.858
0.505
SaProt 650M
0.830
0.956
0.875
0.562
ProtSSN
0.774
0.948
0.807
0.556
ESM2 t30
0.647
0.809
0.667
0.457
ESM2 t33
0.605
0.753
0.669
0.542
ProtBert
0.609
0.790
0.627
0.452
Ankh base
0.647
0.718
0.716
0.499
Macro-F1
GVP-GNN
0,822
0.884
0.757
0.370
SaProt 35M
0,807
0.931
0.849
0.504
SaProt 650M
0.825
0.957
0.863
0.552
ProtSSN
0.764
0.931
0.793
0.556
ESM2 t30
0.815
0.926
0.852
0.875
ESM2 t33
0.810
0.922
0.840
0.898
ProtBert
0.731
0.914
0.827
0.875
Ankh base
0.821
0.906
0.865
0.892
MCC
GVP-GNN
0,906
0.967
0.913
0.791
SaProt 35M
0.926
0.971
0.938
0.892
SaProt 650M
0.926
0.984
0.950
0.921
ProtSSN
0.889
0.967
0.915
0.907
32

---

Published as a conference paper at ICLR 2026
Table 23: Residue-level classification results "MP30" split across Act, Bindl. Evo, Motif, and Dom datasets. "MP30" refers to mixed-family splits with 30% sequence identity filterngg applied at the full-sequence level.
Metric
Model
Act
BindI
Evo
Motif
Dom
ESM2 t30
0.754
0.817
0.664
0.689
0.298
ESM2 133
0.787
0.814
0.703
0.778
0.264
ProtBert
0.593
0.667
0.542
0.586
0.338
Recall
Ankh base
0.794
0.901
0.777
0.787
0.389
Ankh large
0.766
0.835
0.761
0.616
0.392
ProtT5 xl uniref50
0.760
0.894
0.758
0.730
0.465
Table 24: Residue-level classification performance across Act and Evo datasets. "MP50" refers to mixed-family splits with 50% sequence identity filtering applied at full-sequence levels. Metric reported Recall.
Act
Evo
Model
MP30
MP50
MP70
MP90
MP30
MP50
MP70
MP90
ESM2 t30
0.754
0.847
0.887
0.926
0.664
0.818
0.862
0.911
ESM2 (33
0.787
0.870
0.892
0.894
0.703
0.843
0.884
0.932
ProtBert
0.593
0.781
0.810
0.857
0.542
0.677
0.823
0.866
Ankh base
0.794
0.886
0.892
0.901
0.777
0.864
0.909
0.938
Ankh large
0.766
0.862
0.853
0.863
0.761
0.846
0.902
0.926
ProfT5 xl uniref50
0.760
0.886
0.889
0.887
0.758
0.860
0.905
0.939
33


Reviews:
[{"review_id": "1ppSbgNIFZ", "reviewer": "Reviewer_4KXz", "summary": "This paper introduces VENUSX, a pioneering benchmark for assessing deep learning models’ fine-grained functional understanding of proteins, beyond whole-protein tasks. It highlights the limitations of current models in capturing localized mechanisms—crucial for enzyme engineering, annotation, and interpretability—by curating >878,000 samples from InterPro, BioLiP, and SAbDab. VENUSX features three task types: residue-level binary classification, fragment-level multi-class classification, and pairwise similarity scoring, with mix- and cross-family splits at 50/70/90% identity. Baselines (ESM2, SaProt, GVP-GNN, etc.) show global performance does not guarantee local accuracy. All code, data, and leaderboards are open-sourced.", "questions": "1. Given the diversity of annotation sources (InterPro, BioLiP, SAbDab) and task types, could the authors provide an empirical analysis (e.g., correlation of model rankings, overlap in difficult examples, or transfer learning performance) across the seven residue-level and five fragment-level subtasks?\n\n2. The experimental setup uses only frozen encoders with linear probes for all protein language models. Could the authors explain whether fine-tuning was attempted and, if not, provide a rationale (e.e., computational constraints, risk of overfitting on smaller splits, or intent to test pretraining quality)?\n\n3. Proteins are often truncated to meet the input length limitations of the models. How are the protein structures modified in these cases to ensure that each residue is correctly mapped to its spatial location in the sequence+structure models? Providing more details on this process would significantly strengthen confidence in the fairness and rigor of the evaluation.\n\n4.  For many entries, the benchmark relies on predicted structures from the AlphaFold Database rather than experimentally determined structures from PDB. While this is necessary to achieve the benchmark's massive scale, it introduces a potential confounding variable. An analysis of model performance on high-confidence predicted structures versus experimental structures would be a valuable addition.\n\n5. For the classification tasks, GVP-GNN is the primary structure-only model evaluated and is trained from scratch. Including other modern, pre-trained structure-only models in the comparison would provide a more complete picture of that model class.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tVENUSX introduces a new focus on fine-grained protein functional understanding through novel residue-, fragment-, and similarity-based tasks, with cross-family splits to test generalization, addressing a key gap in evaluating localized biological signals.\n\n2.\tThe benchmark is well-constructed from over 878k curated samples, with detailed task definitions, appropriate metrics, diverse baselines, and clear documentation, making it practical and accessible for research.", "weaknesses": "1. The evaluation lacks newer protein language models such as ESM3, which models sequence, structure, and function jointly; its absence limits the benchmark’s ability to reflect current state-of-the-art performance on fine-grained functional tasks.\n\n2. Residue- and fragment-level tasks use only frozen residue embeddings passed through two linear layers; the lack of experiments on fine-tuning language models leaves unaddressed whether supervised adaptation can improve capture of localized functional signals.\n\n3. Many of the tasks require training new classification heads on top of each protein representation model, which may be computationally demanding and could limit the benchmark’s accessibility and widespread adoption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eV2aHJjf1N", "reviewer": "Reviewer_WEHT", "summary": "The paper curates data from a few protein databases to create some benchmarking tasks for protein function prediction. The tasks differ from most prior work in this space in that they focus on annotating sub-sequences of proteins corresponding to active sites, domains, etc. Given the evolutionary relationships between protein sequences, they exhibit clustering behavior, and thus non-trivial train test splits are important to evaluate models' extrapolation. They do a good job of guaranteeing distance between training and evaluation sequences.\n\nThe performance of variety of deep learning and alignment-based baselines are compared.", "questions": "I was surprised to see that fragment-level classification and pairwise similarity scoring tasks are done using localized annotation types such as active sites and binding sites. An enzyme has a small number of active site residues, and they are often non-consecutive. What is the corresponding 'fragment' for this task? In Fig 2, the fragments are fairly long. Did you just take the entire subsequence that includes the active site residues? I don't think this is justifiable as a prediction task, since there are no clear semantics to this subsequence. This issue might also serve as a confounder when comparing the alignment-based baselines on these tasks.\n\nFor the residue-level annotation task, I'm curious how an alignment-based technique would have worked (you do alignment between the query sequence and the training set). I expect that this would perform well for things like active site residues.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "This problem is important for analyzing proteins, both for making predictions for the wide variety of uncharacterized natural proteins and for designing new proteins.\n\nThe benchmark appears to be well organized and accessible for future researchers.", "weaknesses": "I have key questions about how fragments are defined for certain annotation types. See below.\n\nThere are no non-deep-learning baselines for some tasks.\n\nBoth the fragment-level classification and fragment similarity tasks don't seem to be simulating a workflow that practitioners would encounter in practice. How practical is it to be presented with a pre-identified sub-sequence of the protein without knowledge of it's functional label? Usually, this is presented as a sequence segmentation problem, where the input is a full sequence and the output is a set of labeled segments.\n\nThe discussion of related work should have been in the main paper instead of Appendix.\n\nWhen doing such a large exploration of baseline deep learning models, it is always hard to understand how much the results would have changed if more hyperparameter tuning had been done for each.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "32whG5LWul", "reviewer": "Reviewer_7f4D", "summary": "This study presents VenusX, a comprehensive benchmark for assessing protein representation learning models with respect to fine-grained, intra-protein functional understanding. VenusX encompasses three primary task categories: residue-level binary classification, fragment-level multi-class classification, and pairwise functional similarity scoring. The benchmark is built from over 878,000 curated samples sourced from InterPro, BioLiP, and SAbDab, featuring both mixed-family and cross-family data splits at multiple sequence identity thresholds to evaluate in-distribution and out-of-distribution generalization. The authors benchmark a diverse range of models—including sequence-only, sequence-structure, and structure-only approaches—across all tasks. Key observations reveal that sequence-based models excel in in-distribution performance, whereas sequence-structure hybrid models exhibit superior generalization to unseen families, with cross-family prediction remaining a significant challenge. The benchmark, along with its code, data, and leaderboard, is released as open-source.", "questions": "- Could the authors provide further analysis or intuition on why sequence-structure hybrid models, such as SaProt, exhibit superior generalization on cross-family splits compared to sequence-only models? Is this improvement primarily driven by the structural vocabulary, the training objective, or a combination of both factors?\n- The epitope prediction task (Epi) reports very low AUPR scores. What are the specific challenges underlying this task—such as data quality, label definition, or the conformational diversity of epitopes—that contribute to its difficulty? What potential directions could be explored to improve performance in future work?\n- For the pairwise functional similarity scoring task, evaluation is conducted on a fixed sample of 10,000 positive and negative pairs. Was the sensitivity of the AUC score to this particular random sample analyzed? Reporting the variance across different random samples could provide a clearer picture of evaluation robustness.\n- The GVP-GNN model, a structure-only approach, is trained from scratch, whereas other models use frozen pretrained checkpoints. Could the authors elaborate on the rationale for this choice? Were experiments conducted using pretrained GVP-GNN models or treating GVP-GNN as a frozen feature extractor to enable a fairer comparison in terms of parameter efficiency and training data usage?\n- What are the key innovations and contributions of VenusX compared to prior work such as ProteinBench [1]?\n\n[1] Ye, F., Zheng, Z., Xue, D., Shen, Y., Wang, L., Ma, Y., … & Gu, Q. (2024). ProteinBench: A holistic evaluation of protein foundation models. arXiv preprint arXiv:2409.067.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- VenusX represents the first large-scale benchmark specifically targeting fine-grained, sub-protein functional understanding, addressing a critical gap in existing protein benchmarks that largely focus on protein-level properties.\n- The benchmark encompasses a diverse set of tasks (residue-level, fragment-level, and pairwise functional similarity), multiple annotation types (active sites, binding sites, epitopes, etc.), and meticulously designed data splits (mixed-family, cross-family, and multiple sequence identity thresholds), enabling a thorough evaluation of model capabilities.\n- The study conducts an extensive assessment of state-of-the-art models across different modalities (sequence-only, structure-only, and sequence-structure hybrids), providing valuable insights into their respective strengths and limitations.\n- All code, data, and a leaderboard will be released as open-source, facilitating community adoption and future research. The methodology for data curation, task formulation, and evaluation is documented in sufficient detail to ensure reproducibility.", "weaknesses": "- Although a wide range of models is evaluated, the paper provides limited insight into why certain approaches—such as sequence-structure hybrids—perform better on cross-family splits. Incorporating ablation studies or attention visualization could shed light on the learned representations and their functional relevance.\n- Performance on the epitope prediction (Epi) task is notably low (AUPR < 0.3), even for top-performing models. A more detailed discussion of the intrinsic challenges of this task, along with potential reasons for the poor results, would strengthen the analysis.\n- The benchmark’s scale (e.g., pairwise similarity tasks involving 177 billion negative pairs) and reliance on large models (e.g., ESM2-3B, ProtT5) entail substantial computational resources for full evaluation. Addressing the computational burden and discussing accessibility for researchers with limited resources would be valuable.\n- For residue-level tasks on the BioLiP (BindB) and SAbDab (Epi) datasets, only sequence-only models are evaluated (as shown in Table 4). Including structure-aware baselines where feasible would offer a more complete assessment of model performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)