Quantifying Cross-Attention Interaction in Transformers for Interpreting TCR-pMHC Binding
Authors: Jiarui Li
Zixiang Yin
Haley Smith
Zhengming Ding
Samuel J Landry
Ramgopal R. Mettu
Venue: ICLR 2026 Poster (2026)
Keywords: CD4+ T cell response
epitope prediction
explainable AI
multi-modal learning
transformer models
deep learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We introduce QCAI, a post-hoc method that incorporates cross-attention into explanations for encoder-decoder transformers; we apply it to analyze TCR–pMHC binding and show that it can successfully interpret experimentally observed interactions.

Abstract: CD8+ “killer” T cells and CD4+ “helper” T cells play a central role in the adaptive immune system by recognizing antigens presented by Major Histocompatibility Complex (pMHC) molecules via T Cell Receptors (TCRs). Modeling binding between T cells and the pMHC complex is fundamental to understanding basic mechanisms of human immune response as well as in developing therapies. While transformer-based  models such as TULIP have achieved impressive performance in this domain, their black-box nature precludes  interpretability and thus limits a deeper mechanistic understanding of T cell response. 
Most existing post-hoc explainable AI (xAI) methods are confined to encoder-only, co-attention, or model-specific architectures and cannot handle encoder-decoder transformers used in TCR-pMHC modeling. To address this gap, we propose Quantifying Cross-Attention Interaction (QCAI), a new post-hoc method designed to interpret the cross-attention mechanisms in transformer decoders. Quantitative evaluation is a challenge for XAI methods; we have compiled TCR-XAI, a benchmark consisting of 274 experimentally determined TCR-pMHC structures to serve as ground truth for binding. Using these structures we compute physical distances between relevant amino acid residues in the TCR-pMHC interaction region and evaluate how well our method and others estimate the importance of residues in this region across the dataset. We show that QCAI achieves state-of-the-art performance on both interpretability and prediction accuracy under the TCR-XAI benchmark.

Published as a conference paper at ICLR 2026
QUANTIFYING CROSS-ATTENTION INTERACTION IN TRANSFORMERS FOR INTERPRETING TCR-PMHC BINDING
Jiarui Li, *Zixiang Yin, "Haley Smith,
1 *Zhnngming Ding, "Samuel J. Landry, Ramgopal R. Mettu
1 Department of Computer Science, Tulane University
2 Department of Biochemistry and Molecular Biology, Tulane University School Medicine
{j1178 zyin, hsmith1. 2dingl, landry, rmettu)@tulane. edu
"Corresponding author.
https://qcal: jiarui, li/
ABSTRACT
CD8+ "killer" cells and CD4+ "helper" T cells play a central role in the adap- tive immune system by recognizing antigens presented by Major Histocompati- bility Complex (pMHC) molecules via T Cell Receptors (TCRs). Modeling bind- ing between cells and the pMHC complex is fundamental to understanding basic mechanisms of human immune response as well as in developing thera- pies. While transformer- based models such as TULIP have achieved impressive performance in this domain, their black-box nature precludes interpretability and thus limits deeper mechanistic understanding of T cell response. Most existing post-hoc explainable AI (xAI) methods confined encoder-only, co-attention, or model-specific architectures and cannot handle encoder-dccoder transformers used in TCR-pMHC modeling. To address this gap, propose Quantifying Cross-Attention Interaction (QCAD, a new post-hoc method designed to inter- pret the cross-attention mechanisms in transformer decoders. Quantitative evalu- ation is a challenge for XAI methods; we have compiled TCR-XAI, a benchmark consisting of 274 experimentally determined TCR-pMHC structures to ground truth for binding. Using these structures we compute physical distances between relevant amino acid residues the TCR-pMHC interaction region and evaluate how well our method and others estimate the importance of residues in this region across the dataset. We show that QCAI achieves state-of-the-an per- formance on both interpretability and prediction accuracy under the TCR-XAI benchmark.
INTRODUCTION
T cells play a pivotal role in the adaptive immune system by identifying and responding to antigenic proteins, both from pathogens such as viruses, bacteria and cancer cells (Joglekar & Li 2021 as well as in the context of autoimmunity. The final and arguably most critical component of T cell response is binding between the peptide Major Histocompatibility Complex (pMHC) which con- tains an antigenic peptide bound to a MHC molecule and the surface receptor on T cells (TCR). The specificity of this interaction underpins T cell-mediated immunity and is an intense area of search in both the development of therapies and fundamental understanding of immune response. Understanding T cell response is the key to vaccines that confer long-lasting immunity, and can also
enable effective personalized cancer therapies (Rojas al. 2023 Poorebrahim et al. 2021).
Transformer models have recently been use to analyze T cell immunity (Hudson et al. 2023 et al.) 2023 Karthikeyan al., 2023 Driessen et al., 2024 Comwall et al.] (023). Specifically, models have been developed to predict TCR-pMHCC binding such as TULIP (Meynard Pigandea
al. 2024). Cross- I (Koyama al. 2023), TCR-BERT (Wu et al. (024b), and

---

Published as a conference paper at ICLR 2026
CDR$a
AL ALGHSSSSWOLI
Encoder-Dacodee Transformer
Binden binder
CDR3b
Encoder
Decoder
ASSLRTGANSDYT
Peptide GVYATSSAVRLR
QCAI
cention
CDR3a Importance.
0800
aEF
CDR3 importance Peptide Importance
Quantifying Cross-Attention Interaction (QCAI) is post-hoc explanation method designed for cross-attention mechanisms. In this paper, we show that QCAI enables insight into the structural basis for TCR-pMHC binding.
Figure 1:
BERTrand Myronov et al. 2023 However these models are black boxes and suffer from lack of interpretability, which is critically important in elucidating the mechanisms involved in cell response. To address this challenge, post-hoc explanation techniques (Kenny et al. 2021) have been developed to connect elements the input and model to the outputs. However, current existing post- hoc methods (e.g., AttaLRP (Achtibat al. 2024), and TokenTM (Wu al. (024a)) are designed for encoder-only transformer or convolution neural network (CNN) models, while state-of-the-art TCR-pMHC binding predictors adopt encoder-decoder architectures.
The main contribution of this paper is to fill this gap with a novel post hoc explanation method that we call Quantifying Cross-Attention Interaction (QCAI) that enables interpretation of any encoder-decoder transformer model while taking cross-attention into account. Our motivation is the application of TCR-pMHC modeling. but cross-attention used extensively in vision and NLP applications as well and thus QCAI has the potential to be applied to other fields in Appendix A. Figure [ shows how QCAI is used analyze decoder blocks; cross-attention between the CDR3 and peptide sequences is captured used to generate importance scores by residue position. Another important contribution this paper is to provide a way to quantify the performance of XAI methods for TCR-pMHC binding. Typically XAI methods are evaluated qualitatively (c.g. in image analysis), but in the context of immunology, interpretations that match intuition are challenging to justify. We introduce TCR-XAI, compilation of 274 structures of TCR-pMHC complexes. For each complex we determine interaction distance between the CDR3 regions and peptide. Using this benchmark, can determine whether the importance scores over the input produced by any particular method matches the expected interaction shown in the experimental structure.
ceerimeentally-dtermmmnn
X-ray cry crystallography
We performed extensive evaluation of QCAI against other post hoc methods and demonstrate the benefit of incorporating cross-aad tross-attention. We conduct an extensive comparison with other methods over the TCR-XAI benchmark and demonstrate that QCAI achieve state-of-the-ae performance. We also analyze two case studies of TCR TCR-pMHC systems highlight mechanisms identified by QCAI
PRELIMINARIES
In this section we first outline some basic concepts for self-attention and cross-attention and then discuss limitations in existing post-hoc methods. Transformer-based architectures typically consist of L stacked encoder layers, or a combination of encoder and decoder layers. Each layer comprises two primary components: Multi-Head Attention (MHA) and Feed-Forwer Network (FFN), each followed by layer normalization and residual connections (Vaswani et al. 2017). The distinction be- tween encoder and decoder modules lies in their input structure and the type of attention mechanism employed.
In the encoder, each layer takes the output of the previous layer 14-1 E RR Nxd xd and computes hy R where N is the number of tokens and d is the hidden dimension. In contrast, the decoder layer integrates two inputs: h111 E Nxd from the previous decoder layer, and iy E TR from
For comprehensive discussion, please consult Section A.4 of the Appendix.
2

---

Published as a conference paper at ICLR 2026
the corresponding encoder layer. The decoder output remains h4 E RNxd with N' denoting the number source tokens.
These inputs linearly projected into query (Qi), key (Ki), and value (V)) matrices for the MHA computation. For encoder, it could be computed following Q: W, h1-1, K W/K - and Vi = W/y 1--1- For decoder, it could be computed following Q: = w 11-1. K₁ = W R12 and V/2 = w/ ki. Where W,O, W/K WY E Rdxd are trainable projection matrices. For brevity, bias terms are omitted. Considering a single attention head for simplicity, the attention matrix A for layer / is computed as:
[QUk vd
A = softmax
The shape of A RNXN for the encoder and R for the decoder. The output of the attention module is computed as: hı WP (ArVi + h1-1) R&xdd where WP E Rdxd is learnable output projection matrix. Outputs from multiple attention heads are concatenated before being linearly transformed.
2.1 LIMITATIONS OF CURRENT INTERPRETABILITY METHODS FOR TRANSFORMERS
Several post-hoc interpretabilityy methods, such as To (Wu et al. 2024a), Rt (Achtibat al. 2024). and AuCAT (Qiang et al. 2022). have demonstrated reliable performance on encoder- only transformer models that rely on self-attention. However, these methods are not designed to extract the interaction information from cross- attention found in decoder layers. As result, their ap- plicability is limited in models that include decoder components, such as TULIP "Meynard-Piganeee
ctt al. 2024) and MixTCRpred (Croce et al. 2024).
The core distinction between self-attention and cross- voos-aatenttion lies in the source of the key (K) and value (V) matrices. While self-attention derives Q, K, and from the same input, cross-attention uses separate inputs for Q and (K, V). Consequently, the attention matrix A in cross-attention has dimensions I N''N instead of R N where N is the number of query tokens and N' is the number of key tokens. Additionally, A now represents the fused information from both modalities. This asymmetry poses challenge for interpretability: the attention matrix longer provides direct measure of query token importance of one input modality, making it difficult to attribute model predictions to input query tokens.
QUANTIFYING CROSS-ATTENTION INTERACTION
In this section present main contribution, which is way handle the aforementioned asymmetry so that cross-attention can be captured. Since the attention matrix is computed as a scaled dot-product QK which captures the cosine similarity between query and key representations, interpreting the cross-attention mechanism can be structured into three key steps: 1. identifying which components of the attention matrix contribute most significantly the model's s prediction, 2. decomposing these importance values into contributions from the query and key inputs, respectively, and 3. aggregating the cross-attention explanation with other layers' explanation.
Inspired by GradCAM (Selvaraju et al. 2017), we propose compute the importance of the atten- tion matrix A; at layer using the gradient of the loss L with respect to A for a target class c, in conjunction with the attention weights themselves. Specifically, we define the importance score map as:
@OL- aA 3 41)) + +IERNNN
S(A:) = Eh ReLU
where (-) denotes averaging across all attention heads, O represents element- wise multiplication, and / denotes the identity matrix for residue connection. This formulation highlights the attention entries that both have high weights and contribute significantly to the class-specific loss. The next step is to quantify this attention importance map into contributions from the query and key inputs. By analyzing the structure of the attention matrix, which serves as a soft alignment between queries and keys, we aim to attribute the importance scores back to the input tokens in both sequences.
3

---

Published as a conference paper at ICLR 2026
3.1 QUANTIFYING QUERY IMPORTANCE FROM CROSS-ATTENTION
For the query input Qi at layer I, its importance scores with respect to the loss L for class can be estimated in GradCAM-style fashion as:
aQ: o Q
@T.
S(Q)) = ReLU
where @ denotes element multiplication. obtain token-level importance scores from this matrix, we compute the column-wise maximum:
wi arg max S(QI) ERR
where i indexes the feature dimension, j indexes the query tokens, and arg max denotes the max- imum across the feature dimension. However, this importance score is intrinsic to Qi itself and does reflect how Qi is influenced by the attention mechanism. Explaining the attention matrix is a key component of post-hoc methods for interpreting transformer models (Wu et 2024a). In the case of tross-attention, the query and key matrices originate from different inputs, and thus the resulting attention matrix is not necessarily square. To better understand how Qr contributes within the attention process, we define antention-conditiiond importance scores S(Qr; A1), the query importance modulated by the attention matrix A₁- We approximate this as:
OL
S(Q Ar)
Qi.
0A,
where matrix product. From the previous step, we have already obtained the attention importance 2L map TA Ar. We now seek transformation that allows us to infer S(Qi; Ai) from this. The attention matrix is computed via scaled dot-product A QiK? with softmax and vd ignored for simplicity. We can express with linear operations (e.g., ReLU, Ex, and (-) + I) ignored for simplicity.:
OL 04,
S(A,)
Q.K.
To isolate the influence of Qi, we need eliminate K,T from the right-hand side. Since is guaranteed to be square or invertible, we employ the Moore-Penrose pseudoinverse:
0L
QiKT =S(Ai)
0A
aL
Qi = SSA)) Ki(K K2) E Nxd
0A,
This yields a decompasition of attention importance into the query space. Then, the importance scores corresponding to the token part can be obtained following:
0L 0A
w arg max
Q1
E
if
where indexes the feature dimension, j indexes the query tokens, and arg max denotes maximum ; taken feature dimension. However, to ensure robustness, particularly in cases where Qi is also influenced by other layers. We conservatively combine this result with the intrinsic query importance:
S(Q;:A)) = max (wF up)
Here, the maximum is applied element-wise to capture the strongest importance attribution from either source.
3.2 QUANTIFYING KEY IMPORTANCE FROM CROSS-ATTENTION
Similar to the approach used to extract query importance scores, the key matrix importance can also be quantified into components: (1) the intrinsic importance of the key matrix, denoted as
4

---

Published as a conference paper at ICLR 2026
S(K)), and (2) the attention-con conditionnd importance, ((K; At), which reflects how the key matrix contributes to the attention mechanism.
The intrinsic importance of the key matrix with respect to the loss L for class can be estimated using a GradCAM-style formulation:
(or, 3 Ki) ki).
(Ki) = ReLU
OK.
To obtain token-level importance scores from this matrix, we compute the column-wise maximum:
wfr = arg max ((K)), E RN
where indexes the feature dimension, j indexes the key tokens, and arg max denotes the maximum + across the feature dimension (d). Similar to the issue encountered in query attention quantifica- tion, the attention matrix is no longer necessarily square for key attention quantification. However, compared to decomposing query importance, extracting key importance from the attention matrix more straightforward, since attention explicitly maps queries into the key space. Thus, we can directly analyze the attention matrix to determine which key tokens exert the strongest influence on the query representations. Because transformer models rely primarily on token-level outputs, we focus on interpreting token-level activations. The attention matrix E N indicates how each query token (rows) attends to the key tokens (columns). To evaluate the overall importance of each key token guiding the query representations, we compute the maximum relevance each key across all queries and attention heads:
w arg max
EN (ReLU
ReLU
ØL
A ...) ))) R.N.
"A.,"
where Σµ denotes averaging over attention heads and i and j index the queries and keys respectively, and arg max denotes the maximum across the feature dimension.
Finally, combine this attention-derived importance with the intrinsic importance to produce robust estimate of key token relevance:
((K; = max (uft", wf²)
where the maximum is taken element wise to reflect the highest attribution signal from either source.
3.3 AGGREGATION LAYER IMPORTANCE SCORES
Inspired by the attention flow perspective (Abnar & Zuidema 2020). we aggregate token-level im- portance scores across layers to track how relevance propagates from the final output back through the decoder and encoder layers. Let & denote the index of the first decoder layer (with cross- attention) encountered when traversing the model from the output layer backwards. All subsequent layers with smaller indices assumed to be encoder layers with self-attention. To capture how importance propagates through these layers, we define the aggregated token-level importance scores at layer k, denoted by Sk. recursively as follows:
SS(kk Ax) 5x41
(query) (key)
S.
S(KK; Ac) Se+1
In models with multiple decoder blocks that contain cross-attention, importance interactions may diverge and converge different points. To handle such cases, we adopt conservative strategy and aggregate importance via element-wiss wise maximum to retain the most influential attribution signal:
max (S(Qu:A)) $et1)
(query)
S max ((KK; Sk+1)
(key)
These recursive rules ensure that attention importance is correctly traced through both cross- attention and self-attention components. Consequently, if the explanation path contains any decoder block with cross-attention, the final output of our QCAI method will a vector token-level importance scores, indicating the contribution of each input token to the model's prediction.
5

---

Published as a conference paper at ICLR 2026
ROC-AUC with Different Contact Distance Thresholds
CDR3a
CDR3b
epitope
0.600
0.56
0.65
0.575
0.54
0.550
0.52
0.60
ROCAUC
0.525
0.500
0.53
0.475
0.46
0.450
0.50
0.44
0.425
0.42
0.45
Distance Thresholds (N)
Distance Thresholds (A)
Distance e Thresholds (M]
QCA)
AttnLRR
AICCAT
GradCAM
RawAttn
Figure 2: ROC-AUC of predicted importance scores for TCR-pMHC binding site identification
across a threshold of interaction distances demonstrates that QCAI surpasses competing methods in all cases.
4 EXPERIMENTAL RESULTS AND DISCUSSION
We first evaluate proposed QCAI method using state-of-art BERT-based model named TULIP, a transformer architecture tailored for predicting TCR-pMHC binding, which focuses on the role of aross-attention and outperforms one of the widely used bascline models, NetTCR-2.2 (Jensen & Nielsen 2024). TULIP adopts an encoder-decoder design and processes three modalities in parallel: CDR3a, CDR3b, and peptide sequences dently transforms input sequences into latent feature representations, while decoder layers model inter-sequence interactions (Devlin et al.) 2019 Vaswani et al.) 2017). As self-regressive genera- tive model, TULIP estimates the conditional probability distribution of each sequence (e.g., peptide) conditioned on the others (e.g.. CDR3a, CDR3b) (Meynard- Meyynadd-Pganneuu al. 2024).
Prigandd al.) 2024). Each encoder indepen-
We compare our QCAI method against several existing post-hoc interpretability techniques, includ- ing AttnLRP (Achtibat et al., (024). Token (Wu al. (024a). AtCAT (Qiang
2022
Rollout (Abnar & Zuidemal (020). GradCAM (Selvaraju et al. (017). LRP (Binder et al. 2016
and Rtwn (Wiegreffe & Pinter 2019). For methods that require aggregation across all layers and compute on attention matrix, we apply them exclusively to the self-attention layers and omit cross- attention components, as these competing methods do not support cross-attention explanations. All experiments were implemented in Python using the Py Torch framework. Testing was conducted on a local workstation equipped with two NVIDIA A2000 GPUs, 16 Intel Xeon E5 CPU cores, and 64 GB of RAM.
4.1 A BENCHMARK FOR TCR-PMHC BINDING INTERPRETATION
To quantitatively assess the quality of interpretability methods, we constructed benchmark that we call TCR-XAI using structural data from TCR- TCR-PMHC complexes. We collected 274 valid samples from the STCRDab (Leem et al., 2018 and TCR3d 2.0 (Lin et al. 2025) datasets, which consist of 213 (77.7%) MHC-I samples and 61 (22.3%) MHC-II samples. Only samples with complete TCR a and |3 chains, full peptide sequences, intact CDR3 regions, and non-overlapping MHC and peptide chain IDs were retained. Statisties of the benchmark set are discussed in Section A.9 of the Appendix. For each sample, we computed residue-level distances: (1) from each CDR residue the closest atom in the peptide, and (2) from each peptide residue to the closest atom any CDR region. A smaller distance indicates a stronger interaction, and atomic distance as a proxy for ground-truth importance. We believe this simple, yet highly useful assumption since protein folding and, by extension, protein-proter interactions are most routinely evaluated by the close- ness of packing, which signals the exclusion of water molecules and demands the formation of all possible hydrogen bonds (without water molecules). Other types of interatomic interactions such as hydrophobic contacts and ionic bonds contribute to binding, but they generally cannot be re- alized without exclusion of water. Thus, the formation of stable protein-protein interface has sharp distance threshold, above-which the interaction is not likely to be stable. The importance of residue-level distance is evident in prior work, starting with TCRdist /Cddisst|Mayer-aree Slayer-Blackwelll al. (2021)

---

Published as a conference paper at ICLR 2026
classic unsupervised method for TCR-pMHE binding prediction. It defines the TCRdist distance as "the weightt mismatch distance between the potential PMMC-conttctinne loops of the two receptors." Using distance as an indicator is also common in TCR-pMHC transformer model explanations, though typically for qualitative rather than quantitative evaluation (e.g., PISTE (Feng et al. 2024) and TCR-BERT (Wu et al. 2024b)).
4.2 ROC ANALYSIS AND PERTURBATION EXPERIMENTS
To evaluate the explainability of different post-hoc interpretation methods, we quantitatively assess their ability to identify true TCR-pMHC binding sites using the TCR-XAI benchmark. We computed ROC curves (ROC curves for each threshold are shown in Section in the Appendix.) by compar- ing predicted residue importance against ground-truth binding site annotations derived from struc- tural data, where the ground-truth was defined according to distance threshold between and A, with the ROC using predicted importance as the threshold. As shown in Figure 2 QCAI achieves AUCs of 0.5492, 0.5489, and 0.6024 for CDR3a, CDR3b, and peptide respectively and consistently outperforms other methods. Notably, QCAI exceeds 0.6 on the peptide chain, demonstrating strong alignment between its predicted importance scores and the underlying structural binding interac- tions.
We also conducted perturbation studies on to assess whether each method identifies important residues; we adopt two commonly used metries: Log-Odds Score (LOddss and Area Over the Per- turbation Curve (AOPC). AOPC measures explanation quality by averaging the drop in model con- fidence as top-k important features are removed. Higher AOPC indicates better alignment between explanation and model behavior. LOdds computes the change in log-odds of the model's prediction before and after perturbing a feature. A larger LOdds value indicates greater importance of the per- turbed feature. Perturbation is implemented by replacing the k highest-scoring tokens with padding tokens (< PAD >). We evaluate the CDR3a, CDR3b and peptide chains separately, with k=4 for the CDR3a and CDR3b chains, and k=7 for peptides to match the average number of predicted binding residues across TCR-XAI.
Chains
CDR3a km4
CDR3b
Peptide km
km4
LOdds
AOPC
LOdds
AOPC
LOdds
AOPC
QCAI (Ours)
-3.328
0.014
-3.498
0.045
-1.470
0.013
AttaLRP (Achtibtt et al.
2024
-2.481
0.020
-2.662
0.032
-0.017
0.000
Token'IM (Wu et al.
2024a)
-2.195
0.021
-2.383
0.032
-0.736
0.012
AtСAT (Qang al. (2022)
-2.825
0.020
-3.131
0.044
-0.694
0.006
Rollout Abnar & Zuidema,
2020
-2.356
0.022
-2.653
0.032
-0.044
-0.001
GradCAM (Selvaraju et
2017)
-2.700
0.019
-3.112
0.038
-1.004
0.009
LRP BBiner et al., 2016)
-2.938
0.020
-3.127
0.043
-1.167
0.011
RawAltn (Wiegreffe & Pinter
2019)
-2.334
0.015
-3.250
0.039
-0.691
0.010
Table I: Perturbation experiment results using fixed thresholds. Thresholds for the and 3 chains
are 4-4, and for the peptide chain k=77 The average number of binding regions are 3.64, 4.12, and 7.05 for a, 3, and peptide chains respectively.
Table [ shows that QCAI consistently outperforms other methods across most metrics. QCAI achieves the most negative LOdds and highest AOPC scores in the CDR3b and peptide chains, indicating greater disruption to the model's confidence when informative residues are perturbed. Although Rollout outperforms QCAI in AOPC on the CDR3a chain, QCAI still achieves the best LOdds score.
4.3 IDENTIFICATION BINDING REGION RESIDUES WITH IMPORTANCE SCORES
Using the TCR-XAI benchmark we construct evaluation metric that we call Binding Region Hit Rate (BRHR). To compute BRHR, we first choose a percentile threshold t E (0, 1] and identify the top fraction of residues with respect to highest importance scores S. Each of these residues is marked a hit if its interaction distance is in the top fraction of interaction distances. We compute the hit rate for each input sequence type in each sample and take the mean across TCR-XAI obtain

---

Published as a conference paper at ICLR 2026
TCR-pMHC Interaction Hit Rate
CDR3a
CDR3b
épitope
0.8
0.8
0.7
0.7
0.6
Nit rate
0.6
0.6
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.3
0.4
0.5
0.6
0.7
0.3
0.4
0.5
0.6
0.7
0.3
0.4
0.5
0.6
0.7
Percentile Distance/lportance
Percentile Distanceltmsortance
Percentile Distancelmportance
TokenTT
Rollout
LRP RowAttn
AttnLRP AItCAT
GradCAM
Figure 3: Comparison of TCR-pMHC Binding Region Hit Rate (BRHR) across different methods on different chains. At any selected percentile of distance/imporlance,, the higher the hit rate the more closely the importance tracks physical interaction distance. QCAI surpasses other methods in all practical cases.
the final BRHR. This metric reflects the proportion of true binding residues (according to structural proximity) that successfully identified by the explanation method.
our method achieves state-of-the-art performance compared to all other ex- planation methods. For the peptide chain, method consistently outperforms all other methods before the 50th percentile. After this threshold other methods prevail but have high false positive rate of other methods (as seen in the ROC analysis). We postulate that the latter effect due the fact that these methods can only access self-atte attention weights from the encoder and cannot benefit from the regulatory influence of cross- -oossaattention layers.
As shown in Figure D]
4.4 CASE STUDIES
To highlight the ability of QCAI assist in the interpretation of TCR-pMHC binding we discuss two specific examples, one for CD8+ T cells and one for CD4+ T cells. In both cases the anal- ysis of importance using QCAI finds residue positions in CDR3s that form critical contacts with epitope peptides and, by revealing unconstrained positions in longer CDR loops, can explain large differences in TCCR-pepide--HL.a binding affinity.
In the first case study (Figure 4ja)) we consider the immunodominant CD8+ T-cell peptide from the influenza matrix protein which has been used to understand influenza T cell response. Multiple crystal structures (IOGA (Stewart-Jones et al.] 2003) and 5TEZ (Yang al., (017)) of different TCRs recognizing this peptide have revealed a common mode of binding that involves the insertion of single CDR3b sidechain (R98 in the 1OGA structure) into notch between the peptide and the HLA-A2 alpha-2 helix and, otherwise. makes numerous contacts with the HLA-A2, whose shape depends the peptide. In distinct example, the TCR in the 5TE7. structure is rotated by 40 degrees around the HLA-TCR axis to create a very different group of TCCR-HLA-A2 contacts, but this TCR also places CDR3b sidechain (W99 in the STEZ structure) in the notch between peptide and HLA-A2 alpha-2 helix. Consistent with the common aspect of binding, the QCAI evaluation finds importance in the position of the notch-binding residue and in several N-terminal flanking positions of CDR3b. The distinct aspect of binding for the two TCRs arises in the longer and less- constrained CDR3a for the STEZ TCR, which may explain its 25-fold lower affinity than for the 10GA TCR. We note that for both binding orientations, AttnL and Token'IM produce weaker importance scores overall.
The second case study considers self-antigen in the autoimmune disease of rheumatoid arthritis. The HLA-DD4-bound citrullinated peptide, named vimentin-64cit59-711 has been analyzed in the complex with two different TCRs (Loh et al. 2024h (indicated with PDB codes, 8TRR and 8TRQ in Figure
4 b)). The QCAI evaluation finds an overall similar number of important positions in the two TCRs, including concentration of importance along one edge of the hairpin formed by the CDR3a in both TCRs (highlighted with a dark outline in Figure 4b)). The CDR3a contributes the largest direct contact with the peptide in both complexes. Interestingly, the CDR3b of the S-loer-lower-
8

---

Published as a conference paper at ICLR 2026
Distance
QCAI
At
TM
CDR3a
1OGA
CDR3b
R98
R98
F98
R98
epitope
CDR3b
5TEZ
W99
W99
W99
W99
+
022
0.8
epitope
(a)
ostance
0.4
Aepezooo -0.4
1066
8TRR
CDR3a
CDR3b
0.6
00
epitape
1.0
CDR3a
8TRQ
CDR3b
epitope
26 (b)
Figure 4: Case studies on systems from TCR-XAL (a) We consider the same TCR-pMHC bound in two distinct binding orientations. For this system QCAI identifies key residues from both orien- tations. (b) We consider the same pMHC bound to two distinct TCRs. Here QCAI identifies the importance of the hairpin region of CDR3a in both cases.
affinity 8TRQ complex is longer and contains more positions of lower importance, again suggesting that the entropic cost of ordering this loop is responsible for the reduced affinity. For this case study, AttnLRP does not produce meaningful results while TokenTM does not capture the importance of residues in the peptide proximal the CDR3a hairpin.
To investigate how QCAI explanations differ for similar TCR-pMHC complexes, we conducted a case study on two TCR-M TCR-MHCII- pectide structures, 2PXY and 2231, which investigates whether a germline-encoded motif structurally guides TCR recognition of MHC (Feng et al., 2007). They differ by two amino acids in the CDR3b loop (Feng et al. 2007). To convince chain alignment, amino acids were re-indexed starting from for each chain. As shown in Figure 5 QCAI with TULIP assigns similar importance scores to the peptide in both complexes but produces different pattern of importance for the CDR3b loop. Both complexes correctly highlight AS as important contact region, and QCAI identifies additional contact sites in 2PXY. In 2PXY, residues S6 and G7 receive higher scores, whereas the corresponding region in 2231 receives lower scores, where also the contact regions. These results indicate that QCAI can detect critical contact regions even with minor sequence changes. However, such changes affect the overall explanation quality.

---

Published as a conference paper at ICLR 2026
CDRBa
CDR3b
CDR3s
2PXY
2231
A
E
N
N
2PXY
4
5
9
10
12
2PXY
A
6
D
A
A
E
T
A
G
D
A
G
N
Paptide
5
9
CDR3a
CDR3b
Poptide
6
A
Y
2231
6
A
5
5
2
5
7
8
9
10
11
2231
Residue No.
Importance
0.88
Peptide
0.2
Figure 5: Case studies of two closely related TCR-pMHC complexes from TCR-XAL. These com- plexes differ by only two amino acids in the CDR3b, highlighted in the figure with red rectangles.
5 CONCLUSIONS
this paper, we present Quantifying Cross-Attention Interaction (QCAI) to interpret the cross- attention in the decoders of transformer models, aiming to better understand encoder-decoder TCR- pMHC binding prediction models. QCAI quantifies the importance the cross cross-ateention matrix into contributions from query and key inputs, revealing how they influence each other. To rigorously evaluate the explanations, we created a new structural explanation benchmark, TCR-XAI, along with novel evaluation metric, the Binding Region Hit Rate (BRHR). On this benchmark, QCAI achieves state-of-the-art results across perturbation metries (LOdds and AOPC), ROC-AUC, ROC curve analysis, and BRHR.
5.1
FUTURE WORK
In future work, we plan to pursue two primary directions: (1) extending the metrics used to evaluate explainability, and (2) applying QCAI broader range of immunological and protein-protein inter- action tasks. Beyond distance- based measures, energy functions (e.g.. REFI5 (Alffrrd et al., (017)) offer a promising alternative for quantifying explanations in TCR-pMHC binding prediction. Inves- tigating a range of energy-based models to better understand the relationship between explainability and protein energetics will be important next step.
Given the emergence of several cross atoss-attention models for protein-protedd interactions and immuno- logical tasks, such PALM-H3 (He al., 2024) for antigen generation, UniPMT (Zhao 2025) for peptide-MHH prediction, ProtAntB (Liu et al., 2025) for antibody-antigen prediction, and HB- Former (Zhagg et al. 2024) for human- virus interaction identification, QCAI provides method for opening the black box of cross-attentions in these models and revealing their underlying mecha- nisms. In addition, QCAI can be extended beyond these applications. For instance, we have already applied it to CLIP encoders with cross-attention, as discussed in Appendix [A12 Exploring broader applications of QCAI across these tasks and domains is an important direction for future work.
Transformers widely used for TCR- TCR-pMHC binding prediction, but they remain black-box mod- els. While post-hoc methods like QCAI improve explainability, they cannot directly integrate these insights into prediction. Beyond post-hoc methods, important future direction is to develop explain-by by-inn-b--eiignn models that provide inherent explainability and utilize mechanistic TCR-pMHC insights improve predictive performance.
Code Availability: The source code is publicly available at our project website (hrtae::/ppppp::cooo:: jiarui. 117) and GitHub htap::://gttte thub. http:///ggghhhaccoc/o/aaaaa00110
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors acknowledge support from the Harold L. and Heather E. Jurist Center of Excellence for Artificial Intelligence at Tulane University. This work was also supported by the National Institutes Health (U54-CA26058I) through the Tulane University COVID Antibody and Immunity Network (TUCAIN): Tulane SOM Pilot Funding for "MHCII Pathway Processing of SARS-CoV-2 Spike"; the Tulane Center of Excellence for Emerging and Re-emerging Infectious Diseases (CEERID) Pilot Research Program for "Large- Scale Validation of Novel Parallel Algorithm for Computational Epi- tope Prediction"; and the Lavin-Beriick Faculty Grant Proposal Research and Scholarly Activities Support for "Research Trainee Support for Modeling Antigen Processing and HLA Immunopep- tidomics".
REFERENCES
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4190-4197, 2020.
Reduan Achtibat, Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Aakriti Jain, Thomas Wiegand, Sebastian Lapuschkin, and Wojeiech Samek. Attnlrp: Attention-awar layer-wise rele- vance propagation for transformers. In Forty-first International Conference on Machine Learning, 2024.
Rebecca F Alford, Andrew Leaver-Fay, Jeliazko R Jeliazkov, Matthew J O' O'Meara, Frank P DiMaio, Hahnbeom Park, Maxim V Shapovalov, Douglas Renfrew, Vikram K Mulligan, Kalli Kappel, et al. The rosetta all-atom energy function for macromolecular modeling and design. Journal of chemical theory and computation, 13(6):3031-3053 2017.
Ameen Ali, Thomas Schnake, Oliver Eberle, Grégoire Montavon, Klaus-Rober Müller, and Lior Wolf. Xai for transformers: Better explanations through conservative propagation. Interna- tional conference machine learning, pp. 435-451. PMLR, 2022.
Dmitry V Bagaev, Renske MA Vroomans, Jerome Samir, Ulrik Stervbo, Cristina Rius, Garry Dolton, Alexander Greenshields- Watson, Meriem Attaf, Evgeny S Egorov, Ivan V Zvyagin, et al. Vdjdb in 2019: database extension, new analysis infrastructure and a t-cell receptor motif com- pendium. Nucleic acids research, 4801)): 2020.
Avik Bhattacharya, James Wrabl, Samuel Landry, and Ramgopal R Mettu. Parallel computa- tion of conformational stability for cd4+ t-cell epitope prediction. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 88-93. IEEE, 2023.
Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin. Klaus Klaus-Robert Müller, and Wojeiech Samek. Layer-wise relevance propagation for neural networks with local renormalization layers. In Artificial Neural Networks and Machine Learning-ICANt 2016: 25th International Confer- ence on Artificial Neural Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part 25, pp. 63-71. Springer, 2016.
Rémy Bosselut. T cell antigen recognition: Evolution-driven affinities. Proceedings of the National Academy of Sciences, 116(44):2196-2116 2019.
Tysheena Charles, Daniel L Moss, Pawan Bhat, Peyton Moore, Nicholas Kummer, Avik Bhat- tacharya, Samuel J Landry, and Ramgopal R Mettu. Cd4+ t-cell epitope prediction by combined analysis antigen conformational flexibility and peptide-mheht binding affinity. Biochemistry, 61(15):1555-159) 2022.
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE winter conference on applications of computer vision (WACV), pp. 839-847. IEEE, 2018.
Hila Chefer, Shir Gur, and Lior Wolf. Generic attention-mon explainability for interpreting bi- modal and encoder-decoder transformers. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pp. 397-406, 2021.
11

---

Published as a conference paper at ICLR 2026
Junwei Chen, Bowen Zhao, Shenggeng Lin, Heqi Sun, Xueying Mao, Meng Wang, Yanyi Chu, Liang Hong, Dong-Qing Wei, Min Li, et al. Tepcam: Prediction of t-cell receptor receptor-epitope epitope binding specificity via interpretable deep learning. Protein Science, 33( (1)())) :ee481, 2024.
Lewis Cornwall, Grisha Szep, James Day. SR Gokul Krishnan, David Carter, Jamie Blundell, Lilly Wollman, Neil Dalchau, and Aaron Sim. Fine-tuned protein language models capture t cell re- ceptor stochasticity. In NeurIPS 2023 Generative Al and Biology (GenBio) Workshop, 2023.
Giancarlo Croce, Sara Bobisse, Dana Léa Moreno, Julien Schmidt, Philippe Guillame, Alexandre Harari, and David Gfeller Deep learning predictions of tcr-epitope interactions reveal epitope- specific chains in dual alpha cells. Nature Communications, 15(1):3211, 2024.
Pradyot Dash, Andrew J Fiore-Gartland, Tomer Hertz, George C Wang. Shalini Sharma, Aisha Souquette, Jeremy Chase Crawford, E Bridie Clemens, Thi HO Nguyen, Katherine Kedzierska, et al. Quantifiable predictive features define epitope-specific t cell receptor repertoires. Nature, 547(7661):89-93,, 2017.
Mark M Davis and Pamela.J J Bjorkman. T-cell antigen receptor genes and t-cell recognition. Nature, 334(6181):395-402, 1988.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Alice Driessen, Jannis Born, Rocío Castellanos Rueda, Sai T Reddy, and Marianna Rapsomaniki. Modeling car response the single-cell level using conditional ot. In NeurIPS 2024 Workshop on Al for New Drug Modalities, 2024.
Rudresh Dwivedi, Devam Dave, Het Naik, Smiti Singhal, Rana Omer, Pankesh Patel, Bin Qian, Zhenyu Wen, Tejal Shah, Graham Morgan, al. Explainable ai (xai): Core ideas, techniques, and solutions. ACM Computing Surveys, 55(9):1-33, 2023.
Dan Feng, Christopher Bond, Lauren K Ely. Jennifer Maynard, and K Christopher Garcia. Struc- tural evidence for a gernline-encoded cell receptor-major histocompatibility complex interac- tion's cono'o'.'. Nature immunology, 0000997593333 2007.
Ziyan Feng, Jingyang Chen, Youlong Hai, Xuelian Pang, Kun Zheng, Chenglong Xie, Xiujuan Zhang. Shengqing Li, Chengjuan Zhang, Kangdong Liu, et al. Sliding-attention transformer neu- ral architecture for predicting cell sccettr-antigen--umme Machine Intelligence, 6(10): 1216 2024.
leucocyte antigen binding. Nature
Jacob Glanville, Huang Huang, Allison Nau, Olivia Hatton, Lisa Wagar, Florian Rubelt, Xuhuai Ji, Arnold Han, Sheri M Krams, Christina Pettus, et al. Identifying specificity groups in the cell receptor repertoire. Nature, 547(7661):94-98,, 2017.
Haohuai He, Bing He, Lei Guan, Yu Zhao, Feng Jiang, Guanxing Chen, Qingge Zhu, Calvin Yu- Chian Chen, Ting Li, and Jianhua Yao. De novo generation of sars-cov-2 antibody cdrh3 with pre-trained generative large language model. Nature Communications, 15(1):6867, 2024.
Ilka Hoof, Bjoern Peters, John Sidney, Lasse Eggers Pedersen, Alessandro Sette, Ole Lund, Søren Buus, and Morten Nielsen. Netmhepan, method for mhe class i binding prediction beyond humans. Immunogenetics, 61:1-13, 2009.
Huang Huang, Chunlin Wang, Florian Rubelt, Thomas Scriba, and Mark M Davis. Analyzing the mycobacterium tuberculosis immune response by t-cell receptor clustering with gliph2 and genome-wee wide antigen screening. Nature biotechnology, 38(10): 1194 -902 2020.
Dan Hudson, Ricardo A Fernandes, Mark Basham, Graham Ogg, and Hashem Koohy. Can we predict t cell specificity with digital biology and machine learning? Nature Reviews Immunology, 23(8):511-521 2023."
Dan Hudson, Alex Lubbock, Mark Basham, and Hashem Koohy. A comparison of clustering models for inference of t cell receptor antigen specificity. Inmunoliformatics, 13: 100033, 2024.
12

---

Published as a conference paper at ICLR 2026
Mathias Fynbo Jensen and Morten Nielsen. Netter 2.2- i.2-impreeed ter specificity predictions by com- bining pan-and peptide-speciffs training strategies, loss ssaling and integration of sequence simi- larity. eLife, 12. 2024.
Alok V Joglekar and Guideng Li. cell antigen discovery. Nature methods, 18(8):873-88) 880, 2021.
Edita Karosiene, Claus Lundegaard, Ole Lund, and Morten Nielsen. Netmhccons: a consensus method for the major histocompatibility complex class predictions. Immunogenetics, 64:177-
186, 2012.
Dhuvarakesh Karthikeyan, Colin Raffel, Benjamin Vincent, and Alex Rubinsteyn. Conditional gen- eration of antigen specific t-cell receptor sequences. In NeurIPS 2023 Generative Al and Biology (GenBio) Workshop, 2023.
Eoin M Kenny, Courtney Ford. Molly Quinn, and Mark T Keane. Explaining black-box classifiers using post-hoc leeaanations-b--xxmmpp The effect of explanations and error-rates in xai user studies. Artificial Intelligence, 294:103459 2021.
Yohan Kim, John Sidney, Clemencia Pinilla, Alessandro Sette, and Bjoern Peters. Derivation of amino acid similarity matrix for peptide: Mhc binding and its application bayesian prior. BMC bioinformatics, 10:1-11, 2009.
Kyohei Koyama, Kosuke Hashimoto, Chioko Nagao, and Kenji Mizuguchi. Attention network for predicting t-cell receptor-peptide binding can associate attention with interpretable protein structural properties. Frontiers in Bioinformatics, 3:1274599, 2023.
Bjørn PY Kwee, Marius Messemaker, Eric Marcus, Giacomo Oliveira, Wouter Scheper, Catherine Wu, Jonas Teuwen, and Ton N Schumacher. Stapler: efficient learning of tcr-peptide specificity prediction from full-length ter ter-peptide data. bioRxiv, pp. 2023-022200 04, 2023.
Jinwoo Leem, Saulo H P de Oliveira, Konrad Krawczyk, and Charlotte M Deane. Sterdab: the structural t-cell receptor database. Nucleic acids research, 46(D1):D406-D411 2018.
Jiarui Li, Samuel J Landry, and Ramgopal Mettu. Gpu acceleration of conformational stabil- ity computation for cd4+ t-cell epitope prediction. In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 191-196. IEEE, 2024a.
Jiarui Li, Samuel J Landry, and Ramgopal R Mettu. Gpu acceleration for markov chain monte carlo sampling. In 4th International Conference AMML.Systent (AIMLSSystems 2024). ACM, 2024b.
Tianxiao Li, Hongyu Guo, Filippo Grazioli, Mark Gerstein, and Martin Renqiang Min. Disentan- gled wasserstein autoencoder for t-cell receptor engineering. Advances in Neural Information Processing Systems, 36:73604-73632, 2023.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Valerie Lin, Melyssa Cheung, Ragul Gowthaman, Maya Eisenberg, Brian M Baker, and Brian G Pierce. Ter3d 2.0: expanding the t cell receptor structure database with new structures, tools and interactions. Nucleic Acids Research, 53(D1) D6004 2025.
Chen Liu, Mingchen Li, Yang Tan, Wenrui Gou, Guisheng Fan, and Bingxin Zhou. Sequence-only prediction of binding affinity changes: robust and interpretable model for antibody engineering. Bioinformatics, b188).b 2025.
Tiing Jen Loh, Jia Jia Lim, Claerwen M Jones, Hien Thy Dao, Mai Tran, Daniel G Baker, Nicole L La Gruta, Hugh Reid, and Jamie Rossjohn. The molecular basis underlying cell specificity towards citrullinated epitopes presented by hla-dr4. Nature Communicationss, 15(1): 2024.
Claus Lundegaard, Kasper Lamberth, Mikkel Harndahl, Søren Buus. Ole Lund, and Morten Nielsen. Netmhe-3.0 accurate web accessible predictions of human, mouse and monkey mhe class affini- ties for peptides of length 8-11. Nucleic acids research, 36(suppl 2):W599-W51 2008.
13

---

Published as a conference paper at ICLR 2026
Koshlan Mayer-Blackwell, Stefan Schattgen, Liel Cohen-Lavi, Jeremy C Crawford, Aisha Sou- quette, Jessica A Gaevert, Tomer Hertz, Paul G Thomas, Philip Bradley, and Andrew Fiore- Gartland. Ter meta-clonotypes for biomarker discovery with tcrdist3 enabled identification of public, hla-restricted clusters of sars-cov-2 ters. Elife, 10:e68605, 2021.
Ramgopal R Mettu, Tysheena Charles, and Samuel Landry. Cd4+ t-cell epitope prediction using antigen processing constraints. Journal of immunological methods, 432:72-81, 2016.
Barthelemy Meynard-Piganeuu Christoph Feinauer, Martin Weigt, Aleksandra M Walczak, and Thierry Mora. Tulip: A transformer-basee unsupervised language model for interacting peptides and cell receptors that generalizes unseen epitopes. Proceedings of the National Academy of Sciences, 121(24):a23 16401 121, 2024.
Alexander Myronov, Giovanni Mazzocco, Paulina Kröl, and Dariusz Plewezynski. Bertrand Bertrand-peptide: 'er binding prediction using bidirectional encoder representations from transformers augmented with random pairing. Bioinformatics, 39(8) 2023.
Jacques Neefjes, Marlieke LM Jongsma, Petra Paul, and Oddmund Bakke. Towards a systems understanding of mhc class and mhe class antigen presentation. Nature reviews immunology, 11(12):823-836 2011.
Morten Nielsen, Claus Lundegaard, Peder Worning, Sanne Lise Lauemoller, Kasper Lamberth, Søren Buus, Søren Brunak, and Ole Lund. Reliable prediction of t-cell epitopes using neural networks with novel sequence representations. Protein Science, 12(5):0007-1017, 2003.
Morten Nielsen, Claus Lundegaard, Thomas Blicher, Kasper Lamberth, Mikkel Harndahl, Sune Justesen, Gustav Røder, Bjoern Peters, Alessandro Sette, Ole Lund, al. Netmhepan, method for quantitative predictions of peptide binding to any hla-a and-b locus protein of known sequence. PloS one, 2(8):779, 2007.
Morten Nielsen, Massimo Andreatta, Bjoern Peters, and Søren Buus. Immunoinformatics: predict- ing peptide-mhe binding. Annual review of biomedical data science, 3(1):191-215, 2020.
Bjoern Peters and Alessandro Sette. Generating quantitative models describing the sequence speci- ficity of biological processes with the stabilized matrix method. BMC bioinformatics, 6(1):1-9, 2005.
Bjoem Peters, Morten Nielsen, and Alessandro Sette. T cell epitope predictions. Annual Review of Immunology, 38(1):123-145) 2020.
Mansour Poorebrahim, Niloufar Mohammadkhani, Reza Mahmoudi, Monireh Gholizadeh, Elham Fakhr, and Angel Cid-Arregui. Ter-like cars and ter-cars targeting neoepitopes: an emerging potential. Cancer gene therapy, 28(6):581-589, 2021.
Yao Qiang, Deng Pan, Chengyin Li, Xin Li, Rhongho Jang, and Dongxiao Zhu. Attcat: Explaining transformers via attentive class activation tokens. Advances in neural information processing systems, 35:5052-5064, 2022.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language under- standing by generative pre-training. OpenAI, 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 9748-8763. PmLR, 2021.
Luis A Rojas, Zachary Sethna, Kevin C Soares, Cristina Olcese, Nan Pang, Erin Patterson, Jayon Lihm, Nicholas Ceglia, Pablo Guasp, Alexander Chu, et al. Personalized ma neoantigen vaccines stimulate cells in pancreatic cancer. Nature, 61 (177766) 2023.
Saranya and Subhashini. A systematic review of explainable artificial intelligence models and applications: Recent developments and future trends. Decision analytics journal, 7:100230, 2023.
14

---

Published as a conference paper at ICLR 2026
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Ido Springer, Nili Tickotsky, and Yoram Louzoun. Contribution of cell receptor alpha and beta cdr3, mhe typing, and genes to peptide binding prediction. Frontiers in immunology, 12: 664514, 2021.
Guillaume BE Stewart-Jones, Andrew McMichael, John I Bell, David I Stuart, and E Yvonne Jones. A structural basis for immunodominant human cell receptor recognition. Nature im- munology, 4(7):657-663, 2003.
Nili Tickotsky, Tal Sagiv, Jaime Prilusky, Eric Shifrut, and Nir Friedman. Mepas-ter: a manu- ally curated catalogue of pathology-associane I cell receptor sequences. Bioinformatics, 33(18): 2924-2929, 2017.
Sebastiaan Valkiers, Max Van Houcke, Kris Laukens, and Pieter Meysman. Cluster: python interface for rapid clustering of large sets of cdr3 sequences with unknown antigen specificity. Bioinformatics, 37(24))4865-4867, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- tion processing systems, 30, 2017.
Randi Vita, Swapnil Mahajan, James A Overton, Sandeep Kumar Dhanda, Sheridan Martini, Jason R Cantrell, Daniel Wheeler, Alessandro Sette, and Bjoern Peters. The immune epitope database (iedb): 2018 update. Nucleic acids research,
(DD91))333-D3455 2019.
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. -808,, 2019.
Anna Weber, Jannis Born, and María Rodriguez Martínez. Titan: T-cell receptor specificity predic- tion with bimodal attention networks. Bioinformatics, 37(Supplement 1): 1):1237-1244, 2021.
Sarah Wiegreffe and Yuval Pinter. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIP), pp- 11-20, 2019.
Junyi Wu, Bin Duan, Weitai Kang, Hao Tang, and Yan Yan. Token transformation matters: Towards faithful post-hoc explanation for vision transformer. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 10926-10935, 2024a.
Kevin E Wu, Kathryn Yost, Bence Daniel, Julia Belk, Yu Xia, Takeshi Egawa, Ansuman Satpathy, Howard Chang, and James Zou. Ter-bert: learning the grammar of t-cell receptors for flexible antigen-binding analyses. In Machine Leurning in Computational Biology, pp. 194-229, PMLR, 2024b.
Xinbo Yang. Guobing Chen, Nan-ping Weng, and Roy A Mariuzza. Structural basis for clonal diversity of the human t-cell response to a dominant influenza virus epitope. Journal of Biological Chemistry, 292(45):18618 1611--17111881 2017.
Hongyi Zhang, Longchao Liu, Jian Zhang, Jiahui Chen, Jianfeng Ye, Sachet Shukla, Jian Qiao, Xiaowei Zhan, Hao Chen, Catherine Wu, et al. Investigation of antigen-specific t-cell receptor clusters in human cancers. Clinical Cancer Research, 26(6):1359-1377 2020.
Hongyi Zhang, Xiaowei Zhan, and Bo Li. Giana allows compuutaiionalll-efffiiii ter clustering and multi-disease repertoire classification by isometric transformation. Nature communications, 12 (1):4699, 2021.
Liyuan Zhang, Sicong Wang, Yadong Wang, and Tianyi Zhao. Hbformer: single-stream frame- work based on hybrid attention mechanism for identification of numan-virus virus protein-protein in- teractions. Bioinformatics, 40(1 bbtaae,,,, 2024.
15

---

Published as a conference paper at ICLR 2026
Yunxiang Zhao, Jijun Yu, Yixin Su, You Shu, Enhao Ma, Jing Wang. Shuyang Jiang, Congwen Wei, Dongsheng Li, Zhen Huang, et al. unified deep framework for peptide-may histocompatibility complex-t cell receptor binding prediction. Nature Machine Intelligence, pp. 1-11, 2025.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedingy of the IEEE conference on computer vision and pattern recognition, pp. 2921-2929, 2016.
16

---

Published as a conference paper at ICLR 2026
A SUPPLEMENTARY MATERIAL
A.1 POST-HOC EXPLANATION METHODS
A variety of explainable AI (XAI) methods have been developed interpret deep learning mod-
els (Saranya & Subhashini 2023). These methods fall into two broad categories: explain-by-desigen which integrates interpretability into the model architecture (Dwivedi et al.] (023), and post-hoc, which analyzes model behavior after training (Kenny et al., 2021). Post-hoc approaches offer a promising avenue for interpreting TCR-pMHC models and uncovering the underlying factors driv- ing binding predictions. Several families of post-hoc methods have been proposed, including:
The Class Activation Map (CAM) (e.g.. CAM (Zhou et al. 2016), GradCAM Selvaraju
et al.] 2017). GradCAM+ (Chattopadhay et al. 2018))
Layer-wise Relevance Propagation (LRP) (e.g.. LRP (Binder al. 2016 Partial
LRP (Voita et al. 2019). Conservative LRP (Ali et al.
2022). Att
(Achtibat et al.
2024g)
Attention-based methods (c.g., Raw Attention (Wiegreffe & Pinter 2019). Attention Roll-
out (Abnar & Zuidema, 2020), AuCAT (Qiang et al. [0222)
Model-specific hybrid methods (e.g.. Token'IM (Wu et al. 2024a). GAE (Chefer et al. 202I))
These techniques have been successfully applied to TCR-pMHC models. For example, TEPCAM interpret altention-CNNN with attention map (Chen et al. (024). while TCR-BERT relies on tention weight analysis for interpretability (Wu et al. 2024b). These efforts have revealed struc- tural determinants of TCR-pMHC binding. However, existing post-hoc methods primarily support encoder-only co-attention mechanisms (Chefer et al. (202T) limiting their applicability to modern incodrrddecoee models, which consists of cross-attention. This poses major barrier to understand- ing how such models capture TCR-pMHC interactions.
A.2 CLASS ACTIVATION MAPS
Class Activation Map (CaM)-based methods have achieved significant success in explaining Con- volutional Neural Networks (CNNs) by generating class-discriminative localization maps. Grad- CAM (Selvaraju al. 2017), one of the most effective CAM methods, leverages the gradient the class score L' with respect to the feature maps Fa from the last convolutional layer. These gradients are used to compute importance weights for each feature map channel, enabling spatial localization of the regions most relevant for class c. The importance weight u for feature map Fa is computed as:
OL OF.
- = E
where E denotes global average and w represents the global average pooled gradient for feature map Fg. The final CAM is then computed as weighted sum over channels, followed by a ReLU activation:
GradCAM ReLU
Σ UFFF
The resulting heatmap is upsampled to the input resolution to highlight input regions most relevant to the prediction for class c.
A.2.1 ATTENTION ROLLOUT
CAM-based approaches primarily designed for CNNs. To interpret transformer-based models, Attention Rollout was proposed by Abnar & Zuidema (2020), which estimates the flow of attention across layers. This method computes how information propagates through the self-attention mecha- nism across layers. Given the raw attention weights / for layer the augmented attention matrix defined as
1 (W A + I), A
2
17

---

Published as a conference paper at ICLR 2026
where I is the identity matrix, modeling the residual connection. rollout, is then computed recursively:
The cumulative attention, or
[1 if 0
R
AA,
capturing the total attention contribution from input tokens through layer L.
A.3 TCR-PMHE BINDING PREDICTION
T cells are important component of our immune system, which can be mainly catogorized in two CD8+ and CD4+ T cells. CD8+ T cells are initiated through the Major Histocompatibility Com- plex (MHCI) pathway, while CD4+ T cells are initiated through the MHCII pathway. Epitope prediction for CD8+ T cells has had remarkable success, while the mechanisms of CD4+ T cell response are less understood. cell immune response can be viewed as consisting of two stages of recognition. In the first stage, antigen is taken up by antigen-presenting cells (APCs), where undergoes joint processing (i.e., cleavage) and binding to Major Histocompatibilityy Complex II (MHCII) molecules. Peptide-MHC complexes are then presented on the APC cell surface (Davis & Bjorkman 1988; Neefjes et al., 2011). In the second stage, T cell receptors (TCRs) on cells "recognize" pMHC complexes and T cell response is initiated. TCR recognition is mediated by its a and ,3 domains, which consist of variable (V), joining (J), constant (C), and, in the 3 chain, diversity (D) regions (Bosselut) 2019).
Accurate prediction of T cell responses requires a comprehensive understanding of both of these stages (Peters et al., 2020 Nielsen al. 2020). Early efforts in the area of computational epitope prediction focused on characterizing peptide-MHCII binding using allele-specific machine learning
models (Nielsen et al., 2020) with tools such as SMM (Peters Sette 2005; Kim et al. 2009),
NetMHC (Lundegaard et al. 2008 Nielsen et al.] (2003), NetMHCpan (Hoof et al. 22099] Nielsen
ct al. 2007), and NerMHCcons (Karosiene et al. 2012). More recent work has focused on modeling
antigen processing computationally with the Antigen Processing Likelihood (API.) algorithm (Mettu
ct al. 2016, Bhattacharya et al.) 2023] Li al.
22244 bij Charles et al., 2022), which seeks to model
the contributions of antigen structure which peptides are made available for MHCII binding.
Accurately predicting TCR-pMHC binding remains critical for advancing quantitative immunol- ogy and adaptive immunity research (Hudson et al., 2023). For this stage of prediction, both unsupervised and supervised methods have been developed (Hudson et al. 2023 2024). Unsu- pervised methods process cluster TCR sequencing datasets through dimensionality reduction and clustering (Dash et al. 2017 Glanville et al. (017) through a carefully chosen similarity metric (e.g., TCRdist3 (Mayer- Blackwell et al.] (021T) ). These methods cluster TCRs by analyzing their caapllemmmnaartyy-deeemnnii
regions (CDRs) using only TCR sequence data, without requiring binding labels or epitope information (e.g., GIANA (Zhang al. [202)) ClusTCR (Valkiers al.
2021). GLIPH2 (Huang et al.) 2020) iSMART (Zhang et al. 2020)). The resulting cluster labels serve as the output for each input TCR sequence (Hudson et al. 2024) and are typically analyzed by practitioners to guide and supplement experimental methods. In contrast, supervised machine learn- ing techniques make use of large amounts of TCR-pMHC data for training (Hudson et al.. 2023) from databases such as VDJdb (Bagaev et al. 2020), McPAS-TCR (Tickotsky et al. (017) and the IEDB (Vita et al. 2019). Supervised approaches (c.g. TITAN (Weber et al. 20211) STAPLER (Kwee et al. 2023), ERGOR GO2 (Springer et al. 2021), MixTCRpred (Croce et al.) 2024). Ne1TCR2.2 (Jensen & Nielsen 2024), TULIP Meyynadd-Piganeee et al., (0244) use a variety of deep learning models
providing reasonable performance and generalization capability.
A.3.1 TCR-PMHC BINDING PROBLEM FORMATION
The TCR-pMHC binding prediction problem can be formulated classification task: given the TCR alpha (ax) and beta (3) chains, an epitope e, and an MHC molecule m, the model predicts whether the pair binds (binder) or does not bind (non-binder). The TCR chains and the epitope proteins or peptides, typically represented as amino acid sequences. Formally, we define amino acid units as a E A, where A is the set of amino acid characters. For a single TCR-pMHC binding case, No a = [a.] 1=11 3 = [a No and e = Na with N., Ny. No E Z representing the sequence lengths. The MHC allele type is denoted by m E M, where M is the set of all MHC alleles. The
18

---

Published as a conference paper at ICLR 2026
PMHC-TCR binding classification is formulated as conditional probability: Poind P(ela, B, 8. Poind > t, where E [0, 1), the case is classified as positive, otherwise negative.
A.4 TCR-PMHC PREDICTION TRANSFORMER MODELS
Transformers Vaswani al. series of variants such as Bidirectional Encoder Representations from Transformers (BERT) (Devlin al. 2019) and Generative Pre-training Transformers (GPT) (Radford et al.) 2018). These mod- els support multi-sequence inputs and excel in modeling interactions, are well-suited for this task. Because TCR- PMMC interactions are determined by interactions among the TCR a and chains, epitope, and MHC, several and cross- TCR- TCR-interrreter (Koyama et al. 2023), adopt encoder-decoder transformer architectures to learn these complex relationships.
2017 successful deep learning models in different areas, have
models, such TULIP (Meynard Piganeau et al. 2024)
TULIP: TULIP is transformer- encoder-
C3R3
based model with decoder architecture designed for TCR-pMHC binding prediction. operates through three parallel modality processing pipelines, pro- cessing CDR3a, CDR3b, and epitope sequences separately (Meynard- Piganeau et al.) 2024). The encoders transform the imput sequences into feature representations, while the decoders model interactions across different sequences (Devlin al. 2019 Vaswani et al.] 2017). As an auto-regressive generative model, TULIP computes the conditional probability distribution of sequences (e.g.. epitope) given others (e.g., CDR3a, CDR3b, and MHC) during training (Meynard-Piganeau et al. (2024). For evaluation, TULIP retains only the epitope stream to produce the binding score. In this setting, the peptide features serve the query, while the CDR3a and CDR3b features are used as the keys and values in the cross- atts-ttt module. To compute gradients for TULIP, we design an moiino-ccidwwor loss function. The ground truth derived from the TCR alpha, TCR beta, and epitope sequences. These sequences are first one-hot encoded, and the model's predicted probabilities are compared against them using a negative log- likelihoeh (NLL) loss. This formulation allows us attribute importance scores the amino acid level based on how well the model reconstructs each residue.
chain
Encoder
Ceooder BonEa(o) MBCC)
CD83
Encoder
A Decoder
Ingfrl MHC)
Encoder
CDR3 fi dhaan
4 Deoader
Inglateio, 8, MBC)
C
Figure model MeennaddPPiganauu et al. (2024).
6: The architecture figure of TULIP
Cross CrossTCRInternn Intttrrpre:
CrossTCR IntInterpretrr is an encoder- decoder transformer for TCR- PMCC binding prediction (Koyama 2023). It takes the CDR regions of the alpha and beta chains, along with the peptide sequence, as inputs. The CDR alpha and beta chains are con- catenated using a colon (:) to form the TCR input. The TCR and peptide sequences are then independently encoded by an encoder module. Subsequently, cross-attention is employed to model the interaction between the two inputs and predict whether the pair represents a binder non-binder. We apply binary classification loss to extract the model gradients.
al.
Concal
1x0
xx
Meen
Mean
Transherror (Drose Abtrrccc)
Transfomar (Dies Aftention
Lengh2x
L
Thanelormo Encoder (Sel Aitenlio
Transtorerr Eneoser (Zel Atention)
Enteedding
Embedding
Longtz
Longrn
TORATOR-
Preride
Figure 7: The architecture of CrossTCCRInerppetee model (Koyama et al. 2023)
19

---

Published as a conference paper at ICLR 2026
BERTrand: BERTrand is an encoder-only transformer model (Myronov et al. 2023). takes the TCR beta chain and pep- tide sequence inputs. These two sequences are concatenated using <SEP> token and are processed jointly by transformer encoder as an integrated input. Similar to CrossTCRInterpreterr, BERTrand is a classification model designed to predict whether the TCR-pMHC pair a binder or a non-binder. We apply binary classification loss to obtain the model gradients.
Middet statattc
BLAGOLYY CASSCTOD1Y
Figure 8: The architecture figure of BERTrand model (My- ronov et al. 2023
A.5 PERTURBATION EXPERIMENTS
We evaluated the robustness of interpretability methods using perturbation-basel metrics across varying values of k. Figure presents the comparison results for both AOPC and LOdds across all chains.
Area Over the Perturbation Curve (AOPC)
CDR3a
CDR3b
epitope
0.035
0.08
0.030
0.08
0.025
ADPC
0.020
0.04
0.010
0.005
0.000
0.02
7.5
7.5
7.5
k
QCA Attn
TokenTM ARCAT
Rollout
LRP
GradCAM
RowAttn
Log-Odds Score (LOdds)
epitope
LOdbs
-12
2.5
5.0
7.5
10.0
12.5
15.0
2.5
5.0
7.5
10.0
12.5
15.0
2.5
5.0
7.5
10.0
12.5
15.0
TolcaTTM
Rollout
LRP
ArteLRP
AICCT
GradCAM
RawAttn
Figure 9: Comparison of Area Over the Perturbation Curve (AOPC) and Comparison of Aog-Odds Score (LOdds) across different values of & for all chains.
We also conducted with integrated dataset that includes data from VDJdb, IEDB and McPAS-
TCR. For both AOPCs and LOdds, the thresholds for peptide, CDR3a, and CDR3b are 7, 4, and respectively.
20

---

Published as a conference paper at ICLR 2026
AOPCs
LOdds
Chain
Method
TCR-XAI
Integrated
TCR-XAI
Integrated
peptide
QCAI
0.014
0.036
-1.62
-0.84
peptide
TM
0.013
0.026
-0.77
0.09
peptide
AttnLRP
0,012
0,026
-0,42
-0.50
CDR3a
QCAI
0.014
0.020
-3.50
-2.63
CDR3a
TM
0.021
0.020
-2.43
-2.35
CDR3a
AttnLRP
0.020
0.020
-2.72
-2.44
CDR3b
QCAI
0.048
0.027
-3.61
-3.08
CDR3b
To
0.033
0.025
-2.53
-2.78
CDR3b
AttnLRP
0.034
0.024
-2.82
-2.88
Table 2: AOPCs and LOdds comparison on TCR-XAI and Integrated datasets.
A.6 MAXIMUM VS. AVERAGE FOR AGGREGATION
High attention weights indicate meaningful interactions and used maximum across different cross-attention layers to retain all activated signals. Ablation studies the table below show that max generally outperforms average, with small exceptions on peptide BRHR and CDR LOdds.
Chain
Mix
ROC-AUC(3.4)
BRHR.25
AOPCs
LOdds
peptide
Max.
0.60
74.3
0.014
-1.52
peptide
Avg.
0.60
76.7
0.013
-1.51
CDR3a
Max.
0.55
79.1
0.014
-3.37
CDR3a
Avg.
0.50
72.6
0.013
-3.51
CDR3b
Max.
0.55
79.3
0.046
-3.54
CDR3b
Avg.
0.54
75.3
0.045
-3.63
Table 3: Maximum vs. Average for aggregation comparison across chains for ROC-AUC (3.4),
BRHR, AOPCs, and LOdds.
A.7 ROC CURVES RESIDUE-LEVEL IMPORTANCE SCORES FOR BINDING REGION IDENTIFICATION
We compared QCAI with other methods using ROC curves. We mainly set the distance threshold at 3.4, 4, and Á. 3.4 Á is chosen because it corresponds to the der Waals diameter between two carbon atoms, implying that residues within this distance are considered to be in contact.
Curve (Atom Distance Threshold 3.4 A)
CDR3a
CDR3b
opitope
Postive Bate Rate (199)
0.8
0.4
0.0
0.2
0.4
0.6
1.0
0.2
0.4
0.6
0.8
0.0
0.4
0.6
0.8
1.0
Ralso Positive Rate (FPR)
Faise Positive Rate (FPR)
Faise Positive Rato (FFR)
OCN
TakenTM ABCAT
Rellaut
USP RawAttn
SCAM
Figure 10: ROC curve comparison of the alpha, beta, and epitope chains between QCAI and other post-hoc methods. The distance threshold is set to 3,4 Å.
21

---

Published as a conference paper at ICLR 2026
ROC Curve (Atom Distance Threshold 3
CD
CDR3b
epitope
1.0
Thue Positive hate (TPR)
0.8
0.6
0.2
0.0
0.0
0.2
0.4
0.6
1.0
0.0
0.2
0.4
0.8
1.0
0.0
0.2
0.6
0.8
1.0
False Positive Rate (FPR)
Falise Positivv Rate (FPR)
False Positiiee Rate (PPR)
AttnLRP
AtCCAT
GrodCAM
RawAttn
ROC Curve (Atom Distance Threshold 4 A) CDR3b
CDR3a
epitope
1.0
True Positive Bate (TPR)
0.4
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
False Positive Rate (FPR)
False Positive Rate (FPR)
False Positive Rate (FPR)
QCAI AttnLRP
TokenTM AMCAT
Rollout
LRP
GradCAM
RawAttn
ROC Curve (Atom Distance Threshald 5 A) CDR3b
CDR3a
epitope
True Posilive Rate (TPR)
0.8
0.6
0.4
0.2
0.0
0.0
0.7
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.7
0.4
0.6
0.8
1.0
Positive (FPR)
Positive
False Positive (FPR)
QCAI
TokenTM
Rollout
LRP
GradCAM
Figure 11: ROC curve comparison of the alpha, beta, and epitope chains between QCAI and other post-hoc methods. The distance thresholds are set 3, 4, and Á.
A.8 BINDING REGION HIT RATE
We compare the Binding Region Hit Rate (BRHR) across the TCR a and 3 chains as well as the epitope region for different explanation methods. Here, HR./ denotes the hit rate calculated based on the top percentile of importance scores.
22

---

Published as a conference paper at ICLR 2026
Table 4: The Binding Region Hit Rate comparison among TCR alpha and beta chains and epitope between various methods, The HR. denotes the hit rate computed based on the top i percentile.
05.3(1115.7) %
53.7
44 44.4(=-20.7)9)
20.77%
46.5 46.5(
53.7(#15.8)%%
50.6(+19.55
42.2(=+17.3)8))
42.80 42.8( +22.00
66.7 55.2(116..6) %
43.0 43.0(=+19.4%)
19.4)%
50.1
45.0 45.0(
41.1
46.5(=11.005%
46.0 46.0(
44.0 11.11)
56.5(1143))
49.5(+11788)
51.6 18..))%
19.3(11944)9))
48.3 18.30-+18.0)9%
48.66 18.6(=1 (6.60=19.111))
45.1 45.1(-%1..5%%
44.0(++1.88)
HR.50 .0
44.00
66.4 1-1.8) %
60.4(
56.3(+25.9195 90%
66.8(130.0)% 57.9(
11.4(119.8)%
4-249))
51.7(
53.7
54.6 +20.99%
57.1 77.1(=27.1198
54.9 1:2..4)
51.5( 11.5(+20.7)9
56.1(++20.11)
1)%
54.9 121.822
53.1 33.1(1222.19%
67.7 17.22216.2
(7.1 21.8%
60.8 1.22.00%
57.9 7.2.2-23.29%
55.5 55.5(+21..)%%
61.0 +24.3)
54.6 54.60 120.6)%
55.9(+
61.9(
HR.4 .0
60.
72.7(1124.6) %
60 60.6(125.99) 9)%
66 (6.4(+29.44) 4)%
60 60.6(+225.9995 25.
72.2(4-99.59%% 70.9(4-29.3%%
58.0(1-29.87)
83%
65.2(4:26.09%
74.9
63 63.6 +25.11%
60.8 60.8(13002)4)
60.4 60.4(1124.9)8
61.5(424.7755
62.7(+25.5)%
61.7
60,8 00.8(425.339
16.7(1 %
66.1((+23.4)%
69.5 19.5(127.55)) 66.0(1126.7%)
64.9(
67. 67.1(+27.11%)
58.8 18.8(123.11%
(9.3(+-24.3)) 65.0(++22.2)5
64.4(
HR. IR.30
66.
74.3
58.4 (3.4(129.2)8
68.5 68.5(+-29.69% 6).6%
69.1 09.1(=30.88%
58. (8.4(1229.2)9) 9%%
60.8 01.8(3311.49)
68. 68.6(122) 28.6(-27..799)
79.11120.1)%
68.5 244.55
64, 14.4(1-30.8)0
62.7 (2.7(1155.0)%
66. (6.7(%-26.7)%
66.3 (6.3(127.11)8)
65. 8 (
(9.3 19.0)
72.6 2-2.33%
(6.9(-255.9999)
70. 70.4(425.6)9%
71.7(=26.88%)
61.8 26..3%%
66. 06.6(115522
HR.25
RawAlln (Wiegrelle & Pinter 2019
RawAAth Wiegreffe & Pinter 2019
Wiegreffe & Pinter 2019
Rollout Abnar Zuidema 2020
GradCAM varavarauu et al. 2017)
Rollout Abnar & Zundema 2020
GradCAM (Sel varaju et al. 2017)
2020
2017
AttnRR Achtibat et al. 2024
AttnLRP (Achtibat et 2024
AttnLR! RP Achtibat et al. 2024
TM Ww et al. 2024a
AttCAT (Oang et aL. 2022
Token Wu at. 2024a
ÁtCCATT (Qiang et al. a 2022
TokenTM WO et aL. 2024a
AUCAT Qrang et at. 2022
Rollout Abnar & Zuidema
GradCAM Selvaragu et al.
LRP [Binder et al. 2016)
LRP (Binder et aL. 2016
LRP Binder at. 2016)
QCAI (Ours)
QCAI (Ours)
QCAI (Ours)
Raw Rtwn Attn
Method
epitope
CDR3a
CDR3b
Chain
23

---

Published as a conference paper at ICLR 2026
To consider performance relative to training similarity we consider the change in BRHR of samples indexed by the Levenshtein distance each input modality to the TULIP training dataset (this approach is also used in the original TULIP paper). In the Table each cell represents the BRHR of all samples with the minimum Levenshtein distance to the sequences of TULIP training dataset smaller than the given threshold. As sample's distance between the sequences of TCR-XAL and the sequences of training dataset increases we find that the BRHR score of QCAI on TULIP decreases slightly but remains reliable with the BRHR drop within 0.05, which is small relative the BRHR difference between QCAI and other methods. This shows that QCAI's performance is preserved even samples differ from the training set.
Levenshtein
1> d
2> d
3> d
4> d
5> d
6 > d
7 d
Distance (d)
Peptide
.77(.23)
.77(.44)
.80(233
.77(.24)
.75(.25)
.74(.25)
.76(.25)
CDR3a
.83(.7))
(811.88
84(.19)
82(.19)
.79(200)
.79(200)
.79(.20)
CDR3b
.83(.20)
.80(221)
.77(21)
.78(21)
.78(20)
.78(20)
.78(.20)
Table 5: The change in BRHR for samples indexed by their Levenshtein distance.
To examine how model confidence and prediction outcomes affect BRHR, further compare BRHR on positive and negative samples for both TULIP and ICt-Interpreter. Because TULIP provides only relative binding likelihood scores, we perform QCAI analysis on Cross-TCR- Interpreter separately for its predicted positive and negative samples. For TULIP, we additionally set manual threshold by treating the top 50% scoring pairs as positive and the remaining 50% as negative. The BRHR results shown in Table Table[A.8]iniice)) A.8jiniccate that negative samples decreased the quality of explanation comparing to the positive samples.
BRHR
Cross- Meas--CR-Inaee ICtt-Interreeter
TULIP
TULIP
Samples
Positive
Negative
Positive
Negative
Peptide
.61
.55
.75
.74
CDR3a
.41
.46
.79
.79
CDR3b
.87
.87
.81
.79
Table 6: The BRHR of predicted positive and negative samples.
A.9 TCR-XA BENCHMARK
Types
Minimum Distance Each Sample Statistics
202
250
CDR3b
CDR3s
175
200
epitope
125
200
2.0
2.5
3.0
3.4
4.0
4.5
5.0
5.5
6.0
6.5
7.0
NNCH
Sample
Figure 12:
In the example (8TRQ) from the TCR-XAI benchmark, the peptide, CDR3a, and CDR3b regions are highlighted based their residue-level distances to the nearest interacting residues. Additionally, we report statistics for the minimum distance in each sample and MHC distribution.
We have compiled 274 samples from the STCRDab (Leem et al. 2018) and TCR3d 2.0 (Lin et al. 2025) datasets. Only samples with fully provided CDR3 regions and peptide sequences were se- lected. Among them, 213 (77.7% are MHC-I and 61 (22.3%) are MHC-II complexes. For each
24

---

Published as a conference paper at ICLR 2026
sample, computed the distance from each residue in the CDR3 regions to the nearest atom in the peptide, and vice versa from the peptide residues to the CDR3 regions. The resulting dataset in- cludes both the CDR3 and peptide sequences along with their corresponding residue-level distances. Since the model lacks structural input, we allow a one-residue positional tolerance to account for minor attention shifts. To this end, smooth each method's output importance scores by convolv- ing them with the kernel [1/3, 1/3, 1/3] prior to evaluation. The detailed information can be found in Table 9 Compared with the TULIP training dataset, there 176 distinct epitopes, and none appears in more than 3.3% (9) of the samples.
A.10 COMPUTATIONAL EFFICIENCY OF QCAI
We evaluate QCAI efficiency based on datasets including VDJdb, IEDB, McPAS-TCR, and TCR- XAI. All evaluations conducted CPU (32 E5 cores). QCAI involves pseudo-inverss operations making it more computationally expensive than alternative approaches, but it is still relatively ef- ficient per sample basis. For example benchmark sets with thousands of test samples would need on the order of seconds for QCAI evaluation this is far smaller than what would be needed by practitioners.
Method
TCR-XAI
McPAS-TCR
VDJdb
IEDB
QCAI
2.19 ms
2.18 ms
1.30 ms
1.90 ms
Token'I
0.11
0.15
0.05 ms
0.11 ms
AutnLRP
0.04 ms
0.02 ms
0.02 ms
0.04 ms
Table 7: Milliseconds per sample for each method across different datasets.
A.11 ABLATION STUDY: QCAI ON CROSS- vS. SELF-ATTENTION
To investigate whether QCAI applied to attsstattnntion or self-attention contributes more to the final explanation, we compare QCAI applied only to cross-attention, only to self-attention, and to both, using perturbation experiments. Applying QCAI solely to self-attention is equivalent to Rollout. As shown in Figure A. and Table A. the performance of QCAI on cross-attention alone is comparable to applying These results indicate that cross-attention is the main contributor to the final explanation and plays significant role in cross-attention incorporated transformers.
both cross- and self-attention, and both outperform Rollout.
CDR3a 3ak
CDR3b k=
Peptide k=
LOdds
AOPC
LOdds
AOPC
LOdds
AOPC
QCAI
-3.328
0.014
-3,498
0.045
-1.470
0.013
QCAI (Cross-Attention)
-3.728
0.017
-3.511
0.048
-1.417
0.012
Rollout self-AAteetion)
-2.356
0.022
-2.653
0.032
-0,044
-0.001
AttnLRP
-2.481
0.020
-2.662
0.032
-0.017
0.000
Token'TM
-2.195
0.021
-2.383
0.032
-0.736
0.012
Table 8: Comparison of AOPC and LOdds for QCAI applied to cross-attention only, self-attention only (Rollout), and both.
25

---

Published as a conference paper at ICLR 2026
Area Over the Perturbation Curve (AOPC)
CDR3a
CDR3b
peptide
0.10
0.030
0.08
0.025
MDPC
0.06
0.020
0.015
0.04
0.003
0.000
0.02
2.5
5.0
7.5
10.0
12.5
15.0
2.5
5.0
7.5
10.0
12.5
15.0
2.5
5.0
7.5
10.0
12.5
15.0
k
QCAI
Rollout
AttnLRP
TakenTM
SoAlCroosAAteention)
Log-Odds Score (LOdds)
CDR3a
CDR3b
peptide
LOdds
-8
7.5
12.5
10.0
12.5
5.0
7.5
10.0
Figure 13: Comparison of AOPC and LOdds for QCAI applied to cross cross-attention only, eeff-attention only (Rollout), and both.
A.12 APPLICATION OF DISION-LANGAAE MODELS
Since QCAI can be applied broadly to cross-attention modules, we illustrate its use vision- language model (VLM). For this case study, employ CI.IP Radford et al. (202T). widely used vision foundation model. CLIP provides separate vision and text encoders with aligned features, so added a cross-attention layer to fuse image features (as key and value) with text features (as query). We use a subset of the MS- MS-COCO dataset Lin et al. (2014), containing 73,000 images for lulti-label classification. The dataset is split 9:1, with 75,700 training samples and 7,300 test samples. The input consists of an image-text text pair, where the text is generated following the CLIP paper's recommendation as "a photo of cat", "a photo of a couch") ") |Rafford et al.] (2018). Features extracted via cross-attention are used for label prediction. After 100 epochs of training, the model achieves 94.08% accuracy and 0.9997 ROC-AUC on the test set.
with the corresponding labels (e.g., "a photo of
QCAI then applied to analyze the model. Since each image-text pair has multiple labels, gra- dients and QCAI are computed for one label at a time. Figure A. presents case studies on both training and test samples, demonstrating that QCAI can identify interactions between the two input modalities in cross-attention and highlight the relative importance of the image and text for given classification label.
26

---

Published as a conference paper at ICLR 2026
3 photo of person.
photo of cat
photo af perton ahoto surfboand,
photo of person: photo of suriboans.
5 photo of person photo of bicycle
photo of umbrela.
photo of couch.
5 photo of person. photo of umbrelaa
photo of cat. photo of couch
Dexilo persi ahoto a suriboadd
plicto parsott photo of sur surfboans,
pliuto a pet strs
photo of bicycle)
photto horse
focus traffic
focus
focus
focus
8 photo a person,
photo of person
shotto person
photo person.
photo of person
3 photo a cor. photo of
photo of car.
shoto car:
photo of car.
a photo of bicycle photo of horse.
photo of bus aheto
photo of bus.
trailic
Importance
Figure 14: Example of QCAI explaining CLIP with cross-attention.
27

---

Published as a conference paper at ICLR 2026
A.13
TCR-XAI BENCHMARK SAMPLES
PDB
MHC
Peptide
CDRA3
CDRB3
STRQ
МICI
GVYATSSAVRLR
ALGDHISGSWLLI
ASSLRTGANSDYT
40ZI
MICI
QPFPOPELPYP
LYGDGGSFSGGYNKLI
SAGVGGQETOY
2AK4
MHCI
LPEPLPOGQLTAY
ALSGFYNTDKLI
ASPGLAGEYEOY
5EU6
MHCI
YLEPGPVTV
AVLSSGGSNYKLI
ASSFIGGTDTQY
7BBE
MHCI
YLOPRTFLL
VVNINTDKLI
ASSSANSGELF
4Z7W
MHCII PSGEGSPOPSQENPQ
AVGETGANNLF
ASSEARRYNEOF
6V18
MHСI GGYRAPAKAAAT
ALSDSGSENKIT
ASSIDWGGONTLY
SEO8
MHCT
LPFDKATIM
AADGGAGSYQLT
SAGPTSGRTDIO
SWKH
MHCI
GTSGSPINN
GLGDAGNMLT
ASSLGQGLLYGYT
7T2B
MHСI
ATGLAWEWWRTTE
ATDKKGGATNKLL
ASSQGGGEQY
7SG1
MHCI
OPFPOPELPYGSGGO
LVGGLARDMR
SVALGSDTGELF
3WOW
MHCI
RFPLTFGWCF
GTYNQGGKLI
ASSGASHEQY
SWIV
MHCI
VMAPRTLIL
CROPLGGSNYKLT
ASSANPGDSSNEKLLE
6AVE
MHCI
APRGPHGGAASGL
LVGEILDNENKKH
ASSOROEGITTOY
SNHT
MICC
ELAGIGILTV
AVGGGADGLT
ASSQGLAGAGELF
TTPU
MICC
FLSPFWEDI
AVSAKGTGSKLS
ASSDAPGQLY
2P5E
MHCI
SLLLMWITTCC
AVRPLLDGTYIPT
ASSYLGNTGELLF
4P2O
MHCII
PADPLAFFSSAN VFFSSAIKGGGOSLY
AALRATGGNNKLT
ASSLNWSQDTOY
7NME
MHCI
QLPRLFPLL
AEPSGNTGKLI
ASSLHHEQY
SJZI
MHCI
KLVALGINAV
AYGEDDKII
ASRRGPYEQY
8ISC
MHCT
VVGAVGVGK
AARDSNYQLI
ASGDTGGYEQY
4E41
MHCIT
GELIGI GELIGILNAAKVRR NAAVPAA
AVDRGSTI.GRE
ASSQIRETQY
6AM5
MHCT
SMI.GIIIVVV
AVNEGGGKI.
ASSTSFGTTEAF
3E2H
MHCI
QLSPFPFDL
AVSLERPYLT
ASGGGGTLY
3MV8
HPYGEADYFEY
AVQDLGTSGSRLT
ASSARSGELF
3PPR
BEYLKAWTF
ILPLAGGTSYGKLT
ASSLGQAYEQY
SIIYJ
MHCC
AQWGPDPAAA
AMRGDSSYKLI
ASSLWEKLAKNIOY
5KS9
MHCII
APSGEGSFOPSSENNP
AVALNNNAGNMLIT
ASSVAPGSDTOY
7T2D
MHCII
ATGLAWEWWRTVVYE
ALSGSARQLT
ASSHREGETOY
KJ2
MHCI
KVITFIDL
AARYQGGRALI
TCSAAPDWWASSETLY
6RPB
MHCI
SLLMWITQV
AVKSGGSYIPT
ASSYLNRDSALD
6UON
MHCI
GADGVGKSAL
AAAMDSSYKLI
ASSDPGTEAF
IZGL
MHCII
VHFFKNIVTPRIPG
ALSGGDSSYKLI
ASSLADRVNTEAP
3PQY
MHCI
SSLENFRAYV
ILSGGSNYKLT
ASSFGREQY
4Y19
МHСII QPLALEGSLQKRG
AASVYAGCTSYGKLT
ASRPRRDNEQF
6AVG
MHCI APRGPHGGAASGL
LVVDQKLV
ASSGGHTGSNEQQ
6V15
MHCII GGYAPAKAAAT
ALSPSNNTKVV
ASSI.DWVNNTL
4Z7U
MHCII APSGEGSFQPSQENPQ
ILRDRSNOFY
ASSTTPGIGTETQY
7RM4
MHCT
HMTEVVRHC
ALDIYPHDMR
ASSIDPGDTTGEL
4Q0K
MHCI
EAAGIGILTV
AVNVAGKST
AWSETGLGTGELF
3PWP
MHCI
LGYGFVNYI
AVTTDSWGKLQ
ASRPGLAGGRPEQY
7N6E
MHCI
YLOPRTFLL
VVNRNNDMR
AGOVINTGELF
SWUL
MHCI
VVGAVGVGK
AARSSGSWOLI
ASSQDRGDSAHTLY
3DXA
MHCI
BENLLDFVRF
IVWGGYOKVT
ASRYRDDSYNEON
6RP9
MHCI SLLMWITQV
ALTRGPGNQFY
ASSSPGGVSTEAF
4MJI
MHCI TAFTIPSI
ATDDDSARØLT
ASSLTGGGELF
6V19
MHCII GGYAPAKAAAT
ALSDSSSFSKIL
ASSI.DWASQNTLY
7RDV
MHCII EGRVRVNSAYOS
AASDDNNNRIF
ASGGQSNERILF
3E3Q
MHCI OLSPFPEDL
AVSDPPPLLT
ASGGGGTLY
4MS8
MHCI SPAEEAGFFL
AVSAKGTOSKLS
ASSDAPGQLY
2F53
MHCI
SLLMWITQC
AVRPISGGSYIPT
ASSYVGNTGELF
3QDJ
MHCI
AAGIGILTV
AVNFGGGKII
AFSLSFGTEAF
6BJ2
MHCI
IPLTEEAEL
ALSHNSGGGNNYKLT
ASSFRGGKTOY
3QDM MHCI
ELAGIGILTV
AGGTGNOFY
AISEVGVGOPOH
STIL
MHCI
KAPYNEATM
AALYGNEKIT
ASSDAGGRNTLY
6VMX
MHCI
RPPIFIRRL
AFGSSNTGKLI
ASSQDLETGGYT
Table 9: The samples contained in TCRxAI benchmarks
28

---

Published as a conference paper at ICLR 2026
PDB
MHC
Peptide
CDRA3
CDRB3
7RTR
MHCI
YLOPRTFLL
AVNRDDKII
ASSPDIEQY
8EN8
MHCI
LPFDKSTIM
AADGGAGSYQLT
SAGPTSGRTDTOY
YYMM
MHCI
ENPVVHFFKNIVTP
ATDTTSGTYKYI
SARDLTSGANNEQU
SC07
MHCI
YQFGPDEPIA
AMRGDSSSYKLI
ASSLWEKLAKNIOY
3VXU
MHCI
RFPLIFGWCF
GTYNOGGKLI
ASSGASHEQY
6VQO
MHCI
HMTEVVRHC
AMSGLKEDSSYKLI
ASSIQOGADTOY
HJSH
MHCII
PKYVKONTLKLAI
AVSESPFGNEKLT
ASSSTGLPYGYT
8ENH
MHCT
LPFEKSTIM
AADGGAGSYQLT
SAGPTSGRTDTOY
2P5W
MHCI SI.LMWTTCC
AVRPLLDGTTIPT
ASSYLGGNTGEFF
BUTT
MHCI
ALWGPDPAAA
AMRGDSSYKLI
ASSLWEKLAKNIOY
7Q99
MHCI NLSALGIFST
AVNVAGKST
AWSETGLGTGELF
6ZKZ
MICC
RLPAKAPL
AVTNQAGTALI
ASSYSIRGSRGEQF
4GG6
MHCII SGEGSFOPSQENP
ILRDGRGGADGLT
ASSVAVSAGTYYQY
6TMO
MHCI
EAAGIGILTV
AVNDGGRLT
AWSETGLLGMMGWO
3QIU
MHCII ADLIAYLKQATHR
AAEPSSGOKLY
ASSLNNANSDYY
SWKF
MHCI
GTSGSPIVNR
GLGDAGNMIT
ASSIGQGLLYGYT
8SHI
MHCI
VRSRRLRL
ATTALLSGGGADGII
ASSYSEGEDEAF
SD2N
MHCI
NLVPMVATV
ILDNNNDMR
ASSLAPGTTNEKLLF
5KSA
MHCII
QPQQSFPEQEA
AVQEMDSNYQLI
ASSVAGTPSYEQY
6MTM
MHCI
FEDLRVLSF
GTERSGGYQKVT
ASSMSAMGTEAF
2BNQ
MHCI
SLLMWITQV
AVRPTSGGSYIPT
ASSYVGNTGELF
4Z7V
MHCII SGEGSFOPSQENP
ILRDSSAAQILY
ASSAGTSGBYERO
2F54
MHCI SIL.WITQC
AVRITSGGSYIPT
ASSYVGNTGELF
SBS0
MHCI ESDPIVAQY
AVRPGGAGPFEVV
ASSENMATIQOY
6CQR
MHCI REYKTLRABQAST
AFKAAGNKLT
ASSRLAGGMDEQF
SM00
MHCI
KAVANTATM
AALYGNEKIT
ASSDDAAGGGGRNTLY
7N2Q
MHCI
LRVMMLAPF
AVSNENKFY
ASSVATYSTDTQOY
6EQB
MHCI AAAAGGIIGGILTV
AVNDGGRLT
AWSETGLGMMGGWOO
4P2R
MHCII ANGVAFFLIPPKA
AAEASNTNKVY
ASSLNNANSDYT
4P2Q
MHCI ADGLAYFRSSFKGG
AAEASNTNKVV
ASSLNNANSDYT
8DNT
MHCI LLIDRINQL
AVREGAQKLY
ASSLDLGADEEQ
SE61
MHCI GIIGGEVTL
AGPGGSSNTGKII
ASSLIYPGELF
MHCI
KAVYNFATM
AALYGNEKIT
ASSDAGGRNTIY
2J8U
MHCI
ALWGFFPVL
ALFLASSSESKLY
ASSDWVSYEQY
ILP9
MHCI
ALWGFFPVL
ALFLASSSFSKLY
ASSDWVSYEQY
3KPS
MHCI
EEYLQAFTY
ILPLAGGTSYCKLI
ASSLGQAYEQY
2BNR
MHCI
SILMWITQC
AVRPTSGGSYIPT
ASSYVGNTGELF
SWIW
MHCI
VMAPRTLYL
AGQPLGGGSYKKLT
ASSANPGDSSNEKLLF
6CQL
MHCII RFYKTLRAEQQN
AFKAAGNKLT
ASSRLAGGMDEQF
SC09
MHCI YLGGPDFFTI
AMRGDSSYKLI
ASSLWEKLAKNIQY
4MXQ
MHCI SPAPRPLDL
AVSAKGTGSKLS
ASSDAPGQLY
3SJV
MHCI FLRGRAYGL
VVRAGKLI
ASGQGNFDIQY
IQRN
MHCI LLFGYAVYV
AVTTDSWGKLQ
ASRFGLAGGRPEOY
3KXF
MHCI LPEPLPQGQLTAY
ALSGFYNTDKLI
ASPGLAGEYEOY
SCOA
MHCI MVWGPDPLYV
AMRGDSSYKLI
ASSLWEKLAKNIQY
7N2N
MHCI TRLALIAPK
AVLSPYOETSGSRLT
ASSVGLESTDTTO
8ES9
MHCI GVYDGREHTV
AVQPLNAGNNRKI
SAREWGGTEAF
2NX5
MHCI EPL.POQLIAY
AVQASGGSYIPT
ATGTGDSNQQQQ
166R
MHCI STYRYYGI.
AVSGFASALT
ASGGGGTLY
8GVB
MHCI RYPLIFGW
AVGFTGGGNKLT
ASSDRDRVPETOY
STRL
MHCI
EIFDSGNPTGEV
IVNPANTGNQFY
ASRRDYFSYEOY
SD2L
MHCI NLVPMVATY
AFITGNOFY
ASSQTQLWETOY
SWLG
MHCI SQLLNAKYL
ATVYAQGLT
ASSDWGDTGQLY
SNMG
MHCI SLENTIAVL
AVRTNSGYALN
ASSDTVSYEQY
7DZM
MHCI TPQDLNTML
IVRGLNNAGNMIT
ASSLGIDAIY
7BYD
MHCI
GGAI
LYGGGGYVLT
ASSQDL.AAEEYYEO
SHHO
MHCI
GILEFVETL
AGAGSQGNLI
ASSIRSSYEQY
Table 10: The samples contained in TCRxAI benchmarks (continue table 1)
29

---

Published as a conference paper at ICLR 2026
PDB
MHC
Peptide
CDRA3
CDRB3
IQE
MHCT
ILEGYPRYV
AVTTDSWGKIQ
PSRPGLAGGRRFFO
3RGV
MHCT
WIYVYRPMGCGGS
AANSGTYQR
ASGDFWGITTLY
2E7L.
MHCT
SISPFEDDI
AVSHQGRYLT
ASGGGGTLY
3MB
MHCH
GAMKRIGLDDYRGYYSLO
AAEDGGSGNKLI
ASSWDRAGNTLY
5MO1
MHCI
KAPANFATM
AALYGNEKIT
ASSDDA
ASSDDAAAGGGGRRNTLYY
SSWZ
MHCI
ASNENMETM
AASETSGSWQLI
ASSRDLGRDTOY
5NMF
MHCI
SLYNTIATL
AVRTNSGYALN
ASSDTVSYEQY
8GVI
MHCI
RYPLTFGW
AVVFTGGGNKLT
ASSLRDRVPETOY
7N5C
MHCI
SSLCNFRAY.
ILSGGCCNYKLT
ASSFGREQY
STRR
MHCI
GVYATSSAVRLR
ALGDTGNYKYV
ASSAVNSGNTLY
40ZG
МHСI
APOPELPYPOPG
IVLGGADGLT
ASSERFTDTQY
40ZH
МHСI APOPELPYPOPGS
IVWGGATNKLI
ASSVRSTDIQY
20L3
MHCI
SOYYYNSL
AMRGDYGGSONKLI
TCSADRVGNTLY
7QPJ
MHCI
GLYDGMEHL
AVRGTGRRALT
ASSFATEAF
3VXM
MHCI
REPLTFOWCF
AVGAPSGAGSYQLT
ASSPTSGIYEQY
6EQA
MHC1 AAAAGCIIGGIILT
AVNVAGKST
AWSETGLGTGELF
2YPL
МHС KAESPEVIPMP
AVSGGYOKVT
ASTGSYGYT
7RK7
MHCI YMDGTMSQV
LVALNYGGSOGNII
AISPTEEGGLIFRGNT
8WTE
MHСT
VVGAVGVGK
AARSSGSWQLI
ASSQDRGDSAETLYY
BUTS
MICC
ALWGPDPAAA
AMRGDSSYKLI
ASSLWEKLAKNIQY
QSFF
MHCI
LLFGYPVAV
AVTTDSWGKLQ
ASRPGLAGGRPEQY
IOGA
MHCI
GILGEVFTL
AGAGSOGNLI
ASSSRSSYEQY
2GJ6
MHCI
LLFGKPVYV
AVTTDSWGKLQ
ASRPGLAGGRPEQY
3QDG
MHCI
ELAGIGILTV
AVNFGGGKLI
ASSLSFGTEAF
2VLR
MHCI
GILGEVFTL
AGAGSQGNLI
ASSSRASYEQY
7NAS
МHСТ
YGERNVVHI
AVSNYNVLY
ASSQEPGGYAEOF
SCX4
MHСТ
LRVMMIAPE
AVNSPGSGAGSYQL.
ASSVGTYSTITTOY
4PRT
МHСТ
HPVGEADYFEY
ASSARSGELF
SYE4
MHCI
NYNYLYRLF
VVNAHSGAGSYQLT
ASSETGGYEQY
SM02
MHCI
KAPFNEATM
AALYGNEKIT
ASSDAGGRNTLY
2CKB
MHCI
EQYKFYSV
AVSGFASALT
ASGGGGTLY
3TFK
МHСI
QLSDVPMDL
AVSAKGTGSKLS
ASSDAPGOLY
7N2S
MHCI
TRLALIAPK
AVSLGGGAASSYQLT
ASSVGLYSTDTOY
5KSB
MHCII GPOOSFPEQEA
AVQASGGSYIPT
ASSNRGLGTDTQO
2UWE
MHCI
ALWGETPVL
ALFLASSSTSKLV
ASSDWVSYEQY
7Q9A
MHCI
LLL.IGILVL
AVNVAGKST
AWSETGLGTGELF
5C08
MHCI
ROWGPDPAAV
AMRGDSSYKLI
ASSLWEKLAKNIOY
3HGI
MHCI
ELAGIGILTV
AVNVAGKST
AWSETGLGTGELF
815D
MHCI
VVGAVGVGK
AASSGSWOLI
ASSLEGTVEETLY
5JHD
МHCI
GILGEVFTL
AWGVNAGGTSYGKIT
ASSIGVYGYT
7JWJ
МHСТ
ASNENMETM
AAVTGNTGKI.
ASSRGTIHSNTEVE
4MNQ
MHСТ
ILAKF.HWI.
AVDSATALPPGYI
ASSYQGTEAF
6PY2
MHCI APFSEQEQPVIG
ASPQGGSEKIV
ASSSGGWGGGTEAF
7DZN
MHCI TPODLNTML
IVRGLNNAGNMLT
ASSLGIDAIY
4EUP
MHCI
ALGIGILTV
AVSGGGADGLT
ASSFLGTGVEQY
NNI
MHCI
RLQSLQTYV
ALSGFNNAGNMLT
ASSLGGAGGADTOY
QQEQ
MHCI
AAGIGILTV
AGGTGNQFY
AISEVGYGQPQH
21AN
MHCI
GELIGTLNAAKVPAD
AALIQGAQKLY
ASTYHGTGY
2VLJ
МHCI
GILGEVETL
AGAGSOGNLI
ASSSRSSYEQY
6CQN
MHСI
REYKTLRAFQASQ
APKAAGNKLT
ASSGLAGGMDEQF
3VXR
MHCT
RYPLTFGWCF
AVRMDSSYKLI
ASSSWDTGELF
7NMG
MHCI
LWMRLLPLL
AEPSGNTGKLI
ASSLHHEQY
3D3V
MHCI
LLFGPVYV
AVTTDSWGKLQ
ASRPGLAGGRPPEO)
SISZ
MHCI GILGEVFTL
AFDTNAGKST
ASSIFGQREQY
6U3N
MHCI APMPMPELPYP
AVGAGSNYQLI
ASSLEGQGASEQQ
6RSY
MHCT RMFPNAPYI.
IGGGTTSGTYKYI
ASSI.GFRRDVMR
4MVB
MHCT QPAEGGFQL
AVSAKGTGSKI.S
ASSDAPGOLY
Table 11: The samples contained in TCRxAI benchmarks (continue table 2)
30

---

Published as a conference paper at ICLR 2026
PDB
MHC
Peptide
CDRA3
CDRB3
1M15
MHCI
FLRGRAYGL
ILPLAGGTSYGKLT
ASSLGQAYEQY
3VXS
MHCI
RYPLTLGWC GWCF
AVRMDSSYKLI
ASSSWDTGELF
70W5
MHCI
VVVGAGGVGK
AMSVPSGIDGSSOFFT
ASKYGPGOHNSPLI
8GVG
MHCT
RFPLIFGW
AVGFTGGGNKLT
ASSDRDRVPPTQY
2VLK
MHCT
GIIGGEFTL
AGAGSOGNIJ
ASSSRSSYEQY
8GOM
MHCI
RLOSLQTYV
ASSGNTPLV
ASTWGRASTDTOY
ID9K
MHCI
AATGSENKLT
ASGGOGRAEQF
6CQQ
MHCI
RFYKTLRABQASQ
AFKAAGNKLT
ASSRLAGGMDEQF
5HHM
MHCT
GILGLYFTL
AGAGSOGNLI
ASSSSRSSYEQY
4NOC
MHCI MPAGRPWDL
AVSAKGTGSKLS
ASSDAPGQLY
SCOB
MHCI ROFGPDEPTI
AMRGDSSYKLI
ASSLWEKLAKNIOY
7PB2
MHCI VVVGADGVGK
ALSGPSGAGSYOLT
ASSYGPGOHNSPLII
IMWA
MHCI
EQYKFYSV
AVSGFASALT
ASGGGGTLY
4QRP
MHCI
HIKKKKDEL
ALSDPVNDMR
ASSLRGRGDOPQ!I
6RPA
MHCI
SLLMWITQV
AVRDINSGAGSYQLI
SVGGSGGADTOY
7N5P
MHCI
SSLCNFRAYV
ILSGGSNYKLT
ASSFFGREQY
7NIF
MHCI
YLOPRTFLL
AVNRDDKII
ASSPDIEQY
SCOC
MHCT ROFGPDWIva
AMRGDSSYKLI
ASSIWEKLAKNIOG
6D78
MHCI
AAGIGILTV
AVNFGGGKII
ASSWSFGTEAF
4JFF
MHCI FLAGIGILTV
AVNDGGRLT
AWSETGI
DESETGGIGNGGWO
4NSE
MHCI
VPYMAEFGM
AVSAKGTOSKLS
ASSDAPGQLY
4JRX
MHCI
LPEPLPQGQLTAY
ALSGFYNTDKLII
ASPGETEAF
7NMF
MHCI
QLPRLFPLL
AEPSGNTGKLI
ASSLHHEQY
3QIW
MICII
ADLIAYLEQATKG
AAEPSSGQKLY
ASSLNNANSDYT
6ZKX
MHCI
RLPAKAPLLGCG
AVTNQAGTALI
ASSYSIRGSRGEOF
INAM
MHCI
RGYVYQGL
AMRGDYYGGGNNLLI
TCSADRVGNTLY
8PJG
MHCII
PKYVKONTLKLAR
AVSEQDDKII
ATSDESYGYT
8VCX
MHCII
IVSHNAGNMLT
ASSLERETOY
SYXU
MHCI KIVALGINAV
AYGEDDKII
ASRRGSAELY
304L
MHCI GLCTLVAML
AEDNNARLM
SARDGTGNGYT
7SG2
MHCI QPFPOPEQPEPGS
LVGGLARDMR
SVALGSDTGELF
8GON
MHCI RLQSLQIYV
ASSGNTPLV
ASTWGRASTDIQY
2JCC
MHCI ALWGFFPVL
ALFLASSSFSKLY
ASSDWVSYEQY
6G9Q
MHCI
KAPYDYAPI
AALYGNEKIT
ASSDAGGRNTLY
6DKP
MHCI ELAGIGILTV
AVNFGGGKLI
ASSWSFGTEAF
5NQK
MHCI BLAGIGILTV
AGGGGADGLT
ASSQGLAGAGEI
2PYE
MHCI SLLMWITQC
AVRPLLDGTYIPT
ASSYLGNTGELF
6R2L
MHCI SLSKILDTV
AVGGNDWNTDKLI
ASSPLDVSISSYNEOF
2019
MHCI QLSPFPEDL
AVSGFASALT
ASGGGGTLY
8F5A
MHCI
SESIQEQIGW
AVTLNNNAGNMLT
ASSVGGTEAF
7Z50
MHCII AOTLALEVEDDEE
AASVRNYKYV
ASSRQGONTLY
6BGA
MHCII
YVVVPD
AALRATGGNNKLT
ASSLNWSQDTOY
3MV7
MHCI
HPYGEADYFEY
AVVODLGTSGSRLT
ASSARSGELF
8VD2
MHCII
GQVEL DOVELLGGGTPIESC
IVRVAIEGSQGNII
ASSLRRGDTIY
RVCY
MHCII
SEVEELGGSSPPECC
IVSHNAGNMIT
ASSLERETQY
SYXN
MHCI
KLVALGINAV
AYGEDDKII
ASRRGPYEOY
SE9D
MHCI ELAGIGILTV
AVTKYSWGKLO
ASRPGWMAGGVELY
6AMU
MHCI MMWDRGLGMI
AVNFGGGKLI
ASSLSFGTEAF
SBRZ
MHCI EVDPIGHLY
AVRPGGAGPFRVY
ASSENMATGQY
3TJ
MHCI SPLDSLWWI
AVSAKGTGSKLS
ASSDAPGQLY
3H9S
MHCI MLWGYLOYV
AVTTDSWGKI.O
4PRP
MHCI HIPGQAADYFEY
AVQDLGTSGSRLT
ASSARSGELF
SIVX
MHCI RGPGRAFVTI
AASASPGDNSKLI
ASSLGHTEVF
4Y1A
MHCII
SELLLLESSLORG
AASSSAGGTSYGKLIT
ASRPRIPVTQY
21AM
MHCI GELIGILNAAKVPAD
AALIQGAQKLY
ASTYHGTGY
6U30
MHCI AVVQSELPYPEGS
IAFQGAQKLV
ASSFRALAADTQY
Table 12: The samples contained in TCRxAI benchmarks (continue table 3)
31

---

Published as a conference paper at ICLR 2026
PDB
MHC
Peptide
CDRA3
CDRB3
6VOY
MHCI
GGYAPAKAAAT
ALSDSGSFNKLI
ASSLDWGGONTLY
SNME
MHCI
SLYNTVATL
AVRINSGYALN
ASSDTVSYEQY
7T2C
MHCII
TGLAWEWWRIVY
LVGDTGFOKLY
SARDPGGOGSSYEOY
2PXY
MHCII
RGGASQYRPSQ
ALSENYGNEKIT
ASGDASGAETLY
4G9F
MHCI
KRWTIMGLNK
AMRDDLDDNENKE
ASREGLGGTEAF
3QIB
MHCII
ADLIAYLKQATKG
AALRATGGNNKLT
ASSLNWSQDTOY
4G8G
MHCI
KRWIILGINK
AMRDLRDDNNNKE
ASREGLGGTEAF
7N20
MHCI
LRVMMLAPF
AVLSPVQETSGSRLT
ASSVGLESTDTOY
STEZ
MHCI
GILGEVFTL
AASFIIQGAQKLY
ASSLLGGWSEAF
3D39
MHCC
LLFGPVYV
AVTTDSWGKLQ
ASRPGLAGGRPEQY
6ZKW
MHCI
RLPAKAPLL
AVTNQAGTALI
ASSYSIRGSRGEQF
4FTV
MHCI
LLFGYPVYV
AVTTDSWGKLQ
ASRPGLMSAQPEOY
6PX6
MHCII APFSEQEQPVLG
AVHTGARLM
ASSHGASTDTOY
6V1A
MHCII GGYRAPAKAAAT
ALSDSSSFSKII
ASSIDWASONTTLY
1U3H
МHСТ SRGGASQYRPSQ
AASANSGTYQR
ASGDAGGGYEOY
7N4K
MHCI
SSLENFRRAYV
I.SGGSNYKI.
ASSFFGREQY
2Z31
MHCI RGGASQYRPSQ
ALSENYGNEKIT
ASGDASGGNTLY
ESS
MHCI
VMAPRTLIL
IVVRSSNTGKLI
ASSQDRDTQY
SEUO
MHCI
GILGFVFTL
AGAIGPSNTGKLI
ASSIRSSYEQY
4JFD
MHCI
ELAAIGILTV
AVNDGGRLT
AWSETGLGMGGWO
6ZKY
MHCI
RLPAKAPL
AVTNQAGTALI
ASSYSIRGSRGEQF
6TRO
MHCI
GVYDGREHTV
VVNHSGGSYIPT
ASSFLMTSGDDPYOO
7N2P
MHCI
GQVMVVAPR
AVSNENKFY
ASSVATYSTDTOY
7R80
MHCI
QASQEVKNW
AQLNQAGTALI
ASSYCTGINYGYT
IBD2
MHCI
LLFGYPVYV
AAMEGAQKLY
ASSYPGGGFYEOY
4L3E
MHCI
ELAGIGILTV
AVNFGGGKLI
ASSWSFGTEAF
7HH
MHCI
YLEPGPYTV
ATDGSTPMQ
ASSWGAPYEOY
FFFC
MHCI
FLRGRAYGL
AMREDTGNQFY
ASSFTWTSGGATDTOO
4JRY
MHCI
LPEPLPQGQLTAY
AVGGGSNYQLI
ASSRTGSTYEQY
SSWS
MHCI
ASNENMETM
AASEGSGSWQLI
ASSAGL
6UZ1
MHCI
LLFGYPVYV
AVTTDRSGKIQ
ASRPGAAGGRPELY
IFO0
MHCI
INFDENTI
AMRGDYGGSGNKLI
ICSADRVGNTLY
7JWI
MHCI
ASNENMETM
AASETSGSWOLI
ASSRDLGRDTOY
SDSQ
MHCI
LPPGVVEEFE
ALGDPTGANTGKLT
ICSAGRGGYAEOF
6VRM
MHCI
HMTEVVRHC
VYQPGGYOKVT
ASSEGLWOVGDEQQ
7N2R
MHCI
TRLALIAPK
AVSNENKFY
ASSVATYSTDTQQY
IFYT
MHCII
PKYVKONTLKLAT
AVSESPFGNEKLT
ASSSTGLPYGYT
3QFJ
MHCI
LLFGFPVYV
AVTTDSWGKLQ
ASRPGLAGGRPEQY
3GSN
MHCI
NLVPMVATV
ARNTGNOFY
ASSPVIGGIYGYT
6V13
MHCII GGYRAPAKAAAT
ALSPSNTNKVV
ASSLDWGVNTTLY
70W6
MHCI VVVGADGVGK
AMSYPSGDGSYQFT
ASKYGPGOHNSPLA
4OZF
MHCII APOPELPYPOPGS
IAFQGAQKLY
ASSFRALAADTOY
4JFE
MHCI ELAGIGALTV
AVNDGGRLT
AWSETGLGMGOWO
3MV9
MHCI HPYGEADYFEY
AVQDLGTSGSRLT
ASSARSGELF
6Q3S
MHCI SLLMWITQV
AVRPTSGGSYIPT
ASSYVGNTGGLLF
5MEN
MHCI ILAKFLHWL
AVDSATSGTYKYI
ASSYQGTEAF
IA07 MHCI LLFGYPVYV
AVTTDSWGKLQ
4HIL
MHCI QHIRCNIPKRISA
AVGASGNTGKLI
ASSLRDGYTGELF
Table 13: The samples contained in TCRxAI benchmarks (continue table 4)
32

---

Published as a conference paper at ICLR 2026
B REPRODUCIBILITY STATEMENT
The source code is publicly available at our project website (https://qcal jiarui. and on GitHub http:////g.cuuu..cccco//aa/a////
C LARGE LANGUAGE MODEL USAGE STATEMENT
We employed large language models (LLMs), primarily ChatGPT, in limited ways:
coding assistant, and for polishing written text.
Coding Assistant LLMs were consulted to clarify documentation, organize API references, and suggest debugging strategies. All code, documentation, and fixes obtained were manually reviewed and verified by the authors.
Polishing Article LLMs used only refine the clarity and style of sentences written by the authors and to format tables from raw data. No raw text or substantive content was generated by LLMs. All refined content was manually checked and further revised by the authors.
33


Reviews:
[{"review_id": "HZ5M6yxCi0", "reviewer": "Reviewer_iex5", "summary": "The paper introduces a novel method to interpret the features learned by an encoder-decoder transformer with cross attention. The method relies on the evaluation of attention scores for query and key values, and aggregates them across the layers of the decoder. The method is then tested against others on the task of predicting contact residues from models trained on protein binding (TCR-pMHC), showing increase in performance. In order to do so, the authors provide an annotated benchmark dataset.", "questions": "- I understand the the method proposed outperforms the others, but how should I read the overall low AUCs in Fig. 2? Are most methods performing worse than random chance? Can the authors comment on why they consider an AUC of .6 as “demonstrating strong alignment” between importance scores and binding interactions?\n- Can the authors test their method on tasks where explainable approaches work more convincingly?\n- The method is applied to TULIP, a state-of-the-art model for TCR-pHMC binding prediction. Are the benchmark data curated by the authors in the training set of TULIP, or not?\n \nMinor:\n\nThe notation for element-wise multiplication is not consistent throughout the paper ($\\odot$ vs. $\\cdot$)", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Methods to explain cross-attention in transformers are needed to go beyond their use as “black boxes”. The tested case is timely and interesting for the community working in molecular biology. The benchmark dataset curated by the authors can also be useful.", "weaknesses": "If I understand correctly, the task in which the authors test their method is rather hard: all the competitors perform in a way that is comparable or worse than random chance (Fig. 2), showing biased assessment. For a method which is “potential to be applied to other fields”, as the authors claim, other applications, where explainable methods work more reliably, should be tested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LKQ537Kp44", "reviewer": "Reviewer_rFG4", "summary": "Transformer-based models, while effective for predicting TCR-pMHC binding, are \"black boxes\" because existing explainable AI methods are not designed for their encoder-decoder architectures. This paper introduces Quantifying Cross-Attention Interaction, a post-hoc explanation method specifically designed to interpret the cross-attention mechanisms in the decoder part of the transformer. To quantitatively evaluate this new method, the authors also compiled TCR-XAI, a benchmark of 274 TCR-PMHC crystal structures derived from STCRDab and TCR3d 2.0 that uses physical residue distance as a ground truth for binding interaction. The authors demonstrate that QCAI achieves good performance on this benchmark across multiple metrics, including ROC-AUC, perturbation studies (LOdds and AOPC), and a new metric called Binding Region Hit Rate.", "questions": "- In the introduction, TULIP is mentioned as an example of a BERT-style model, but as mentioned in other parts of the paper, TULIP is an autoregressive encoder-decoder model, not an encoder-only BERT-like model.\n- The 2017 transformer paper should be cited in section 2\n- Did the author consider evaluating their method on a transformer trained on natural language?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The primary strength is providing an explainable AI method for encoder-decoder transformers, like TULIP, which current methods designed for encoder-only models cannot adequately interpret.\n- QCAI's performance is validated against a suite of competing methods (e.g., AttnLRP, TokenTM, Rollout) using ROC analysis, two different perturbation metrics (AOPC and LOdds) , and the authors' own BRHR metric, demonstrating SOTA results across most of them.", "weaknesses": "- The paper explicitly states that QCAI is up to 50x slower per sample than other methods due to the necessary pseudo-inverse operations.\n- The new TCR-XAI benchmark is heavily skewed towards MHC-I samples, which may limit the generalizability of the findings for MHC-II complexes.\n- The benchmark relies on atomic distance as a \"proxy for ground-truth importance\". An assessment based on an energy function would have been more appropriate", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q08AXOOYwN", "reviewer": "Reviewer_XjQm", "summary": "This work presents a method for the interpretation of TCR-pMHC binding prediction models with encoder-decoder structure and cross attention mechanisms, as well as a benchmark dataset of ground truth interacting residues defined by residue distance.. Specifically, the proposed method, quantifying cross-attention interaction (QCAI), combines a GradCAM-style intrinsic importance and a relevance score derived from the attention matrix. The authors demonstrated superior performance of the proposed importance in the task of binding site retrieval by log-odds and area under perturbation metrics.", "questions": "1\\. Since the experiments are only performed on known, \"positive\" TCR-pMHC pairs (those already with a stable structure), what bias might that introduce? Please discuss.\n\n2\\. Could the proposed method be applied to downstream prediction tasks such as impacts of sequence or structure perturbations, or other protein-protein interaction types?  Please discuss.\n\n3\\. Since the method requires 3D structure of the interaction complex, how would predicted structures (such as by AlphaFold base model or fine-tuned on TCR-pMHC) be used when structures are not available? From another perspective, is it possible to use the method for the evaluation of predicted structures or unknown interaction pairs?\n\n4\\. Does the range and scale of the residue importance show any relationship to the TULIP model's confidence of prediction?\n\n5\\. Since some samples in PDB are close neighbours (differing by only one or two residues), such as 2PXY and 2Z3, does the importance score show any different or similar patterns between the different residues?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1\\. This work offers insights into the important problem of interaction interpretation in immune-proteins. Considering the scarcity of data and limited prediction performance of existing models, a reliable interpretation method would allow maximal use of available data and models. The method and findings could potentially guide rational design of TCRs and respective immunotherapy.\n\n2\\. The authors provide solid theoretical justifications of the methodology as well as practical insights.\n\n3\\. The work offers a useful benchmark dataset for future work.\n\n4\\. Case studies with visualization clearly demonstrate the biochemical implications of the proposed importance scores.", "weaknesses": "Despite the sound problem setup and results, my major concern is the limited application both within and beyond the domain of TCR-pMHC. Specifically:\n\n1\\. The scope of the defined task and the respective benchmark dataset is somewhat limited. Distance is not the only indicator of interaction and only weakly indicates \"importance\" overall, considering residue contributions to TCR-pMHC interactions are somewhat additive (smaller, weaker interactions than dominating hotspots). Though that may be hard to analyze with the current tools, it should be briefly discussed.\n\n2\\. The method relies on TCR-pMHC structures, which is rather rare compared to other interaction types.\n\n3\\. TCR-pMHC is an important and complex subject, the proposed method could be widely applied to many other interaction types. Generalizability needs to be shown, or at least discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v83Loke7B6", "reviewer": "Reviewer_dbL7", "summary": "This paper proposed QCAI to interpret encoder-decoder transformer models for the TCR-pMHC binding task, where existing explainability (XAI) methods are inadequate for the asymmetric nature of cross-attention. It combines gradient-based scoring with a Moore-Penrose pseudoinverse decomposition and element-wise score aggregation to attribute quantitative contributions back to the distinct query and key input tokens. The paper establishes the TCR-XAI benchmark, a curated dataset of 274 experimentally determined protein structures that utilizes atomic distance as a ground truth for interaction importance. Also  included is direct-comparison metric, the Binding Region Hit Rate (BRHR), to confirm QCAI's ability to generate quantitativeuperior and biologically plausible insights.", "questions": "Although transformer models offer promising results, its positional encoding scheme is not suited for modelling the positional relations in interacting sequences, because (1) relative distance between residues across sequences are unknown and have to be inferred first; (2) existing positional encoding vectors are not physically meaningful in the first place. Therefore, while it is useful to provide explanations on a complex, data-driven black-box model, it could be more significant to devise a model that itself accord with first principles (or at least provide physically meaningful intermediate results). The author may refer to the recent work entitled \"Sliding-attention transformer neural architecture for predicting T cell receptor-antigen-human leucocyte antigen binding\" in Nature Machine Intelligence, 2024, 6(10): 1216-1230. \n\nTypos:\nLines 197–198: The phrase ‘denotes calculate max among feature dimension’ is ungrammatical.\nLines 407–408: The phrase ‘critical contacts with peptide peptides’ contains a redundant repetition of ‘peptide.’\nLine 805: The text ‘was proposedAbnar & Zuidema’ is missing both a space and the preposition ‘by’.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method, which interprets the complex cross-attention mechanism, effectively addresses the application gap between the existing post-hoc explainable AI methods and scientific research problems conceptually.\n\n3. The development of the TCR-XAI benchmark is useful for evaluation, grounding XAI performance in objective, physical atomic distances rather than subjective metrics.   \n\n4. A comprehensive validation framework is used that employs a suite of metrics, including ground-truth correspondence, perturbation influence, and the novel Binding Region Hit Rate.", "weaknesses": "1. **Ambiguity in Cross-Attention Inputs:** The paper does not clearly define the roles of the input sequences (CDR and Peptide) within the TULIP model's cross-attention mechanism (Figure 1). It is ambiguous which sequence provides the query vectors and which provides the key and value vectors. Given that the TULIP model is autoregressive and can predict any one sequence conditioned on the others, these roles may be dynamic. A detailed visualization of this process, **explicitly tagging each component with its corresponding input entity (e.g., CDR3α as query, Peptide as key)** , would be highly beneficial for interpreting the model's mechanics and the application of the QCAI method.\n\n2. **Clarification of the Core Methodological Distinction:** The description of the gradient-based importance score does not clearly articulate how its computation differs for cross-attention versus self-attention. In particular, the manuscript does not say what is **unique or nontrivial about handling cross-attention compared to standard self-attention**. A more explicit treatment of these differences would help clarify the specific methodological contribution of QCAI, beyond applying generic gradient-weighted attention analysis.\n\n3. **Experimental Setting around Attention Modules:** In the current experiments, the baseline methods are evaluated only on self-attention layers, with cross-attention omitted, whereas QCAI is evaluated on both self-attention and cross-attention. This raises two concerns. First, part of QCAI’s performance gain may simply come from exploiting additional cross-attention information that the baselines are not allowed to use, which makes the comparison potentially unfair. Second, because QCAI is only reported in its combined form, it is not possible to disentangle how much of the improvement is due to self-attention versus cross-attention. It would substantially strengthen the empirical analysis to report ROC-AUC, LOdds, and AOPC for **(i) QCAI applied only to self-attention and (ii) QCAI applied only to cross-attention**, in order to quantify the individual contribution of each attention component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)