Quantifying Cross-Attention Interaction in Transformers for Interpreting TCR-pMHC Binding

## Abstract

Abstract CD8+ “killer” T cells and CD4+ “helper” T cells play a central role in the adaptive immune system by recognizing antigens presented by Major Histocompatibility Complex (pMHC) molecules via T Cell Receptors (TCRs). Modeling binding between T cells and the pMHC complex is fundamental to understanding basic mechanisms of human immune response as well as in developing therapies. While transformer-based models such as TULIP have achieved impressive performance in this domain, their black-box nature precludes interpretability and thus limits a deeper mechanistic understanding of T cell response.
Most existing post-hoc explainable AI (XAI) methods are confined to encoder-only, co-attention, or model-specific architectures and cannot handle encoder-decoder transformers used in TCR-pMHC modeling. To address this gap, we propose Quantifying Cross-Attention Interaction (QCAI), a new post-hoc method designed to interpret the cross-attention mechanisms in transformer decoders. Quantitative evaluation is a challenge for XAI methods; we have compiled TCR-XAI, a benchmark consisting of 274 experimentally determined TCR-pMHC structures to serve as ground truth for binding. Using these structures we compute physical distances between relevant amino acid residues in the TCR-pMHC interaction region and evaluate how well our method and others estimate the importance of residues in this region across the dataset. We show that QCAI achieves state-of-the-art performance on both interpretability and prediction accuracy under the TCR-XAI benchmark.

## 1 Introduction

T cells play a pivotal role in the adaptive immune system by identifying and responding to antigenic proteins, both from pathogens such as viruses, bacteria and cancer cells joglekar2021t as well as in the context of autoimmunity. The final and arguably most critical component of T cell response is binding between the peptide Major Histocompatibility Complex (pMHC) which contains an antigenic peptide bound to a MHC molecule and the surface receptor on T cells (TCR). The specificity of this interaction underpins T cell-mediated immunity and is an intense area of research in both the development of therapies and fundamental understanding of immune response. Understanding T cell response is the key to vaccines that confer long-lasting immunity, and can also enable effective personalized cancer therapies rojas2023personalized; poorebrahim2021tcr.

CD8+ T cells are initiated through the Major Histocompatibility Complex I (MHCI) pathway, while CD4+ T cells are initiated through the MHCII pathway. Epitope prediction for CD8+ T cells has had remarkable success, while the mechanisms of CD4+ T cell response are less understood. T cell immune response can be viewed as consisting of two stages of recognition. In the first stage, a antigen is taken up by antigen-presenting cells (APCs), where it undergoes joint processing (i.e., cleavage) and binding to Major Histocompatibility Complex II (MHCII) molecules. Peptide-MHC complexes are then presented on the APC cell surface davis1988t; neefjes2011towards. In the second stage, T cell receptors (TCRs) on T cells “recognize” pMHC complexes and a T cell response is initiated. TCR recognition is mediated by its $\alpha$ and $\beta$ domains, which consist of variable (V), joining (J), constant (C), and, in the $\beta$ chain, diversity (D) regions bosselut2019t.

Accurate prediction of T cell responses requires a comprehensive understanding of both of these stages peters2020t; nielsen2020immunoinformatics. Early efforts in the area of computational epitope prediction focused on characterizing peptide-MHCII binding using allele-specific machine learning models nielsen2020immunoinformatics with tools such as SMM peters2005generating; kim2009derivation, NetMHC lundegaard2008netmhc; nielsen2003reliable, NetMHCpan hoof2009netmhcpan; nielsen2007netmhcpan, and NetMHCcons karosiene2012netmhccons. More recent work has focused on modeling antigen processing computationally with the Antigen Processing Likelihood (APL) algorithm mettu2016cd4+; bhattacharya2023parallel; jiarui2024gpu; jiarui2024gpumcmc; charles2022cd4+, which seeks to model the contributions of antigen structure on which peptides are made available for MHCII binding.

Accurately predicting TCR-pMHC binding remains critical for advancing quantitative immunology and adaptive immunity research hudson2023can. For this stage of prediction, both unsupervised and supervised methods have been developed hudson2023can; hudson2024comparison.
Unsupervised methods process cluster TCR sequencing datasets through dimensionality reduction and clustering dash2017quantifiable; glanville2017identifying through a carefully chosen similarity metric (e.g., TCRdist3 mayer2021tcr). These methods cluster TCRs by analyzing their complementarity-determining regions (CDRs) using only TCR sequence data, without requiring binding labels or epitope information (e.g., GIANA zhang2021giana, ClusTCR valkiers2021clustcr, GLIPH2 huang2020analyzing iSMART zhang2020investigation). The resulting cluster labels serve as the output for each input TCR sequence hudson2024comparison and are typically analyzed by practitioners to guide and supplement experimental methods.
In contrast, supervised machine learning techniques make use of large amounts of TCR-pMHC data for training hudson2023can from databases such as VDJdb bagaev2020vdjdb, McPAS-TCR tickotsky2017mcpas and the IEDB vita2019immune. Supervised approaches (e.g. TITAN weber2021titan, STAPLER kwee2023stapler, ERGO2 springer2021contribution, MixTCRpred croce2024deep, NetTCR2.2 jensen2023nettcr, TULIP meynard2024tulip) use a variety of deep learning models providing reasonable performance and generalization capability.

Transformer models have recently been use to analyze T cell immunity  hudson2023can; li2023disentangled; karthikeyan2023conditional; driessen2024modeling; cornwall2023fine.Specifically, models have been developed to predict TCR-pMHC binding, with an emphasis on BERT-based architectures such as TULIP meynard2024tulip, Cross-TCR-Interpreter koyama2023attention, TCR-BERT wu2024tcr, and BERTrand myronov2023bertrand.
However these models are black boxes and suffer from a lack of interpretability, which is critically important in elucidating the mechanisms involved in T cell response. To address this challenge, post-hoc explanation techniques kenny2021explaining have been developed to connect elements of the input and model to the outputs.
However, current existing post-hoc methods (e.g., AttnLRP achtibatattnlrp, TokenTM wu2024token, and TEPCAM chen2024tepcam) are designed for encoder-only transformer or convolution neural network (CNN) models, while state-of-the-art TCR-pMHC binding predictors adopt encoder–decoder architectures.

Figure: Figure 1: Quantifying Cross-Attention Interaction (QCAI) is a post-hoc explanation method designed for cross-attention mechanisms. In this paper, we show that QCAI enables insight into the structural basis for TCR–pMHC binding.
Refer to caption: extracted/6594550/figures/intro.png

The main contribution of this paper is to fill this gap with a novel post hoc explanation method that we call Quantifying Cross-Attention Interaction (QCAI) that enables interpretation of any encoder–decoder transformer model while taking cross-attention into account. Our motivation is the application of TCR-pMHC modeling, but cross-attention is used extensively in vision and NLP applications as well and thus QCAI is a general-purpose tool. Figure [1](https://arxiv.org/html/2507.03197v1#S1.F1) shows how QCAI is used to analyze decoder blocks; cross-attention between the CDR3 and epitope sequences is captured used to generate importance scores by residue position. Another important contribution of this paper is to provide a way to quantify the performance of XAI methods for TCR-pMHC binding. Typically XAI methods are evaluated qualitatively (e.g. in image analysis), but in the context of immunology, interpretations that match intuition are challenging to justify. We introduce TCR-XAI, a compilation of 274 experimentally-determined X-ray crystallography structures of TCR-pMHC complexes. For each complex we determine interaction distance between the CDR3 regions and epitope. Using this benchmark, we can determine whether the importance scores over the input produced by any particular method matches the expected interaction shown in the experimental structure.

We performed extensive evaluation of QCAI against other post hoc methods and demonstrate the benefit of incorporating cross-attention. We conduct an extensive comparison with other methods over the TCR-XAI benchmark and demonstrate that QCAI achieve state-of-the-art performance. We also analyze two case studies of TCR-pMHC systems to highlight mechanisms identified by QCAI.

## 2 Related Works

In this section we first define the TCR-pMHC binding problem. Then, we outline some basic concepts for self-attention and cross-attention, and introduce the transformer models for TCR-pMHC binding prediction. Next, the existed post-hoc explanation methods are introduced. Finally, we discuss limitations in existing post-hoc methods.

### 2.1 TCR-pMHC Binding Problem Formation

The TCR-pMHC binding prediction problem can be formulated as a classification task: given the TCR alpha ($\alpha$) and beta ($\beta$) chains, an epitope $e$, and an MHC molecule $m$, the model predicts whether the pair binds (binder) or does not bind (non-binder). The TCR chains and the epitope are proteins or peptides, typically represented as amino acid sequences. Formally, we define amino acid units as $a\in\mathbb{A}$, where $\mathbb{A}$ is the set of amino acid characters. For a single TCR-pMHC binding case, $\alpha=[a_{i}^{\alpha}]_{i=1}^{N_{\alpha}}$, $\beta=[a_{i}^{\beta}]_{i=1}^{N_{\beta}}$, and $e=[a_{i}^{e}]_{i=1}^{N_{e}}$, with $N_{\alpha},N_{\beta},N_{e}\in\mathbb{Z}^{+}$ representing the sequence lengths. The MHC allele type is denoted by $m\in M$, where $M$ is the set of all MHC alleles. The pMHC-TCR binding classification is formulated as a conditional probability: $p_{\text{bind}}=P(e|\alpha,\beta,m)$.
If $p_{\text{bind}}>t$, where $t\in[0,1]$, the case is classified as positive, otherwise negative.

#### 2.1.1 Transformer Models

Transformer-based architectures typically consist of $L$ stacked encoder layers, or a combination of encoder and decoder layers. Each layer comprises two primary components: Multi-Head Attention (MHA) and a Feed-Forward Network (FFN), each followed by layer normalization and residual connections. The distinction between encoder and decoder modules lies in their input structure and the type of attention mechanism employed.

In the encoder, each layer takes the output of the previous layer $h_{l-1}\in\mathbb{R}^{N\times d}$ and computes $h_{l}\in\mathbb{R}^{N\times d}$, where $N$ is the number of tokens and $d$ is the hidden dimension. In contrast, the decoder layer integrates two inputs: $h_{l-1}\in\mathbb{R}^{N\times d}$ from the previous decoder layer, and $h_{l}^{\prime}\in\mathbb{R}^{N^{\prime}\times d}$ from the corresponding encoder layer. The decoder output remains $h_{l}\in\mathbb{R}^{N\times d}$, with $N^{\prime}$ denoting the number of source tokens.

These inputs are linearly projected into query ($Q_{l}$), key ($K_{l}$), and value ($V_{l}$) matrices for the MHA computation. For encoder, it could be computed following $Q_{l}=W^{Q}_{l}h_{l-1}$, $K_{l}=W^{K}_{l}h_{l-1}$, and $V_{l}=W^{V}_{l}h_{l-1}$. For decoder, it could be computed following $Q_{l}=W^{Q}_{l}h_{l-1}$, $K_{l}=W^{K}_{l}h_{l}^{\prime}$, and $V_{l}=W^{V}_{l}h_{l}^{\prime}$.
Where $W^{Q}_{l},W^{K}_{l},W^{V}_{l}\in\mathbb{R}^{d\times d}$ are trainable projection matrices. For brevity, bias terms are omitted.
Considering a single attention head for simplicity, the attention matrix $A_{l}$ for layer $l$ is computed as:

$$ $\displaystyle A_{l}=\text{softmax}\left(\frac{Q_{l}K_{l}^{\top}}{\sqrt{d}} \right).$ $$

The shape of $A_{l}$ is $\mathbb{R}^{N\times N}$ for the encoder and $\mathbb{R}^{N\times N^{\prime}}$ for the decoder. The output of the attention module is computed as: $h_{l}=W_{l}^{O}\left(A_{l}V_{l}+h_{l-1}\right)\in\mathbb{R}^{N\times d},$
where $W_{l}^{O}\in\mathbb{R}^{d\times d}$ is a learnable output projection matrix. Outputs from multiple attention heads are concatenated before being linearly transformed.

#### 2.1.2 TCR-pMHC Prediction Transformer Models

Transformers vaswani2017attention, as a successful deep learning models in different areas, have a series of variants such as Bidirectional Encoder Representations from
Transformers (BERT) devlin2019bert and Generative Pre-training Transformers (GPT) radford2018improving. These models support multi-sequence inputs and excel in modeling interactions, are well-suited for this task.
Because TCR-pMHC interactions are determined by interactions among the TCR $\alpha$ and $\beta$ chains, epitope, and MHC, several state-of-the-art models, such as TULIP meynard2024tulip and cross-TCR-interpreter koyama2023attention, adopt encoder-decoder transformer architectures to learn these complex relationships.

### 2.2 Post-hoc Explanation Methods

A variety of explainable AI (XAI) methods have been developed to interpret deep learning models saranya2023systematic. These methods fall into two broad categories: explain-by-design, which integrates interpretability into the model architecture dwivedi2023explainable, and post-hoc, which analyzes model behavior after training kenny2021explaining. Post-hoc approaches offer a promising avenue for interpreting TCR-pMHC models and uncovering the underlying factors driving binding predictions.
Several families of post-hoc methods have been proposed, including:

- •
The Class Activation Map (CAM) (e.g., CAM zhou2016learning, GradCAM selvaraju2017grad, GradCAM++ chattopadhay2018grad)
- •
Layer-wise Relevance Propagation (LRP) (e.g., LRP binder2016layer, Partial LRP voita2019analyzing, Conservative LRP ali2022xai, AttnLRP achtibatattnlrp)
- •
Attention-based methods (e.g., Raw Attention wiegreffe2019attention, Attention Rollout abnar2020quantifying, AttCAT qiang2022attcat)
- •
Model-specific hybrid methods (e.g., TokenTM wu2024token, GAE chefer2021generic)

These techniques have been successfully applied to TCR-pMHC models. For example, TEPCAM uses CAM to interpret a CNN-based predictor chen2024tepcam, while TCR-BERT relies on attention weight analysis for interpretability wu2024tcr. These efforts have revealed structural determinants of TCR-pMHC binding. However, existing post-hoc methods primarily support encoder-only or co-attention mechanisms chefer2021generic, limiting their applicability to modern encoder-decoder models, which consists of cross-attention. This poses a major barrier to understanding how such models capture TCR-pMHC interactions.

#### 2.2.1 Class Activation Maps

Class Activation Map (CAM)-based methods have achieved significant success in explaining Convolutional Neural Networks (CNNs) by generating class-discriminative localization maps. Grad-CAM selvaraju2017grad, one of the most effective CAM methods, leverages the gradient of the class score $L^{c}$ with respect to the feature maps $F_{d}$ from the last convolutional layer. These gradients are used to compute importance weights for each feature map channel, enabling spatial localization of the regions most relevant for class $c$. The importance weight $w_{d}^{c}$ for feature map $F_{d}$ is computed as:

$$ $\displaystyle w_{d}^{c}=\mathbb{E}\left(\frac{\partial L^{c}}{\partial F_{d}} \right),$ $$

where $\mathbb{E}$ denotes global average and $w_{d}^{c}$ represents the global average pooled gradient for feature map $F_{d}$. The final CAM is then computed as a weighted sum over channels, followed by a ReLU activation:

$$ $\displaystyle\text{GradCAM}^{c}=\text{ReLU}\left(\sum_{d}w_{d}^{c}F_{d}\right).$ $$

The resulting heatmap is upsampled to the input resolution to highlight input regions most relevant to the prediction for class $c$.

#### 2.2.2 Attention Rollout

CAM-based approaches are primarily designed for CNNs. To interpret transformer-based models, Attention Rollout was proposedabnar2020quantifying, which estimates the flow of attention across layers. This method computes how information propagates through the self-attention mechanism across layers.
Given the raw attention weights $W^{A}_{l}$ for layer $l$, the augmented attention matrix is defined as

$$ $\displaystyle A_{l}=\frac{1}{2}(W^{A}_{l}+I),$ $$

where $I$ is the identity matrix, modeling the residual connection. The cumulative attention, or rollout, is then computed recursively:

$$ $\displaystyle R_{l}=\begin{cases}A_{l}R_{l-1},&\text{if }l>0\\ A_{l},&\text{if }l=0\end{cases},$ $$

capturing the total attention contribution from input tokens through layer $l$.

### 2.3 Limitations of Current Interpretability Methods for Transformers

Several post-hoc interpretability methods, such as TokenTM wu2024token, AttnLRP achtibatattnlrp, and AttCAT qiang2022attcat, have demonstrated reliable performance on encoder-only transformer models that rely on self-attention. However, these methods are not designed to handle the cross-attention mechanisms found in decoder layers. As a result, their applicability is limited in models that include decoder components, such as TULIP meynard2024tulip and MixTCRpred croce2024deep.

The core distinction between self-attention and cross-attention lies in the source of the key ($K$) and value ($V$) matrices. While self-attention derives $Q$, $K$, and $V$ from the same input, cross-attention uses separate inputs for $Q$ and $(K,V)$. Consequently, the attention matrix $A$ in cross-attention has dimensions $\mathbb{R}^{N\times N^{\prime}}$ instead of $\mathbb{R}^{N\times N}$, where $N$ is the number of query tokens and $N^{\prime}$ is the number of key tokens. This asymmetry poses a challenge for interpretability: the attention matrix no longer provides a direct measure of query token importance, making it difficult to attribute model predictions to input query tokens.

## 3 Quantifying Cross-Attention Interaction

In this section we present our main contribution, which is a way to handle the aforementioned asymmetry so that cross-attention can be captured. Since the attention matrix is computed as a scaled dot-product $QK^{\top}$, which captures the cosine similarity between query and key representations, interpreting the cross-attention mechanism can be structured into three key steps: 1. identifying which components of the attention matrix contribute most significantly to the model’s prediction, 2. decomposing these importance values into contributions from the query and key inputs, respectively, and 3. aggregating the cross-attention explanation with other layers’ explanation.

Inspired by GradCAM selvaraju2017grad, we propose to compute the importance of the attention matrix $A_{l}$ at layer $l$ using the gradient of the loss $L^{c}$ with respect to $A_{l}$ for a target class $c$, in conjunction with the attention weights themselves. Specifically, we define the importance score map as:

$$ $\displaystyle\mathbf{S}(A_{l})=\mathbb{E}_{H}\left(\text{ReLU}\left(\frac{ \partial L^{c}}{\partial A_{l}}\odot A_{l}\right)\right)+I,$ $$

where $\mathbb{E}_{H}(\cdot)$ denotes averaging across all attention heads, $\odot$ represents element-wise multiplication, and $I$ denotes the identity matrix for residue connection. This formulation highlights the attention entries that both have high weights and contribute significantly to the class-specific loss. The next step is to quantify this attention importance map into contributions from the query and key inputs. By analyzing the structure of the attention matrix, which serves as a soft alignment between queries and keys, we aim to attribute the importance scores back to the input tokens in both sequences.

### 3.1 Quantifying Query Importance from Attention

For the query input $Q_{l}$ at layer $l$, its importance scores with respect to the loss $L^{c}$ for class $c$ can be estimated in a GradCAM-style fashion as:

$$ $\displaystyle\mathbf{S}(Q_{l})=\text{ReLU}\left(\frac{\partial L^{c}}{\partial Q _{l}}\odot Q_{l}\right),$ $$

where $\odot$ denotes element-wise multiplication.
To obtain token-level importance scores from this matrix, we compute the column-wise maximum:

$$ $\displaystyle\omega_{l}^{Q}=\arg\max\limits_{i}\mathbf{S}(Q_{l})_{i,j}\in \mathbb{R}^{N},$ $$

where $i$ indexes the feature dimension, $j$ indexes the query tokens, and $\arg\max\limits_{i}$ denotes the maximum across the token dimension.
However, this importance score is intrinsic to $Q_{l}$ itself and does not reflect how $Q_{l}$ is influenced by the attention mechanism.
Explaining the attention matrix is a key component of post-hoc methods for interpreting transformer models wu2024token. In the case of cross-attention, the query and key matrices originate from different inputs, and thus the resulting attention matrix is not necessarily square.
To better understand how $Q_{l}$ contributes within the attention process, we define its attention-conditioned importance scores as $\mathbf{S}(Q_{l};A_{l})$, the query importance modulated by the attention matrix $A_{l}$. We approximate this as:

$$ $\displaystyle\mathbf{S}(Q_{l};A_{l})\propto\frac{\partial L^{c}}{\partial A_{l }}\cdot Q_{l}\,.$ $$

From the previous step, we have already obtained the attention importance map $\frac{\partial L^{c}}{\partial A_{l}}\odot A_{l}$. We now seek a transformation that allows us to infer $\mathbf{S}(Q_{l};A_{l})$ from this.
Assuming the attention matrix is computed via scaled dot-product as $A_{l}\sim Q_{l}K_{l}^{\top}$, we can express with linear operations (e.g., ReLU, $\mathbb{E}_{H}$, and $(\cdot)+I$) ignored for simplicity.:

$$ $\displaystyle\mathbf{S}(A_{l})\sim\frac{\partial L^{c}}{\partial A_{l}}\cdot Q _{l}K_{l}^{\top},$ $$

where $\sim$ denotes approximation.
To isolate the influence of $Q_{l}$, we need to eliminate $K_{l}^{\top}$ from the right-hand side. Since $K_{l}$ is not guaranteed to be square or invertible, we employ the Moore-Penrose pseudoinverse:

$$ $\displaystyle\frac{\partial L^{c}}{\partial A_{l}}\cdot Q_{l}$ $\displaystyle\sim\mathbf{S}(A_{l})\cdot K_{l}(K_{l}^{\top}K_{l})^{-1}\,.$ $$

This yields a decomposition of attention importance into the query space. Then, the importance scores corresponding to the token part can be obtained following:

$$ $\displaystyle\omega_{l}^{A}=\arg\max_{i}\left(\frac{\partial L^{c}}{\partial A _{l}}\cdot Q_{l}\right)_{i,j}\in\mathbb{R}^{N},$ $$

where $i$ indexes the feature dimension, $j$ indexes the query tokens, and $\arg\max\limits_{i}$ denotes calculate max among token dimension.
However, to ensure robustness, particularly in cases where $Q_{l}$ is also influenced by other layers. We conservatively combine this result with the intrinsic query importance:

$$ $\displaystyle\mathbf{S}(Q_{l};A_{l})=\max\left(\omega_{l}^{A},\omega_{l}^{Q} \right)\,.$ $$

Here, the maximum is applied element-wise to capture the strongest importance attribution from either source.

### 3.2 Quantifying Key Importance from Attention

Similar to the approach used to extract query importance scores, the key matrix importance can also be quantified into two components: (1) the intrinsic importance of the key matrix, denoted as $\mathbf{S}(K_{l})$, and (2) the attention-conditioned importance, $\mathbf{S}(K_{l};A_{l})$, which reflects how the key matrix contributes to the attention mechanism.

The intrinsic importance of the key matrix with respect to the loss $L^{c}$ for class $c$ can be estimated using a GradCAM-style formulation:

$$ $\displaystyle\mathbf{S}(K_{l})=\text{ReLU}\left(\frac{\partial L^{c}}{\partial K _{l}}\odot K_{l}\right)\,.$ $$

To obtain token-level importance scores from this matrix, we compute the column-wise maximum:

$$ $\displaystyle\omega_{l}^{K}=\arg\max_{i}\mathbf{S}(K_{l})_{i,j}\in\mathbb{R}^{ N^{\prime}}\,.$ $$

where $i$ indexes the feature dimension, $j$ indexes the key tokens, and $\arg\max\limits_{i}$ denotes the maximum across the token dimension.
Similar to the issue encountered in query attention quantification, the attention matrix is no longer necessarily square for key attention quantification. However, compared to decomposing query importance, extracting key importance from the attention matrix is more straightforward, since attention explicitly maps queries into the key space. Thus, we can directly analyze the attention matrix to determine which key tokens exert the strongest influence on the query representations. Because transformer models rely primarily on token-level outputs, we focus on interpreting token-level activations. The attention matrix $A\in\mathbb{R}^{N\times N^{\prime}}$ indicates how each query token (rows) attends to the key tokens (columns). To evaluate the overall importance of each key token in guiding the query representations, we compute the maximum relevance of each key across all queries and attention heads:

$$ $\displaystyle{\omega_{l}^{A}}^{\prime}=\arg\max_{i}\left(\mathbb{E}_{H}\left( \text{ReLU}\left(\frac{\partial L^{c}}{\partial A_{i,j}}\cdot A_{i,j}\right) \right)\right)\in\mathbb{R}^{N^{\prime}},$ $$

where $\mathbb{E}_{H}$ denotes averaging over attention heads and $i$ and $j$ index the queries and keys respectively, and $\arg\max\limits_{i}$ denotes the maximum across the feature dimension.

Finally, we combine this attention-derived importance with the intrinsic importance to produce a robust estimate of key token relevance:

$$ $\displaystyle\mathbf{S}(K_{l};A_{l})=\max\left({\omega_{l}^{A}}^{\prime},\; \omega_{l}^{K}\right),$ $$

where the maximum is taken element-wise to reflect the highest attribution signal from either source.

### 3.3 Aggregation of Layer Importance Scores

Inspired by the attention flow perspective abnar2020quantifying, we aggregate token-level importance scores across layers to track how relevance propagates from the final output back through the decoder and encoder layers. Let $k$ denote the index of the first decoder layer (with cross-attention) encountered when traversing the model from the output layer backwards. All subsequent layers with smaller indices are assumed to be encoder layers with self-attention. To capture how importance propagates through these layers, we define the aggregated token-level importance scores at layer $k$, denoted by $\tilde{\mathbf{S}}_{k}$, recursively as follows:

$$ $\displaystyle\tilde{\mathbf{S}}_{k}=\begin{cases}\mathbf{S}(Q_{k};A_{k})\cdot \tilde{\mathbf{S}}_{k+1}&\quad\text{(query)}\\ \mathbf{S}(K_{k};A_{k})\cdot\tilde{\mathbf{S}}_{k+1}&\quad\text{(key)}\end{ cases}\,.$ $$

In models with multiple decoder blocks that contain cross-attention, importance interactions may diverge and converge at different points. To handle such cases, we adopt a conservative strategy and aggregate importance via element-wise maximum to retain the most influential attribution signal:

$$ $\displaystyle\tilde{\mathbf{S}}_{k}=\begin{cases}\max\left(\mathbf{S}(Q_{k};A_ {k}),\;\tilde{\mathbf{S}}_{k+1}\right)&\quad\text{(query)}\\ \max\left(\mathbf{S}(K_{k};A_{k}),\;\tilde{\mathbf{S}}_{k+1}\right)&\quad\text {(key)}\end{cases}\,.$ $$

These recursive rules ensure that attention importance is correctly traced through both cross-attention and self-attention components. Consequently, if the explanation path contains any decoder block with cross-attention, the final output of our QCAI method will be a vector of token-level importance scores, indicating the contribution of each input token to the model’s prediction.

## 4 Experimental Results and Discussion

We first evaluate our proposed QCAI method using a state-of-art BERT-based model named TULIP, a transformer architecture tailored for predicting TCR–pMHC binding. TULIP adopts an encoder-decoder design and processes three modalities in parallel: CDR3a, CDR3b, and epitope sequences meynard2024tulip. Each encoder independently transforms input sequences into latent feature representations, while decoder layers model inter-sequence interactions devlin2018bert; vaswani2017attention. As a self-regressive generative model, TULIP estimates the conditional probability distribution of each sequence (e.g., epitope) conditioned on the others (e.g., CDR3a, CDR3b) meynard2024tulip.

We compare our QCAI method against several existing post-hoc interpretability techniques, including AttnLRP achtibatattnlrp, TokenTM wu2024token, AttCAT qiang2022attcat, Rollout abnar2020quantifying, GradCAM selvaraju2017grad, LRP binder2016layer, and RawAttn wiegreffe2019attention. For methods that require aggregation across all layers and compute on attention matrix, we apply them exclusively to the self-attention layers and omit cross-attention components, as these competing methods do not support cross-attention explanations. All experiments were implemented in Python using the PyTorch framework. Testing was conducted on a local workstation equipped with two NVIDIA A2000 GPUs, 16 Intel Xeon E5 CPU cores, and 64 GB of RAM.

### 4.1 A Benchmark for TCR-pMHC Binding Interpretation

To quantitatively assess the quality of interpretability methods, we constructed a benchmark that we call TCR-XAI using structural data from TCR-pMHC complexes. We collected 274 valid samples from the STCRDab leem2018stcrdab and TCR3d 2.0 lin2025tcr3d datasets, which consist of 213 (77.7%) MHC-I samples and 61 (22.3%) MHC-II samples. Only samples with complete TCR $\alpha$ and $\beta$ chains, full epitope sequences, intact CDR3 regions, and non-overlapping MHC and epitope chain IDs were retained.

Figure: Figure 2: In the example (8TRQ) from the TCR-XAI benchmark, the peptide, CDR3a, and CDR3b regions are highlighted based on their residue-level distances to the nearest interacting residues. Additionally, we report statistics for the minimum distance in each sample and MHC distribution.
Refer to caption: extracted/6594550/figures/benchmark.png

Among them, 213 (77.7%) are MHC-I and 61 (22.3%) are MHC-II complexes. For each sample, we computed residue-level distances: (1) from each CDR residue to the closest atom in the epitope, and (2) from each epitope residue to the closest atom in any CDR region. A smaller distance indicates a stronger interaction, which we use as a proxy for ground-truth importance. The resulting dataset includes both the CDR3 and peptide sequences along with their corresponding residue-level distances.
Since the model lacks structural input, we allow a one-residue positional tolerance to account for minor attention shifts. To this end, we smooth each method’s output importance scores by convolving them with the kernel $[1/3,\,1/3,\,1/3]$ prior to evaluation. The detailed information can be found in Table [3](https://arxiv.org/html/2507.03197v1#A1.T3).

### 4.2 ROC Analysis

Figure: Figure 3: ROC curve comparison of the alpha, beta, and epitope chains between QCAI and other post-hoc methods. The distance thresholds are set to 3.4, 4, and 5 Å.
Refer to caption: x1.png

Figure: Figure 4: ROC-AUC of predicted importance scores for TCR-pMHC binding site identification across a threshold of interaction distances demonstrates that QCAI surpasses competing methods in all cases.
Refer to caption: x2.png

To evaluate the explainability of different post-hoc interpretation methods, we quantitatively assess their ability to identify true TCR-pMHC binding sites using the TCR-XAI benchmark. We computed ROC curves shown in Figure [4](https://arxiv.org/html/2507.03197v1#S4.F4) by comparing predicted residue importance against ground-truth binding site annotations derived from structural data, where the ground-truth was defined according to distance threshold between 3.4 and 6 Å, with the ROC using predicted importance as the threshold. As shown in Figure [4](https://arxiv.org/html/2507.03197v1#S4.F4) and Figure [12](https://arxiv.org/html/2507.03197v1#A1.F12), QCAI achieves AUCs of 0.5492, 0.5489, and 0.6024 for CDR3a, CDR3b, and epitope respectively and consistently outperforms other methods. Notably, QCAI exceeds 0.6 on the epitope chain, demonstrating strong alignment between its predicted importance scores and the underlying structural binding interactions.

### 4.3 Perturbation Experiments

We also conducted perturbation studies to assess whether each method identifies important residues; we adopt two commonly used metrics: Log-Odds Score (LOdds) and Area Over the Perturbation Curve (AOPC). Perturbation is implemented by replacing the $k$ highest-scoring tokens with padding tokens (<PAD>). We evaluate the CDR3a, CDR3b and epitope chains separately, with $k{=}4$ for the CDR3a and CDR3b chains, and $k{=}7$ for epitopes to match the average number of predicted binding residues across TCRXAI.

**Table 1: Perturbation experiment results using fixed thresholds. Thresholds for the $\alpha$ and $\beta$ chains are $k{=}4$, and for the epitope chain $k{=}7$. The average number of binding regions are 3.64, 4.12, and 7.05 for $\alpha$, $\beta$, and epitope chains respectively.**
| Chains | CDR3a k=4 | CDR3b k=4 | Epitope k=7 |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | LOdds$\downarrow$ | AOPC$\uparrow$ | LOdds$\downarrow$ | AOPC$\uparrow$ | LOdds$\downarrow$ | AOPC$\uparrow$ |
| QCAI (Ours) | -3.328 | 0.014 | -3.498 | 0.045 | -1.470 | 0.013 |
| AttnLRP achtibatattnlrp | -2.481 | 0.020 | -2.662 | 0.032 | -0.017 | 0.000 |
| TokenTM wu2024token | -2.195 | 0.021 | -2.383 | 0.032 | -0.736 | 0.012 |
| AttCAT qiang2022attcat | -2.825 | 0.020 | -3.131 | 0.044 | -0.694 | 0.006 |
| Rollout abnar2020quantifying | -2.356 | 0.022 | -2.653 | 0.032 | -0.044 | -0.001 |
| GradCAM selvaraju2017grad | -2.700 | 0.019 | -3.112 | 0.038 | -1.004 | 0.009 |
| LRP binder2016layer | -2.938 | 0.020 | -3.127 | 0.043 | -1.167 | 0.011 |
| RawAttn wiegreffe2019attention | -2.734 | 0.015 | -3.250 | 0.039 | -0.691 | 0.010 |

Figure: Figure 5: Comparison of Area Over the Perturbation Curve (AOPC) and Comparison of Log-Odds Score (LOdds) across different values of $k$ for all chains.
Refer to caption: x3.png

Table [1](https://arxiv.org/html/2507.03197v1#S4.T1) and Figure [5](https://arxiv.org/html/2507.03197v1#S4.F5) shows that QCAI consistently outperforms other methods across most metrics. QCAI achieves the most negative LOdds and highest AOPC scores in the CDR3b and epitope chains, indicating greater disruption to the model’s confidence when informative residues are perturbed. Although Rollout outperforms QCAI in AOPC on the CDR3a chain, QCAI still achieves the best LOdds score.

### 4.4 Identification of Binding Region Residues with Importance Scores

Figure: Figure 6: Comparison of TCR–pMHC Binding Region Hit Rate (BRHR) across different methods on different chains. At any selected percentile of distance/importance, the higher the hit rate the more closely the importance tracks physical interaction distance. QCAI surpasses other methods in all practical cases.
Refer to caption: x5.png

**Table 2: The Binding Region Hit Rate comparison among TCR alpha and beta chains and epitope between various methods. The $\text{HR}.t$ denotes the hit rate computed based on the top $t$ percentile.**
| Chain | Method | HR.25 | HR.30 | HR.40 | HR.50 |
| --- | --- | --- | --- | --- | --- |
| epitope | QCAI (Ours) | 74.3($\pm$24.5)% | 72.7($\pm$24.6)% | 66.4($\pm$19.8)% | 55.3($\pm$15.7)% |
| AttnLRP achtibatattnlrp | 58.4($\pm$29.2)% | 60.6($\pm$25.9)% | 60.4($\pm$19.8)% | 53.7($\pm$15.8)% |  |
| TokenTM wu2024token | 68.5($\pm$29.6)% | 66.4($\pm$29.4)% | 56.3($\pm$25.9)% | 44.4($\pm$20.7)% |  |
| AttCAT qiang2022attcat | 69.1($\pm$30.8)% | 66.8($\pm$30.0)% | 57.9($\pm$23.9)% | 46.5($\pm$18.4)% |  |
| Rollout abnar2020quantifying | 58.4($\pm$29.2)% | 60.6($\pm$25.9)% | 60.4($\pm$19.8)% | 53.7($\pm$15.8)% |  |
| GradCAM selvaraju2017grad | 72.2($\pm$29.5)% | 70.9($\pm$29.3)% | 61.9($\pm$24.9)% | 50.6($\pm$19.9)% |  |
| LRP binder2016layer | 60.8($\pm$31.4)% | 58.0($\pm$29.8)% | 51.7($\pm$21.4)% | 42.2($\pm$17.3)% |  |
| RawAttn wiegreffe2019attention | 68.6($\pm$27.7)% | 65.2($\pm$26.0)% | 53.7($\pm$23.5)% | 42.8($\pm$20.0)% |  |
| CDR3a | QCAI (Ours) | 79.1($\pm$20.1)% | 74.9($\pm$19.4)% | 66.7($\pm$16.7)% | 55.2($\pm$16.6)% |
| AttnLRP achtibatattnlrp | 68.5($\pm$24.5)% | 63.6($\pm$25.1)% | 54.6($\pm$20.9)% | 43.0($\pm$19.4)% |  |
| TokenTM wu2024token | 64.4($\pm$30.8)% | 60.8($\pm$30.2)% | 57.1($\pm$27.1)% | 50.1($\pm$20.9)% |  |
| AttCATT qiang2022attcat | 62.7($\pm$25.0)% | 60.4($\pm$24.9)% | 54.9($\pm$23.4)% | 45.0($\pm$19.9)% |  |
| Rollout abnar2020quantifying | 66.6($\pm$25.2)% | 61.5($\pm$24.7)% | 51.5($\pm$20.7)% | 41.1($\pm$18.8)% |  |
| GradCAM selvaraju2017grad | 66.7($\pm$26.7)% | 62.7($\pm$25.5)% | 56.1($\pm$20.1)% | 46.5($\pm$17.0)% |  |
| LRP binder2016layer | 66.3($\pm$27.1)% | 61.7($\pm$26.6)% | 54.9($\pm$21.8)% | 46.0($\pm$19.0)% |  |
| RawAttn wiegreffe2019attention | 65.8($\pm$27.2)% | 60.8($\pm$25.3)% | 53.1($\pm$22.5)% | 44.0($\pm$17.1)% |  |
| CDR3b | QCAI (Ours) | 79.3($\pm$19.0)% | 76.7($\pm$18.9)% | 67.7($\pm$16.2)% | 56.5($\pm$14.9)% |
| AttnLRP achtibatattnlrp | 72.6($\pm$25.3)% | 66.1($\pm$23.4)% | 57.1($\pm$21.8)% | 49.5($\pm$17.8)% |  |
| TokenTM wu2024token | 69.5($\pm$27.5)% | 66.0($\pm$26.7)% | 60.8($\pm$22.0)% | 51.6($\pm$18.3)% |  |
| AttCAT qiang2022attcat | 66.9($\pm$25.9)% | 64.9($\pm$24.4)% | 57.9($\pm$23.2)% | 49.3($\pm$19.4)% |  |
| Rollout abnar2020quantifying | 70.4($\pm$25.6)% | 64.4($\pm$23.3)% | 55.5($\pm$21.5)% | 48.3($\pm$18.0)% |  |
| GradCAM selvaraju2017grad | 71.7($\pm$26.8)% | 67.1($\pm$27.1)% | 61.0($\pm$24.3)% | 48.6($\pm$19.1)% |  |
| LRP binder2016layer | 61.8($\pm$26.3)% | 58.8($\pm$23.1)% | 54.6($\pm$20.6)% | 45.1($\pm$18.5)% |  |
| RawAttn wiegreffe2019attention | 69.3($\pm$24.3)% | 65.0($\pm$22.2)% | 55.9($\pm$19.8)% | 44.0($\pm$17.8)% |  |

Using the TCR-XAI benchmark we construct an evaluation metric that we call Binding Region Hit Rate (BRHR). To compute BRHR, we first choose a percentile threshold $t\in(0,1]$ and identify the top $t$ fraction of residues with respect to highest importance scores $\mathbf{S}$. Each of these residues is marked a hit if its interaction distance is in the top $t$ fraction of interaction distances. We compute the hit rate for each input sequence type in each sample and take the mean across TCR-XAI to obtain the final BRHR. This metric reflects the proportion of true binding residues (according to structural proximity) that are successfully identified by the explanation method.

As shown in Figure [6](https://arxiv.org/html/2507.03197v1#S4.F6) and Table [2](https://arxiv.org/html/2507.03197v1#S4.T2), our method achieves state-of-the-art performance compared to all other explanation methods. For the epitope chain, our method consistently outperforms all other methods before the 50th percentile. After this threshold other methods prevail but have high false positive rate of other methods (as seen in the ROC analysis). We postulate that the latter effect is due to the fact that these methods can only access self-attention weights from the encoder and cannot benefit from the regulatory influence of cross-attention layers.

### 4.5 Case Studies

To highlight the ability of QCAI to assist in the interpretation of TCR-pMHC binding we discuss two specific examples, one for CD8+ T cells and one for CD4+ T cells. In both cases the analysis of importance using QCAI finds residue positions in TCRs that form critical contacts with epitope peptides and, by revealing unconstrained positions in longer CDR loops, can explain large differences in TCR-peptide-HLA binding affinity.

Figure: Figure 7: Case studies on systems from TCR-XAI. (a) We consider the same TCR-pMHC bound in two distinct binding orientations. For this system QCAI identifies key residues from both orientations. (b) We consider the same pMHC bound to two distinct TCRs. Here QCAI identifies the importance of the hairpin region of CDR3a in both cases.
Refer to caption: extracted/6594550/figures/results/case_study.png

In the first case study (Figure [7](https://arxiv.org/html/2507.03197v1#S4.F7)(a)) we consider the immunodominant CD8+ T-cell epitope from the influenza matrix protein which has been used to understand influenza T cell response. Multiple crystal structures (1OGA stewart2003structural and 5TEZ yang2017structural) of different TCRs recognizing this epitope have revealed a common mode of binding that involves the insertion of a single CDR3b sidechain (R98 in the 1OGA structure) into a notch between the peptide and the HLA-A2 alpha-2 helix and, otherwise, makes numerous contacts with the HLA-A2, whose shape depends on the peptide. In one distinct example, the TCR in the 5TEZ structure is rotated by 40 degrees around the HLA-TCR axis to create a very different group of TCR-HLA-A2 contacts, but this TCR also places a CDR3b sidechain (W99 in the 5TEZ structure) in the notch between peptide and HLA-A2 alpha-2 helix. Consistent with the common aspect of binding, the QCAI evaluation finds importance in the position of the notch-binding residue and in several N-terminal flanking positions of CDR3b.
The distinct aspect of binding for the two TCRs arises in the longer and less-constrained CDR3a for the 5TEZ TCR, which may explain its 25-fold lower affinity than for the 1OGA TCR.
We note that for both binding orientations, AttnLRP and TokenTM produce weaker importance scores overall.

The second case study considers a self-antigen in the autoimmune disease of rheumatoid arthritis. The HLA-DR4-bound citrullinated epitope, named vimentin-64cit59-71, has been analyzed in the complex with two different TCRs loh2024molecular (indicated with PDB codes, 8TRR and 8TRQ in Figure [7](https://arxiv.org/html/2507.03197v1#S4.F7)(b)). The QCAI evaluation finds an overall similar number of important positions in the two TCRs, including a concentration of importance along one edge of the hairpin formed by the CDR3a in both TCRs (highlighted with a dark outline in Figure [7](https://arxiv.org/html/2507.03197v1#S4.F7)(b)). The CDR3a contributes the largest direct contact with the peptide in both complexes. Interestingly, the CDR3b of the 5-fold-lower-affinity 8TRQ complex is longer and contains more positions of lower importance, again suggesting that the entropic cost of ordering this loop is responsible for the reduced affinity. For this case study, AttnLRP does not produce meaningful results while TokenTM does not capture the importance of residues in the epitope proximal to the CDR3a hairpin.

### 4.6 Cross-Attention Enables Understanding of Interactions Between Chains

Figure: Figure 8: BRHR and ROC-AUC comparison with different contact distance thresholds among TULIP, CrossTCRInterpreter, and BERTrand. TULIP and CrossTCRInterpreter are encoder-decoder transformers explained by QCAI.
Refer to caption: x6.png

To assess whether cross-attention truly empowers the model to analyze interactions between different chains, we further compared explanations with BERTrand myronov2023bertrand, an encoder-only transformer model. Since BERTrand lacks cross-attention, we used AttnLRP achtibatattnlrp, the best-performing method after QCAI in previous evaluations, to obtain importance scores for all, negative (false negative), and positive (true positive) samples.
As shown in Figure [8](https://arxiv.org/html/2507.03197v1#S4.F8), BERTrand demonstrates weaker understanding for both beta chain and epitope, which shows about 0.01 to 0.05 and 0.05 to 0.025 lower ROC-AUC compared to TULIP respectively.
Notably, for positive samples, BERTrand shows improved understanding of the epitope.
These results suggest that cross-attention effectively guides models to better capture the interaction patterns between different chains, beyond simply memorizing single-chain features.

### 4.7 Self-Regression versus Classification Loss

TULIP frames the TCR-pMHC binding problem as a self-regression task rather than a classification task, leveraging cross-attention. To further investigate the impact of self-regression loss versus classification loss in helping the model understand TCR-pMHC binding, we extended our analysis beyond TULIP to CrossTCRInterpreter koyama2023attention, which is an encoder-decoder BERT model trained on TCR-pMHC binding prediction using both TCR alpha and beta chains and epitope sequences, optimized with a binder/non-binder classification loss koyama2023attention.
We observed that CrossTCRInterpreter significantly outperformed TULIP on the TCR beta chain about 0.05 to 0.1 ROC-AUC lower for both positive and negative samples as shown in Figure [8](https://arxiv.org/html/2507.03197v1#S4.F8). However, it performed poorly on the alpha chain and epitope, which are about 0.1 to 0.15 lower ROC-AUC. This suggests that the self-regression loss used in TULIP encourages the model to explore the underlying structural relationships across all TCR and epitope chains, whereas the classification loss primarily helps the model capture discriminative patterns, particularly in the beta chain, patterns of which are relatively limited and can be clustered (e.g., TCRdist3 mayer2021tcr, GIANA zhang2021giana, and ClusTCR valkiers2021clustcr).
Also, we observed that the positive samples of CrossTCRInterpreter showed better performance on the epitope chain compared to negative samples as BERTrand.

## 5 Limitations

While we have shown the superiority of QCAI over competing methods, we note some limitations and opportunities for future study. QCAI currently focuses on identifying regions with positive attention weights, lacking the capability to capture negatively contributing or inhibitory signals. Furthermore, although the TCR-XAI benchmark emphasizes structural binding regions, TCR–pMHC interactions are also influenced by other biological factors, such as mechanosensor mechanisms choi2023catch and conformational dynamics within CDR loops van2023tcr. In terms of evaluation, while we can assess whether the predicted binding regions overlap with known structural contacts, we cannot verify whether the model has correctly learned the interaction dependencies among these regions without examining each case carefully.

## 6 Conclusions

In this paper, we present Quantifying Cross-Attention Flow (QCAI) to interpret the cross-attention in the decoders of transformer models, aiming to better understand encoder-decoder TCR-pMHC binding prediction models. QCAI quantifies the importance of the cross-attention matrix into contributions from query and key inputs, revealing how they influence each other. Our method can be combined with other post-hoc explanation techniques and is applicable to any encoder-decoder transformer model. To rigorously evaluate the explanations, we created a new structural explanation benchmark, TCR-XAI, along with a novel evaluation metric, the Binding Region Hit Rate (BRHR). On this benchmark, QCAI achieves state-of-the-art results across perturbation metrics (LOdds and AOPC), ROC-AUC, ROC curve analysis, and BRHR.

By applying QCAI to TULIP (an encoder-decoder model with self-regression loss) and CrossTCRInterpreter (an encoder-decoder model with classification loss), we found that self-regression loss and cross-attention enable models to better learn chain formation rules and interactions. Meanwhile, classification loss helps the model focus more effectively on understanding the TCR beta chain. Compared to BERTrand, an encoder-only model, our findings show that cross-attention significantly enhances the model’s ability to analyze inter-chain interactions. These results suggest that post-hoc explanations of TCR-pMHC binding models can guide both binding structure prediction and the rational design of future predictive models.

### 6.1 Future Work

In future work, we aim to extend QCAI to decompose both positive and negative attention interactions. Additionally, we plan to integrate QCAI with other encoder-only methods, such as AttnLRP and TokenTM, to fully realize its explanatory potential. Additionally, we are going to expand the TCR-XAI benchmark to include more binding-related factors, and leverage QCAI to guide the design of next-generation TCR-pMHC binding models with improved interpretability and predictive performance.

## Acknowledgments and Disclosure of Funding

We acknowledge support from Harold L. and Heather E.
Jurist Center of Excellence for Artificial Intelligence at Tulane University.
We also acknowledge support from the Amazon AWS Cloud Credit for Research program for providing necessary GPU resources.

## References

- [1]
Samira Abnar and Willem Zuidema.
Quantifying attention flow in transformers.
In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190–4197, 2020.
- [2]
Reduan Achtibat, Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Aakriti Jain, Thomas Wiegand, Sebastian Lapuschkin, and Wojciech Samek.
Attnlrp: Attention-aware layer-wise relevance propagation for transformers.
In Forty-first International Conference on Machine Learning, 2024.
- [3]
Ameen Ali, Thomas Schnake, Oliver Eberle, Grégoire Montavon, Klaus-Robert Müller, and Lior Wolf.
Xai for transformers: Better explanations through conservative propagation.
In International conference on machine learning, pages 435–451. PMLR, 2022.
- [4]
Dmitry V Bagaev, Renske MA Vroomans, Jerome Samir, Ulrik Stervbo, Cristina Rius, Garry Dolton, Alexander Greenshields-Watson, Meriem Attaf, Evgeny S Egorov, Ivan V Zvyagin, et al.
Vdjdb in 2019: database extension, new analysis infrastructure and a t-cell receptor motif compendium.
Nucleic acids research, 48(D1):D1057–D1062, 2020.
- [5]
Avik Bhattacharya, James O Wrabl, Samuel J Landry, and Ramgopal R Mettu.
Parallel computation of conformational stability for cd4+ t-cell epitope prediction.
In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 88–93. IEEE, 2023.
- [6]
Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojciech Samek.
Layer-wise relevance propagation for neural networks with local renormalization layers.
In Artificial Neural Networks and Machine Learning–ICANN 2016: 25th International Conference on Artificial Neural Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part II 25, pages 63–71. Springer, 2016.
- [7]
Rémy Bosselut.
T cell antigen recognition: Evolution-driven affinities.
Proceedings of the National Academy of Sciences, 116(44):21969–21971, 2019.
- [8]
Tysheena Charles, Daniel L Moss, Pawan Bhat, Peyton W Moore, Nicholas A Kummer, Avik Bhattacharya, Samuel J Landry, and Ramgopal R Mettu.
Cd4+ t-cell epitope prediction by combined analysis of antigen conformational flexibility and peptide-mhcii binding affinity.
Biochemistry, 61(15):1585–1599, 2022.
- [9]
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian.
Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.
In 2018 IEEE winter conference on applications of computer vision (WACV), pages 839–847. IEEE, 2018.
- [10]
Hila Chefer, Shir Gur, and Lior Wolf.
Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers.
In Proceedings of the IEEE/CVF international conference on computer vision, pages 397–406, 2021.
- [11]
Junwei Chen, Bowen Zhao, Shenggeng Lin, Heqi Sun, Xueying Mao, Meng Wang, Yanyi Chu, Liang Hong, Dong-Qing Wei, Min Li, et al.
Tepcam: Prediction of t-cell receptor–epitope binding specificity via interpretable deep learning.
Protein Science, 33(1):e4841, 2024.
- [12]
Hyun-Kyu Choi, Peiwen Cong, Chenghao Ge, Aswin Natarajan, Baoyu Liu, Yong Zhang, Kaitao Li, Muaz Nik Rushdi, Wei Chen, Jizhong Lou, et al.
Catch bond models may explain how force amplifies tcr signaling and antigen discrimination.
Nature communications, 14(1):2616, 2023.
- [13]
Lewis Cornwall, Grisha Szep, James Day, SR Gokul Krishnan, David Carter, Jamie Blundell, Lilly Wollman, Neil Dalchau, and Aaron Sim.
Fine-tuned protein language models capture t cell receptor stochasticity.
In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop, 2023.
- [14]
Giancarlo Croce, Sara Bobisse, Dana Léa Moreno, Julien Schmidt, Philippe Guillame, Alexandre Harari, and David Gfeller.
Deep learning predictions of tcr-epitope interactions reveal epitope-specific chains in dual alpha t cells.
Nature Communications, 15(1):3211, 2024.
- [15]
Pradyot Dash, Andrew J Fiore-Gartland, Tomer Hertz, George C Wang, Shalini Sharma, Aisha Souquette, Jeremy Chase Crawford, E Bridie Clemens, Thi HO Nguyen, Katherine Kedzierska, et al.
Quantifiable predictive features define epitope-specific t cell receptor repertoires.
Nature, 547(7661):89–93, 2017.
- [16]
Mark M Davis and Pamela J Bjorkman.
T-cell antigen receptor genes and t-cell recognition.
Nature, 334(6181):395–402, 1988.
- [17]
Jacob Devlin.
Bert: Pre-training of deep bidirectional transformers for language understanding.
arXiv preprint arXiv:1810.04805, 2018.
- [18]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.
Bert: Pre-training of deep bidirectional transformers for language understanding.
In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019.
- [19]
Alice Driessen, Jannis Born, Rocío Castellanos Rueda, Sai T Reddy, and Marianna Rapsomaniki.
Modeling car response at the single-cell level using conditional ot.
In NeurIPS 2024 Workshop on AI for New Drug Modalities, 2024.
- [20]
Rudresh Dwivedi, Devam Dave, Het Naik, Smiti Singhal, Rana Omer, Pankesh Patel, Bin Qian, Zhenyu Wen, Tejal Shah, Graham Morgan, et al.
Explainable ai (xai): Core ideas, techniques, and solutions.
ACM Computing Surveys, 55(9):1–33, 2023.
- [21]
Jacob Glanville, Huang Huang, Allison Nau, Olivia Hatton, Lisa E Wagar, Florian Rubelt, Xuhuai Ji, Arnold Han, Sheri M Krams, Christina Pettus, et al.
Identifying specificity groups in the t cell receptor repertoire.
Nature, 547(7661):94–98, 2017.
- [22]
Ilka Hoof, Bjoern Peters, John Sidney, Lasse Eggers Pedersen, Alessandro Sette, Ole Lund, Søren Buus, and Morten Nielsen.
Netmhcpan, a method for mhc class i binding prediction beyond humans.
Immunogenetics, 61:1–13, 2009.
- [23]
Huang Huang, Chunlin Wang, Florian Rubelt, Thomas J Scriba, and Mark M Davis.
Analyzing the mycobacterium tuberculosis immune response by t-cell receptor clustering with gliph2 and genome-wide antigen screening.
Nature biotechnology, 38(10):1194–1202, 2020.
- [24]
Dan Hudson, Ricardo A Fernandes, Mark Basham, Graham Ogg, and Hashem Koohy.
Can we predict t cell specificity with digital biology and machine learning?
Nature Reviews Immunology, 23(8):511–521, 2023.
- [25]
Dan Hudson, Alex Lubbock, Mark Basham, and Hashem Koohy.
A comparison of clustering models for inference of t cell receptor antigen specificity.
ImmunoInformatics, 13:100033, 2024.
- [26]
Mathias Fynbo Jensen and Morten Nielsen.
Nettcr 2.2-improved tcr specificity predictions by combining pan-and peptide-specific training strategies, loss-scaling and integration of sequence similarity.
bioRxiv, pages 2023–10, 2023.
- [27]
Alok V Joglekar and Guideng Li.
T cell antigen discovery.
Nature methods, 18(8):873–880, 2021.
- [28]
Edita Karosiene, Claus Lundegaard, Ole Lund, and Morten Nielsen.
Netmhccons: a consensus method for the major histocompatibility complex class i predictions.
Immunogenetics, 64:177–186, 2012.
- [29]
Dhuvarakesh Karthikeyan, Colin Raffel, Benjamin Vincent, and Alex Rubinsteyn.
Conditional generation of antigen specific t-cell receptor sequences.
In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop, 2023.
- [30]
Eoin M Kenny, Courtney Ford, Molly Quinn, and Mark T Keane.
Explaining black-box classifiers using post-hoc explanations-by-example: The effect of explanations and error-rates in xai user studies.
Artificial Intelligence, 294:103459, 2021.
- [31]
Yohan Kim, John Sidney, Clemencia Pinilla, Alessandro Sette, and Bjoern Peters.
Derivation of an amino acid similarity matrix for peptide: Mhc binding and its application as a bayesian prior.
BMC bioinformatics, 10:1–11, 2009.
- [32]
Kyohei Koyama, Kosuke Hashimoto, Chioko Nagao, and Kenji Mizuguchi.
Attention network for predicting t-cell receptor–peptide binding can associate attention with interpretable protein structural properties.
Frontiers in Bioinformatics, 3:1274599, 2023.
- [33]
Bjørn PY Kwee, Marius Messemaker, Eric Marcus, Giacomo Oliveira, Wouter Scheper, Catherine J Wu, Jonas Teuwen, and Ton N Schumacher.
Stapler: efficient learning of tcr-peptide specificity prediction from full-length tcr-peptide data.
bioRxiv, pages 2023–04, 2023.
- [34]
Jinwoo Leem, Saulo H P de Oliveira, Konrad Krawczyk, and Charlotte M Deane.
Stcrdab: the structural t-cell receptor database.
Nucleic acids research, 46(D1):D406–D412, 2018.
- [35]
Jiarui Li, Samuel J Landry, and Ramgopal R Mettu.
Gpu acceleration for markov chain monte carlo sampling.
In 4th International Conference on AIML Systems (AIMLSystems 2024). ACM, 2024.
- [36]
Jiarui Li, Samuel J Landry, and Ramgopal R Mettu.
Gpu acceleration of conformational stability computation for cd4+ t-cell epitope prediction.
In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 191–196. IEEE, 2024.
- [37]
Tianxiao Li, Hongyu Guo, Filippo Grazioli, Mark Gerstein, and Martin Renqiang Min.
Disentangled wasserstein autoencoder for t-cell receptor engineering.
Advances in Neural Information Processing Systems, 36:73604–73632, 2023.
- [38]
Valerie Lin, Melyssa Cheung, Ragul Gowthaman, Maya Eisenberg, Brian M Baker, and Brian G Pierce.
Tcr3d 2.0: expanding the t cell receptor structure database with new structures, tools and interactions.
Nucleic Acids Research, 53(D1):D604–D608, 2025.
- [39]
Tiing Jen Loh, Jia Jia Lim, Claerwen M Jones, Hien Thy Dao, Mai T Tran, Daniel G Baker, Nicole L La Gruta, Hugh H Reid, and Jamie Rossjohn.
The molecular basis underlying t cell specificity towards citrullinated epitopes presented by hla-dr4.
Nature Communications, 15(1):6201, 2024.
- [40]
Claus Lundegaard, Kasper Lamberth, Mikkel Harndahl, Søren Buus, Ole Lund, and Morten Nielsen.
Netmhc-3.0: accurate web accessible predictions of human, mouse and monkey mhc class i affinities for peptides of length 8–11.
Nucleic acids research, 36(suppl_2):W509–W512, 2008.
- [41]
Koshlan Mayer-Blackwell, Stefan Schattgen, Liel Cohen-Lavi, Jeremy C Crawford, Aisha Souquette, Jessica A Gaevert, Tomer Hertz, Paul G Thomas, Philip Bradley, and Andrew Fiore-Gartland.
Tcr meta-clonotypes for biomarker discovery with tcrdist3 enabled identification of public, hla-restricted clusters of sars-cov-2 tcrs.
Elife, 10:e68605, 2021.
- [42]
Ramgopal R Mettu, Tysheena Charles, and Samuel J Landry.
Cd4+ t-cell epitope prediction using antigen processing constraints.
Journal of immunological methods, 432:72–81, 2016.
- [43]
Barthelemy Meynard-Piganeau, Christoph Feinauer, Martin Weigt, Aleksandra M Walczak, and Thierry Mora.
Tulip: A transformer-based unsupervised language model for interacting peptides and t cell receptors that generalizes to unseen epitopes.
Proceedings of the National Academy of Sciences, 121(24):e2316401121, 2024.
- [44]
Alexander Myronov, Giovanni Mazzocco, Paulina Król, and Dariusz Plewczynski.
Bertrand—peptide: Tcr binding prediction using bidirectional encoder representations from transformers augmented with random tcr pairing.
Bioinformatics, 39(8):btad468, 2023.
- [45]
Jacques Neefjes, Marlieke LM Jongsma, Petra Paul, and Oddmund Bakke.
Towards a systems understanding of mhc class i and mhc class ii antigen presentation.
Nature reviews immunology, 11(12):823–836, 2011.
- [46]
Morten Nielsen, Massimo Andreatta, Bjoern Peters, and Søren Buus.
Immunoinformatics: predicting peptide–mhc binding.
Annual review of biomedical data science, 3(1):191–215, 2020.
- [47]
Morten Nielsen, Claus Lundegaard, Thomas Blicher, Kasper Lamberth, Mikkel Harndahl, Sune Justesen, Gustav Røder, Bjoern Peters, Alessandro Sette, Ole Lund, et al.
Netmhcpan, a method for quantitative predictions of peptide binding to any hla-a and-b locus protein of known sequence.
PloS one, 2(8):e796, 2007.
- [48]
Morten Nielsen, Claus Lundegaard, Peder Worning, Sanne Lise Lauemøller, Kasper Lamberth, Søren Buus, Søren Brunak, and Ole Lund.
Reliable prediction of t-cell epitopes using neural networks with novel sequence representations.
Protein Science, 12(5):1007–1017, 2003.
- [49]
Bjoern Peters, Morten Nielsen, and Alessandro Sette.
T cell epitope predictions.
Annual Review of Immunology, 38(1):123–145, 2020.
- [50]
Bjoern Peters and Alessandro Sette.
Generating quantitative models describing the sequence specificity of biological processes with the stabilized matrix method.
BMC bioinformatics, 6(1):1–9, 2005.
- [51]
Mansour Poorebrahim, Niloufar Mohammadkhani, Reza Mahmoudi, Monireh Gholizadeh, Elham Fakhr, and Angel Cid-Arregui.
Tcr-like cars and tcr-cars targeting neoepitopes: an emerging potential.
Cancer gene therapy, 28(6):581–589, 2021.
- [52]
Yao Qiang, Deng Pan, Chengyin Li, Xin Li, Rhongho Jang, and Dongxiao Zhu.
Attcat: Explaining transformers via attentive class activation tokens.
Advances in neural information processing systems, 35:5052–5064, 2022.
- [53]
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al.
Improving language understanding by generative pre-training.
OpenAI, 2018.
- [54]
Luis A Rojas, Zachary Sethna, Kevin C Soares, Cristina Olcese, Nan Pang, Erin Patterson, Jayon Lihm, Nicholas Ceglia, Pablo Guasp, Alexander Chu, et al.
Personalized rna neoantigen vaccines stimulate t cells in pancreatic cancer.
Nature, 618(7963):144–150, 2023.
- [55]
A Saranya and R Subhashini.
A systematic review of explainable artificial intelligence models and applications: Recent developments and future trends.
Decision analytics journal, 7:100230, 2023.
- [56]
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.
Grad-cam: Visual explanations from deep networks via gradient-based localization.
In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
- [57]
Ido Springer, Nili Tickotsky, and Yoram Louzoun.
Contribution of t cell receptor alpha and beta cdr3, mhc typing, v and j genes to peptide binding prediction.
Frontiers in immunology, 12:664514, 2021.
- [58]
Guillaume BE Stewart-Jones, Andrew J McMichael, John I Bell, David I Stuart, and E Yvonne Jones.
A structural basis for immunodominant human t cell receptor recognition.
Nature immunology, 4(7):657–663, 2003.
- [59]
Nili Tickotsky, Tal Sagiv, Jaime Prilusky, Eric Shifrut, and Nir Friedman.
Mcpas-tcr: a manually curated catalogue of pathology-associated t cell receptor sequences.
Bioinformatics, 33(18):2924–2929, 2017.
- [60]
Sebastiaan Valkiers, Max Van Houcke, Kris Laukens, and Pieter Meysman.
Clustcr: a python interface for rapid clustering of large sets of cdr3 sequences with unknown antigen specificity.
Bioinformatics, 37(24):4865–4867, 2021.
- [61]
Floris J van Eerden, Aalaa Alrahman Sherif, Mara Anais Llamas-Covarrubias, Arthur Millius, Xiuyuan Lu, Shigenari Ishizuka, Sho Yamasaki, and Daron M Standley.
Tcr-pmhc complex formation triggers cd3 dynamics.
eLife, 12, 2023.
- [62]
A Vaswani.
Attention is all you need.
Advances in Neural Information Processing Systems, 2017.
- [63]
Randi Vita, Swapnil Mahajan, James A Overton, Sandeep Kumar Dhanda, Sheridan Martini, Jason R Cantrell, Daniel K Wheeler, Alessandro Sette, and Bjoern Peters.
The immune epitope database (iedb): 2018 update.
Nucleic acids research, 47(D1):D339–D343, 2019.
- [64]
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov.
Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned.
In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5797–5808, 2019.
- [65]
Anna Weber, Jannis Born, and María Rodriguez Martínez.
Titan: T-cell receptor specificity prediction with bimodal attention networks.
Bioinformatics, 37(Supplement_1):i237–i244, 2021.
- [66]
Sarah Wiegreffe and Yuval Pinter.
Attention is not not explanation.
In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 11–20, 2019.
- [67]
Junyi Wu, Bin Duan, Weitai Kang, Hao Tang, and Yan Yan.
Token transformation matters: Towards faithful post-hoc explanation for vision transformer.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10926–10935, 2024.
- [68]
Kevin E Wu, Kathryn Yost, Bence Daniel, Julia Belk, Yu Xia, Takeshi Egawa, Ansuman Satpathy, Howard Chang, and James Zou.
Tcr-bert: learning the grammar of t-cell receptors for flexible antigen-binding analyses.
In Machine Learning in Computational Biology, pages 194–229. PMLR, 2024.
- [69]
Xinbo Yang, Guobing Chen, Nan-ping Weng, and Roy A Mariuzza.
Structural basis for clonal diversity of the human t-cell response to a dominant influenza virus epitope.
Journal of Biological Chemistry, 292(45):18618–18627, 2017.
- [70]
Hongyi Zhang, Longchao Liu, Jian Zhang, Jiahui Chen, Jianfeng Ye, Sachet Shukla, Jian Qiao, Xiaowei Zhan, Hao Chen, Catherine J Wu, et al.
Investigation of antigen-specific t-cell receptor clusters in human cancers.
Clinical Cancer Research, 26(6):1359–1371, 2020.
- [71]
Hongyi Zhang, Xiaowei Zhan, and Bo Li.
Giana allows computationally-efficient tcr clustering and multi-disease repertoire classification by isometric transformation.
Nature communications, 12(1):4699, 2021.
- [72]
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.
Learning deep features for discriminative localization.
In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016.

## Appendix A Technical Appendices and Supplementary Material

### A.1 TCR-pMHC Prediction Transformer Models

Transformers [62], as a successful deep learning models in different areas, have a series of variants such as Bidirectional Encoder Representations from
Transformers (BERT) [18] and Generative Pre-training Transformers (GPT) [53]. These models support multi-sequence inputs and excel in modeling interactions, are well-suited for this task.
Because TCR-pMHC interactions are determined by interactions among the TCR $\alpha$ and $\beta$ chains, epitope, and MHC, several state-of-the-art models, such as TULIP [43] and cross-TCR-interpreter [32], adopt encoder-decoder transformer architectures to learn these complex relationships.

Figure: Figure 9: The architecture figure of TULIP model [43].
Refer to caption: extracted/6594550/figures/appendix/tulip.png

TULIP:
TULIP is a transformer-based model with an encoder-decoder architecture designed for TCR-pMHC binding prediction. It operates through three parallel modality processing pipelines, processing CDR3a, CDR3b, and epitope sequences separately [43]. The encoders transform the input sequences into feature representations, while the decoders model interactions across different sequences [17, 62]. As an auto-regressive generative model, TULIP computes the conditional probability distribution of sequences (e.g., epitope) given others (e.g., CDR3a, CDR3b, and MHC) [43].
To compute gradients for TULIP, we design an amino-acid-wise loss function. The ground truth is derived from the TCR alpha, TCR beta, and epitope sequences. These sequences are first one-hot encoded, and the model’s predicted probabilities are compared against them using a negative log-likelihood (NLL) loss. This formulation allows us to attribute importance scores at the amino acid level based on how well the model reconstructs each residue.

Figure: Figure 10: The architecture figure of BERTrand model [44].
Refer to caption: extracted/6594550/figures/appendix/bertrand.jpeg

BERTrand:
BERTrand is an encoder-only transformer model [44]. It takes the TCR beta chain and peptide sequence as inputs. These two sequences are concatenated using a <SEP> token and are processed jointly by a transformer encoder as an integrated input. Similar to CrossTCRInterpreter, BERTrand is a classification model designed to predict whether the TCR-pMHC pair is a binder or a non-binder. We apply a binary classification loss to obtain the model gradients.

Figure: Figure 11: The architecture of CrossTCRInterpreter model [32].
Refer to caption: extracted/6594550/figures/appendix/crosstcrinterpreter.jpg

CrossTCRInterpreter:
CrossTCRInterpreter is an encoder-decoder transformer for TCR-pMHC binding prediction [32]. It takes the CDR regions of the alpha and beta chains, along with the peptide sequence, as inputs. The CDR alpha and beta chains are concatenated using a colon ($:$) to form the TCR input. The TCR and peptide sequences are then independently encoded by an encoder module. Subsequently, cross-attention is employed to model the interaction between the two inputs and predict whether the pair represents a binder or a non-binder. We apply a binary classification loss to extract the model gradients.

#### A.1.1 ROC Analysis

Figure: Figure 12: ROC curve comparison of the alpha, beta, and epitope chains between QCAI and other post-hoc methods. The distance thresholds are set to 4, 5, and 6 Å.
Refer to caption: x8.png

### A.2 TCR-XAI Benchmark

**Table 3: The samples contained in TCRxAI benchmarks**
| PDB | MHC | Peptide | CDRA3 | CDRB3 |
| --- | --- | --- | --- | --- |
| 5EU6 | MHCI | YLEPGPVTV | AVLSSGGSNYKLT | ASSFIGGTDTQY |
| 7PBE | MHCI | YLQPRTFLL | VVNINTDKLI | ASSSANSGELF |
| 4Z7W | MHCII | PSGEGSFQPSQENPQ | AVGETGANNLF | ASSEARRYNEQF |
| 6V18 | MHCII | GGYRAPAKAAAT | ALSDSGSFNKLT | ASSLDWGGQNTLY |
| 8EO8 | MHCI | LPFDKATIM | AADGGAGSYQLT | SAGPTSGRTDTQY |
| 5WKH | MHCI | GTSGSPIINR | GLGDAGNMLT | ASSLGQGLLYGYT |
| 7T2B | MHCII | ATGLAWEWWRTVYE | ATDKKGGATNKLI | ASSQGGGEQY |
| 7SG1 | MHCII | QPFPQPELPYGSGGS | LVGGLARDMR | SVALGSDTGELF |
| 3W0W | MHCI | RFPLTFGWCF | GTYNQGGKLI | ASSGASHEQY |
| 5W1V | MHCI | VMAPRTLIL | AGQPLGGSNYKLT | ASSANPGDSSNEKLF |
| 6AVF | MHCI | APRGPHGGAASGL | LVGEILDNFNKFY | ASSQRQEGDTQY |
| 5NHT | MHCI | ELAGIGILTV | AVGGGADGLT | ASSQGLAGAGELF |
| 3TPU | MHCI | FLSPFWFDI | AVSAKGTGSKLS | ASSDAPGQLY |
| 2P5E | MHCI | SLLLMWIITQC | AVRPLLDGTYIPT | ASSYLGNTGELLF |
| 4P2O | MHCII | PADPLAFFSSAIKGGGGSLV | AALRATGGNNKLT | ASSLNWSQDTQY |
| 7NME | MHCI | QLPRLFPLL | AEPSGNTGKLI | ASSLHHEQY |
| 5JZI | MHCI | KLVALGINAV | AYGEDDKII | ASRRGPYEQY |
| 8I5C | MHCI | VVGAVGVGK | AARDSNYQLI | ASGDTGGYEQY |
| 4E41 | MHCII | GELIGILNAAKVPAD | AVDRGSTLGRLY | ASSQIRETQY |
| 6AM5 | MHCI | SMLGIGIVPV | AVNFGGGKLI | ASSLSFGTEAF |
| 3E2H | MHCI | QLSPFPFDL | AVSLERPYLT | ASGGGGTLY |
| 3MV8 | MHCI | HPVGEADYFEY | AVQDLGTSGSRLT | ASSARSGELF |
| 3KPR | MHCI | EEYLKAWTF | ILPLAGGTSYGKLT | ASSLGQAYEQY |
| 5HYJ | MHCI | AQWGPDPAAA | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 5KS9 | MHCII | APSGEGSFQPSQENPQ | AVALNNNAGNMLT | ASSVAPGSDTQY |
| 7T2D | MHCII | ATGLAWEWWRTVYE | ALSGSARQLT | ASSHREGETQY |
| 1KJ2 | MHCI | KVITFIDL | AARYQGGRALI | TCSAAPDWGASAETLY |
| 6RPB | MHCI | SLLMWITQV | AVKSGGSYIPT | ASSYLNRDSALD |
| 6UON | MHCI | GADGVGKSAL | AAAMDSSYKLI | ASSDPGTEAF |
| 1ZGL | MHCII | VHFFKNIVTPRTPG | ALSGGDSSYKLI | ASSLADRVNTEAF |
| 3PQY | MHCI | SSLENFRAYV | ILSGGSNYKLT | ASSFGREQY |
| 4Y19 | MHCII | QPLALEGSLQKRG | AASVYAGGTSYGKLT | ASRPRRDNEQF |
| 6AVG | MHCI | APRGPHGGAASGL | LVVDQKLV | ASSGGHTGSNEQF |
| 6V15 | MHCII | GGYAPAKAAAT | ALSPSNTNKVV | ASSLDWGVNTLY |
| 4Z7U | MHCII | APSGEGSFQPSQENPQ | ILRDRSNQFY | ASSTTPGTGTETQY |
| 7RM4 | MHCI | HMTEVVRHC | ALDIYPHDMR | ASSLDPGDTGELF |
| 4QOK | MHCI | EAAGIGILTV | AVNVAGKST | AWSETGLGTGELF |
| 3PWP | MHCI | LGYGFVNYI | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 7N6E | MHCI | YLQPRTFLL | VVNRNNDMR | AGQVTNTGELF |
| 8WUL | MHCI | VVGAVGVGK | AARSSGSWQLI | ASSQDRGDSAHTLY |
| 3DXA | MHCI | EENLLDFVRF | IVWGGYQKVT | ASRYRDDSYNEQF |
| 6RP9 | MHCI | SLLMWITQV | ALTRGPGNQFY | ASSSPGGVSTEAF |
| 4MJI | MHCI | TAFTIPSI | ATDDDSARQLT | ASSLTGGGELF |
| 6V19 | MHCII | GGYAPAKAAAT | ALSDSSSFSKLV | ASSLDWASQNTLY |
| 7RDV | MHCII | EGRVRVNSAYQS | AASDDNNNRIF | ASGGQSNERLF |
| 3E3Q | MHCI | QLSPFPFDL | AVSDPPPLLT | ASGGGGTLY |
| 4MS8 | MHCI | SPAEEAGFFL | AVSAKGTGSKLS | ASSDAPGQLY |
| 2F53 | MHCI | SLLMWITQC | AVRPTSGGSYIPT | ASSYVGNTGELF |
| 3QDJ | MHCI | AAGIGILTV | AVNFGGGKLI | ASSLSFGTEAF |
| 6BJ2 | MHCI | IPLTEEAEL | ALSHNSGGSNYKLT | ASSFRGGKTQY |
| 3QDM | MHCI | ELAGIGILTV | AGGTGNQFY | AISEVGVGQPQH |
| 5TIL | MHCI | KAPYNFATM | AALYGNEKIT | ASSDAGGRNTLY |
| 6VMX | MHCI | RPPIFIRRL | AFGSSNTGKLI | ASSQDLFTGGYT |

**Table 4: The samples contained in TCRxAI benchmarks (continue table 1)**
| PDB | MHC | Peptide | CDRA3 | CDRB3 |
| --- | --- | --- | --- | --- |
| 7RTR | MHCI | YLQPRTFLL | AVNRDDKII | ASSPDIEQY |
| 8EN8 | MHCI | LPFDKSTIM | AADGGAGSYQLT | SAGPTSGRTDTQY |
| 1YMM | MHCII | ENPVVHFFKNIVTP | ATDTTSGTYKYI | SARDLTSGANNEQF |
| 5C07 | MHCI | YQFGPDFPIA | AMRGDSSSYKLI | ASSLWEKLAKNIQY |
| 3VXU | MHCI | RFPLTFGWCF | GTYNQGGKLI | ASSGASHEQY |
| 6VQO | MHCI | HMTEVVRHC | AMSGLKEDSSYKLI | ASSIQQGADTQY |
| 1J8H | MHCII | PKYVKQNTLKLAT | AVSESPFGNEKLT | ASSSTGLPYGYT |
| 8ENH | MHCI | LPFEKSTIM | AADGGAGSYQLT | SAGPTSGRTDTQY |
| 2P5W | MHCI | SLLMWITQC | AVRPLLDGTYIPT | ASSYLGNTGELF |
| 3UTT | MHCI | ALWGPDPAAA | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 7Q99 | MHCI | NLSALGIFST | AVNVAGKST | AWSETGLGTGELF |
| 6ZKZ | MHCI | RLPAKAPL | AVTNQAGTALI | ASSYSIRGSRGEQF |
| 4GG6 | MHCII | SGEGSFQPSQENP | ILRDGRGGADGLT | ASSVAVSAGTYEQY |
| 6TMO | MHCI | EAAGIGILTV | AVNDGGRLT | AWSETGLGMGGWQ |
| 3QIU | MHCII | ADLIAYLKQATKG | AAEPSSGQKLV | ASSLNNANSDYT |
| 5WKF | MHCI | GTSGSPIVNR | GLGDAGNMLT | ASSLGQGLLYGYT |
| 8SHI | MHCI | VRSRRLRL | ATDALYSGGGADGLT | ASSYSEGEDEAF |
| 5D2N | MHCI | NLVPMVATV | ILDNNNDMR | ASSLAPGTTNEKLF |
| 5KSA | MHCII | QPQQSFPEQEA | AVQFMDSNYQLI | ASSVAGTPSYEQY |
| 6MTM | MHCI | FEDLRVLSF | GTERSGGYQKVT | ASSMSAMGTEAF |
| 2BNQ | MHCI | SLLMWITQV | AVRPTSGGSYIPT | ASSYVGNTGELF |
| 4Z7V | MHCII | SGEGSFQPSQENP | ILRDSRAQKLV | ASSAGTSGEYEQY |
| 2F54 | MHCI | SLLMWITQC | AVRPTSGGSYIPT | ASSYVGNTGELF |
| 5BS0 | MHCI | ESDPIVAQY | AVRPGGAGPFFVV | ASSFNMATGQY |
| 6CQR | MHCII | RFYKTLRAEQASQ | AFKAAGNKLT | ASSRLAGGMDEQF |
| 5M00 | MHCI | KAVANFATM | AALYGNEKIT | ASSDDAAGGGGRNTLY |
| 7N2Q | MHCI | LRVMMLAPF | AVSNFNKFY | ASSVATYSTDTQY |
| 6EQB | MHCI | AAAAGGIIGGIILTV | AVNDGGRLT | AWSETGLGMGGWQQ |
| 4P2R | MHCII | ANGVAFFLTPFKA | AAEASNTNKVV | ASSLNNANSDYT |
| 4P2Q | MHCII | ADGLAYFRSSFKGG | AAEASNTNKVV | ASSLNNANSDYT |
| 8DNT | MHCI | LLLDRLNQL | AVREGAQKLV | ASSLDLGADEQF |
| 5E6I | MHCI | GILGFVFTL | AGPGGSSNTGKLI | ASSLIYPGELF |
| 5TJE | MHCI | KAVYNFATM | AALYGNEKIT | ASSDAGGRNTLY |
| 2J8U | MHCI | ALWGFFPVL | ALFLASSSFSKLV | ASSDWVSYEQY |
| 1LP9 | MHCI | ALWGFFPVL | ALFLASSSFSKLV | ASSDWVSYEQY |
| 3KPS | MHCI | EEYLQAFTY | ILPLAGGTSYGKLT | ASSLGQAYEQY |
| 2BNR | MHCI | SLLMWITQC | AVRPTSGGSYIPT | ASSYVGNTGELF |
| 5W1W | MHCI | VMAPRTLVL | AGQPLGGSNYKLT | ASSANPGDSSNEKLF |
| 6CQL | MHCII | RFYKTLRAEQASQ | AFKAAGNKLT | ASSRLAGGMDEQF |
| 5C09 | MHCI | YLGGPDFPTI | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 4MXQ | MHCI | SPAPRPLDL | AVSAKGTGSKLS | ASSDAPGQLY |
| 3SJV | MHCI | FLRGRAYGL | VVRAGKLI | ASGQGNFDIQY |
| 1QRN | MHCI | LLFGYAVYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 3KXF | MHCI | LPEPLPQGQLTAY | ALSGFYNTDKLI | ASPGLAGEYEQY |
| 5C0A | MHCI | MVWGPDPLYV | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 7N2N | MHCI | TRLALIAPK | AVLSPVQETSGSRLT | ASSVGLFSTDTQY |
| 8ES9 | MHCI | GVYDGREHTV | AVQPLNAGNNRKLI | SAREWGGTEAF |
| 2NX5 | MHCI | EPLPQGQLTAY | AVQASGGSYIPT | ATGTGDSNQPQH |
| 1G6R | MHCI | SIYRYYGL | AVSGFASALT | ASGGGGTLY |
| 8GVB | MHCI | RYPLTFGW | AVGFTGGGNKLT | ASSDRDRVPETQY |
| 8TRL | MHCII | EIFDSGNPTGEV | IVNPANTGNQFY | ASRRDYFSYEQY |
| 5D2L | MHCI | NLVPMVATV | AFITGNQFY | ASSQTQLWETQY |
| 5WLG | MHCI | SQLLNAKYL | ATVYAQGLT | ASSDWGDTGQLY |
| 5NMG | MHCI | SLFNTIAVL | AVRTNSGYALN | ASSDTVSYEQY |
| 7DZM | MHCI | TPQDLNTML | IVRGLNNAGNMLT | ASSLGIDAIY |
| 7BYD | MHCI | GGAI | LVGGGGYVLT | ASSQDLGAGEVYEQY |
| 5HHO | MHCI | GILEFVFTL | AGAGSQGNLI | ASSIRSSYEQY |

**Table 5: The samples contained in TCRxAI benchmarks (continue table 2)**
| PDB | MHC | Peptide | CDRA3 | CDRB3 |
| --- | --- | --- | --- | --- |
| 1QSE | MHCI | LLFGYPRYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 3RGV | MHCI | WIYVYRPMGCGGS | AANSGTYQR | ASGDFWGDTLY |
| 2E7L | MHCI | QLSPFPFDL | AVSHQGRYLT | ASGGGGTLY |
| 3MBE | MHCII | GAMKRHGLDNYRGYSLG | AAEDGGSGNKLI | ASSWDRAGNTLY |
| 5M01 | MHCI | KAPANFATM | AALYGNEKIT | ASSDDAAGGGGRRNTLY |
| 5SWZ | MHCI | ASNENMETM | AASETSGSWQLI | ASSRDLGRDTQY |
| 5NMF | MHCI | SLYNTIATL | AVRTNSGYALN | ASSDTVSYEQY |
| 8GVI | MHCI | RYPLTFGW | AVVFTGGGNKLT | ASSLRDRVPETQY |
| 7N5C | MHCI | SSLCNFRAYV | ILSGGCCNYKLT | ASSFGREQY |
| 8TRR | MHCII | GVYATSSAVRLR | ALGDTGNYKYV | ASSAVNSGNTLY |
| 4OZG | MHCII | APQPELPYPQPG | IVLGGADGLT | ASSFRFTDTQY |
| 4OZH | MHCII | APQPELPYPQPGS | IVWGGATNKLI | ASSVRSTDTQY |
| 2OL3 | MHCI | SQYYYNSL | AMRGDYGGSGNKLI | TCSADRVGNTLY |
| 7QPJ | MHCI | GLYDGMEHL | AVRGTGRRALT | ASSFATEAF |
| 3VXM | MHCI | RFPLTFGWCF | AVGAPSGAGSYQLT | ASSPTSGIYEQY |
| 6EQA | MHCI | AAAAGGIIGGIILTV | AVNVAGKST | AWSETGLGTGELF |
| 2YPL | MHCI | KAFSPEVIPMF | AVSGGYQKVT | ASTGSYGYT |
| 7RK7 | MHCI | YMDGTMSQV | LVALNYGGSQGNLI | AISPTEEGGLIFPGNTIY |
| 8WTE | MHCI | VVGAVGVGK | AARSSGSWQLI | ASSQDRGDSAETLY |
| 3UTS | MHCI | ALWGPDPAAA | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 1QSF | MHCI | LLFGYPVAV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 1OGA | MHCI | GILGFVFTL | AGAGSQGNLI | ASSSRSSYEQY |
| 2GJ6 | MHCI | LLFGKPVYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 3QDG | MHCI | ELAGIGILTV | AVNFGGGKLI | ASSLSFGTEAF |
| 2VLR | MHCI | GILGFVFTL | AGAGSQGNLI | ASSSRASYEQY |
| 7NA5 | MHCI | YGFRNVVHI | AVSNYNVLY | ASSQEPGGYAEQF |
| 8CX4 | MHCI | LRVMMLAPF | AVNSPGSGAGSYQLT | ASSVGTYSTDTQY |
| 4PRI | MHCI | HPVGEADYFEY | AVQDLGTSGSRLT | ASSARSGELF |
| 8YE4 | MHCI | NYNYLYRLF | VVNAHSGAGSYQLT | ASSETGGYEQY |
| 5M02 | MHCI | KAPFNFATM | AALYGNEKIT | ASSDAGGRNTLY |
| 2CKB | MHCI | EQYKFYSV | AVSGFASALT | ASGGGGTLY |
| 3TFK | MHCI | QLSDVPMDL | AVSAKGTGSKLS | ASSDAPGQLY |
| 7N2S | MHCI | TRLALIAPK | AVSLGTGAGSYQLT | ASSVGLYSTDTQY |
| 5KSB | MHCII | GPQQSFPEQEA | AVQASGGSYIPT | ASSNRGLGTDTQY |
| 2UWE | MHCI | ALWGFFPVL | ALFLASSSFSKLV | ASSDWVSYEQY |
| 7Q9A | MHCI | LLLGIGILVL | AVNVAGKST | AWSETGLGTGELF |
| 5C08 | MHCI | RQWGPDPAAV | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 3HG1 | MHCI | ELAGIGILTV | AVNVAGKST | AWSETGLGTGELF |
| 8I5D | MHCI | VVGAVGVGK | AASSGSWQLI | ASSLEGTVEETLY |
| 5JHD | MHCI | GILGFVFTL | AWGVNAGGTSYGKLT | ASSIGVYGYT |
| 7JWJ | MHCI | ASNENMETM | AAVTGNTGKLI | ASSRGTIHSNTEVF |
| 4MNQ | MHCI | ILAKFLHWL | AVDSATALPYGYI | ASSYQGTEAF |
| 6PY2 | MHCII | APFSEQEQPVLG | ASPQGGSEKLV | ASSSGGWGGGTEAF |
| 7DZN | MHCI | TPQDLNTML | IVRGLNNAGNMLT | ASSLGIDAIY |
| 4EUP | MHCI | ALGIGILTV | AVSGGGADGLT | ASSFLGTGVEQY |
| 7N1E | MHCI | RLQSLQTYV | ALSGFNNAGNMLT | ASSLGGAGGADTQY |
| 3QEQ | MHCI | AAGIGILTV | AGGTGNQFY | AISEVGVGQPQH |
| 2IAN | MHCII | GELIGTLNAAKVPAD | AALIQGAQKLV | ASTYHGTGY |
| 2VLJ | MHCI | GILGFVFTL | AGAGSQGNLI | ASSSRSSYEQY |
| 6CQN | MHCII | RFYKTLRAEQASQ | AFKAAGNKLT | ASSGLAGGMDEQF |
| 3VXR | MHCI | RYPLTFGWCF | AVRMDSSYKLI | ASSSWDTGELF |
| 7NMG | MHCI | LWMRLLPLL | AEPSGNTGKLI | ASSLHHEQY |
| 3D3V | MHCI | LLFGPVYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 5ISZ | MHCI | GILGFVFTL | AFDTNAGKST | ASSIFGQREQY |
| 6U3N | MHCII | APMPMPELPYP | AVGAGSNYQLI | ASSLEGQGASEQF |
| 6RSY | MHCI | RMFPNAPYL | IGGGTTSGTYKYI | ASSLGFGRDVMR |
| 4MVB | MHCI | QPAEGGFQL | AVSAKGTGSKLS | ASSDAPGQLY |

**Table 6: The samples contained in TCRxAI benchmarks (continue table 3)**
| PDB | MHC | Peptide | CDRA3 | CDRB3 |
| --- | --- | --- | --- | --- |
| 1MI5 | MHCI | FLRGRAYGL | ILPLAGGTSYGKLT | ASSLGQAYEQY |
| 3VXS | MHCI | RYPLTLGWCF | AVRMDSSYKLI | ASSSWDTGELF |
| 7OW5 | MHCI | VVVGAGGVGK | AMSVPSGDGSYQFT | ASKVGPGQHNSPLH |
| 8GVG | MHCI | RFPLTFGW | AVGFTGGGNKLT | ASSDRDRVPETQY |
| 2VLK | MHCI | GILGFVFTL | AGAGSQGNLI | ASSSRSSYEQY |
| 8GOM | MHCI | RLQSLQTYV | ASSGNTPLV | ASTWGRASTDTQY |
| 1D9K | MHCII | GNSHRGAIEWEGIESG | AATGSFNKLT | ASGGQGRAEQF |
| 6CQQ | MHCII | RFYKTLRAEQASQ | AFKAAGNKLT | ASSRLAGGMDEQF |
| 5HHM | MHCI | GILGLVFTL | AGAGSQGNLI | ASSSSRSSYEQY |
| 4N0C | MHCI | MPAGRPWDL | AVSAKGTGSKLS | ASSDAPGQLY |
| 5C0B | MHCI | RQFGPDFPTI | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 7PB2 | MHCI | VVVGADGVGK | ALSGPSGAGSYQLT | ASSYGPGQHNSPLH |
| 1MWA | MHCI | EQYKFYSV | AVSGFASALT | ASGGGGTLY |
| 4QRP | MHCI | HSKKKCDEL | ALSDPVNDMR | ASSLRGRGDQPQH |
| 6RPA | MHCI | SLLMWITQV | AVRDINSGAGSYQLT | SVGGSGGADTQY |
| 7N5P | MHCI | SSLCNFRAYV | ILSGGSNYKLT | ASSFFGREQY |
| 7N1F | MHCI | YLQPRTFLL | AVNRDDKII | ASSPDIEQY |
| 5C0C | MHCI | RQFGPDWIVA | AMRGDSSYKLI | ASSLWEKLAKNIQY |
| 6D78 | MHCI | AAGIGILTV | AVNFGGGKLI | ASSWSFGTEAF |
| 4JFF | MHCI | ELAGIGILTV | AVNDGGRLT | AWSETGLGMGGWQ |
| 4N5E | MHCI | VPYMAEFGM | AVSAKGTGSKLS | ASSDAPGQLY |
| 4JRX | MHCI | LPEPLPQGQLTAY | ALSGFYNTDKLII | ASPGETEAF |
| 7NMF | MHCI | QLPRLFPLL | AEPSGNTGKLI | ASSLHHEQY |
| 3QIW | MHCII | ADLIAYLEQATKG | AAEPSSGQKLV | ASSLNNANSDYT |
| 6ZKX | MHCI | RLPAKAPLLGCG | AVTNQAGTALI | ASSYSIRGSRGEQF |
| 1NAM | MHCI | RGYVYQGL | AMRGDYGGSGNKLI | TCSADRVGNTLY |
| 8PJG | MHCII | PKYVKQNTLKLAR | AVSEQDDKII | ATSDESYGYT |
| 8VCX | MHCII | GQVELGGGPGAESCQ | IVSHNAGNMLT | ASSLERETQY |
| 5YXU | MHCI | KLVALGINAV | AYGEDDKII | ASRRGSAELY |
| 3O4L | MHCI | GLCTLVAML | AEDNNARLM | SARDGTGNGYT |
| 7SG2 | MHCII | QPFPQPEQPFPGS | LVGGLARDMR | SVALGSDTGELF |
| 8GON | MHCI | RLQSLQIYV | ASSGNTPLV | ASTWGRASTDTQY |
| 2JCC | MHCI | ALWGFFPVL | ALFLASSSFSKLV | ASSDWVSYEQY |
| 6G9Q | MHCI | KAPYDYAPI | AALYGNEKIT | ASSDAGGRNTLY |
| 6DKP | MHCI | ELAGIGILTV | AVNFGGGKLI | ASSWSFGTEAF |
| 5NQK | MHCI | ELAGIGILTV | AGGGGADGLT | ASSQGLAGAGELF |
| 2PYE | MHCI | SLLMWITQC | AVRPLLDGTYIPT | ASSYLGNTGELF |
| 6R2L | MHCI | SLSKILDTV | AVGGNDWNTDKLI | ASSPLDVSISSYNEQF |
| 2OI9 | MHCI | QLSPFPFDL | AVSGFASALT | ASGGGGTLY |
| 8F5A | MHCI | TSTLQEQIGW | AVTLNNNAGNMLT | ASSVGGTEAF |
| 7Z50 | MHCII | LQTLALEVEDDPC | AASVRNYKYV | ASSRQGQNTLY |
| 6BGA | MHCII | YVVVPD | AALRATGGNNKLT | ASSLNWSQDTQY |
| 3MV7 | MHCI | HPVGEADYFEY | AVVQDLGTSGSRLT | ASSARSGELF |
| 8VD2 | MHCII | GQVELGGGTPIESC | IVRVAIEGSQGNLI | ASSLRRGDTIY |
| 8VCY | MHCII | GQVELGGGSSPETCI | IVSHNAGNMLT | ASSLERETQY |
| 5YXN | MHCI | KLVALGINAV | AYGEDDKII | ASRRGPYEQY |
| 5E9D | MHCI | ELAGIGILTV | AVTKYSWGKLQ | ASRPGWMAGGVELY |
| 6AMU | MHCI | MMWDRGLGMM | AVNFGGGKLI | ASSLSFGTEAF |
| 5BRZ | MHCI | EVDPIGHLY | AVRPGGAGPFFVV | ASSFNMATGQY |
| 3TJH | MHCI | SPLDSLWWI | AVSAKGTGSKLS | ASSDAPGQLY |
| 3H9S | MHCI | MLWGYLQYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 4PRP | MHCI | HPVGQADYFEY | AVQDLGTSGSRLT | ASSARSGELF |
| 5IVX | MHCI | RGPGRAFVTI | AASASFGDNSKLI | ASSLGHTEVF |
| 4Y1A | MHCII | LQPLALEGSLQKRG | AASSSAGGTSYGKLT | ASRPRDPVTQY |
| 2IAM | MHCII | GELIGILNAAKVPAD | AALIQGAQKLV | ASTYHGTGY |
| 6U3O | MHCII | AVVQSELPYPEGS | IAFQGAQKLV | ASSFRALAADTQY |

**Table 7: The samples contained in TCRxAI benchmarks (continue table 4)**
| PDB | MHC | Peptide | CDRA3 | CDRB3 |
| --- | --- | --- | --- | --- |
| 6V0Y | MHCII | GGYAPAKAAAT | ALSDSGSFNKLT | ASSLDWGGQNTLY |
| 5NME | MHCI | SLYNTVATL | AVRTNSGYALN | ASSDTVSYEQY |
| 7T2C | MHCII | TGLAWEWWRTVY | LVGDTGFQKLV | SARDPGGGGSSYEQY |
| 2PXY | MHCII | RGGASQYRPSQ | ALSENYGNEKIT | ASGDASGAETLY |
| 4G9F | MHCI | KRWIIMGLNK | AMRDLRDNFNKFY | ASREGLGGTEAF |
| 3QIB | MHCII | ADLIAYLKQATKG | AALRATGGNNKLT | ASSLNWSQDTQY |
| 4G8G | MHCI | KRWIILGLNK | AMRDLRDNFNKFY | ASREGLGGTEAF |
| 7N2O | MHCI | LRVMMLAPF | AVLSPVQETSGSRLT | ASSVGLFSTDTQY |
| 5TEZ | MHCI | GILGFVFTL | AASFIIQGAQKLV | ASSLLGGWSEAF |
| 3D39 | MHCI | LLFGPVYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 6ZKW | MHCI | RLPAKAPLL | AVTNQAGTALI | ASSYSIRGSRGEQF |
| 4FTV | MHCI | LLFGYPVYV | AVTTDSWGKLQ | ASRPGLMSAQPEQY |
| 6PX6 | MHCII | APFSEQEQPVLG | AVHTGARLM | ASSHGASTDTQY |
| 6V1A | MHCII | GGYRAPAKAAAT | ALSDSSSFSKLV | ASSLDWASQNTLY |
| 1U3H | MHCII | SRGGASQYRPSQ | AASANSGTYQR | ASGDAGGGYEQY |
| 7N4K | MHCI | SSLENFRRAYV | ILSGGSNYKLT | ASSFFGREQY |
| 2Z31 | MHCII | RGGASQYRPSQ | ALSENYGNEKIT | ASGDASGGNTLY |
| 2ESV | MHCI | VMAPRTLIL | IVVRSSNTGKLI | ASSQDRDTQY |
| 5EUO | MHCI | GILGFVFTL | AGAIGPSNTGKLI | ASSIRSSYEQY |
| 4JFD | MHCI | ELAAIGILTV | AVNDGGRLT | AWSETGLGMGGWQ |
| 6ZKY | MHCI | RLPAKAPL | AVTNQAGTALI | ASSYSIRGSRGEQF |
| 6TRO | MHCI | GVYDGREHTV | VVNHSGGSYIPT | ASSFLMTSGDPYEQY |
| 7N2P | MHCI | GQVMVVAPR | AVSNFNKFY | ASSVATYSTDTQY |
| 7R80 | MHCI | QASQEVKNW | AQLNQAGTALI | ASSYGTGINYGYT |
| 1BD2 | MHCI | LLFGYPVYV | AAMEGAQKLV | ASSYPGGGFYEQY |
| 4L3E | MHCI | ELAGIGILTV | AVNFGGGKLI | ASSWSFGTEAF |
| 7PHR | MHCI | YLEPGPVTV | ATDGSTPMQ | ASSWGAPYEQY |
| 3FFC | MHCI | FLRGRAYGL | AMREDTGNQFY | ASSFTWTSGGATDTQY |
| 4JRY | MHCI | LPEPLPQGQLTAY | AVGGGSNYQLI | ASSRTGSTYEQY |
| 5SWS | MHCI | ASNENMETM | AASEGSGSWQLI | ASSAGLDAEQY |
| 6UZ1 | MHCI | LLFGYPVYV | AVTTDRSGKLQ | ASRPGAAGGRPELY |
| 1FO0 | MHCI | INFDFNTI | AMRGDYGGSGNKLI | TCSADRVGNTLY |
| 7JWI | MHCI | ASNENMETM | AASETSGSWQLI | ASSRDLGRDTQY |
| 8D5Q | MHCI | HPGSVNEFDF | ALGDPTGANTGKLT | TCSAGRGGYAEQF |
| 6VRM | MHCI | HMTEVVRHC | VVQPGGYQKVT | ASSEGLWQVGDEQY |
| 7N2R | MHCI | TRLALIAPK | AVSNFNKFY | ASSVATYSTDTQY |
| 1FYT | MHCII | PKYVKQNTLKLAT | AVSESPFGNEKLT | ASSSTGLPYGYT |
| 3QFJ | MHCI | LLFGFPVYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 3GSN | MHCI | NLVPMVATV | ARNTGNQFY | ASSPVTGGIYGYT |
| 6V13 | MHCII | GGYRAPAKAAAT | ALSPSNTNKVV | ASSLDWGVNTLY |
| 7OW6 | MHCI | VVVGADGVGK | AMSVPSGDGSYQFT | ASKVGPGQHNSPLH |
| 4OZF | MHCII | APQPELPYPQPGS | IAFQGAQKLV | ASSFRALAADTQY |
| 4JFE | MHCI | ELAGIGALTV | AVNDGGRLT | AWSETGLGMGGWQ |
| 3MV9 | MHCI | HPVGEADYFEY | AVQDLGTSGSRLT | ASSARSGELF |
| 6Q3S | MHCI | SLLMWITQV | AVRPTSGGSYIPT | ASSYVGNTGELF |
| 5MEN | MHCI | ILAKFLHWL | AVDSATSGTYKYI | ASSYQGTEAF |
| 1AO7 | MHCI | LLFGYPVYV | AVTTDSWGKLQ | ASRPGLAGGRPEQY |
| 4H1L | MHCII | QHIRCNIPKRISA | AVGASGNTGKLI | ASSLRDGYTGELF |
| 8TRQ | MHCII | GVYATSSAVRLR | ALGDHSGSWQLI | ASSLRTGANSDYT |
| 4OZI | MHCII | QPFPQPELPYP | LVGDGGSFSGGYNKLI | SAGVGGQETQY |
| 2AK4 | MHCI | LPEPLPQGQLTAY | ALSGFYNTDKLI | ASPGLAGEYEQY |

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 4
presentation: 3
questions: - I understand the the method proposed outperforms the others, but how should I read the overall low AUCs in Fig. 2? Are most methods performing worse than random chance? Can the authors comment on why they consider an AUC of .6 as “demonstrating strong alignment” between importance scores and binding interactions?
- Can the authors test their method on tasks where explainable approaches work more convincingly?
- The method is applied to TULIP, a state-of-the-art model for TCR-pHMC binding prediction. Are the benchmark data curated by the authors in the training set of TULIP, or not?
 
Minor:

The notation for element-wise multiplication is not consistent throughout the paper ($\odot$ vs. $\cdot$)
soundness: 3
strengths: Methods to explain cross-attention in transformers are needed to go beyond their use as “black boxes”. The tested case is timely and interesting for the community working in molecular biology. The benchmark dataset curated by the authors can also be useful.
summary: The paper introduces a novel method to interpret the features learned by an encoder-decoder transformer with cross attention. The method relies on the evaluation of attention scores for query and key values, and aggregates them across the layers of the decoder. The method is then tested against others on the task of predicting contact residues from models trained on protein binding (TCR-pMHC), showing increase in performance. In order to do so, the authors provide an annotated benchmark dataset.
weaknesses: If I understand correctly, the task in which the authors test their method is rather hard: all the competitors perform in a way that is comparable or worse than random chance (Fig. 2), showing biased assessment. For a method which is “potential to be applied to other fields”, as the authors claim, other applications, where explainable methods work more reliably, should be tested.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: - In the introduction, TULIP is mentioned as an example of a BERT-style model, but as mentioned in other parts of the paper, TULIP is an autoregressive encoder-decoder model, not an encoder-only BERT-like model.
- The 2017 transformer paper should be cited in section 2
- Did the author consider evaluating their method on a transformer trained on natural language?
soundness: 3
strengths: - The primary strength is providing an explainable AI method for encoder-decoder transformers, like TULIP, which current methods designed for encoder-only models cannot adequately interpret.
- QCAI's performance is validated against a suite of competing methods (e.g., AttnLRP, TokenTM, Rollout) using ROC analysis, two different perturbation metrics (AOPC and LOdds) , and the authors' own BRHR metric, demonstrating SOTA results across most of them.
summary: Transformer-based models, while effective for predicting TCR-pMHC binding, are "black boxes" because existing explainable AI methods are not designed for their encoder-decoder architectures. This paper introduces Quantifying Cross-Attention Interaction, a post-hoc explanation method specifically designed to interpret the cross-attention mechanisms in the decoder part of the transformer. To quantitatively evaluate this new method, the authors also compiled TCR-XAI, a benchmark of 274 TCR-PMHC crystal structures derived from STCRDab and TCR3d 2.0 that uses physical residue distance as a ground truth for binding interaction. The authors demonstrate that QCAI achieves good performance on this benchmark across multiple metrics, including ROC-AUC, perturbation studies (LOdds and AOPC), and a new metric called Binding Region Hit Rate.
weaknesses: - The paper explicitly states that QCAI is up to 50x slower per sample than other methods due to the necessary pseudo-inverse operations.
- The new TCR-XAI benchmark is heavily skewed towards MHC-I samples, which may limit the generalizability of the findings for MHC-II complexes.
- The benchmark relies on atomic distance as a "proxy for ground-truth importance". An assessment based on an energy function would have been more appropriate

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 4
questions: 1\. Since the experiments are only performed on known, "positive" TCR-pMHC pairs (those already with a stable structure), what bias might that introduce? Please discuss.

2\. Could the proposed method be applied to downstream prediction tasks such as impacts of sequence or structure perturbations, or other protein-protein interaction types?  Please discuss.

3\. Since the method requires 3D structure of the interaction complex, how would predicted structures (such as by AlphaFold base model or fine-tuned on TCR-pMHC) be used when structures are not available? From another perspective, is it possible to use the method for the evaluation of predicted structures or unknown interaction pairs?

4\. Does the range and scale of the residue importance show any relationship to the TULIP model's confidence of prediction?

5\. Since some samples in PDB are close neighbours (differing by only one or two residues), such as 2PXY and 2Z3, does the importance score show any different or similar patterns between the different residues?
soundness: 4
strengths: 1\. This work offers insights into the important problem of interaction interpretation in immune-proteins. Considering the scarcity of data and limited prediction performance of existing models, a reliable interpretation method would allow maximal use of available data and models. The method and findings could potentially guide rational design of TCRs and respective immunotherapy.

2\. The authors provide solid theoretical justifications of the methodology as well as practical insights.

3\. The work offers a useful benchmark dataset for future work.

4\. Case studies with visualization clearly demonstrate the biochemical implications of the proposed importance scores.
summary: This work presents a method for the interpretation of TCR-pMHC binding prediction models with encoder-decoder structure and cross attention mechanisms, as well as a benchmark dataset of ground truth interacting residues defined by residue distance.. Specifically, the proposed method, quantifying cross-attention interaction (QCAI), combines a GradCAM-style intrinsic importance and a relevance score derived from the attention matrix. The authors demonstrated superior performance of the proposed importance in the task of binding site retrieval by log-odds and area under perturbation metrics.
weaknesses: Despite the sound problem setup and results, my major concern is the limited application both within and beyond the domain of TCR-pMHC. Specifically:

1\. The scope of the defined task and the respective benchmark dataset is somewhat limited. Distance is not the only indicator of interaction and only weakly indicates "importance" overall, considering residue contributions to TCR-pMHC interactions are somewhat additive (smaller, weaker interactions than dominating hotspots). Though that may be hard to analyze with the current tools, it should be briefly discussed.

2\. The method relies on TCR-pMHC structures, which is rather rare compared to other interaction types.

3\. TCR-pMHC is an important and complex subject, the proposed method could be widely applied to many other interaction types. Generalizability needs to be shown, or at least discussed.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Although transformer models offer promising results, its positional encoding scheme is not suited for modelling the positional relations in interacting sequences, because (1) relative distance between residues across sequences are unknown and have to be inferred first; (2) existing positional encoding vectors are not physically meaningful in the first place. Therefore, while it is useful to provide explanations on a complex, data-driven black-box model, it could be more significant to devise a model that itself accord with first principles (or at least provide physically meaningful intermediate results). The author may refer to the recent work entitled "Sliding-attention transformer neural architecture for predicting T cell receptor-antigen-human leucocyte antigen binding" in Nature Machine Intelligence, 2024, 6(10): 1216-1230. 

Typos:
Lines 197–198: The phrase ‘denotes calculate max among feature dimension’ is ungrammatical.
Lines 407–408: The phrase ‘critical contacts with peptide peptides’ contains a redundant repetition of ‘peptide.’
Line 805: The text ‘was proposedAbnar & Zuidema’ is missing both a space and the preposition ‘by’.
soundness: 3
strengths: 1. The proposed method, which interprets the complex cross-attention mechanism, effectively addresses the application gap between the existing post-hoc explainable AI methods and scientific research problems conceptually.

3. The development of the TCR-XAI benchmark is useful for evaluation, grounding XAI performance in objective, physical atomic distances rather than subjective metrics.   

4. A comprehensive validation framework is used that employs a suite of metrics, including ground-truth correspondence, perturbation influence, and the novel Binding Region Hit Rate.
summary: This paper proposed QCAI to interpret encoder-decoder transformer models for the TCR-pMHC binding task, where existing explainability (XAI) methods are inadequate for the asymmetric nature of cross-attention. It combines gradient-based scoring with a Moore-Penrose pseudoinverse decomposition and element-wise score aggregation to attribute quantitative contributions back to the distinct query and key input tokens. The paper establishes the TCR-XAI benchmark, a curated dataset of 274 experimentally determined protein structures that utilizes atomic distance as a ground truth for interaction importance. Also  included is direct-comparison metric, the Binding Region Hit Rate (BRHR), to confirm QCAI's ability to generate quantitativeuperior and biologically plausible insights.
weaknesses: 1. **Ambiguity in Cross-Attention Inputs:** The paper does not clearly define the roles of the input sequences (CDR and Peptide) within the TULIP model's cross-attention mechanism (Figure 1). It is ambiguous which sequence provides the query vectors and which provides the key and value vectors. Given that the TULIP model is autoregressive and can predict any one sequence conditioned on the others, these roles may be dynamic. A detailed visualization of this process, **explicitly tagging each component with its corresponding input entity (e.g., CDR3α as query, Peptide as key)** , would be highly beneficial for interpreting the model's mechanics and the application of the QCAI method.

2. **Clarification of the Core Methodological Distinction:** The description of the gradient-based importance score does not clearly articulate how its computation differs for cross-attention versus self-attention. In particular, the manuscript does not say what is **unique or nontrivial about handling cross-attention compared to standard self-attention**. A more explicit treatment of these differences would help clarify the specific methodological contribution of QCAI, beyond applying generic gradient-weighted attention analysis.

3. **Experimental Setting around Attention Modules:** In the current experiments, the baseline methods are evaluated only on self-attention layers, with cross-attention omitted, whereas QCAI is evaluated on both self-attention and cross-attention. This raises two concerns. First, part of QCAI’s performance gain may simply come from exploiting additional cross-attention information that the baselines are not allowed to use, which makes the comparison potentially unfair. Second, because QCAI is only reported in its combined form, it is not possible to disentangle how much of the improvement is due to self-attention versus cross-attention. It would substantially strengthen the empirical analysis to report ROC-AUC, LOdds, and AOPC for **(i) QCAI applied only to self-attention and (ii) QCAI applied only to cross-attention**, in order to quantify the individual contribution of each attention component.


Decision: Accept (Poster)