xTrimoABFold: Improving Antibody Structure Prediction without Multiple Sequence Alignments 
Authors: Yining Wang
Xumeng Gong
Shaochuan Li
Bing Yang
YiWu Sun
Chuan Shi
Hui Li
Yangang Wang
Cheng Yang
Le Song
Venue: Submitted to ICLR 2023 (2023)
Keywords: Protein structure prediction
antibody structure prediction
amino acid sequence
homologous structure

Abstract: Antibody, used by the immune system to identify and neutralize foreign objects such as pathogenic bacteria and viruses, plays an important role in immune system. In the field of drug engineering, the essential task is designing a novel antibody to make sure its paratope (substructures in the antibody) binds to the epitope of the specific antigen with high precision. Also, understanding the structure of antibody and its paratope can facilitate a mechanistic understanding of the function. Therefore, antibody structure prediction has always been a highly valuable problem for drug discovery. AlphaFold2, a breakthrough in the field of structural biology, provides a feasible solution to predict protein structure based on protein sequences and computationally expensive coevolutionary multiple sequence alignments (MSAs). However, the computational efficiency and undesirable prediction accuracy on antibody, especially on the complementarity-determining regions (CDRs) of antibody limit its applications on the industrially high-throughput drug design. In this paper, we present a novel method named xTrimoABFold to predict antibody structure from antibody sequence based on a pretrained antibody language model (ALM) as well as homologous templates, which are searched from protein database (PDB) via fast and cheap algorithms. xTrimoABFold outperforms the MSA-based AlphaFold2 and the protein language model based SOTAs, e.g., OmegaFold, HelixFold-Single and IgFold with a large significant margin (30+% improvement on RMSD) while performs 151x faster than AlphaFold2. To the best of our knowledge, xTrimoABFold is the best antibody structure predictor to date in the world.

Under review as a conference paper at ICLR 2023
XTRIMOABFOLD: IMPROVING ANTIBODY STRUC- TURE PREDICTION WITHOUT MULTIPLE SEQUENCE ALIGNMENTS
Anonymous authors Paper under double-bling review
ABSTRACT
Antibody, used by the immune system to identify and neutralize foreign objects such as pathogenic bacteria and viruses, plays an important role in immune sys- tem. In the field of drug engineering, the essential task is designing novel anti- body to make sure its paratope (substructures in the antibody) binds to the epitope of the specific antigen with high precision. Also, understanding the structure of antibody and its paratope can facilitate a mechanistic understanding of the func- tion. Therefore, antibody structure prediction has always been a highly valuable problem for drug discovery. Alphafold2, a breakthrough in the field of structural biology, provides feasible solution to predict protein structure based pro- tein sequences and computationally expensive coevolutionary multiple sequence alignments (MSAs). However, the computational efficiency and undesirable pre- diction accuracy on antibody, especially the campllementarttyddeeemmm gions (CDRs) of antibody limit its applications on the industrially high-throughput drug design. In this paper, present novel method named xTrimoABBolk to predict antibody structure from antibody sequence based on a pretrained antibody language model (ALM) as well as homologous templates, which searched from protein database (PDB) via fast and cheap algorithms. XTrimoABFold out- performs the MSA-based AlphaFold2 and the protein language model based SO- TAs, e.g., OmegaFold, HelixFold-Sing and IgFold with large significant mar- gin (30+% improvement RMSD) while performs 151x faster than AlphaFold2. To the best of our knowledge, x ABrlo is the best antibody structure pre- dictor date in the world.
re-
INTRODUCTION
Antibody is an important type of proteins for disease diagnosis and treatment. The structures of antibodies closely related to their functions, that antibody structure prediction, which aims to predicting the 3D coordinates of atoms in antibody, is essential in the biological and medical applications such as protein engineering, modifying the antigen binding affinity, and identifying an epitope of specific antibody. However, manual experimental methods such as X-ray crystallography are time-consuming and expensive.
Recently, deep learning methods have achieved great success in protein structure prediction (Jumper al., 2021; Back al., [02T;] Li et al.) (2022). In short, these methods incorporate evolutional and geometric information of protein structures and deep neural networks. In particular, Al- phaFold2 (Jumper et al. 202T) introduces the architecture to jointly model the multiple sequence alignments (MSAs) and pairwise information, which is able to end-to-end predict the protein struc- tures in near experimental accuracy. Nevertheless, unlike general proteins, antibodies do not evolve naturally but rather they bind to specific antigens and evolve specifically (fast and one-way evolv- ing), the MSAs of antibodies especially on wwapllemennaatydddeemmiiii always available or reliable, which hurts the accuracy of models antibody data.
regions (CDRs) are not
With the development large-scale pretrained language model, many protein language models (Rao al. 2020 Elnaggar al. 2022 Rives et al. 2021 Rao al. 2021 Ruffolo al. 2021 Ofer al.

---

Under review as a conference paper at ICLR 2023
2021
2022) have been developed to generate the representation of protein sequence and Wu et al.
show promising performance on contact prediction (luchi et al. [021 Rao et al. 2020). functional
properties prediction (Meier et al.) [202]] Hie et al. 2021) and structure prediction from single
quence (Hong et al. 2022] Wu et al. 2022 Fang et al. 2022 Chowdhury et al. 2021 Ruffolo & [ (2022). These single- Sequence- based structure predictors typically follows a two-stage frame- work that i) trains a protein language model (PLM) on large-scale unlabeled protein databases, c.g., UniRe150, UniRef90, BFD or Observed Antibody Space (OAS) database (Olsen et al. 2022a), ii) employs the evoformer variants and structure module variants to predict protein structures from the learned representation from the pretrained PLM. Their experimental results show comparable accu- racy with the standard AlphaFold2 and perform much more efficient because of skipping the compu- tationally expensive CPU-based MSA searching stage. Although large-scale PLM show promising results, neither of PI.M-based methods outperform AlphaFold2 on both general protein databases and antibody databases.
Contribution. In this paper, focus one the of the most important problems in the field of drug discovery: antibody structure prediction. We claim that when conducting structure prediction on antibody, the general protein language model (PLM) is not the best optional. In contrast, we employ an antibody language model (ALM) pretrained on the large-scale OAS database and use evoformer and structure modules to learn the antibody structures in end-to-end fashion. Also, we design fast and cheap template searching algorithms based on two modalities of both sequence and structures. The searched templates help XBrol learn from a good starting point. We construct two large antibody databases of 19K antibody structure database and 501K protein structure database from RCSB PDB [Berman al., 2000) Experimental results show that our XTri per- forms much better than all the latest SOTAs, e.g., AlphaFold2, OmegaFold, HelixFold-Sin -ingle and IgFold, with significant margin (30+% improvement on RMSD). To the best of our knowledge, XTrimoA BFold is currently the most accurate antibody structure prediction model of the world. We believe such large improvement on antibody prediction from x ABFll will make a substantive impact drug discovery.
2 RELATED WORKS
Protein & Antibody Structure Prediction. Protein structure prediction aims to getting the 3D coordinates from an amino acid sequence, which has been an important open research problems for over 50 years (Dill et al. |008 [Anfinsen| (973). In recent years, deep learning methods have been widely used in protein structure prediction and considerable progress has been made by using the co-evolution information from Multiple Sequence AMiS))) such like AlphaFold (Se- nior et al. 2019 2020). AlphaFold2 (Jumper et al. (021). OpenFold (Ahdritz et al., [021] and RoseTT (Back et al., 202I). However, these methods are time-consuning and strictly de- pendent on MSAs, which remains a challenge for the structure prediction of orphan proteins with less homologous information or antibody for which MSAs are not always useful on account of a fast evolving nature. Recently, Lin et al. (2022), Fang et al.] (2022) and Wu et al. (2022) make protein structure prediction on large protein language models which are no longer dependent on MSAs, which drastically reduce computation time but incur certain loss of prediction precision. In particular, models like DeepAb (Ruffolo et al. 2022), ABlooper (Abanades et al. 2022 and
IgFold (Ruffolo & Gray 2022 are specifically developed for antibody structure.
pretrained Language Model on general protein and antibody.
General Protein Language Model (PLM). Typically, protein language models (Rao et al.,
2020
Elnaggar et al., 2022 Rives et al.] 2021 Rao al.; 202T Ruffolo al.) 2021 Ofer
et al.
2021: Wu et al., (0222) employ the popular transformer neural architecture vari- Vaswani et al. 2017) and train on different protein databases, such as UniRef50,
UniRet90 and BFD, etc. For example, |Roo et al. (202T) introduce ESM variants and use axial attention to learn row and column representation from MSA. Elnaggar al. (2022) train several PLMs with different number of parameters on UniRef and BFD datasets. Lin et al.] (2022) extends ESM and proposed ESM-2, which used relative position encoding to capture the intrinsic interaction of amino acids in the sequence. Wu et al. (2022) pro-
'The two datahase splitted by release datetime January 17th, 2022.
2

---

Under review as a conference paper at ICLR 2023
Pretrain Antibody Language Model
Residue Encoding
Single Repit
Single Repr.
Evo forner 48 Blocks
Attention Weights
+ Structure Module
A
Sequence
Residue Sequence
Searching
507k Protein
Structures Detahase
Pair Repr.
Strueture Seurching
Pair Hept.
Blocks
Ayiibody Datebase
Recycling
course- course-grained prediction
l'emplates
Template Features
fine grabned prediction
Figure I: The architecture of XrrimoABFold, which takes single sequence and coarse-grained struc- tural prediction as input, and uses pretrained antibody language model and cross-modal templates to model homologous sequences and structures respectively. Then, the combination of Evoformer (encoder) and Structure Module (decoder) of AlphaFold2 is employed to predict the fine- fine-graing greegra prediction 3D structure of antibody.
pose OmegaPI.M and use gated attention module (GAT) to replace self-attention layers in transformer.
Antibody Language Model (ALM). For antibody problems, language model trained on antibody sequences may learn more specific language information and can perform more powerful representations than general PLM. Sapiens (Prihoda al.) (022) consisting of two Transformer models were trained on 20M BCR heavy chains and 19M BCR light chains. DeepAb a long short-term memory (LSTM) network which is trained on 100k paired BCR sequences of Observed Antibody Space (OAS) database (Olsen et al. (022a). AbLang (Olsen et al. 20226) contains two models for either light chains or heavy chains trained OAS. AntiBERTa (Choi 2022) and AntiBERTy Ruffolo et al. (2021) both BERT-based pretrained language model for antibody trained on OAS with no distinction light and heavy chains.
Template Searching Algorithms and Tools For protein structure prediction, templates struc- tures can be a kind of auxiliary information to improve the quality structure models. HH- Search (Steinegger et 2019) applied by AlphaFold2 for template searching is an open source tool mainly used to template detection by HMM-HMM alignments between query and target database. Different from HHSearch by making search based on MSAs, FoldSeek (van Kempen al. 2022). can support a fast and accurate structure searching on datasets by using structure as query.
METHODOLOGY
In this section, we propose xTrimoABFold, an antibody structure prediction pipeline based on the AlphaFold2 architectures, but without the computationally expensive CPU-based MSA searching. Specifically, XTrimooBFold uses pretrained antibody language model generate residue encoding and pair attentions to initialize single and pair representations respectively, which can compensate MSAs. Also, xTrimoABFold adopts cross-modal tem-
the loss of homologous information plate searching algorithm, which can quickly search homologous structures in both sequential and structural modals. The overview of XTriooAFolld is shown in Figure.
3.1 SINGLE SEQUENCE MODELING WITH PRETRAINED ANTIBODY LANGUAGE MODEL
The excellent performance on many downstream tasks e.g., protein structure prediction, drug-target interaction of pretrained protein language models (PLMs) shows that PLMs can mine homologous
3

---

Under review as a conference paper at ICLR 2023
sequence information without complex manual preparation of MSAs. Therefore, XTrimoABFold uses a pretrained antibody language model to generate single and pair representations instead of MSAs.
Single Representation. By default, pretrained antibody language model generates residue (token) level representations with a single sequence as input, which can be used as the initial single repre- sentation of following evoformer (a transformer-based encoder) by proper transformation. Since the language model adopts the mechanism of multi-head self-attention, each token can get information from other tokens, which can be seen pair2residue communication. Formally, given residue sequence of an antibody, the computation of single representation is as follows:
2 PLMas(()) 8 Linear(z). E R.Nxd s0 ERNx
where N refers to the number of residues in the given protein, dim d, are the hidden sizes of lan- guage model and following evoformer respectively, PLMas is pretrained antibody language model, æ = (Ii, T2,-. ..... .NN) denotes the sequence of residues, and Linear refers to the linear layer of neural network. Then, 8 is used as the initial single representation of evoformer.
Pair Representation. The attention weights of multi-head self-attention mechanism in anti- body language model are rich in prior knowl- edge about the relation between residues such position information, which can be natu- rally combined as the initial pair representation through adaptive transformation. Specifically, the generation of initial pair representation can be formalized follows:
Query
A-V
V
Residuu2Patr
Weights
Figure 2: An illustration of the residue2pair com- munication.
di Concat (1
Qi (K anj dim
where p = Ldinear(gs)) a RNNNNHL L and H denote the number of layers and attention heads respectively, ke! and K are the query and key vectors of residues and j in I-th layer and k-th head respectively.aij denotes the relative position encoding between and j. and pd Nxx,, is the initial pair representation.
The above calculation can be regarded residue2pair communication because of multi- query- key product of residue pairs are involved in this step.
3.2 STRUCTURE PRIOR: CROSS-MODA HOMOLOGOUS STRUCTURE SEARCHING
Structure templates typically provide good prior for structure prediction. The previous works such as AlphaFold2 search templates by MSAs-based algorithms, e.g., MMseqs2 (Steinegger & Söding 2017). HHBlits (Remmert et al. 2012). which expensive, inefficient and highly depend on MSAs quality. In this paper, we provide an memory and computation efficient cross-modal template search- ing algorithm, which introduces two perspectives of sequence and structure to search templates without MSAs. Before conducting template search, we construct a novel antibody database named Antibody Data Bank (ADB), which is derived from RCSB Protein Data Bank (PDB) (Berman al. 2000) and contains 501K proteins structures.
Sequential Modal Searching. Taking into account the idea that similar antibody sequences are likely to have similar 3D structures we use sequence alignment based similarity score to search the structures with similar sequences target antibody from the database as the templates. The similarity score function can formalized as:
Atthough this idea is work everywhere, helieve it could provide well prior empirically.
4

---

Under review as a conference paper at ICLR 2023
Sim( 222) Align((1, #2))/ ma len(2:)))
where #1 and 2g are residue sequences, and (lign(.,.) is the sequence alignment, which denotes the maximum matched residues between two amino acid sequences (e.g., (iign""GV", 'GIV") 2), we use the Needleman- Wunsch algorithm (Likic) 2008) for sequence alignment computation. Sequential modal searching first screens out all sequences whose similarity scores in the range of (0.4, 0.95), and restricts the available templates to up to 10 with the maximum similarity scores to target sequence database. After that, these top 10 structures will be considered part of template candidates for the following training or inference.
Structural Modal Searching. Structural modal searching focuses finding similar structures in the database based on the coarse-grained structural prediction of the target antibody even though the sequences of these structures may not match the target. We use target sample in AlphaFold Protein Structure Database as the coarse-grained structural prior. Similar to sequential modal searching. we compute the alignment scores between structure of target antibody and structures in database and remove the target itself and the structures with too high similarity to the target structure from database. We add the top 10 structures the template candidate set. We use FoldSeek (van Kempen et al. 2022). a fast tool suitable for structure-pairwise alignment to calculate alignment scores.
Template Features After cross-modal template searching, we can get T template candidates, T less than or equal 20 because of the duplication of two modal search results. We choose templates from candidate set of top-2 sequential modal templates and top-2 structural modal tem- plates at inference time. In the training step, we randomly select muiiUnnffrra (0, T), 4) templates out of this restricted set of T templates, and we believe that the structures selected by two searching algorithms contain more homologous structure information, so that we assign higher sampling prob- ability to these structures. We employ the template encoder of AlphaFold2 to encode the template structures into two types template features of template angle features and template pair features, and add them to the initial single and pair representations respectively, which can be formalized as follows:
8 Concat(s Sta), D0- - dd Stpi
where fra E RAxNxd, 8 R p ftp E NxNxx, fra and fip the template angle and pair features respectively, and g0 and 6o are the initial single and pair representations with template features.
3.3 XTRIMOABFOLD
Evoformer and Structure Module. We use the Evoformer of AlphaFold2 to model complex in- formation initial single and pair representations. 8 and 6 can be naturally taken as the input of Evoformer. Note that the column- gated self-attention of Evoformer is necessary, which can exchange the sequence information modeled by the antibody language model with the structure in- formation of templates. We follow the other important components of AlphaFold2, i.c., Structure Module, which employs a number geometric transformation operators such as Invariant Point Attention predict the 3D structures of the protein end-to-end. Moreover, recycling mechanism is employed to refine the predicted structures iteratively followed AlphaFold2.
Training Objective. xTrimoABFold is trained end- and-to-end by variation of the loss function pro- posed by AlphaFold2 of Frame Aligned Point Error and a number of auxiliary losses, which can be formalized as follows:
Liin = 0.5LFAPE + 0.5hanz + 0.3 + 0.01L
(5)
'One can tune number of selected templates accordingly. "hatps//alphafooll.biiee http://alpppooll.........////
5

---

Under review as a conference paper at ICLR 2023
Table 1: Statistics of datasets
Dataset
Protein Count
Residue Count
Resolution Statistic
min
max
mean
std
min
max
mean
std
Training BCR Set
18470
879
191
46
0.92
9.00
2.82
0.95
Test BCR Set
470
236
1.14
7.51
2.81
0.78
RCSB PDB Set
501533
4433
259
199
0.48
9.00
2.59
1.00
where CFAPE refers to the framed aligned point error (FAPE) over all atoms (Jumper et al. 2021) Lauz are the averaged FAPE and torsion losses the inter-mediate structures over Ca only, Ldist an averaged cross-entropy loss for distogram prediction, Leon is the model confidence loss, these four loss functions are proposed by AlphaFold2. Compared to AlphaFold2, TrimoAABool removes the loss on masked MSA, because we don't I need MSA anymore.
Fine-tuning with CDR focal loss Since the structure of complementarity determining region (CDR) in antibody usually hard to predict than other framework regions (FR), we fine-tune xTri-- moABFold with a CDR focal loss after training. Formally, the CDR focal loss is denoted
Bij TT wi, Fij
Time
TT (IR 3x3 (R3x,, w. mie R x
dij
v | mm +G, c= 10
LIsCOR
Σ
Σ
min miedeaammmps dis)) delamp; 2 10A (8)
CDR N
Nirames jE Nuanes)
atoms
tE(1," NCDR
Lime Lirain XLFzCOR
where 2;, az!e are the prediction and ground-truth 3D coordinates of atom in CDR regions re- spectively, Tymmeme are the SE(3) transformations, NCCRR denotes the number of atoms in CDR regions of antibodies, and Nirames is the number of local frames. Fine-tuning with Limo-tune helps XTrid pay more attention to the difficult CDR regions.
EXPERIMENTS
4.1 EXPERIMENTAL. SETUP
Datasets. In the experiments, we created two large datasets. The first one is 19K antibody structure dataset. We got total of 18937 antibody data consisting of both amino acid sequences and structures selected from RCSB Protein Data Bank (PDB) (Berman et al. 2000) released before April 13th, 2022. The specific selections focusing the structures and sequences split each PDB file into single chains, and then make the selection. On the one hand, among the whole 19736 BCR chains from PDB, samples which have no structure resolution values those of which the structure resolution is larger than 9Å were filtered out to keep the quality of structure data. On the other hand, as for the sequences, we filtered out the samples whose sequence empty or those the repetition rate of a kind of amino acid is more than 90 percent in a sequence. Besides, we also conducted deduplication on the sequence and kept the samples which have lower structure resolution. After these filtering processes, we got 18937 antibody data our dataset. Among these, we collected data released before January 17th, 2022 as the training set which contains 18470 samples and other 470 samples to be the test set. The second dataset is our 501K protein structure database. We got the whole protein database downloaded from RCSB PDB. Filtering out the missing structure file, we got total of 593491 protein chains. Later, deleted the parts out of specification on structure resolution and sequence similarity mentioned before. After that, we finally get rid of the repeated examples getting total 501533 protein chains. T detailed statisties.
as follows. We firstly

---

Under review as a conference paper at ICLR 2023
Baseline. We compare XTrimoABFold method with 4 latest of-t protein structure pre- diction methods: AlphaFold2 (Jumper et al. | PLM-based HelixFold-Sine -ingle (Fang et al. 2022) and OmegaFold (Wu et al. 2022), ALM-based IgFold (Ruffolo & Gray 2022) For AlphaFold2, made the inference using five different models and picked up the structures with highest pL.DDT confidence for benchmarking. We also trained variant of our model called XTrimoABFold-ESSM which replaces the antibody language model to a general protein language model of ESM2 et al. worse than XTrimoABoold, which demon- strates that antibody language model is a better optional than general protein language model.
(2022). The performance of <TrimoABFold-ESM
Evaluation metrics. To evaluate the quality of antibody structure prediction, we used root-meam squared-deviatinn (RMSD). TM-Score (Zhang & Skolnick 2004), GDT .S and GDT HA Zemla et al. 1999) as the evaluation metric. Both two values are calculated over backbone heavy atoms after alignment of the respective framework residues by DeepAlign (Wang al. 2013). In order to evaluate the performance on CDR loops which are considered difficult for model to predict (Ruf-) folo & Gray 2022), we calculated these two metries on both antibody structures and CDR loop structures.
Leommon
1
TM-Score max
Ltarget
2
da da(Iiturget)
where Taarget is the sequence length of target protein and Ircommon is the number of residues that appear in both the template and target structures.
Hyperparameter Settings. We used AntiBERTy (Version 0.0.5, installed from PyPD), a BERT- based pretrained protein language model trained OAS with 558M antibody natural sequences to generate residue level representations, The hidden dimension of the model is 512 and the feedward dimension is 2048 AntiBERTy contains 8 layers, with attention heads per layer. In total, AntiB- ERTy contains approximately 26M trainable parameters (Ruffolo et al., (021). In the training part, we block the gradient backpropagation of antibody language model and just train the Evoformer and Structure Module. We use the Adam Optimizer (Kingma & Ba (014)) with the learning rate of le-3, 31 = 0.9, 32 0.999, e 8 and weight decay of O. Moreover, We also clip the gradient using the threshold of 10e9. Our model was trained for 25 epochs in 46 hours 8 NVIDIA A100 GPUs with a stayed batch size of 8. The same as AlphaFold2, the crop size of sequence is set into 256. On ac- count of the replacing of multi- sequence alignments(MSA) representation with the single sequence representation of antibody language model, we removed Imputdmbeddeer, ExtraMSA Embedder and ExtraMSAStack as well as the masked MSA loss compared to AlphaFold2. When making structural modal searching, Foldseck which enables fast and sensitive comparisons of large structure sets was used. While searching templates from PDB, the following flags were set non-default value for this tool. We chose 3Di Coloh--mmit-Waaterann as the alignment type and set max-seq into 2000.
4.2 RESULTS OF MAIN EXPERIMENTS
In this subsection, we show the main results and compare our XTimoABFold with four baselines. Main results contain two part, one of which is the model performance on evaluation metrics, and another is for the time efficiency. Table 2 and respectively show the accuracy performance of all models on antibody structure prediction and CDR loop structure prediction. For brevity. we only present RMSD and TM-score for 3 CDR loops. Moreover, we present the performance with respect the antibody structure prediction time of each methods on different length of amino acid sequence from the test dataset in Figure
Performance on antibody structure. XTrimoABFold significantly outperforms all baselines on test dataset for antibody structure prediction. In terms of RMSD, XTrimoABFold makes 32.60%. 35.42%, 28.97%, 85.21% improvements over AlphaFold2, OmegaFold, HelixFold-Single and Ig- Fold. In the meanwhile, this trend continues on other evaluation metrics. That is to say, our xT- rimoABFold achieves state-of-art performance on the antibody structure prediction compared with
"IIDDDT predicted local distance difference test.

---

Under review as a conference paper at ICLR 2023
1,400
1,050
Median time (5xc)
350
100)
900)
sequence longth AlphaFolc2
XTrtmoABFold
Figure 3: Median time MSA search, AlphaFold2 and XTrimoABFold. XTrimoABFold is 151 faster than AlphaFold2.
Table 2: Experimental results of antibody structure prediction on test dataset with 95% confidence interval. XFrimoABFold-ESS refers the similar approach like xTrimoABFold expect for replacing the pretrained ALM to the pretrained PLM: ESM2 (Lin et al. (0222 with 15b parameters (the largest PLM to date). The results show ALM is more suitable for antibody structure prediction.
Method
RMSD
TM-Score
GDT-TS
GDT_HA t
AlphaFold2
3.1254 0.1410
0,8385 = 0.0055
0.7948 = 0.0057
0.6548 = 0.0063
OmegaFold
3,2610 + 0.1463
0.8384 = 0.0057
0.7925 = 0.0059
0.6586 = 0.0063
HelixFold-Single
2.9648 0.0997
0.8328 = 0.0055
0.7805 0.0057
0.6225 + 0.0060
IgFold
14.2407 0.4115
0.5035 0.0143
0.4495 0.0157
0.3474 0.0148
XTrimoABFold
2.1059 0.0812
0.8936 # 0.0053
0.8655 = 0.0058
0.7439 # 0.0068
XTrimoABFold-EST.
4.8790 + 0.1299
0.7057 = 0.0061
0.6727 = 0.0063
0,4992 = 0.0062
not only PLM-based but also MSA-based protein structure prediction methods. To the best of our knowledge, our x ABFld has achieved optimal performance antibody structure prediction.
Performance CDR loops. As for the structure prediction of CDR loops, which well-known as difficult domains for model to make accurate prediction, our ATrimoABoold also performs well. In detail, ATold has relatively improvements over HelixFold-Single and IgFold, which are trained based on large scale protein language model and antibody language model on CDR1 and CDR2 loop. Concretely, we notice that our ABold yields the best performance in CDR3 loop which has been proven difficult domain to predict because of the highly variable and conformationally diverse.
Time efficiency on structure prediction XTrimoABFold performed a fast antibody structure pre- diction. AlphaFold2 make protein structure prediction according MSAs, which results in massive time consumption. Compared with AlphaFold2, our xTiomoA is a MSA-free model which predict the protein structure by single amino acid sequence with MSA searching. From Figure we can find that, our model performs 15IX faster than AlphaFold2, which overcomes the bottle- neck of time efficiency in protein structure prediction by AlphaFold2, enables large-scale antibody structures prediction in fast speed.
4.3 ABLATION STUDY OF MODEL COMPONENTS
In this subsection, make ablation studies to evaluate the performance improvement brought by our the introducing of pretrained antibody language model Antiberty and the added CDR focal loss when fine-tuning the model.
8

---

Under review as a conference paper at ICLR 2023
Table 3: Experimental results of Antibody CDR loop structure prediction test dataset with 95%
confidence interval.
Method
CDRI
CDR2
CDR3
RMSD
TM-Score
RMSD 1
TM Soure
RMSD 1
TM SMMoS f
AlphaFold2
0.7151 = 0.0409
0.6396 + 0.0070
0.6733 + 0.0360
0.6669 = 0.0069
1,4794 0.0682
0.5075 + 0.0000
OmegaFold
0.7202 0.0443
0.6457 0.0073
0.6804 +: 0.0406
0.6750 0.00669
1.4861 0.0685
0.5114 +: 0,0092
HelixFold-Single
1.5964 = 1093
0,3361 0.0131
1,1659 + 0.0894
0.3564 = 0.0137
1.6303 O. 1088
0.2132 + 0.01
IgFold
2.7411 0.1059
0,3138 0,0093
2,6774 0,1022
0.3339 0.0093
3.3222 1055
0.2104 0,0078
TrimooAFoola
0.6064 0.0294
0.6585 0.0072
0.5899 0.0278
0.6823 0.0067
1,2539 0.0584
0.5517 0.0095
1
RMSD
7a0w_E
7e8o_H
7b8p_D
7bh8_E
7bh8_M
7aDw_C
7e8o_L
7ekuS
7ek0_H
?enf_P
Figure Model performance on RMSD w/o CDR focal loss
Performance gains from pretrained antibody language model antibody language model, AntiBERTy to generate residue-level representations, which contains more specific antibody information compared to general protein language model like OmegaPI.M ESM-2(Rives et al., 2021) etc. Here fixed the other part of x xTrimoFold and use ESM-2, large- scale protein language model trained on 250 million protein sequences to validate our choice of antibody language model rather than regular protein language model. This model was trained on the same setting of xTrimoABFole and get worse performance compared to XTrimoABFold, the prediction performance can be seen in Table 2
XTrimoABFold used pretrained
Performance gains from CDR focal loss From test dataset, we randomly selected ten samples and compare the improvements of model performance before and after adding CDR focal loss. In these examples, observed various degree of decrease of RMSD value compared from predicted structure to the ground truth Figure 4, meaning that the our focal loss truly makes sense the anti- body structure prediction especially for the CDR loops which seems difficult to predict for regular models.
CONCLUSION
In this paper, we have proposed promising model, xTrimoABFold, for antibody structure predic- tion. On the one hand, XTrimoABFold employs the pretrained antibody language model to extract the information of single sequence, which is performs well than traditional protein language mod- cls. On the other hand, xTrimoABFold uses efficient template searching algorithm based on two modalities of both sequences and structures.
Limitations and Broader Impact. The limitation of this work is that only address the antibody structure prediction although it's important on drug discovery. the future, we will extend this work general protein prediction and the complex prediction.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Brennan Abanades, Guy Georges, Alexander Bujotzek, and Charlotte M Deane. Ablooper: Fast accurate antibody cdr loop structure prediction with accuracy estimation. Bioinformatics, 38(7): 1877-1880, 2022.
Gustaf Ahdritz, Nazim Bouatta, Sachin Kadyan, Qinghui Xia, William Gerecke, and Mo- hammed AlQuraishi. OpenFold, 11 2021. URL openfold
Christian Anfinsen. Principles that govern the folding of protein chains. Science, 181(40))): 223-230, 1973.
Minkyung Back, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov. Gyu Rie Lee. Jue Wang. Qian Cong. Lisa N Kinch, R Dustin Schaeffer, et al. Accurate prediction of protein structures and interactions using three-track neural network. Science, (73(6557):871- 876, 2021.
Helen M Berman, John Westbrook, Zukang Feng, Gary Gilliland, Talapady N Bhat, Helge Weissig, Ilya N Shindyalov, and Philip Bourne. The protein data bank. Nucleic acids research, 28(1): 235-242, 2000.
Yoonjoo Choi. Artificial intelligence for antibody reading comprehension: Antiberta. Patterns, (0::10053,,022
Ratul Chowdhury, Nazim Bouatta. Surojit Biswas, Charlotte Rochereau, George Church, Peter K Sorger, and Mohammed AlQuraishi. Single-sequence protein structure prediction using language models from deep learning. bioRxiv, 2021.
Ken A Dill, S Banu Ozkan, M Scott Shell, and Thomas R Weikl. The protein folding problem. Annual review of biophysics, 37:289, 2008
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, Debsindhu Bhowmik, and Burkhard Rost. Prottrans: Toward understanding the language of life through self-supervised learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(10):7112-1127, 2022. doi: 10.1109TPAM1.2211300333..
Xiaomin Fang, Fan Wang, Lihang Liu, Jingzhou He, Dayong Lin, Yingfei Xiang, Xiaonan Zhang, Hua Wu, Hui Li, and Le Song. Helixfold-single: Msa-free protein structure prediction by using protein language model as an alternative. arXiv preprint arXiv:2207. 13921, 2022.
Brian Hie, Ellen D Zhong, Bonnie Berger, and Bryan Bryson. Learning the language of viral evolu- tion and escape. Science, (7(66))) 2021.
Yiyu Hong, Juyong Lee, and Junsu Ko. A-prot: protein structure modeling using msa transformer. BMC bioinformatics, 23(1):1-11, 2022.
Hitoshi Iuchi, Taro Matsutani, Keisuke Yamada, Natsuki Iwano, Shunsuke Sumi, Shion Hosoda, Shitao Zhao, Tsukasa Fukunaga, and Michiaki Hamada. Representation learning applications in biological sequence analysis. Computational and Structural Biotechnology Journal, 19:3198- 3208, 2021.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583
2021. Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization, arXiv preprint arXiv:1412.6980 2014.
Ziyao Li, Xuyang Liu, Weijie Chen, Fan Shen, Hangrui Bi, Guolin Ke, and Linfeng Zhang Uni- fold: An open-source platform for developing protein folding models beyond alphafold. bioRxiv, 2022.
10

---

Under review as a conference paper at ICLR 2023
Vladimir Likic. The saalleman-wunseee algorithm for sequence alignment. Lecture given at the 7th Melbourne Bioinformatics Course, Bi021 Molecular Science and Biotechnology Institute, University of Melbourne, pp. 1-46, 2008.
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos San- Tom Sercu, Sal Candido, et al. Language models of protein
tos Costa, Maryam sequences at the scale of evolution enable accurate structure prediction. bioRxiv, 2022.
Joshua Meier, Roshan Rao, Robert Verkuil, Jason Liu, Tom Sercu, and Alex Rives. Language models enable zero-shot prediction of the effects of mutations protein function. Advances in Neural Information Processing Systems, 34:29287-29303 2021.
Dan Ofer, Nadav Brandes, and Michal Linial. The language of proteins: NIp, machine learning & protein sequences. Computational and Structural Biotechnology Journal, 19; 19:1750-1758, 2021.
Tobias Olsen, Fergus Boyles, and Charlotte M Deane. Observed antibody space: A diverse
database of cleaned, annotated, and translated unpaired and paired antibody sequences. Protein Science, 31(1):141-146, 2022a.
Tobias H Olsen, lain H Moal, and Charlotte M Deane. Ablang: An antibody language model for completing antibody sequences. bioRxiv, 2022b.
David Prihoda, Jad Maamary, Andrew Waight, Veronica Juan, Laurence Fayadat-Dilman, Daniel Svozil, and Danny A Bitton. Biophi: A platform for antibody design, humanization, and human- ness evaluation based on natural antibody repertoires and deep learning. In MAbs, volume 14, pp. 2020203. Taylor & Francis, 2022.
Roshan Rao, Joshua Meier, Tom Sercu, Sergey Ovchinnikov, and Alexander Rives. Transformer protein language models are unsupervised structure learners. Biorxiv, 2020.
Roshan M Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Pieter Abbeel, Tom Sercu, and Alexander Rives. Msa transformer. In International Conference on Machine Learning, pp. 8844-8856. PMLR, 2021.
Michael Remmert, Andreas Biegert, Andreas Hauser, and Johannes Söding. Hhblits: lightning-fan iterative protein sequence searching by tmmm-hmm alignment. Nature methods, 9(2):173-175 2012.
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, Lawrence Zitnick, Jerry Ma, al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, (18(15)e20162231118 2021.
Jeffrey A Ruffolo and Jeffrey J Gray. Fast, accurate antibody structure prediction from deep learning massive of natural antibodies. Biophysical Journal, 121(3):15a-116a, 2022.
Jeffrey Ruffolo, Jeffrey J Gray, and Jeremias Sulam. Deciphering antibody affinity maturation with language models and weakly supervised learning. arXiv preprint arXiv:21 2021.
Jeffrey A Ruffolo, Jeremias Sulam, and Jeffrey Gray. Antibody structure prediction using inter- pretable deep learning. Patlerns, 3(2): 100466, 2022.
Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Židek, Alexander WR Nelson, Alex Bridgland, et al. Protein structure prediction using multiple deep neural networks in the 13th critical assessment of protein structure prediction (casp13). Proteins: Structure, Function, and Bioinformatics, 87(12): 87(12)11141-1155 141-1148, 2019.
Andrew W Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Zidek, Alexander WR Nelson, Alex Bridgland, et al. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792)706-7100 2020.
Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology, 35(1 10(11)::026-028. 2017.

---

Under review as a conference paper at ICLR 2023
Martin Steinegger. Markus Meier, Milot Mirdita, Harald Vöhringer, Stephan Haunsberger, and Johannes Söding. Hh-suite3 for fast remote homology detection and deep protein annotation. BMC bioinformatics, 20(1):1-15, 2019.
Michel van Kempen, Stephanie Kim, Charlotte Tumescheit, Milot Mirdita, Johannes Söding, and Martin Steinegger. Foldseek: fast and accurate protein structure search. bioRxiv, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- tion processing systems, 30, 2017.
Sheng Wang, Jianzhu Ma, Jian Peng, and Jinbo Xu. Protein structure alignment beyond spatial proximity, Scientific Reports, 3, 2013.
Ruidong Wu, Fan Ding, Rui Wang, Rui Shen, Xiwen Zhang, Shitong Luo, Chenpeng Su, Zuofan Wu, Qi Xie, Bonnie Berger, al. High-resolution de novo structure prediction from primary sequence. bioRxiv, 2022.
Adam Zemla, Česlovas Venelovas, John Moult, and Krzysztof Fidelis. Processing and analysis of casp3 protein structure predictions. Proteins: Structure, Function, and Bioinformaties, 37(S3): 22-29, 1999.
Yang Zhang and Jeffrey Skolnick. Scoring function for automated assessment of protein structure template quality. Proteins: Structure, Function, and Bioinformatics, 57(4))702-770, 2004.
12


Reviews:
[{"review_id": "Oj0GUrCuxf", "reviewer": "Reviewer_bKCt", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This is computational biology paper which reports a novel protein structure folding technique. The key idea is to improve the prediction of antibody protein structures using an antibody sequence model, which they call antibody language model. For this, the authors create a database of routine structures from the PDB. The authors then use recurrent neural networks similar to transformers for creating the antibody language model. The authors use the antibody language model along with the Alphafold2 to predict protein structures. The authors also claim to design a fast multi-modal sequence search technique.\n\nThe author show improvement on TMscore and RM is the over many recent baselines, on a database comprising of only antibody proteins.", "summary_of_the_review": "In summary, I am not sure if ICLR is the right venue for the paper. While the paper uses some deep learning techniques, it's contributions are in the domain of computational biology. Another problem is that the document is not very accessible to someone not working on the particular problem of protein folding.\n\nOn the positive side, the authors show improvements over some very recent and impressive baselines. ", "strength_and_weaknesses": "The two claimed strengths are:\n- Antibody language model\n- Fast multi-modal search technique.\n\nHowever, I don't see any solid technical contribution in these. I am not actively working on this problem, so I cannot comment about the practical implications of the techniques.\n\nWeakness:\n- While the current method uses many deep learning techniques, the actual contribution seems to be entirely in the computational biology domain.\n- The writing of the method is not friendly to a beginner. For example, it is not clear to me how the PLM in the subsequent model.\n- Finally, the authors only compare with existing methods on the antibody dataset. I am not sure if the existing methods are specialized for the antibody structure prediction problem, which makes the comparison unfair.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MG14r2u_d2g", "reviewer": "Reviewer_1hLD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an antibody folding model based on existing architectures, including a protein language model (AntiBERTy), the EvoFormer and structure module from alphafold. The authors searched for templates from PDBs using Foldseek and extracted features using a similar workflow to AlphaFold. The method outperformed Alphafold and other folding algorithms based on protein language models in terms of RMSD (both framework and individual CDR regions).", "summary_of_the_review": "Despite its empirical value as a better antibody folding tool, I believe this paper should be submitted to a biophysics journal and reviewed by more domain experts who can properly interpret the significance of the results. The algorithmic innovation of this paper is weak, thus I vote for rejection.", "strength_and_weaknesses": "The major strength of this paper is the superior performance over existing state-of-the-art methods like AlphaFold, OmegaFold and IgFold. However, the weakness of this paper is its novelty. The proposed workflow is almost the same as IgFold. In fact, it uses the same antibody language model and the same architecture as AlphaFold. The only difference is perhaps a better search of templates using foldseek that may explain the mode better performance. However, even foldseek is developed by another group. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "3Ki6_Bn110", "reviewer": "Reviewer_hjqN", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, the authors propose a set of changes to the AlphaFold2 architecture to improve its performance on antibody folding. Most importantly, they show how to replace the MSA component of AlphaFold2 with a much faster encoder approach. Consequently, the proposed approach achieves competitive or better performance while reducing the runtime by two orders of magnitude.\n", "summary_of_the_review": "The paper shows that various engineering approaches indeed improve antibody folding prediction.", "strength_and_weaknesses": "The main strength of the paper is the empirical performance compared to existing state-of-the-art methods. In particular, the wall clock time comparison shows that the approach might meaningfully change what is feasible in terms of structure prediction and comparison.\n\nThe main weakness is the lack of novelty. The current work basically just plugs in existing approaches into an existing framework. I appreciate that choosing the right plug-ins isn’t easy, but the theoretical advances seem hard to pinpoint.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2P2SX3uKFE", "reviewer": "Reviewer_pjvj", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Antibody structure prediction is a highly sought after problem in industrial drug discovery pipelines. As with standard proteins, accurate predictions can lead to a better understanding of an antibody's function. Unfortunately, current methods for antibody structure prediction are not yet able to produce high-resolution predictions, particularly at the variable CDR regions of the antibodies. The paper presents xTrimoABFold, which is able to achieve high accuracy on a curated antibody structure dataset at much lower computational cost.", "summary_of_the_review": "The paper is quite poorly written. Besides the numerous grammatical errors and typos, the content of the paper, as it currently stands, would not qualify for publication. There is very little background knowledge provided, e.g. why do we care about CDRs for antibodies and how does this make the problem very different than standard protein structure prediction. The method itself is unoriginal as it uses the exact same architecture (without MSAs) and loss function (again, without MSA contribution) as AlphaFold2. The authors use FoldSeek for template search. The analysis of the results is also quite shallow. I have never seen a biological structure-prediction paper without a figure that displays 3D structures. The results in Table 2 and 3, while impressive, look almost too impressive. In addition, the results for IgFold look almost unreal as IgFold could be considered the current state-of-the-art for antibody structure prediction. ", "strength_and_weaknesses": "Strengths:\n- The results in Tables 2 and 3 are astounding, though some of the numbers may require further exploration/explanation.\n\nWeaknesses:\n- The method itself is not novel at all. The authors essentially reuse the loss function and architecture from AlphaFold2 (without MSAs). \n- The analysis of the results are quite shallow. For example, a structure prediction paper with no figures with structures is quite strange and unusual.\n- The results are almost too good to be true. In addition, the results for IgFold look confusingly poor. IgFold can be considered the current state-of-the-art. \n- Please include more background information on antibodies and proteins as a whole. For example, why do we even care about CDRs? There was not a single mention in the paper about what the CDR actually is and what is so critical about the CDR to an antibody's function.\n- Many grammatical errors and typos, to the point that the paper looks like it was rushed and ill-prepared. I understand that the authors may not be fluent in English, but many of these errors could easily have been fixed by copy-and-pasting into standard text editors/word processors with spelling and grammar checking.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject