Protein-Nucleic Acid Complex Modeling with Frame Averaging Transformer
Authors: Tinglin Huang
Zhenqiao Song
Rex Ying
Wengong Jin
Venue: NeurIPS 2024 poster (2024)
Keywords: Protein
Nucleic acid
Aptamer
3D Structure
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Nucleic acid-based drugs like aptamers have recently demonstrated great therapeutic potential. However, experimental platforms for aptamer screening are costly, and the scarcity of labeled data presents a challenge for supervised methods to learn protein-aptamer binding. To this end, we develop an unsupervised learning approach based on the predicted pairwise contact map between a protein and a nucleic acid and demonstrate its effectiveness in protein-aptamer binding prediction. Our model is based on FAFormer, a novel equivariant transformer architecture that seamlessly integrates frame averaging (FA) within each transformer block. This integration allows our model to infuse geometric information into node features while preserving the spatial semantics of coordinates, leading to greater expressive power than standard FA models. Our results show that FAFormer outperforms existing equivariant models in contact map prediction across three protein complex datasets, with over 10% relative improvement. Moreover, we curate five real-world protein-aptamer interaction datasets and show that the contact map predicted by FAFormer serves as a strong binding indicator for aptamer screening.

Protein-Nucleic Acid Complex Modeling with Frame Averaging Transformer
Tinglin Huang
I*
Zhenqiao Song?
Rex Ying'
Wengong 3.4
1 Yale University, *Carnegie Mellon University, Northeastern University, Khoury College of Computer Sciences 'Broad Institute of MIT and Harvard
Abstract
Nucleic acid-based drugs like aptamers have recently demonstrated great therapeu- tic potential. However, experimental platforms for aptamer screening are costly, and the scarcity of labeled data presents challenge for supervised methods learn protein-aptamer binding. To this end, we develop an unsupervised learning approach based on the predicted pairwise contact map between a protein and nu- cleic acid and demonstrate its effectiveness in protein-aptamer binding prediction. Our model is based FAFormer", novel equivariant transformer architecture that seamlessly integrates frame averaging (FA) within each transformer block. This integration allows our model to infuse geometric information into node features while preserving the spatial semantics of coordinates, leading to greater expressive power than standard FA models. Our results show that FAFormer outperforms existing equivariant models in contact map prediction across three protein complex datasets, with over 10% relative improvement. Moreover, we curate five world protein-aptamer interaction datasets and show that the contact map predicted by FAFormer serves as a strong binding indicator for aptamer screening.
Introduction
Nucleic acids have recently shown significant potential in drug discovery, shown by the success of mRNA vaccines [26, 61, 60) and aptamers [15, 32, 41]. Aptamers are single-stranded nucleic acids wide range of molecules, including previously undruggable targets [20, 11]. Currently, aptamer discovery is driven by high-throughput screening, which is mone-oonumingg and labor-intensive. While machine learning can potentially accelerate this process, the limited availability of labeled data presents a significant challenge in ML-guided aptamer discovery [57, 44, 16). Given this challenge, our goal is to build unsupervised protein-nucleic acid interaction predictor for large-scale aptamer screening.
capable of binding
Motivated by previous work unsupervised protein-proteit interaction prediction [34], we focus predicting the contact map between proteins and nucleic acids at the residuc/muclcotiiee level. The main idea is that predicted contact map offers insights into the likelihood of protein forming complex with an aptamer, thereby encoding the binding affinity between them. Concretely, as shown in Figure I(a), model is trained identify specific contact pairs between residues and nucleotides when forming a complex. The maximum contact probability across all pairs is then interpreted as the binding affinity, which subsequently used for aptamer screening.
"Correpondence to tinglin. huang@yale.....m edu hhttps://githin coclb..com//ee fom/Graph--a and Geometric- wwwpeeeeeeaannnnraaaeeeeeeeee Caaneniinn-aaasseeoe Transformes
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) Pipeline Contact Map Prediction and Aptamer Screening
(b) Architecture Comparison
Protein
Representation
Contact Map Prediction
Aptamer Screening
Foore Fereeiion
Sran four
RNNC
RNA
Representation
RNKZ 0.60
Transfomar
RNA
FAFormer
Averagig
Reprissenaaton
Represertation
+ Transformer
FAF orneer
Figure 1: (a) The pipeline of contact map prediction between protein and nucleic acid, and applying the predicted results for screening in an unsupervised manner. The affinity score quantified as the maximum contact probability over all pairs. (b) Comparison between Transformer with vanilla frame averaging framework and FAFormer, where the blue cells indicate FA-related modules.
One key factor contributing to the accuracy of contact map prediction models is their capacity to learn equivariant transformations for symmetry groups [67, 39, 66, 74, 52]. A novel line of research focuses on adapting Transformer [83] equivariant frameworks, leveraging its great expressive power. However, these studies have encountered issues with either (1) high computational overhead with spherical harmonics-ben based models [47, 27], which complicate encoding by introducing irreducible representations; or (2) limited expressive capability with frame averaging (FA) [63], which diminishes geo-information exploitation by simply concatenating coordinates with node representations.
In light of this, we propose FAFormer, an equivariant Transformer architecture that integrates FA as geometric module within each layer. FA geometric component offers flexibility to effectively integrate geometrie information into node representations while preserving the spatial semantics of coordinates, eliminating the need for complex geometric feature extraction. FAFormer consists of a Local Frame Edge Module that embeds local pairwise interactions between each node and its neighbors; a Biased MLP Attention Module that integrates relational bias from edge representation within MLP attention and equivariantly updates coordinates; and Global Frame FFN that integrates geometric features into node representations within the global context.
To validate the advantage of our model architecture, we evaluate FAFormer on two tasks: (1) protein- nucleic acid contact prediction and (2) unsupervised aptamer virtual screening. In the first task, our model consistently surpasses state-of-the-art equivariant models with over 10% relative improvement across three protein complex datasets. For the second task, we collected five real-world protein- aptamer interaction datasets with experimental binding labels. Our results show that FAFormer, trained on the contact map prediction task, is an effective binding indicator for aptamer screening. Compared to TTFFoldAA large pretrained model for complex structure prediction, FAFormer achieves comparable performance on contact map prediction and better results on aptamer screening. while offering 20-30x speedup.
2 Related Work
Aptamer screening Aptamers are single-stranded RNA DNA oligonucleotides that bind various molecules with high affinity and specificity [14, 15, 32, 41]. SELEX (Systematic Evolution of Ligands by EXponential Enrichment) is conventional technique used for tigh-throughput screening of aptamers [23, 72, 23J, which iteratively selects and amplifies target-bound sequences. Despite its effectiveness, SELEX needs take substantial time to identify small number of aptamers [73, 51].
There are some recent studies applying machine learning techniques to aptamer research [18], including generating aptamer structures [36], optimizing SELEX protocols [6], and predicting and modeling protein-aptamer interactions [24, 46, 68, 57]. However, these methods require the user to provide labeled data from time-consuming SELEX assays, thus not apply to new targets without any SELEX data. Our work focuses predicting the contact map between protein and nucleic acids using 3D structures and conducting unsupervised screening based on the predicted contact maps, which has not yet been thoroughly explored.
2

---

Protein complex modeling The prediction and understanding of interactions between proteins and molecules play crucial role in biomedicine. For example, some prior studies focus on developing geometric learning method to predict the conformation of small molecule when it binds to protein target [71, 19, 53, 50]. for the protein- protiin complex, [25, 29] explore the application of machine learning in predicting the structure of protein multimer. Some studies [77, 56] investigate the protein protein interface prediction in the physical space. Jin et al. [38] studies the protein- protein affinity prediction in an unsupervised manner. For protein-nucleic acid, some prior works explore the identification of the nucleic-acid-
residues on protein [65, 86, 89, 37, 85] or predict the binding probability of RNA proteins [79, 82, 87, 54]. Some previous studies focus on modeling protein- nucleic acid complex by computational method [81, 80, 28]. AlphaFold3 [1] and ToseTTAFoldN [5] are the recent progresses in this field, which both are pretrained language models for complex structure prediction.
Geometric deep learning Recently, geometric deep learning achieved great success in chemistry, biology, and physics domains [13, 90, 40, 55, 52, 64, 10, 70]. The previous methods roughly fall into four categories: 1) Invariant methods extract invariant geometric features from the molecules, such as pairwise distance and torsion angles, to exhibit invariant transformations [67, 31, 30]; 2) Spherical harmonics-based models leverage the functions derived from spherical harmonics and irreducible representations to transform data equivariantly [27, 47, 75]; 3) Some methods encode the coordinates and node features in separate branches, interacting these features through the norm of coordinates [66, 39]; 4) Frame averaging (FA) [63, 21] framework proposes to model the coordinates in eight different frames extracted by PCA, achieving equivariance by averaging the encoded representations.
The proposed FAFormer combines the strengths of FA and the third category of methods by encoding and interacting coordinates with node features using FA-bau PA-based components. Besides, FAFormer can also be viewed combination of GNN and Transformer architectures, the edge representation calculation functions similarly to message passing in GNN. This integration made possible by the flexibility provided by FA as integrated component.
3 Frame Averaging Transformer
In this section, we present our proposed FAFormer, a frame averaging (FA)-based transformer architecture. We first introduce the FA framework in Section 3.1 and elaborate the proposed FAFormer in Section 3.2. Discussion on the cquivariance is provided in Section 3.3, and the computational complexity analysis can be found in Appendix B.
3.1 Background: Frame Averaging
Frame averaging (FA) [63] is an aggosti- framework that can make a given encoder equiv- ariant to the Euclidean symmetry group. FA applies the principle components derived via Principal Component Analysis (PCA) to construct the frame capable of achieving F(3) equivariance. Specifi- cally, the frame function F(-) maps a given set of coordinates X to eight transformations:
F(X)= = {UU CU) = [aq u1, 02222, [gug), ar E (-1,1))
where W1, U2, Uy are the three principal components of X, U E 3R3x3 denotes the rotation matrix based on the principal components, and E R is the centroid of X. The main idea of FA is encode the coordinates as projected by the transformations, followed by averaging these representations. We introduce fx(-) represent the projections of given coordinates via F(-):
fx(X)= = (xX c)U (U, F(X))
(2)
(x())
where (s) denotes the coordinates transformed by g-th transformations. We can apply any encoder (.)) to the projected coordinates and achieve equivariance by averaging, which can be formulated as an inverse mapping f-):(-):
Fz-1 (10(x(0))}>)
| Σ 5
) +c
(0) U
(3)
3

---

FA based fundiion
Coordinate: X &.
2
X
Z
[X.
Engo
(Buleen
X
Edge reprosentation:
Zo
1 R 3-x00
Libear
on
Modkse
Linear
Resikue Connection
+ (inness)
Litser
AEG
Foame Erge
(c) Local Edge
Global frame nodes
X
AX
Tharel ITM
Liooarz
Lineaz
Linser
2 2
A2
AX
AZ
(a) FAFormer Architecture
(b) Blased MILP Attn Module
(4) Global Frame
Gate Function
Local/Global Frame
Figure 2: Overview of FAFormer architecture. The input consists of the node features, coordinates. and edge representations, which are processed by a stack of (b) Biased MLP Attention Module, (c) Local Frame Edge Module, (d) Global Frame FFN, and (e) Gate Function. Σ deontes aggregation, is multiplication, + is addition, and || indicates concatenation. Purple cells indicate the operation related to FA. (f) illustrates the difference between the local and global frames, where the local frame captures local interactions among the immediate neighbors for each node, while the global frame captures long-range correlations among all nodes.
where U-1 is the inverse matrix of g-th transformations and the result exhibit E(3) equivariance. The outcome is invariant when simply averaging the representations without inverse matrix.
3.2 Model Architecture
Instead of serving FA external encoder wrapper, propose instantiating FA as an integral geometric component within Transformer. This integration preserves equivariance and enables the model to encode coordinates effectively in the latent space, ensuring compatibility with Transformer architecture. The overall architecture is illustrated in Figure 2.
Graph Construction Each molecule can be naturally represented graph [47, 35, 43] where the residues/nucleic acids are the nodes and the interactions between them represent the edges. To efficiently model the macro-mo mooeeullsss (i.e., protein and nucleic acid), restrict the attention for each node i to its K-nearest neighbors Nup- K (i), within predetermined distance cutoff c:
Nii Lilde and j Niw (i))
where we use (()) to denote the valid neighbors of node i, and d.j denotes the distance between node and j. The long-range context information can be captured by iterative attention within the local neighbors for each node i.
Overall Architecture As shown in Figure 2(a), the input of FAFormer comprises the node features Z ND coordinates X e RNx3, and edge representations E e xxNxXD derived by our proposed edge module, where N is the number nodes and D the hidden size. FAFormer processes and updates the input features at each layer:
Z (1+1) x (1+1) E(++) (()) (Z X E())
(5)
where (.) represents /-th layer FAForrer. each model layer, we first update coordinates and node features using the Biased MLP Attention Module. Then, these features are fed into Local Frame Edge Module to refine the edge representations. Finally, the node representations undergo further updates through Global Frame FFN.

---

FA Linear Module Based on FA, we generalize the vanilla linear module to encode coordinates in the latent space invariantly:
L = [(()] Norm(X (x (z) W
(6)
where (x fo is obtained using (2), W g E R3xD is a learnable matrix for g-th transformations,
and Norm X) :- X/ XI is the normalization which scales the coordinates such that their
squurre norm is one [39]. Linear,(+) (-) is an invariant transformation and will serve as building block within each layer of the model.
Local Frame Edge Module We explicitly embed the interactions between each node and its neighbors the edge representation E Nx where K is the number of the neighbors. encodes the relational information and represents the bond interaction between nodes, which is critical in understanding the conformation of molecules [40, 4. 35].
As shown in Figure 2(f), unlike the vanilla FA which globally encodes the geometric context of the entire molecule, the edge module builds frame locally around each node's 's neighbors. Specifically, given a node i and its neighbor E Nii,, the geometric context is encoded within the local neighborhood:
(X (Xi+)) Lineary ((xra
x x)v()))
where (X; (i)N()) denotes the direction vectors from center node to its neighbors, and X T is the encoded representation. With the local frame, the spatial information sent from one source node depends on the target node, which is compatible with the attention mechanism. Then the node features engaged with geometric features, and the edge representation finalized through the residual connection with the gate mechanism:
myj Linear( Z 1 X and E 9 mij + Eij
(8)
where (-11-) is the concatenation operation, and the calculated gate 9ij (Lincar((my)) provides flexibility in regulating the impact updated edge representation.
The encoded edge representation in FAFormer plays crucial role in modeling the pairwise rela- tionships between nodes, especially for nucleic acid due to their specific base pairing rules [58, 45]. The incorporation of FA facilitates the encoding of the pairwise relationships in geometric context, resulting in an expressive representation.
Biased MIP Attention Module As shown in Figure 2(b), the attention module of FAFormer first transforms the node features Z into query, key, and value representations:
Zo ZW Q ZK ZW K. Zv ZWy
where Wo, W K. Wy DxD are the learnable projections. We adopt MLP attention [12] to derive the attention weight between node pairs, which can effectively capture any attention pattern. The relational information from the edge representation is integrated as an additional bias term:
aij Softmax; (Linear( Q 2 Kx,) + biy)
(10)
where bij Linear(LN( (E))) represents the scalar bias term based on the edge representation, dij denotes the attention score between i-th and j-th nodes, Z., is i-th representation of the matrix Z., Softmax,( (.) is the softmax function operated on the attention scores of node i's neighbors, and LN(-) is layernorm function [3].
Besides the value embeddings, the edge representation will also be aggregated to serve as the context for the update of node feature in FAFormer:
z
Σ JEN(()
Bij Zj E
Σ
Gij Eij.
(11)
JEN())
Z LN (Linear( Z + Z;
(12)
5

---

where Z is the update representation of node i. The above attention can be extended to multi-head fashion by performing multiple parallel attention functions. For the coordinates, we employ an equivariant aggregation function that supports multi-head attention:
x f--1 [[A (0) X A
H X W)F)
(13)
where 1=
) H is the number of attention heads, [-] is the tensor stack operation and W R. a linear transformation for aggregating coordinates in different heads. An additional gate function that uses node representations as input to modulate the aggregation is applied:
attt X + (1 (1- 9amm) X
(14)
where Jatn = o 2)) is the vector-wise gate designed to modulate the integration between the aggregated and the original coordinates. This introduced gate mechanism further encourages the communication between node features and geometric features.
Global Frame FFN To further exploit the interaction between node features and coordinates, we extend the conventional FFN to Global Frame FFN which integrates spatial locations with node features through FA, which illustrated in Figure 2(d):
X 9 Linear=(X) and Z' FFN + Z
(15)
where (-) denotes two-layer fully connected feed-forware network. This integration of spatial information X into the feature vectors enables the self-attention mechanism to operate geometric- aware manner. Unlike the edge module that focuses on each node's local neighbors, global frame FFN encodes the coordinates all nodes, thereby capturing the long-range correlation among nodes.
3.3 Equivariance
The function Lineary(()) exhibits invariance since results simply averaged across different trans- formations. light of this, the node representation generated by the edge module and FFN are also invariant. The biased attention based on the scalar features so the output is always invariant.
The update of coordinates within FAFormer leverages multi-head attention aggregation with gate function. Both functions while gate function is linear and also exhibits E(3) eqqiivarrri,, with a formal proof in Appendix C.
equiivaara: attention aggregation is based frame averaging,
In conclusion, FAFormer is symmetry-awand which exhibits invariance for node representations and eqquivaiaae for coordinates.
Experiments
In this section, present three protein complex datasets and five aptamer datasets to explore protein complex interactions and evaluate the effectiveness of FAFormer. More details regarding the experiments and datasets can be found in Appendix and Additional experiments, including the ablation studies, and the comparison with AlphaFold3, can be found in Appendix E. All the datasets used in this study are included our anonymous repository.
4.1 Dataset
Protein Complexes We cleaned up and constructed three 3D structure datasets of protein complexes from multiple sources [8, 7, 2, 77]. residue- nucieooide pair determined to be contact if any of their atoms are within 6Å from each other [77, 78). We conduct dataset splitting based the protein sequence identity, using threshold of 50% for protein-RNY RNA/DNA complexes' and 30% for protein- ppotein complexes. The details of all datasets are shown Table 1.
The protein's 's and nucleic acid's structures in the validation/test sets are generated by ESMFold [48] (proteins)
(nucleic acids). This offers a more realistic scenario, given that the crystal structures are often unavailable.
Note that don't 't use 30% the threshold since it results in very limited validation and test

---

Table 1: Protein complex dataset statistics.
Table 2: Aptamer dataset statistics.
#Train
#Val
#Test
Label
Target
GFP
NELF HNRNPC
CHK2
UBLCPI
Protein-RNA
1,009
115
118
1.517%
#Positive
520
797
233
1,255
892
Protein-DNA
2,590
134
134
1.215%
#Candidate 1,875
9,833
3,328
10,000
10,000
Protein-Protein
4,402
544
545
0.469%
Aptamers Our aptamer datasets come from the previous studies [76, 46, 73), including five protein targets and their corresponding aptamer candidates. The affinity of each candidate to the target is experimentally determined. Each dataset is equally split into validation and test sets.
We construct the protein- -Necinn-NNAAAA training set by excluding complexes from our collected dataset with over 30% protein sequence identity to these protein targets, resulting in 1,238 training cases. The 3D structures of proteins are obtained from AlphaFold Database [5], and the structures of RNAs are generated by RosdNAAAFoldNA without using MSAs. The statistics are presented Table 2.
Feature The coordinates of the Ca atoms from residue and the C3 atoms from nucleotide are used as coordinate features. For node feature generation, we employ ESM2 [49] for proteins and RNA-FM [17] for RNA. The one-hot embedding is utilized as DNA's node feature.
4.2 Contact Map Prediction
As shown in Figure I(a), this task aims predict the exact contact pairs between protein {S,) n and nucleic acid { which conducts binary classification over all pairs:
(,, {
1, S contacts with S Other
(16)
0,
Baselines We compare FAFormer with four classes of methods: 1) Vanilla Transformer [83] which doesn't utilize 3D structure; 2) Spherical harmonics- bbssed models Equiformer [47] and SE(3YTransformer [27]; 3) GNN-based models EGNN [66] and GVP-GNN [39]; 4) Transformer with FA [63]. The protein and nucleic acid will be separately encoded with two encoders to avoid label leakage. The representations of residues and nucleotides are concatenated from all pairs and fed into a MLP classifier conduct prediction.
Results To comprehensively evaluate the performance label-imbalanced datasets, we apply F1 and PRAUC as the evaluation metrics. The comparison results are presented in Table 3 from which FAFormer reaches the best performance over all the baselines with relative improvement over 10%.
Additionally, some geometric methods fail outperform the vanilla Transformer in certain cases. We attribute this to overfitting on crystal structures during training, which hinders their ability to generalize well to unbound structures during evaluation. Compared with serving FA external equivariant framework on Transformer, the performance gain on FAFormer verifies the effectiveness of embedding FA as geometric component within Transformer.
Table 3: Comparison results on three datasets of contact map prediction task.
Metric
Transformer
Tran/ransfomeer
Equiformer
EGNN
GVP-GNN
FA
FAForme
Prolein-RNA
Fl PRAUC
0,1021 our
0.0816 oor
0.0390
0.1083
0.1091
0.1150
0.1284
0.101 one
0.0881 on
0.0985 004
0.0964 oce
0.1008 004
0.0965
0.1113 004
F1 0.0963.co
0.0824
0.0925
0.1208
0.1225
0.1283
0.1457
Protein-DNA
PRAUC
0.1111 one
0.1022 our
0.0913 o02
0.1139 010
0.1 00G
0.10 one
0.1279 one
Fl PRAUC
0.0756
0.1147
0.1039
0.1461
0.1302
0.1011
0.1396
Protein-Protein
0.0707 onz
0.0906 ons
0.0834 ope
D.1245 nc1
0.1181 002
0.0830 op
0.1463 ona
4.3 Binding Site Prediction
In addition to contact map prediction, we examine the nucleic acid binding site prediction task, which node-level task, to comprehensively evaluate our model. This task solely takes protein {S.)w
7

---

as input and aims to identify the cacllei--aci--iinniie residues on the protein:
(S.) = 1
1, O, S, contacts with nucleic acid Other
(17)
Predicting the nucleic acid binding site offers promising therapeutic potential for undruggable targets by conventional small molecule drug [33, 9, 88). expanding the range of potential therapeutic targets.
Baselines We compare FAFormer with two state-of-the-art geometric deep learning models: Graph- Bind [86] and GraphSite [89] in this task. The protein structure is embedded with the geometric encoder and the residue's s representations are fed into classifier for prediction.
Results Fl and PRAUC are applied as the evaluation metrics and we report the average results over three different seeds in Table 4. We can observe that FAFormer achieves the best performance over all the baselines, demonstrating the effectiveness of FAFormer on nucleic acid-related tasks and modeling the geometric 3D structure.
Table 4: Comparison results on binding site prediction.
Metric
GraphBind
GraphSite
FAFormer
F1
0.492.
0.416 o07
0.506 oos
Protein-DNA
PRAUC
0.520.oos
0.541.co no1
0.549. 004
F1
0.449.om 0044
0.400 007
0.472, oos
Protein-RNA
PRAUC
0.471.005
0.479.00
0.507.um
4.4 Unsupervised Aptamer Screening
This task aims to screen the positive aptamers from large number of candidates for a given protein target. We quantify the binding affinities between RNA and the protein target as the highest contact probability among the probability of contact are very likely to form complex [34]. The models are first trained on the protein-RNA complexes training set using the contact map prediction, then the aptamer candidates ranked based the calculated highest contact probabilities.
pairs. The main idea is that two molecules with a high
Results Top-10 precision, Top-50 precision, and PRAUC are used as the metrics. As shown in Table 5, the geometric encoders outperform sequence-basee Transformer in most cases, and FAFormer generally reaches the best performance. This demonstrates the great potential of an accurate interaction predictor in determining unsupervisedly promising aptamers.
Table 5: Comparison results of zero-shot aptamer screening.
Metric
Transformer
SBC3)Transfomer
Equiformer
EGNN
GVP-GNN
FA
FAFormer
Top10 Prec.
0.2333
0. 0.2000..a
0.3000.co
0.2666
0.3000
0.3000
0.4000.
GFP
Top50 Proc.
0.2733 ets
0. 0,2606.co cer
0.3606 oaa
0.3400 asi
0.3799 as
0. 0.3333 om
0.1133 o41
PRAUC
0.2881 o1s
0. 0.2883.cm
0.3106
0.3076
0.3
0.2895
0. 0.3224
Top10 Prec.
8.0000 coa
1a100.ca
0. 0.2333, 124
2,2666 121
0,2333 124
0.0066 oer
0.3333 seo
HNRNPC
Top50 Prec.
0.0266 ous
0.1533
0.1666
0,2266 a1z
0.2266
0.1533
0.2399
PRAUC
0. 0.0641 cos
0.1191
0. 1225.0000
0.1434
0.0913,gm gms
0.1628 ose
Top10 Prec.
0.1606
0,2000 Las
0.1666
0.2000,
0.0666
0,1686
2.333
NELF
Top50 Prec.
0.1866 cam
0.1509 ein
0.2000 egg
0.1333 Xx7
0.1133 oo:
0.1533 pm
U. 0.2999 ea2
PRAUC
0.0972.cos cos
0.0931.cm
0.1065 ons
0,0969 ava
0.0850.am
0.0952 mm
0. 0.0963.
Top10 Prec.
0.1000 cen
0.1666 ex17
0.2000 and
0,1333, 124
0. 0,26666 124
0.1606 per
0.1000 os1
CHK2
Top50 Prec.
0. 1.1066.com car
0. 0.0933.cm ooe
0.1533 autt
0.1199
0.1466
0.1333.nn
0.1733 ost
PRAUC
0 0.1273
0. 1.1253.com
0.1271 on13
0.1268 a022
0.1249..ma
0.1251
U. 0.1297 oos
Top10 Prec.
.o0
0. 0,009 oas
0.0606
0.1266
0.0790
0.1000 ano
0.1800 oo9
UBL.CP
Top50 Proc.
0 e.1
0. 10000000m cas
0.1266 023
0.11499 ocr
0.1116.ax0
0. 0.1133
0.1500 ose
PRAUC
0.1004 cec
0.0955 cos
0.1022 oor
0.0977. aco
0.0968 arz
0.0977 ono
0.1070 cm

---

4.5 Comparison with RoseTTAFOIdNA
In this section, we investigate the performance of RoseTTAFoIdNA [5] which is pretrained pro- tein complex structure prediction model and compare it with FAFormer. The performance of FAFormer evaluated on the individual predicted protein and nucleic acid structures by ESMFold and Rose'T ITAFoo.NA. We additionally test the performance of AlphaFold3 [1] on a subset of the screening tasks due AlphaFold3 server submission limits (Appendix E).
Dataset For the contact map prediction task, we select the test cases used in RoseTTAFoldNA to create the test set, yielding 86 protein-DNA and 16 protein-RNA cases, Furthermore, the complexes from our collected dataset that have more than 30% protein sequence identity to these test examples are removed. This leads 1.962 training cases for protein- -NN and 1,094 for protein- -NNA, which are used for training FAFormer. The MSAs of proteins and RNAs are retrieved for Rose ITAFoldAA
For the aptamer screening task, construct smaller candidate for each protein target by randomly sampling 10% candidates, given that the inference of ITAlldAA with MSA searching time-consuming. The datasets will be equally split into validation and test sets. More details of these datasets can be found Appendix D.
RoseTTAFolsENA
RoseTTAFoldZNA
PRAUC
F1
0.14
Protain-DMA Complexes
Complexes
Figure Contact map prediction on RoscTTAFoldNA set.
Table Comparison results with TTAFolNA using the sampled datasets, which accounts for the performance differences of FAFormer shown in Table 5.
Metric
GFP
HNRNPC
NELF
CHKK
UBLCPI
Top10 Pree.
0.4000
0.1000
TTAFolAA
Top50 Prec.
0.3600
0.0599
0.1000
PRAUC
0.3926
0.1452
0.0481
0.1176
0.0722
Top Prec.
0.4000.p
0. 1666.124
0.1666.
0.1333. 124
0. 1000 694
FAFonner
Top50 Prec.
0.3800 o1.
0.0800. 024
0.0866.or 018
0.1266 eae
0.0866 onn
PRAUC
0.4027 022
0.1781 .so
0.1044.o18
0.1374 013
0.0762 016
Residue
Residue
10
20
30
40
50
60
80
10
20
30
40
50
Nucleotide
7DVV
(a) Protein-DNA Contact Map Comparison
Residue
Residue
20
60
80
100
20
60
80
100
Nucleotide
13
7KX9
(b) Protein-RNA Contact Map Comparison
Figure 4: Case study based
complex examples (PDB id: 7DVV and 7KX9), where each heatmap entry represents the contact probability between the nucleotide and residue. In each row, the figure the left displays the ground truth contact maps, while the figure on the right displays the results predicted by FAFormer.
The comparison results of contact map prediction are presented in Figure 3, where FAFormer can achieve comparable performance to RosoTTAFO using unbounded structures.
Results

---

Specifically, our method has higher Fl scores for Pootein-DNA complexes (0.103 vs. 0.087) and performs comparably for protein-RNA complexes (0.108 vs. 0.12). Besides, Table shows that RoseTTAAFlldNA fails to receive positive aptamers for some targets, e.g., NELF and UBLCPI, while FAFormer consistently outperforms RoseTTAFoldNA for all the targets.
Aligning Figure with Table 6, we observe that while FAFormer does not surpass RoseTTAFOLANA in contact map prediction, it significantly excels in aptamer screening. We attribute this to two reasons: 1) Similar to AlphaFold3 (Table 10), RoseTTAFFldh
as a foundational structure prediction model is optimized for general complex structures, which might bias its performance on some specific protein targets. For example, it can achieve good performance on proteins GFP and HNRNPC but fails for NELF. 2) The protein-RNA test set used by RoseTTAFOIdNA for contact map prediction is limited, which may not comprehensively evaluate FAFormer.
Case Study Two examples of protein-DNA (PDB id: 7DVV) and protein-RNA (PDB id: 7KX9) complexes are provided in Figure 4, which shows a visual comparison between the actual (left) and the predicted (right) contact maps. Note that only the residues involved in the actual contact map are presented for a clear demonstration' The complete contact map can be found in Appendix E. We can find that despite the sparsity of contact pairs, the predicted contact maps show high degree of accuracy when compared to the ground truth.
Time Comparison Table shows the aver- age and total inference time of FAFormer and RoseTTAFoldNA on the test cases for the con- tact map prediction task, including the time for predicting the unbound structures for FAFormer. The predicted structures used for the evaluation of FAFormer are generated without protein and
Protein-DNA
Protein-RNA
Avg.
Total
Avg.
Total
RoseTTAFoldNA
0.175h
15.12h
0.440h
7.04h
FAFormer
32.65s
0.78h
51.75s
0.23h
Table 7: Inference time on contact map prediction, denoted in seconds ("s") and hours ("h").
RNA MSAs, demonstrating significantly faster inference speed by orders of magnitude. For the unsupervised aptamer screening task, while Rose TTAFoldAA requires only single MSA search for each protein target, it needs to search MSAs for each RNA candidate sequence separately. Besides, the inclusion of MSAs results in large input sequence matrix, leading to a time-consuming folding process of Rose TTAFoldAA.
5 Conclusion
This research focuses on predicting contact maps for protein complexes in the physical space and reformulates the task of large-scale unsupervised aptamer screening as contact map prediction task. To this end, we propose FAFormer, a frame averaging-bar based Transformer, with the main idea of incorporating frame averaging within each layer of Transformer. Our empirical results demonstrate the superior performance of FAFomeer on contact map prediction and unsupervised aptamer screening tasks, which outperforms eight baseline methods on all the tasks.
Broader Impacts The proposed paradigm for aptamer screening can be extended other modali- ties, such as protein-small molecules and antibody-antigen. Moreover, the strong correlation between contact prediction and affinity estimation demonstrated in our paper can guide future model develop- ment. Besides, FAFormer introduces a novel approach to equivariant model design by leveraging the flexibility of FA. This idea opens up numerous possibilities for future research, including exploring different ways to integrate FA with various neural network architectures.
Limitation In this study, the geometric features utilized in the encoders are limited the coordinates of the C. and C3 atoms. Features extracted from the backbone or sidechains have not been used, which may limit the performance of the geometric encoders.
We sort the residue IDs alongside the ID that the contact map appears diagnostic.
10

---

6 Acknowledgements
We thank Yangtian Zhang, Junchi Yu, Weikang Qiu, and anonymous reviewers for their valuable feedback on the manuscript. This work was supported by the BroadIgnite Award, the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard, NSF IIS Div Of Information Intelligent Systems 2403317, and Amazon research.
References
[1] Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction biomolecular interactions with alphafold 3. Nature, pages 1-3, 2024.
[2] Bartosz Adamezyk, Maciej Antezak, and Marta Szachniuk. Rnasolo: a repository of cleaned pdb-dlerieed ma 3d structures. Bioinformatics, 38( (8(14))3688-367) 2022.
[3] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey arXiv:667700445 2016.
Hinton. Layer normalization.
arXiv preprint
[4] Minkyung Back, Ivan Anishchenko, lan Humphreys, Qian Cong, David Baker, and Frank DiMaio. Efficient and accurate prediction of protein structure using rosettafold2. bioRviv, pages 2023-022 05, 2023.
[5] Minkyung Back, Ryan McHlugh, Ivan Anishchenko, David Baker, and Frank DiMaio. Accurate prediction of nucleic acid and protein-mucleic acid complexes using rosettafoldna. bioRxiv, pages 2022-09, 2022.
[6] Ali Bashir. Qin Yang. Jinpeng Wang. Stephan Hoyer, Wenchuan Chou, Cory McLean, Geoff Davis, Qiang Gong, 7an Armstrong, Junghoon Jang, et al. Machine learning guided aptamer refinement and discovery. Nature Communications, 12(1) 12(1),,366, 2021.
[7] Helen M Berman, Catherine Lawson, and Bohdan Schneider. Developing community resources for nucleic acid structures. Life, 12(4):540, 2022.
[8] Helen M Berman, John Westbrook, Zakang Feng, Gary Gilliland, Talapady N Bhat, Helge Weissig, Ilya N Shindyalov, and Philip Bourne. The protein data bank. Nucleic acids research. 28(1):235 2222, 2000.
Nicholas Boyd, Brandon M Anderson, Brent Townshend, Ryan Chow, Connor Stephens, Ramya Rangan, Matias Kaplan, Meredith Corley, Akshay Tambe, Yuzu Ido, et al. Atom-1: A foundation model for ma structure and function built on chemical mapping data. bioRxiv, pages 2023-12, 2023.
[10] Johann Brehmer, Pim De Haan, Sönke Behrends, and Taco S Cohen. Geometric algebra transformer. Advances in Neural Information Processing Systems, 36, 2024.
[11] Edward N Brody and Larry Gold, Aptamers as therapeutic and diagnostic agents. Reviews in Molecular Biotechnology, 74(1):5-13, 2000.
[12] Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? arXiv preprint arXiv:2105. 14491, 2021.
[13] Michael M Bronstein, Joan Bruna. Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478, 2021.
[14] Andrey A Buglak, Alexey V Samokhvalov, Anatoly V Zherdev, and Boris B Dzantiev. Methods and applications of in silico aptamer design and modeling. International Journal Molecular Sciences, 21(22)88400, 2020.
[15] Jonghoe Byun. Recent progress and opportunities for nucleic acid aptamers. Life. 11(3):193, 2021.
[16] Chetan Chandola and Muniasamy Neerathilingam. Aptamers for targeted delivery: current challenges and future opportunities. Role of novel drug delivery vehicles in nanobiomedicine, pages 1-22, 2019.
[17] Jiayang Chen, Zhihang Hu, Siqi Sun, Qingxiong Tan, Yixuan Wang, Qinze Yu, Licheng Zong, Liang Hong, Jin Xiao, Tao Shen, et al. Interpretable ma foundation model from unannotated data for highly accurate ma structure and function predictions. bioRxiv, pages 2022-08, 2022.
[18] Zihao Chen, Long Hu, Mho-Ting& Zhang, Aiping Lu, Yaofeng Wang. Yuanyuan Yu, and Ge Zhang. Artificial intelligence in aptamer- target binding prediction. International journal of molecular sciences, 22(7):3605, 2021.

---

[19] Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking. arXiv preprini arXiv:2210.01776, 2022.
[20] Maria Francisca Coutinho, Liliana Matos, Juliana Inês Santos, and Sandra Alves. Rua therapeutics: how far have gone? The mRNA Metabolism in Human Disease, pages 133 177, 2019.
[21] Alexandre Agm Duval, Victor Schmidt, Alex Hernández- a, Santiago Miret, Fragkiskos D Malliaros, Yoshua Bengio, and David Rolnick. Facnet: Frame averaging equivariant gnn for materials modeling. In International Conference on Machine Learning, pages 9013-9033, PMLR, 2023.
[22] Stefan Elfwing, Eiji Uchibe, and Kenji Doya. weighdd linear units for neural network function approximation in reinforcement learning. Neural nenworks, 107:3-11, 2018.
[23] Andrew D Ellington and Jack W Szostak. In vitro selection of ma molecules that bind specific ligands. nature, (466(6887)818-8277 1990.
[24] Neda Emami and Reza Ferdousi. Aplanet as deep learning approach for aptamer-protein interaction prediction. Scientific reports, 11(1):6074, 2021.
[25] Richard Evans, Michael O' O'Neill, Alexander Pritzel, Natasha Antropova, Andrew Senior, Tim Green, Augustin Zidek, Russ Bates, Sam Blackwell, Jason Yim, et al. Protein complex prediction with alphafold- multimer. biorxiv, pages 2021-10, 2021.
[26] Enyue Fang, Xiaohui Liu, Miao Li, Zelun Zhang, Lifang Song, Baiyu Zhu, Xiaohong Wu, Jingjing Liu, Danhua Zhao, and Yuhua Li. Advances in covid-19 mrna vaccine development. Signal transduction and targeted therapy. 7(1):94, 2022.
[27] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling, Se (3>-transformers: 3d roto-translation equivariant attention networks. Advances in neural information processing systems, 33:1970-1988 2020.
[28] Michal J Gajda, Irina Tuszynska, Marta Kaczor, Anastasia Yu Bakulina, and Janusz M Bujnicki. Filtrest3d: discrimination of structural models using restraints from experimental data. Bioinformatics, 26(23):2986- 2987, 2010.
[29] Octavian-Eugee Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi Jaakkola, and Andreas Krause. Independent se (3)-equivariant models for end-to-end rigid protein docking. arXiv preprint vrXiv:2111.07700 2021.
[30] Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34:6790-6802, 2021.
[31] Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty- aware directional message passing for non-equilibrium molecules. arXiv preprint a5X1v:2011.14115.
2020.
[32] Larry Gold, Nebojsa Janjie, Thale Jarvis, Dan Schneider, Jeftrey Walker, Sheri K Wilcox, and Dom Zichi. Aptamers and the ma world, past and present. Cold Spring Harbor perspectives in biology, 4(3):0003582, 2012.
[33] James P Hughes, Stephen Rees, S Barrett Kalindjian, and Karen Philpott. Principles of carly drug discovery. British journal of pharmacology, 162(6): 1239-12299999 2011.
[34] IR Humphreys, J Pei, M Back, A Krishnakumar, I Anishchenko, S Ovchinnikov, J /Zang, TJ Ness, Banjade, S Bagde, et al. Structures core cukaryotic protein complexes. biorxiv 2021: 2021.09. 30.462231.
[35] John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph-based protein design. Advances in neural information processing systems, 32, 2019.
[36] Natsuki Iwano, Tatsuo Adachi, Kazuteru Aoki, Yoshikazu Nakamura, and Michiaki Hamada. Generative aptamer discovery using raptgen. Nature Computational Science, 276)3378-386 2022.
[37] Zheng Jiang, Yue- Yue Shen, and Rong Liu. Structure- based prediction of nucleic acid binding residues by merging deep learning-and template-based approaches. PLOS Computational Biology, 19(9)= 1428, 2023.
[38] Wengong Jin, Siranush Sarkizova, Xun Chen. Nir Hacohen, and Caroline Uhler. Unsupervised protein- ligand binding energy prediction via neural euler's 's rotation equation. arXiv preprint arXiv:201..10114 2023.
12

---

[39] Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael JL Townshend, and Ron Dror. Learning from protein structure with geometric vector perceptrons. arXiv preprint urXiv:2009.01411, 2020.
[40] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figumov, Olaf Ronneherger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873) 2021.
[41] Anthony D Keefe, Supriya Pai, and Andrew Ellington. Aptamers therapeutics. Nature reviews Drug discovery, 977)5337-550, 2010.
[42] Diederik Kingma and Jimmy Ba. Adam: method stochastic optimization. arXiv preprint 14Xiv:1412.6900 2014.
[43] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint (et.:02090000) 2016.
[44] AV Lakhin, Vyacheslav Zalmanovich Tarantul, and LV3890987 Gening, Aptamers: problems, solutions and prospects. Acta Naturae, 5(4 (19)):34-43, 2013.
[45] Neocles B Leontis and Eric Westhof. Geometric nomenclature and classification rma base pairs. Rna, 7(4):499 512, 2001.
[46] Shuya Li, Fanghong Dong, Yuexin Wu, Sai Zhang, Chen Zhang, Xiao Liu, Tao Jiang, and Jianyang Zeng. A deep boosting based approach for capturing the sequence binding preferences of rna-binding proteins from tightthroughput clip-seq data. Nucleic acids research, 45 45(14)x=12--e129 2017.
[47] Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. arXiv preprint arXiv:2206.11 11990, 2022.
[48] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel- Fazel-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022:500902. 2022.
[49] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637): 37936637):1123-11130 2023.
[50] Meng Liu, Youzhi Luo, Kanji Uchino, Koji Maruhashi, and Shuiwang Ji. Generating 3d molecules for target protein binding. arXiv preprint arXiv:2204,0941 2022.
[51] Qingxiu Liu, Wei Zhang, Siying Chen, Zhenjing Zhuang, Yi Zhang, Lingli Jiang, and Jun Sheng Lin. Selex tool: novel and convenient gel-based diffusion method for monitoring of aptamer-target binding. Journal of biological engineering, 14:1-13, 2020.
[52] Shengchao Liu, Weitao Du, Yanjing Li, Zhuoxinran Li, Zhiling Zheng, Chenru Duan, Zhiming Ma, Omar Yaghi, Anima Anandkumar, Christian Borgs, et al. Symmetry-informed geometric representation for molecules, proteins, and crystalline materials. arXiv preprint 2306.0230. 2023.
[53] Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. A 3d generative model for structure-based drug design. Advances in Neural Information Processing Systems, 34:6229 6239, 2021.
[54] Daniel Maticzka, Sita J Lange. Fabrizio Costa, and Rolf Backofen. Graphprot: modeling binding preferences of rna-binding proteins. Genome biology, 18()):1-18, 2014.
[55] Amil Merchant, Simon Batzner. Samuel S. Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery. Nature, 2023.
[56] Alex Morehead, Chen Chen, Ada Sedova, and Jianlin Cheng. Dips-plus: The enhanced database of interacting protein structures for interface prediction. Scientific Data, 10(1):509, 2023.
[57] Flemming Morsch, Iswarya Lalitha Umasankar, Lys Sanz Moreta, Paridhi Latawa, Danny B Lange, Jesper Wengel, Huram Konjen, and Christian Code. Aptabert: Predicting aptamer binding interactions. bioRxiv, pages 2023-12 11, 2023.
[58] Wilma K Olson, Manju Bansal, Stephen K Burley, Richard E Dickerson, Mark Gerstein, Stephen C Harvey, Udo Heinemann, Xiang-Jun Lu, Stephen Neidle, Zippora Shakked, et al. A standard reference frame for the description of nucleic acid base-pair geometry. Journal of molecular biology, 3 313(1):229 237, 2001.
13

---

[59] John M Pagano, Hojoong Kwak, Colin T Waters, Rebekka O Sprouse, Brian S White, Abdullah Ozer. Kylan Szeto, David Shalloway. Harold G Craighead, and John T Lis. Defining nelf-e ma binding in hiv-1 and promoter-proximat pause regions. PLoS genetics, 10(1)=e004000, 2014.
[60] Norbert Pardi, Michael J Hogan, Frederick W Porter, and Drew Weissman. mrna vaccines- new era in vaccinology. Nature reviews Drug discovery, 17(4):261-279, 2018.
[61] Jung Woo Park, Philip NP Lagniton, Yu Liu, and Ren-He Xu. mrna vaccines for covid- 19: what, why and how. International journal of biological sciences, 17(6):1446 2021.
[62] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[63] Omri Puny, Matan Atzmon, Heli Ben-Hamu, Ishan Misra, Aditya Grover. Edward J Smith, and Yaron Lipman. Frame averaging for invariant and equivariant network design. arXiv preprint arXiv:210.03333 2021.
[64] Weikang Qiu, Huangrui Chu, Selena Wang, Haolan Zuo, Xiaoxiao Li, Yize Zhao, and Rex Ying. Learning high-order relationships of brain regions. In Forty-first International Conference on Muchine Learning. 2023.
[65] Rahmatullah Roche, Bernard Moussad, Md Hossain Shuvo, Sumit Tarafder, and Debswapna Bhattacharya. Equipnas: improved protein-nucle acid binding site prediction using Laneeii--lnuagee--ae model-informiel equivariant deep graph neural networks. bioRxiv, pages 2023-09, 2023.
[66] Victor Garcia Satorras, Emiel Hoogehoom, and Max Welling. E (n) equivariant graph neural networks. In International conference machine learning, pages 9323-9332. PMLR, 2021.
[67] Kristof T Schütt, Huzicl E Sauceda, P-J Kindermans, Alexandre Tkatchenko, and K-R Müller. Schnet-a deep lcarning architecture for molecules and materials. The Journal of Chemical Physics, 148(24), 2018.
[68] Incheol Shin, Keumseok Kang, Juseong Kim, Sanghun Sel. Jeonghoon Choi, Jae- Book Lee, Ho Young Kang, and Giltac Song. Aptatrans: a deep neural network for predicting aptamer-protein interaction using pretrained encoders. BMC bioinformaties, 24(1):447, 2023.
[69] Shui, Abdullah Ozer, Warren Zipfel. Nevedita Sahu, Avtar Singh, John Lis, Hua Shi, and Michael I Kotlikoff. Rna aptamers that functionally interact with green fluorescent protein and its derivatives. Nucleic acids research, 40(5)x33-e33 2012.
[70] Zhenqiao Song, Tinglin Huang, Lei Li, and Wengong Jin. Surfpro: Functional protein design based on continuous surface. arXiv preprint arXiv:240500660 2024.
[71] Hannes Stärk, Octavian Ganea, Lagnajit Pattanaik, Regina Barzilay, and Tommi Jaakkola. Equibind: Geometric deep learning for drug binding structure prediction. In International conference on machine learning, pages 20503-20521. PMLR, 2022.
[72] Regina Stoltenburg, Christine Reinemann, and Beate Strehlitz. Selex-a a (r) evolutionary method to generate high-affinity nucleic acid ligands. Biomolecular engineering. 24(4):381-405 2007.
[73] Kylan Szeto, David R Latulippe, Abdullah Ozer. John M Pagano, Brian S White, David Shalloway, John T Lis, and Harold G Craighead. Rapid-selex for ma aptamers. PloS one, 8(12)e82667, 2013.
[74] Philipp Thölke and Gianni De Fabritiis. Torchmd-net: equivariant transformers for neural network hased molecular potentials. arXiv preprint 0rXiv:2202.02541, 2022.
[75] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv: 802,
a 08219, 2018.
[76] Jacob M Tome, Abdullah Ozer, John M Pagano, Dan Gheba, Gary P Schroth, and John T Lis. Compre- hensive analysis of ma-prnn interactions by high-throughput sequencing ma affinity profiling. Nature methods, 1 1166)688-6688 2014.
[77] Raphael Townshend, Rishi Bedi, Patricia Suriana, and Ron Dror. End-to-end learning on 3d protein structure for interface prediction. Advances in Neural Information Processing Systems, 32, 2019.
[78] Raphael JL Townshend, Martin Vögele, Patricia Suriana, Alexander Derry. Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Bowen Jing, Brandon Anderson, Stephan Eismann, et al. Atom3d: Tasks molecules in three dimensions. arXiv preprint arX05,2012.04035, 2020.
14

---

[79] Ameni Trabelsi, Mohamed Chaabane, and Asa Ben-Hur. Comprehensive evaluation of deep learning architectures for prediction of dna/ma sequence binding specificities. Bioinformatics, 35(14):129-1177, 2019.
[80] Irina Tuszynska. Marcin Magnus, Katarzyna Jonak, Wayne Dawson, and Janusz M Bujnicki. Npdock: web server for protein protein-nucleic acid docking. Nucleic acids research, 13(W1)::422) W430, 2015.
[81] Irina Tuszynska, Dorota Matelska, Marcin Magnus, Grzegorz Chojnowski, Joanna Kasprzak, Lukasz P Kozlowski, Stanislaw Dunin Horka-Ho Janusz M Bujnicki. Computational modeling of protein ma complex structures. Methods, 65(3):310 (1333910-319, 2014.
[82] Michael Uhl, Van Dinh Tran, Florian Heyl, Rolf Backofen. Rnaprot: an efficient and feature-rich ma binding protein binding site predictor. GigaScience, 10(8):gia0054, 2021.
[83] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems,
30, 2017.
[84] Pauli Virtanen, Ralf Gommers, Travis Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. Scipy 1.0: fundamental algorithms for scientific computing python. Nature methods, 17(3):261 272, 2020.
[85] Junkang Wei. Siyuan Chen, Licheng Zong Xin Gao, and Yu Li. Protein ma interaction prediction with deep lcarning: structure matters. Briefings in bioinformatics, 23(1)xbb, 2022.
[86] Ying Xia. Chun Qiu Xia, Xiaoyong Pan. and Hong-Bip Bin Shen. Graphbind: protein structural context embedded rules learned by hierarchical graph neural networks for recognizing aeleic-acid-bindiie residues. Nucleic acids research, 49(9):e51- e51, 2021.
[87] Yiran Xu, Jianghui Zhu, Wenze Huang, Kui Xu, Rui Yang, Qiangfeng Cliff Zhang, and Lei Sun. Prismnet: predicting protein- ma interaction using vivo ma structural information, Nucleic Acids Research, page gkad353, 2023.
[88] Zichao Yan. William Hamilton, and Mathieu Blanchette. Neural representation and generation for ma sccondary structures. arXiv preprint arXiv:2102.00023. 2021.
889 Qianmu Yuan, Sheng Chen, Jiahua Rao, Shuangjia Zheng, Huiying Zhao, and Yuedong Yang. Alphafold2- aware protein- dna binding site prediction using graph transformer. Briefings in Bioinformatics, 23(2):bbab564, 2022.
[90] Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, et al. Artificial intelligence for science in quantum, atomistic, and continuum systems, arXiv preprint arXiv:2007.0842, 2023.
15

---

A Experimental Details
Running environment. The experiments are conducted on a single Linux server with The AMD EPYC 7513-32 Core Processor, 1024G RAM, and 4 Tesla A40-48GB. Our method implemented on PyTorch 1.13.1 and Python 3.9.6.
Training details. For all the baseline models and FAFormer, we fix the batch size as 8, the number of layers as 3, the dimension of node representation as 64, and the optimizer as Adam [42]. Binary cross-entropy loss used for contact map identification tasks with a positive weight of 4. The gradient norm is clipped to 1.0 in each training step to ensure learning stability. We report the model's performance on the test set using the pest-performing model selected based on performance on the validation set. All the results are reported based on three different random seeds.
The learning rate is tuned within { (le-3, 5e-4, le-4) and is yields the best performance. For each model, we search the hyperparameters in the following ranges: dropout in [0, 0.5), the number of nearest neighbors for the GNN-based methods in (10, 20, 30), and the number of attention heads in {1. 2, 4, 8/. The hyperparameters used in each method are shown below:
le-3 by default, as it generally
FAFormer: The number of attention heads, dropout rate, and attention dropout rate are 4, 0.2, and 0.2 respectively. We initialize the weight of the gate function with zero weights, and bias with a constant value of ensuring mostly-opened gate. SiLU [22] used as the activation function. The distance threshold e is set as le5A and the number of neighbors is 30.
GVP-GNNº:
The dimensions of node/edge scalar features and node/edge vector features are set as 64 and 16 respectively. The dropout rate is fixed at 0.2. For fair comparison, we only extract the geometric feature based on C.. i.e., the forward and reverse unit vectors oriented in the direction Ca between neighbor residues.
The number of neighbors is set 30. Besides, apply gate attention to each edge update module and residue connection to the node update module. SiLU [22] is used as the activation function.
EGNN
Equiformer" and Tran)rrassformer:: The number of attention heads, and the hidden size of each attention head are 100Á and set the number of neighbors 30. Based spherical harmonics to 1, as higher degrees tend to lead to performance collapse according to our experiments.
and 16. We exclude the neighbor nodes with distance greater than
experiments, set the degree of
Transformer and F The Transformer is applied FA's backbone encoder. The dropout and attention dropout rates are 0.2 and 0.2. The number of attention heads is set 4.
GraphBind 10; The dropout ratio and the number of neighbors are 0.5 and 30. We apply addition aggregation to the node and edge update module, following the suggested setting presented in the paper.
The number of neighbors and dropout ratio are 30 and 0.2. The number of attention layers and attention heads are and 4 respectively. Besides, we additionally use the DSSP features as the node features, as suggested in the paper.
GraphSite":;
We employ the released pretrained weight of RoseTTAFoldNA and set up all the required databases, including UniRef, BFD, structure templates, Rfam, and RNAcentral following the instructions.
RoseTTAFolddA
"attpp///ggtabb..oo////aatrrrrrr 7
http:///www..ooooo..........o/aa html
16

---

B Efficiency Analysis
B.1 Computational Complexity
As a core component in the model, frame averaging's time complexity mainly comes from the calculation of PCA among the input coordinates. This operation is practically efficient due to the low dimensionality of the input (only 3 for coordinates), Besides, the calculation of eigenvalue decomposition can be significantly accelerated by some libraries, such PyTorch [62] and SciPy [84]. We ignore the complexity used for calculating projected coordinates in the following analysis.
As for the local frame edge module in FAFormer, linear transformations the pairwise representation (Equ. (7) and Equ. (8)), and an additional gate is applied to regulate these messages (Equ. (8)), resulting in computational complexity of O K D + N +D). Considering the residue connection, the overall complexity of the local frame edge module is O(NKD+NKD) N D).
employed to compute
As for the self-attention module, linear transformations are performed on token embeddings and edge representations (Equ. (9)), and the multi-head MLP attention computation is limited to nearest neighbors (Equ. (10)), leading to the complexity of ( D2 + N K HHD) where H is the number of attention heads. Further operations, including aggregation, linear projection, and residual connections (Equ. (11) and Equ. (12)), add O(NKHD NK + DD. Moreover, applying gate function to the combination of aggregated coordinates and original coordinates (Equ. (13) and Equ. (14)) adds O(NKE KH + ND + DD). As a result, the total complexity for the self-attention module is O(NKHD NKHD2 + NKH ND) + ND²).
Regarding the FFN, most operations are linear transformations that have the complexity of D2N)..
The gate function and linear combination of coordinates contribute O(N D2 ND. So the total computational complexity of FFN is O D + ND).
B.2 Wall-Clock Time Performance Comparison
We conduct a comparison of wall-clock time performance between FAFormer and other geometric baseline models under the same computational environment. Specifically, we measure the average training time for one epoch of each model, with the results illustrated in Figure 5. It can be observed that FAFormer demonstrates greater efficiency compared to spherical barmonics-basee models and achieves performance comparable to the GNN-based method GVP-GNN Such efficiency is attributed the utilization of top-K K neighbor graphs and FA-based modules in FAFormer, which enable efficient modeling of coordinates through linear transformations.
SE13/Ttansformrr
Equitormer
EGNN
GVPGNNN
FA
FAFormer
200
Timeet)
60
150
40
50
Model
Model
Figure 5: Training time comparison between FAFormer and the other baseline models.
C
Equivariance Proof
In this section, we investigate the equivariance of the gate function (Figure 2(e)), which can be formulated as:
Gate X fx(;)) Z,) g, AX x;);;;;
(18)
X' =
(19)
where X; represents the coordinates of i-th node, 9. = a (Linear( (Z,)) the gate score based on i-th node's feature, and AX;; fa(X)) denotes the coordinate update function, i.c., the multi-head
17

---

aggregation function Equ.(13) which is based on the frame averaging and thus is E(3) equivariant:
QAX, + f
(20)
where t TR3 is translation vector and Q R3333 is orthogonal matrix.
We aim to prove that the gate function is E3) equivariant, meaning it is translation equivariant for any translation vector & RR and rotation/reflection equivariant for any orthogonal matrix Q E gxx3 Specifically, we want to show:
QX + Gate Gate - 1.0xx-L,440 10/xQX + t), Z.)
(21)
Derivation.
Calc(QXi+t + Z)) +
(22)
=9QQXX, + ( (1-9))Q + + (1-9.)t
(23)
= =Q + (1- +t
(24)
= -
(25)
Therefore, we have proven that applying rotation and translation to X; results in the identical rotation and translation being applied to X
D Dataset Descriptions
Protein Complex Datasets We collect the complexes from PDB [8], NDB [7], RNASolo [2] and DIPS [77] databases. Complexes are excluded if they have protein sequences shorter than 5 longer than 800 residues, or nucleic acid sequences shorter than or longer than 500 nucleotides. The redundant proteins with over 90% sequence similarity other sequences within the datasets are removed. The protein and the binder structures will be separated and decentered.
Aptamer Datasets Detailed information on each protein target and aptamer candidates is pre- sented below. The threshold for categorizing sequences as positive or negative aptamers is determined by referencing previous studies or identifying natural cutoffs the affinity score distributions.
GFp13; Green fluorescent protein. The aptamer candidates mutants of GFPapt [69], with Ka values ranging from OnM to 125nM as affinity measures. Candidates with Ka values lower than 10nM considered positive cases.
NELF Negative elongation factor E. The aptamer candidates mutants of NELFapt [59], with Ka values ranging from OnM to 183nM as affinity measures. Candidates with K₂ values lower than 5nM considered positive cases.
Heterogeneous nuclear ribonucleoproteins C1/C2. The aptamer candidates the ran- domly generated RNA 7mers and we apply the affinity values provided by the previous studies [46]. Candidates with affinity scores lower than -0.5 are positive.
IN
'6 Serine/threonine- protein kinase Chk2. Szeto al [73] applied SELEX (Systematic Evolution of Ligands by EXponential Enrichment) [23] to screen aptamers from a large library of random nucleic acid sequences through multiple rounds. During each round, the bound sequences were amplified and isolated. We use the final round of sequences as the candidates, with sequences having multiplicities over 100 considered positive aptamers.
UBICP1 17; Ubiquitin-like domain-containing CTD phosphatase 1, Similar CHK2, the final round of sequences are considered candidates, with sequences having multiplicities over 200 considered positive aptamers.
http:/:://wwwwuuippo/ooo
otg//miipoottb//0022///azt
"https://www uniprot. cetpot.rggg/nnninnnne
otg//aiiprotkb//9220//azzt
1httpps///ee hestpa://wwwuniiprott uniprot. orgg/mmiprottb///7910///a
"hhtppa//www uniprot.
http:pp://wwww..npooooooo/////// org/mmiprottb//0007/eeezz
17 "nttps://ww..uni htg.//mippotubb///W//////tttt
18

---

Table 8: Sampled aptamer dataset statistics.
Target
GFP
NELF HNRNPC CHK2
UBLCPI
#Positive
55
62
20
122
66
#Candidate
188
981
328
1,000
1,000
Sampled Aptamer Datasets We construct smaller aptamer datasets for the comparison between ITAFOlNAA and FAFormer by randomly sampling 10% candidates from the original datasets. The statistics are shown in Table 8. We equally split each dataset into validation set and test set. The performance FAFormer on the test set reported with the best performance on the validation set. RoseTTAFoldNA is directly evaluated on the test set.
Test datasets of Rose ITAFolNNA The test datasets used to evaluate RoseTTAFoldNA available at this accessible link. We downloaded the dataset and filtered out non-dimer complexes, resulting in 86 protein-DNA and 16 Protein-RNA complexes.
E
Additional Experiments
E.1
Case Study
Figure presents the complete groundtruth and predicted contact maps of the cases used in Figure 4, where the model accurately captures the sparse pattern.
Residue
50
100
150
200
250
Nucleatide 58555
4..
+
Residue
50
100
150
200
250
Nucleotide B5G6600
(a) Protein-DNA Contact Map Comparison (Top: Groundtruth, Down: Prediction)
Residut
(b) Protein-RNA Contact Map Comparison (Top: Groundtruth, Down: Prediction)
Figure 6: Case study based on two complex examples (PDB id: 7DVV and 7KX9).
E.2
Ablation Study
In this section, we conduct ablation study to investigate the impact of FAFormer's core modules. Specifically, we individually disable the edge module and attention mechanism, and replace the proposed FFN with the conventional FFN in FAFormer. The results are presented in Table 9.
Table 9: Ablation study.
Protein-DNA
Protein-RNA
Protein-Protein
Fl
PRAUC
Fl
PRAUC
PRAUC
FAFormer
0.1457 .o5
0.1279. .006
0.1284.oo 003
0.1113.og .04
0.1596 002
0.1463.oom 003
w/o Edge
0.1171.00 004
0.1225, 002
0.1048.000 007
0.0983.oos
0.1100.m oos
0.0972.002 002
w/o Attention
0.1401.oo 0o1
0.1250.cm
oos
0.1059. o02
0.0973.oo
0.1325.m np1
0.1121.oo
w/o FAFFN
03122..co
0.1211
002
0.1078.
0.0958.cm
0.1474.om
0.1334.oo
19

---

Removing any core module results in a significant performance decline. Notably, the model degrades either an attention-only or Sarpaage-assinn--on architecture when the edge or attention module removed, both of which lead to significant declines in performance. This demonstrates the advantage of combining these two architectures with FA.
E.3 Aptamer Screening with AlphaFold3
AlphaFold3 [1] represents the latest advancement in biomolecular complex structure prediction and accessible through an online server' Due to limited quota (20 jobs per day), we evaluated on two of the smallest sampled aptamer datasets, GFP and HNRNPC. The statistics for these datasets are presented Table 8. We used the contact probability produced by the server and the maximum probability as the estimated affinity. The results are shown in Table 10. Although AlphaFold3 performs well in predicting the complex structure, fails identify promising aptamers our cases. We attribute this to fine-gra tne-grained optimization and overfitting molecule interaction patterns in the structure prediction task, which have biased its screening performance on specific protein targets.
Table 10: Comparison results with AlphaFold3.
Metric
AlphaFold3
RoseTTAFoldNA FAFormer
Top Prec.
0.3000
0.4000
0.4000.
GFP
Top50 Prec.
0.3199
0.3600
0.3800,ox o18
PRAUC
0.3132
0.3926
0.4027 ,222
Top10 Prec.
0.1000
0.1000
0.1666. 124
HNRNPC
Top50 Prec.
0.0799
0.0599
0.0800 .224
PRAUC
0.1355
0.1452
0.1781 .099
"nttps://golgi.aant google.com/
20

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: As presented in the abstract and introduction, focus on learning the interac- tions protein complexes and extending it to the zero-shot aptamer screening application. Besides, propose FAFormer, which is an equivariant Transformer architecture based on frame averaging, achieves the best performance on all the tasks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We've included a section in the paper specifically focusing on modeling the interactions of protein complexes and zero-shot aptamer screening without evaluating FAFormer on other tasks. We will continue to work on this and aim benchmark the method more comprehensively.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
21

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide proof verifying the equivariance of the gate function, including a detailed derivation demonstrating how the gate function maintains equivariance with orthogonal matrices and translation vectors.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We' ve included a code link in the paper for review, which contains the pipelines for all tasks and datasets. Additionally, we present the data sources and processing methods for all datasets.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
22

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: A code link is included in the paper, containing the training pipelines for all tasks. Additionally, we have detailed the dataset preprocessing methods and provided the corresponding scripts in the code link.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/nips. wttpccpppppppppeeeeeceeccccceeet wwwpc//ggie/////ee httpg/dddd///odeuuonnnn //e////ooduSssssseeeeeeee SondeSSumissionPone ssiiiiosccc) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: As shown in Appendix and Appendix D, we have introduced the dataset split- ting strategies based protein sequence identity, the search range of each hyperparameter, and the optimal hyperparameters for each model.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
23

---

Justification: We report the variance for all results by repeating the experiments with three different random seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi-
dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: As mentioned in the Appendix A, the experiments are conducted on a single Linux server with The AMD EPYC 7513-32 Core Processor, 1024G RAM, and Tesla A40- 48GB. Our method is implemented on PyTorch 1.13.1 and Python 3.9.6. The computational complexity and running time comparison are shown in Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https::/neurt
Answer: [Yes]
Justification:
We've followed the instructions in the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).

---

The answer NA means that the paper poses such risks. 10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper presents work whose goal is to advance the field of geometric deep learning on protein complex modeling. There are some potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The datasets and models used in this study don't 't have such risks.
Guidelines:
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
25

---

Justification: All the involved models and datasets, including pretrained protein/RNA se- quence models, pretrained structure prediction models (RoseTTAFoldNA and AlphaFold3), complex datasets, and aptamer datasets, are properly credited and cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce new assets; it only uses existing models and datasets that are properly credited and cited.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
26

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing experiments research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "ezgEAZlPiu", "reviewer": "Reviewer_DGm4", "summary": "The authors propose a novel equivariant model, FAFormer, which leverages the frame averaging operation as an integral geometric component within the Transformer. The authors prove the invariance and equivariance of the architecture. They further conduct experiments showing that FAFormer performs well in contact map prediction and could serve as a strong binding indicator for aptamer screening.", "questions": "- The module structures with formulas are a bit difficult to follow. Also, could you provide more intuition on the design of these modules?\n    \n- Since we are constructing a KNN graph and always leveraging the coordinate X and edge E information, why should the model be called a “former” instead of a GNN, which could be misleading?\n    \n- As far as I know, RosettaFoldNA is used mainly for protein and NA structure prediction. However, FAFormer uses coordinate information as input. Is that a fair comparison?\n    \n- I am curious about the efficiency of FAFormer compared with other equivariant NNs. Could you provide some analyses?\n    \n- During experiments, do other methods include the same featurization? And are you training all models from scratch?", "limitations": "The work provides an innovative framework, FAFormer, and performs well on contact map prediction and aptamer screening tasks. However, the experimental setting is not convincing enough and requires refinement.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- FAFormer is novel in incorporating frame averaging within each layer of the Transformer. The FAlinear module provides a novel method for creating invariant models.\n    \n- Empirical results demonstrate the superior performance of FAFormer on contact map prediction and aptamer screening tasks. FAFormer excels in screening aptamers and outperforms AlphaFold3.\n    \n- The authors provide a detailed proof of the invariance and equivariance property.", "weaknesses": "- The F1 score for contact map prediction tasks is about 0.1, which is very low. I question whether this score can be used to judge which model is better. The model potentially performs worse due to separate modeling of nucleic acid and protein, the linear prediction head, and the difficulty of the pairwise prediction task. I suggest involving other node-level and graph-level prediction tasks for comparing FAFormer with other equivariant neural networks.\n    \n- The main idea of the model is confusing. If we already know the coordinates of the protein and nucleic acid, why do we need further encoding and prediction for contact prediction, instead of directly counting the contacts?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "```\nWeakness: The F1 score for contact map prediction tasks is about 0.1, which is very low. I question whether this score can be used to judge which model is better. I suggest involving other node-level and graph-level prediction tasks for comparing FAFormer with other equivariant neural networks.\n```\n**Ans**: We would like to clarify that no matter the contact map prediction performance, the **main focus** of this paper is unsupervised aptamer screening. Contact map prediction is proposed as a prerequisite task due to its strong correlation with the screening task. While contact map prediction is challenging due to sparse labels, all comparisons are fair, with the same input features and learning pipelines. For benchmarking:\n\n- We have provided results on **binding site prediction** in *Appendix E.1*. This task predicts the NA-binding residues for a given protein, which is a **node-level task**. The metrics exceed 0.4 since this task is much easier than the contact map prediction.\n- The **aptamer screening task** is a **graph-level task** aimed at retrieving the positive RNAs for a protein.\n\nMoreover, we present FAFormer’s performance on two protein-related tasks:\n\n- Fold prediction: Predict the fold class (1195 classes) for protein, with three test sets (Fold, Family, and Superfamily). Accuracy is the evaluation metric.\n- Reaction prediction: Predict the reaction class catalyzed by a given protein (384 classes). Accuracy is the evaluation metric.\n\nFollowing the experimental settings of ProteinWorkshop[1], including dataset splits and Ca-only input features, the comparison results are shown below. FAFormer demonstrates the best performance across most tasks:\n\n||Fold(Fold)|Fold(Family)|Fold(Superfamily)|Reaction|Average|\n|---|---|---|---|---|---|\n|SchNet|0.2071|0.7670|0.2375|0.5894|0.4502|\n|GearNet|0.3090|0.9340|0.4465|0.7814|0.6177|\n|EGNN|0.2577|0.9193|0.3568|0.6578|0.5479|\n|GCPNet|**0.3276**|0.9359|0.4110|0.6697|0.5860|\n|TFN|0.2512|0.9188|0.3426|0.6922|0.5512|\n|FAFormer|0.2451|**0.9661**|**0.4883**|**0.7970**|**0.6241**|\n\nDue to limited time and computational resources, we only show these two tasks' results. More results will be completed and added to the updated manuscript.\n\n[1] ICLR2024-Evaluating representation learning on the protein structure universe\n```\nWeakness: If we already know the coordinates of the protein and nucleic acid, why do we need further encoding and prediction for contact prediction, instead of directly counting the contacts?\n```\n**Ans**: We use the individually **predicted structures** of proteins/nucleic acids during the evaluation (*Lines 207-209*), and all the input structures are **decentralized** (*Line 597*), meaning the coordinates are moved to the zero centers to avoid label leakage. Consequently, the distances between two input structures cannot be used to directly indicate the complex's contacts.\n\n```\nQuestion: Could you provide more intuition on the design of these modules?\n```\n**Ans:** FA is a general framework that endows a given encoder with equivariance, allowing for the flexible design of equivariant modules. Using FA as a module rather than a model wrapper also avoids an 8x increase in computation. We are glad to elaborate more on each module’s intuition:\n\n- Local Frame Edge Module: In biomolecules, atoms interact through chemical bonds and electronic forces. Embedding these pairwise relationships allows the model to **represent these intricate dependencies** explicitly. We **differentiate the spatial context** around each node by considering the local point cloud centered on each target node. FA embeds the directed vectors between source and target nodes, capturing the direction of interactions.\n- Biased MLP Attention Module: The attention mechanism is extended to include edge representations, biasing the attention map to **prioritize/deprioritize certain weights** based on interactions. Node coordinates are also updated to **simulate conformational changes** during molecular interactions, which is crucial for optimal binding.\n- Global Frame FFN: We extend the FFN by incorporating FA to update representations in a geometry-aware context. The attention between two atom representations, combined with coordinate information, **functions similarly to distance calculations**.\n\nWe will add these discussions to the revised manuscript.\n```\nQuestion: Why should the model be called a “former” instead of a GNN, which could be misleading?\n```\n**Ans**: The core module of FAFormer is the biased attention module, which is the primary reason for considering our model a Transformer. We limit the attention to k-nearest neighbors to reduce the computation, which is commonly used in previous geometric Transformers.\nIn the paper (*Lines 90-94*), we have summarized that FAFormer is a hybrid of GNN and Transformer, as the edge module functions similarly to GNN aggregation.\n```\nQuestion: As far as I know, RosettaFoldNA is used mainly for protein and NA structure prediction. However, FAFormer uses coordinate information as input. Is that a fair comparison?\n```\n**Ans**: Besides the input sequence, RosettaFoldNA utilizes **searched MSAs** and **protein structure templates** as input. The protein structure templates are the coordinates of the MSAs. In contrast, the input structures for FAFormer are predicted solely based on input sequences.\n```\nQuestion: I am curious about the efficiency of FAFormer compared with other equivariant NNs.\n```\n**Ans**: We have provided an analysis in *Appendix B*, which includes computational complexity and wall-clock time comparisons. In summary, our proposed FAFormer demonstrates greater efficiency than spherical harmonics and FA-based Transformers.\n```\nQuestion: Do other methods include the same featurization? And are you training all models from scratch?\n```\n**Ans**: Yes, all the baseline models use the same features and are all trained from scratch. The hyperparameter and training details are provided in *Appendix A*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KlUBjgQFhT", "reviewer": "Reviewer_qqru", "summary": "This paper mainly focuses on protein-nucleic acid contact prediction and unsupervised aptamer virtual screening. The latter is based on an unsupervised learning approach by predicting the contact maps.  An equivariant architecture that integrates frame averaging and transformer blocks is proposed for this task. Experiments on the two tasks demonstrate the superiority of the proposed architecture compared with other geometric deep learning models.", "questions": "See the above weaknesses.", "limitations": "The authors have pointed out the limitations due to only modeling certain atoms in the related tasks.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The proposed architecture is effective on the contact map prediction tasks. The tasks are novel and not well-studied in the community of machine learning.\n- The unsupervised learning approach that predicts contact maps is effective on aptamer screening tasks. This perspective is novel. \n- The authors have provided codes and reproducibility can be ensured.", "weaknesses": "- The novelty of the proposed architecture is limited. The proposed architecture simply combines frame averaging and transformer blocks. The idea that builds the attention module based on SE(3)-invariant features is proposed in Equiformer. The dataset and benchmark track may be more suitable for this paper.\n- The expressiveness of the proposed architecture is demonstrated in the contact prediction tasks. More experiments on more general tasks are needed to further show the effectiveness of the proposed architecture, such as protein-protein docking and others.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "```\nWeakness: The novelty of the proposed architecture is limited. The proposed architecture simply combines frame averaging and transformer blocks. The idea that builds the attention module based on SE(3)-invariant features is proposed in Equiformer. The dataset and benchmark track may be more suitable for this paper.\n```\n\n**Ans**: We respectfully disagree with the statements and would like to emphasize the contributions of our paper:\n\nFirst, this is the first work designing a new equivariant Transformer based on the Frame Averaging (FA). Instead of simply combining them, each module is **dedicatedly integrated** with FA, which has **effectively enhanced model performance**, as demonstrated in our ablation study:\n\n- The local frame edge module focuses on local spatial context by constructing the frames on the point cloud centered around each node;\n- The biased MLP attention module applies FA to enable equivariant multi-head attention on the geometric features;\n- Global frame FFN extends the FFN by incorporating geometric information within node representations using FA, allowing the attention to be conducted in a geometry-aware manner.\n\nSecond, the way FAFormer achieves equivariance is **completely different** from Equiformer which relies on spherical harmonics operations. As discussed in the Introduction (*Lines 32-38*), models like Equiformer **compromise efficiency** due to the complexity of irreducible representations, as shown in our efficiency comparison (*Appendix B.2*). We believe that incorporating FA within the model opens new possibilities for designing equivariant architectures in this domain.\n\nThird, besides the architectural contributions, we propose **a new paradigm** **for unsupervised aptamer screening**. This paradigm connects contact map prediction and affinity estimation between two molecules, going beyond simply collecting datasets and constructing benchmarks. \n\n```\nWeakness: The expressiveness of the proposed architecture is demonstrated in the contact prediction tasks. More experiments on more general tasks are needed to further show the effectiveness of the proposed architecture, such as protein-protein docking and others.\n```\n\n**Ans**: The contact map prediction result between proteins is presented in our paper (*Table 3*). Besides, we have provided results on **binding site prediction** in *Appendix E.1*. This task solely takes a protein as input and aims to predict the NA-binding residues on the protein, which is a node-level prediction task.\n\nTo further demonstrate the performance of FAFormer, we here present its performance on two protein understanding tasks, including:\n\n- Fold prediction: This task aims to predict the fold class (a total of 1195 classes) for each protein, with three different test sets (Fold, Family, and Super-Family). Accuracy is used as the evaluation metric.\n- Reaction prediction: This task aims to predict the class of reactions for a given protein catalyzes (a total of 384 classes). Accuracy is used as the evaluation metric.\n\nWe follow the experimental settings in ProteinWorkshop[1], including dataset splits and input features (Ca-only). The statistics of the datasets are shown below:\n\n|  | Fold | Reaction |\n| --- | --- | --- |\n| #Train | 12.3K | 29.2K |\n| #Valid | 0.7K | 2.6K |\n| #Test | 1.3/0.7/1.3K | 5.6K |\n\nThe comparison results are presented below from which we can observe the best performance of FAFormer across most of the tasks:\n\n||Fold(Fold)|Fold(Family)|Fold(Superfamily)|Reaction|Average|\n|---|---|---|---|---|---|\n|SchNet|0.2071|0.7670|0.2375|0.5894|0.4502|\n|GearNet|0.3090|0.9340|0.4465|0.7814|0.6177|\n|EGNN|0.2577|0.9193|0.3568|0.6578|0.5479|\n|GCPNet|**0.3276**|0.9359|0.4110|0.6697|0.5860|\n|TFN|0.2512|0.9188|0.3426|0.6922|0.5512|\n|FAFormer|0.2451|**0.9661**|**0.4883**|**0.7970**|**0.6241**|\n\nDue to the limited time and computational resources, we can only show the results of these two tasks. More benchmarking results will be completed and added to the updated manuscript.\n\n[1] ICLR2024-Evaluating representation learning on the protein structure universe", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ltpsrlfhsX", "reviewer": "Reviewer_npWJ", "summary": "The authors propose am method for predicting contacts between proteins and aptamers based on frame averaging transformers. They showcase their methods on contact prediction and unsupervised aptamer screening, showing improvements over some baselines. The authors also compare to RoseTTAFoldNA, which seems to outperform their method.", "questions": "- Could the authors expand section 4.4 and discuss the combination of Figure 3 and Table 5?\n- Do the times in Table 6 include MSA search for Rosetta? If the protein is unchanged,  could you make it more efficient by doing the MSA search only once?\n- Could the authors expand a bit on possible future research (e.g. applying the same method to other screening tasks)?", "limitations": "Authors discuss some scientific limitations. I do not foresee any negative societal impact.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Frame averaging is a very interesting idea (published in another venue before) and applying it to this specific problem seems to be very promising\n- The presentation is very clear and the paper is well written\n- The problem addressed (aptamer screening) is very important and practically relevant\n- The authors compare to a large number of different architectures (see Table 4)", "weaknesses": "Comparing to a pre-trained structure prediction method is clearly something to be explored well, and the authors do that in section 4.4 (with additional results with AF3 in the appendix). At the same time, this part seems to me not very well developed in the paper. I am especially confused by Figure 3, which the authors comment with \"[..] where FAFormer can achieve comparable performance to RoseTTAFoldNA using unbounded structures.\" Maybe I am not understanding something but the right hand side of the plot seems to clearly indicate that at least for RNAs, RoseTTAFoldNA strongly outperforms the authors' method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "```\nWeakness: Comparing to a pre-trained structure prediction method is clearly something to be explored well, and the authors do that in section 4.4 (with additional results with AF3 in the appendix). At the same time, this part seems to me not very well developed in the paper. I am especially confused by Figure 3, which the authors comment with \"[..] where FAFormer can achieve comparable performance to RoseTTAFoldNA using unbounded structures.\" Maybe I am not understanding something but the right hand side of the plot seems to clearly indicate that at least for RNAs, RoseTTAFoldNA strongly outperforms the authors' method.\n```\n\n**Ans**: We apologize for any confusion and will make this clearer in our updated manuscript. Our claim is based on the F1 scores, which show better performance for protein-DNA complexes (0.103 vs 0.087) and comparable performance for protein-RNA complexes (0.108 vs 0.12). It should be noted that the number of test protein-RNA complexes from RoseTTAFoldNA is limited (16 complexes), which might not be able to comprehensively assess our method's performance.\n\n```\nQuestion: Could the authors expand section 4.4 and discuss the combination of Figure 3 and Table 5?\n```\n\n**Ans**: Aligning Figure 3 with Table 5, we observe that while FAFormer does not achieve higher performance than RoseTTAFoldNA in terms of contact map prediction, it significantly outperforms RoseTTAFoldNA in aptamer screening tasks. We attribute this to two main reasons:\n\n- RoseTTAFoldNA as a foundational structure prediction model is optimized for generally predicting the protein-RNA complex structure, which might bias its performance on some specific protein targets. For example, it can achieve good performance on proteins GFP and HNRNPC but fails for NELF.\n- For the contact map prediction comparison, the protein-RNA test set used by RoseTTAFoldNA is limited (16 complexes), which may not comprehensively evaluate the performance of FAFormer.\n\nAll these discussions will be added to our revised manuscript.\n\n```\nQuestion: Do the times in Table 6 include MSA search for Rosetta? If the protein is unchanged, could you make it more efficient by doing the MSA search only once?\n```\n\n**Ans**: The times reported in Table 6 include the time required for searching MSAs. For the screening task, RoseTTAFoldNA only needs to search MSAs for protein targets once. However, it still needs to search MSAs for each RNA sequence individually. Besides, the forward process of RoseTTAFoldNA is computationally expensive because the inclusion of MSAs results in a large input sequence matrix. We will add this discussion to our revised manuscript.\n\n```\nQuestion: Could the authors expand a bit on possible future research (e.g. applying the same method to other screening tasks)?\n```\n\n**Ans**: For future applications, the proposed paradigm for aptamer screening can be extended to other modalities, such as protein-small molecules and antibody-antigen. Moreover, the strong correlation between contact prediction and affinity estimation demonstrated in our paper can guide future model development. Specifically, this correlation suggests promising directions for designing new objectives and collecting datasets that better capture the nuances of molecular interactions.\n\nRegarding the architecture, FAFormer introduces a novel approach to equivariant model design by leveraging the flexibility of Frame Averaging (FA). This idea opens up numerous possibilities for future research, including exploring different ways to integrate FA with various neural network architectures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the reviewers for noting that we propose a novel model (DGm4,npWJ) to address a meaningful problem (DGm4,qqru,npWJ) with a comprehensive evaluation (DGm4,npWJ). We further summarize our key contributions as follows:

1. We explore a new angle to conduct aptamer screening in an unsupervised manner by leveraging the strong correlation with the contact map prediction task.
2. We propose a new equivariant Transformer architecture, FAFormer, by integrating Frame Averaging (FA) within each module. FA as an integral component offers the flexibility to design expressive and equivariant modules, highlighting a new possibility for geometric encoder design in this domain.
3. We construct three protein complex datasets (Protein-RNA/DNA/Protein) and five aptamer datasets to evaluate the models. Our proposed architecture achieves SOTA performance over baselines, including RoseTTAFoldNA and AlphaFold3.

Additionally, two protein understanding tasks are included to further demonstrate the effectiveness of FAFormer (DGm4,qqru). We thank all the reviewers for their valuable comments, and the corresponding refinements will be added to our updated manuscript.

Meta-review: This work introduces FAFormer, a novel equivariant transformer architecture that seamlessly integrates frame averaging within each transformer block. By incorporating FA as a core component, the architecture allows for the design of expressive and equivariant modules. Results demonstrate that FAFormer outperforms existing equivariant models in contact map prediction across three protein complex datasets. The reviewers found this framework generally interesting and innovative, with valuable yet underexplored application scenarios. Despite some minor concerns, this is a strong paper that I recommend for acceptance to NeurIPS.
Decision: Accept (poster)