Association Pattern-aware Fusion for Biological Entity Relationship Prediction
Authors: Lingxiang Jia
Yuchen Ying
Zunlei Feng
Zipeng Zhong
Shaolun Yao
Jiacong Hu
Mingjiang Duan
Xingen Wang
Jie Song
Mingli Song
Venue: NeurIPS 2024 poster (2024)
Keywords: association pattern
deep learning
interpretable machine learning
biological network
Primary area: machine_learning_for_healthcare

Abstract: Deep learning-based methods significantly advance the exploration of associations among triple-wise biological entities (e.g., drug-target protein-adverse reaction), thereby facilitating drug discovery and safeguarding human health. However, existing researches only focus on entity-centric information mapping and aggregation, neglecting the crucial role of potential association patterns among different entities. To address the above limitation, we propose a novel association pattern-aware fusion method for biological entity relationship prediction, which effectively integrates the related association pattern information into entity representation learning. Additionally, to enhance the missing information of the low-order message passing, we devise a bind-relation module that considers the strong bind of low-order entity associations. Extensive experiments conducted on three biological datasets quantitatively demonstrate that the proposed method achieves about 4%-23% hit@1 improvements compared with state-of-the-art baselines. Furthermore, the interpretability of association patterns is elucidated in detail, thus revealing the intrinsic biological mechanisms and promoting it to be deployed in real-world scenarios. Our data and code are available at https://github.com/hry98kki/PatternBERP.

Association Pattern-aware Fusion for Biological Entity Relationship Prediction
Lingxiang Jia
Yuchen Ying' Zunlei Fengl.2 Zipeng Zhong'
Shaolun Yao
Jiacong Hu
Mingjiang Duan'
Xingen Wang!.3 Jie Song' Mingli Song
State Key Laboratory of Blockchain and Data Security, Zhejiang University Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security Banggsheng Technology Co, Ltd.
Abstract
Deep bearning-based methods significantly advance the exploration of associations among triple-wise biological entities (e.g., drug-target protein-adverse reaction), thereby facilitating drug discovery and safeguarding human health. However, exist- ing researches only focus entity-centric information mapping and aggregation, neglecting the crucial role of potential association patterns among different entities. To address the above limitation, we propose novel association pattern-aware fusion method for biological entity relationship prediction, which effectively inte- grates the related association pattern information into entity representation learning. Additionally, to enhance the missing information of the low-order message passing, we devise a bind-relation module that considers the strong bind of low-order entity associations, Extensive experiments conducted on three biological datasets quan- titatively demonstrate that the proposed method achieves about 43%23% @it@1 improvements compared with state-of -f-the--rr- baselines. Furthermore, the inter- pretability of association patterns is elucidated in detail, thus revealing the intrinsic biological mechanisms and promoting it to be deployed in real-world scenarios. Our data and code are available https:/ // //gi thub. commhrry9kkk//atteere
Introduction
Exploring potential associations among triple-wise biological entities (e.g., drug-target protein- adverse reaction) [1-6] holds significant implications for elucidating underlying biological mecha- nisms and advancing personalized therapies [7-9], thus promoting pharmaceutical innovation and ensuring human health. Recent deep learning-based methods have propelled auxiliary prediction tasks concerning biological entity relationship, with most focusing on binary associations (c.g., drug- ttrgett protein), while only a few methods offer insights for more complex triple-wise associations. Existing solutions for the association prediction task can be broadly categorized into three types: (1) non-graph methods [10-12]; (2) graph- methods [13-16]; (3) sxpergraph-based methods [17-22].
As illustrated in Figure 1, non-graph methods typically concatenate the features of different entities, which are independently mapped by their respective entity encoders, to serve as representations. Graph-based methods adopt nodes and edges of the graph represent entities and their relationships, and leverage the graph structure for feature propagation, thereby achieving effective representation methods employ the hypergraph
learning of entity nodes. Similar yet distinct, hypergraph-basee structure to obtain entity representations using the complex feature aggregation strategy. However, none of the aforementioned methods consider the significance of path patterns in the graph structure, which contain a vast amount of crucial information including hidden context and co-occurrence.
Corresponneece to zunlei wimleiifnnozzju.eee edu.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Encoder
(b) Graph-based Methods
(c) Hypergraph-based Methods
Encoder
Pattern Interaction
I-hop
Entity
2-hop
Entity
Fincoder
3-hop
A Entity c
4-hop
x Target Eutity
(a) Non-graph Methods
(d) Association Pattern-aware Method
Figure I: Comparisons of feature update strategy among non-graph methods, graph- methods, hypergraph-bast based methods, and the proposed association pattern-aware method. Unlike existing methods that map or aggregate node features, the proposed method mines and then fuses association patterns for each target entity node in the graph to enhance the model's 's representative ability.
To this end, we introduce novel association pattern-aware message propagation strategy as shown in Figure (d). The module leverages the potential relationships, such as commonality and diversity, of association patterns as the rule for facilitating message passing among entity features, which can efficiently expand the potential for representing complex interactions within the perspective of both basic graph structure and intrinsic biological mechanisms. Specifically, the related patterns within the graph structure are sampled through the pre-defined distance relation for each entity node. The message passing is driven by the interaction of its assigned patterns. i.c., the feature of the entity node is subsequently updated through feature fusion using adaptive coefficients that capture pattern commonality, generated during the pattern interaction stage. This process is followed by the acquisition of potential common patterns with genuine biological significance for various entities.
In this paper, propose a novel Association Pattern-awand Fusion method for Biological Entity Relationship Prediction, namely Pattern-BERP. First, we devise an association pattern-aware strategy to solve the limitation caused by entity-centric feature mapping and aggregation. The strategy utilizes the association patterns related to each entity node within the graph to extract the common feature based on the attention mechanism for these patterns, thus expanding the ability to represent hidden complex interactions. In addition, to preserve the information interaction of different entities, a hypergraph-based block is incorporated with the association pattern-aware fusion module, thereby enhancing the model ability to capture relationships among various types of entities. Furthermore, to explore low-order associations of biological bind relations, introduce bind-relation enhancement module which can reconstruct missing feature of bind-relation entities and thus generate harder nega- tive sample than random selection. Experimental results conducted on different biological datasets show that the proposed method achieves superior performance compared to advanced baselines, demonstrating its effectiveness and robustness in handling various biological entity relationships. More importantly, the obtained association patterns for the relationship of drug, microbe, and disease are quantitatively visualized with the following biological verification in detail.
Our main contributions are summarized as follow:
We propose novel association pattern-aware fusion method for biological entity relationship prediction. The introduce of association pattern-aware strategy can enhance the representation of complex interactions by aggregate features with potential association patterns.
A bind-relation enhancement module is devised to acquire low-order associations that reveal the biological bind relations, which is essential for reconstructing missing bind-relation entity features and generating challenging negative triplets to enhance the model training.
Extensive experiments are conducted to verify the superiority of Pattern-BERR demonstrating its robustness for various biological scenarios. Uniquely compared to the other methods, the interpretability of association patterns is explained to reveal intrinsic biological mechanisms.

---

2 Related Work
In this section, we elaborate on the related work from two distinct yet interconnected perspectives: Biological Entity Relationship Prediction and Network Search and Mining, Each perceptive represents a fundamental aspect of our research, addressing specific challenges and methodologies in applying machine learning techniques to the prediction task.
Biological Entity Relationship Prediction. The latest advancements in artificial intelligence have motivated researchers to employ deep learning methodologies for predicting triple-wise biological entity associations. Hypergraph neural network (HGNN)- based methods [17-22] have become the mainstream research direction in this field. Tu et al. [17] proposed deep hyper-network embedding model to preserve both local and global proximities in the embedding space. Building upon this, Jiang et al. [18] incorporated a dynamic hypergraph construction strategy to capture the hidden and important relations in data structures. Zhang et al. [19] developed a self-attention-baseee graph neural network applicable to homogeneous and heterogeneous hypergraphs with variable hyperedge sizes. Liu et al. [20] proposed a multi-way relation-enhanced hypergraph representation learning method to predict anti-cancer drug synergy. Liu et al. [21] proposed multi-view contrastive hypergraph model for drug- leingmmccrobe-dieee
association prediction. In addition, Chen and Li [23] attempted to adopt the tensor decomposition strategy to predict which target a drug binds to when administered to a disease, and further proposed neural tensor network model [24] that seamlessly combines tensor algebra and deep neural networks to effectively capture the complex nonlinear dependencies among drugs, targets. and diseases.
Network Search and Mining. Network search and mining techniques, particularly those utilizing path information including random walks [25-28] and meta-path [29-32], have been widely employed to extract local structural information from networks. These methods have found applications in areas such as content recommendation and community detection [33-37]. Brin and Page [25] introduced a classic ranking algorithm PageRank to determine the importance of web pages based on their link structure. Jeh and Widom [26] adopted similarity measure based on pairwise random walk, which can capture the structural similarity between nodes, and further extended PageRank with a personalized version [27]. Perozzi et al. [28] proposed Deepwalk that leverages local random walk information to learn vertex latent representations based on deep learning techniques. These above methods are applicable to homogencous networks and cannot fully utilize the rich semantic information in heterogeneous networks. To address the limitation, Sun et al. [29] introduced meta path-bases similarity framework for heterogeneous information networks, which can capture the subtle semantics of similarity among objects of the same type. Dong et al. [30] proposed deep learning-based heterogeneous network representation learning method that automatically learns hidden meta-path semantics, generating general node embedding representations. Wang et al. [31] introduced a graph-based fraud detection method that addresses the issue of low homophily by integrating label information generate distinguishable neighhorhood information. Furthermore within the bioinformatics field, Chen et al. [32] proposed a computational algorithm that performs random walks on an integrated network infer potential relations between proteins and ADRs.
In line with the above methods, Pattern-BERS BERP utilizes path information for triple-wise heterogeneous biological network mining. By leveraging the fusion of association patterns, it facilitates message passing among various biological entities, which will be described comprehensively in Section 4.2.
3 Problem Formulation
Given three distinct entity types in biological networks, termed as A = {a1,,0,," (0,,002,"" ",,,", , A B = {b,, bg.... by. .b11) and C = (c1, 02. ,m. Cm.. (c.) their Cartesian product = x B x C is a set of all possible triple-wise biological entity associations. For simplicity, the relation A-B-C is used to represent complex relational semantics for biological entities, such as miee-mmerobbeiiiee
"synergistic drug-drug-cell line" or "drug-target protein- reaction".
For each triplet (a,, Cm. E S, we assign a label E [0, 13. label of p = indicates that the existence of certain association has been confirmed, while = O represents an unknown association which denotes that the association is not yet known and could potentially exist. The objective is to develop a credible model that predict these potential associations from unknown ones.
3

---

Bio-encoding
Association Pattern-aware Fusion
Positive
Predictor
Patlern Commorality
I hop LEA
HGNN Q D
hop
ssoociatoon
Negative
788-3
4.hop p
XX|V
g - (V. 8)
Pattem Interaction
Pw--1
GAS
Negative Sampling
O+ O+
Eniry A
Emiiy A
Entity
Element Add
X
Target Entity
GBc
Bind-relation
Negative Sample
Bind-relation Enhancement
Figure 2: Overall framework of PatternBEER.. First, entity attributes are initialized with different types of bio-encoders. Then, these existing associations are constructed into one hypergraph 9 and two bipartite graphs GA.B. GB.C. After that, the hypergraph is encoded with Association Pattern-awand aware Fusion module based on the pattern commonality, thereby affecting target entity representation. In addition, the bipartite graphs are encoded to output the missing bind-relation feature and thus generate hard negative samples. Finally, the integrated entity feature are used for final association prediction.
4 Pattern-BERP
To enhance the ability to represent complex interactions, we propose the first association pattern- aware method, termed Partern-BERS to extract the rich semantic information embedded within the intricate structures of biological networks. As illustrated in Figure 2, the section is divided into four parts: First, these entity relationships are represented in hypergraph and bipartite graph structures for subsequent module input. Next, the association pattern-aware fusion module proposed to update entity node feature through association pattern-aware interaction. In addition, the bind-relation enhancement module is introduced to reconstruct bind-relation feature and thus generate hard negative samples. Finally, the detailed summary of loss function and complexity analysis provided.
4.1 Graph Construction
Given the adjacency relationships among entity nodes, the hypergraph and bipartite graph structures constructed to facilitate the subsequent extraction of structural information and relational associa- tion patterns within the respective graphs, which serve for association pattern-aware fusion module and bind-relation enhancement module.
The attributes of each entity are initialized as node features on the graphs with the domain knowledge of bio-entities through its own type. Finally, the initialized node attributes of X E R consist of features X X and Xe, where d denotes the feature dimensional of initialized entity attributes. Details about entity attribute are provided at Appendix A.
Hypergraph Construction. Triple-wise biological entity associations can be modeled as a hyper- graph 9 (V,&), which includes vertex represents all known associations. Technically, G is further formulated as an attributed hypergraph with node attributes X e xd and an incidence matrix H E {0, 1) (P1))),), which is defined as:
A U B U C. and hyperedge & C S that
M(v,e) -
{b.
if i
module is to decom- pose the original triple-wise associations to separately obtain pairwise relations of different types of entities. Since the original triplet is in the form of (a,, bj, Cn., and generally the interaction relationships between the three entities are hierarchical (e.g. drug a; acts on protein b, and the activated by then leads to certain adverse reaction cm)) Hence in this paper, construct two bipartite
Bipartite Graph Construction. The premise of constructing the

---

graphs: GA.B, for entity A + entity B1,, G3, for entity Bg + entity C. Then the edge sets of the bipartite graphs can be formulated as:
EA.B. {(a, b) a b E B,, Be E H such that a, b e), EBgC {(b. c) b E B2, C. H such that b, E e}.
(2)
(3)
Note that, the relationship of A C' is under consideration due there is no direct connection in this context. The constructed bipartite graphs GA.B, 9B..C serve as input for bind-relation module.
4.2 Association Pattern-aware Fusion
To comprehensively account for the feature interactions within association patterns, the proposed Association Pattern-aware Fusion (APF) method comprises two fundamental components: Firstly, Association Pattern Sampling (APS) block is designed to sample association pattems by utilizing the distance tokens relevant to target nodes. Secondly, Association Pattern-awan Interaction (API) block introduced to update node features by message interaction within the sampled association patterss and mine the intrinsic pattern commonality with biological support.
4.2.1 Association Pattern Sampling
Definition 1. Given certain entity node, the distance token between the node and one hyperedge or association pattern is defined as u-hop. Here, I-hop patterns represent the hyperedges directly covering the node, 2-hop patterns represent the hyperedges directly covering all the I-hop neighbor nodes of the node, and the relation continues for higher hop counts. If the node is unreachable when reaches the max step U, we define these patterns no-relation and set the distance -00.
Based on the above definition', we generate the distance tokens between each node and all hyperedges, ultimately obtaining a distance matrix D E I | defined as follows:
if node is u-hop away from pattern
d(u,e) e) -00, if node is unreachable from pattern e
(4)
where d(v, e) represents the defined distance from node v to pattern e; indicates the number of hops from node u to pattern e.
To represent an entity node with association patterns, the general principle is to prioritize and retain patterns that are closer for each node based on the distance tokens, with a total of N patterns sampled, Formally, given an entity node v, let D. E IRN and P. E x3d represent the distance tokens and feature vectors of the selected sampled patterns, where each sampled pattern consists of three entity nodes and the d-dimensional feature embeddings of each node are defined within the initial node embeddings X. Then, the output pattern feature 2, for node considers the relative position of these related patterns, which is thus formulated as z, = P + POS(D.), where POS(-) denotes the position encoding layer that maps from RV - R N (3d), Finally, the integrated embedding Z, E TR. N x3d with distance information is produced for use in the subsequent API block.
4.2.2 Association Pattern-aware Interaction
Before put into the API block, we adopt hypergraph convolution layer implemented by [40] on the constructed hypergraph
facilitate neighbor- information propagation among different entities. Then the updated node feature X* after hypergraph convolutions is used to construct the feature of patterns mentioned the APS block, thereby yielding z, for the information interaction of these sampled patterns. Details of hypergraph convolution are provided at Appendix.
Pattern Interaction. The API block is designed to search for and extract commonalities among different sampled patterns related to a specific entity node, and thus consists of a composition of Transformer layers [38]. Each Transformer layer has two modules: multi-head self-attention mechanism (MHA) and a position-wise feed-forward network (FFN). For simplicity, we consider the single-head setting, and the extension to multi-head attention is standard and straightforward.
Dil Specificallyy, let ()) [z] 2 (0) E V denotes the input of the self-attention module
2To facilitate distinction, we set the entity term B; = Bg =
Thh definition is analogous to positional embedding in Transformer [38] and VIT [39].
5

---

where E R (33d) is the representation for node The imput Z, is projected by three matrices We E [dd WK E I2 (3d)xxdx, and Wv [dd]xdy to obtain the corresponding query, key, and value representations Q.. K.. and V. y for node u:
Q.K, Atttu(z.) softmax (A V (5) vdx
Q = Z, W.. K. z. W. V = Z. Wv. A.
where A, N is a matrix capturing the similarity between queries and keys; dk. dy denotes the feature dimensional of the key representations K, and the value representations V. Then we will get the output of the self-attention module z E Nxdd) To summarize the process of transformer layer, the output of association pattern-awars attention block is computed as:
z (T) = (HA((LN(( + ((111) z FFN(TNN(Z (I)) + 21),,
(6)
where LN denotes layer normalization, and (z) is the output of the current transformer layer. After (Li)] Xxv)xxx(3dd) and
L1 -laye transformers, we get the encoding output ( (z((a))
then apply the mean function similar entities in N association patterns for each node to obtain the learned embedding Z, E F.
Pattern Commonality. To mine the commonality of N sampled patterns, a score is defined to represent the quantitative relation these association patterns, termed by Pattern Commonality Coefficient, based the attention scores A. of trained API block, which is formulated as follows:
1
A softmax(A.), C.
Σ Im.:]. N
where C. N and [a|n (0, 1) indicates the commonality coefficient of the n-th pattern. Pat- terns with relatively high commonality coefficients tend to share the same or similar pathways, while showing significant differences response compared low commonality patterns. Corresponding biological validations are presented in the Section 5.3.
4.3 Bind-relation Enhancement
In the context of multi-entity relationships, there often exist strong pairwise bind relation between entities, such as drug Aspirin to treat common cold [2]. To mitigate the weakening or overlooking of low-order bind relation, Bind-relation Enhancement (BE) module is designed effectively reconstruct the missing feature by capturing these important pairwise associations. Additionally, the module can generate confident and challenging negative samples to aid the training.
Bind-relation Feature Reconstruction. To efficiently learn entity representations in pairwise bind relations, we introduce an edge prediction classification task on the bipartite graphs EA,B, εB,c with
the initial embeddings x (0) = XAA, (0) X ( X8, x (0) Xc. Specifically, L2-layer self- A
E
supervised BGNN model [43] is employed learn node features on the bipartite graphs, followed by the Multi-layer Perception (MLP) [10] layer to output association probabilities, which is defined as:
x (1+1) x (++1) BGNN(X()) MLP (La) || x
(1) X B GAB,,) X ((+1) x(++1) BGNN(X (1) B3 ( (B,,),
B
(L2) P(2..)) MLP((x (L2) || x((x)))
P(a.11)
where ( X Bg represent the node features at layer (; (Lz) (Lz) (Lz) (Lz) denote ,X/, x22
(1)
the final learned representations of entity a, by, b2, respectively; P(a,bı), j. P(bz.c) represent the esti- mated probability association between entities a, by and entities b2, c respectively. After that, the loss of the supervised prediction task can be formulated as:
LA,,I/LB = E.A. 1/1EB..c
Σ
(pe log + (1 Pe) log (1 pp))
(10)
/
The loss LBE of bind-relation task is defined as LBE = aLABB + (1 ao,L where a the balancing coefficient for the two losses.
In the single-hees head setting, dx dy 3d.

---

Hard Negative Sampling, Based on the above bind-relation task, negative samples are adaptively generated for triple-wise associations, instead randomly selecting from the vast sample space. Moreover, the generated negative samples are challenging, which contributes to efficient learning. As illustrated in Figure 2, three kinds of negative samples are considered as follows:
Eg {(a, b', (P(a.br) < 7) V (P(W..) < <7)).
(11)
where 8' E B represents another entity with random selection that differs from entity b in the original triplet to form the negative sample (a. b.,b)e):: Eg is the set of generated negative triplets; y represents
the threshold for the prediction probability to determine whether the association exists.
4.4 Total Loss
Base on the above modules, Z, of node v learned by the APF module is updated with reconstructed features X from the BE module according the entity type, thereby generate the enhanced em- bedding of with n = 2, + X, for the association predictor network. Hence, for the triple-wise association prediction, we utilize the learned embeddings z 2 and z of entity a, b, e output the probability of the association ş through scoring function P(o.b.c) = (Z, z || zz The loss of the association prediction task for true sample set and negative sample set &g can formulated as:
CAPF = E E
Σ
(pe log Pe + - Pe) log (1 - pp)).
(12)
CE(ELEE)
Hence, the total loss L is computed through an alternating training strategy of the two modules, where the BE module trained for the first 4 epochs of every 5-epoch cycle, followed by the APF module, which is trained for the final epoch of each cycle. During the training of each module, the parameters of the other one are frozen. The equation for L is defined as follows:
o mod 5
) CAPF(O) mod 5
(()) (nr(o)
(13)
4
where o represents the current epoch:
[.] denotes floor function; mod denotes modulo operation.
4.5 Complexity Analysis
Considering the significantly higher complexity of the APF module in comparison to other network components, we only consider APF which aggregates multiple patterns across various entity nodes. Specifically, for one entity node with N sampled patterns from the APS module, the input and hidden features in the MHA layers are of dimension fM, and hidden features in the FFN layers are fr. In APF, the query, key, and value matrices are derived from the same input sequence and share length N. The primary operations for APF include scaled dot- not-product attention, multiplication of attention weights and values, MHA linear transformation, and FFN linear projection. The time complexity is expressed as O(N2. fM + N fh + N fM fF). Hence, for the entire graph, the total complexity O (IVI (N2 fM + N fh N fM fr)). where [V] is the number of entity nodes.
5
Experiment
5.1 Experimental Settings
Datasets. In this paper, we adopt three biological entity association datasets with significant biological meaning, namely DMD Bong-Miirohe--issase DDC (synergistic Drug-Drug-Cell line) and DPA (Drug-target Protein-Adverse reaction), among which DPA dataset is directly constructed. In line with DMD and DDC, we utilize preprocessing tools provided by [21] to deal with the original data from ADReCS- Target [44], and collect total of 1,079 triplets that structured into the data schema <drug, protein, adr>. Appendix Table 3 presents the statistics and characteristics of these datasets.
Baselines. To verify the effectiveness of Pattern-BERP, we compare it with three kinds of methods: (1) Non- Mon-graph methods. Following the work from [21], five non con-graph methods are adopted including Random Forest (RF) [45], MLP [10], CP [11]. Tucker [11]. CoSTCo [12]; (2) Graph-based methods.
We select four classical architectures of graph neural network, including GCN [13), GraphSAGE methods. Recent hypergraph learning methods to address triple-wise biological entities associations or similar tasks are considered as the baselines,
[14], GAT [15], GIN [16]; (3) Hypergraph-
including DHNE [17], HyperSAGNN [19], HGSynergy [20]. MCHNN [21].
7

---

Table Performance comparison on three datasets of different biological entity associations. Each result of these methods is from the average of 5-fold cross-validation experiments with four scenarios. The best result for each dataset and metric marked in bold. All the presented hits scores are in
DMD
DDC
DPA
Methods
hits @
hits@
5
hits @
hits@
hits@5
hits@
hits@3
hits
RF
34.06
57.28
68.31
8.93
18.63
28.29
26.62
37.45
44.95
MLP
42.72
65.40
75.33
13.27
27.98
39.88
27.55
40.65
48.80
CP
44.73
66.74
76.47
13.71
28.67
41.03
31.30
45.46
54.03
Tucker
45.27
66.54
76.98
13.24
28.55
39.77
28.80
43.10
50.74
CoSTCo
38.69
60.38
71.77
10.93
22.95
33.60
31.06
41.95
47.87
GCN
62.66
76,57
79.74
25.86
47.06
58.27
18.38
30.56
38,66
GraphSAGE
56.98
73.52
77.42
22.23
41.73
52.98
12.36
23.84
31.48
GAT
47.13
67.90
75.24
21.60
43,91
54.41
21.53
33.24
41.53
GIN
40.08
60,93
69.29
12.68
28.41
37.52
16.44
30.05
39.12
DHNE
81.86
93.66
96.02
43,42
62.61
72.01
32,64
47.69
56.30
HyNe
87.04
93.99
96.09
41.31
66.38
76.29
33.24
49.58
58.38
HGSynergy
88.68
92.10
94.51
41.07
60.74
70.76
28.19
40.36
48.44
MCHNN
90.04
93.92
95.19
41.91
61.12
72.10
32.27
43.29
50.32
Pattern-BERI
93,94
97.53
98.24
48.01
68.40
76.39
43.52
57.36
63.89
+3.90
+3.54
+2.15
+4.59
+2.02
+0.10
+10.28
+7.78
+5.51
Implementation Details. To accurately evaluate model performance and prevent overfitting, 5-fold cross-validation is used evaluate the performance. Specifically, we randomly split the dataset into 90% pross-valddation set and 10% independent test set. On the cross-validation set, the 5-fold cross- vrlss-validation is implemented. Moreover, the independent testing, in which the model is trained on the cross validation set and tested the independent test set, is conducted to obtain the prediction results. In the training stage, Binary Cross Entropy loss is adopted to measure model performance and Adam optimizer is adopted to optimize all of model parameters with learning rate of 0.001
Evaluations. To evaluate the prediction performance on the triplet associations, hit ratio @hit@n) and normalized discounted cumulative gain (ndeg @ n), which are widely used in recommendation tasks [46, 47], are employed for model ability to provide comprehensive assessment for the task.
5.2 Performance Comparison with Advanced Baselines
Table 1 summarizes the prediction performances of Battern-BERU in comparison with other baselines across the DMD, DDC and DPA datasets. It is evident that Pattern-Bii -ERR significantly surpasses the previous state-of-the-aue baselines, including non-graph, graph-based, and hypergraph- methods, across all three datasets by a large margin, with a particularly notable hit@ improvement of approximately 23.6% the DPA dataset (from 33.24 to 43.52). The results underscore the broad accuracy and applicability of the proposed method various biological association scenarios.
A salient observation is that hypergraph- based methods achieve superior performance than those of the other two types. This phenomenon empirically demonstrates the advantages of utilizing high-order structure information over the other methods that we compare. Additionally, we observe that graph- based methods exceed non-graph methods for DMD and DDC datasets. The relatively large number of associations and proportion of DMD and DDC datasets, shown in Appendix Table 3, indicate a higher density and stronger interconnectivity within the underlying entity relationships. Hence, graph-based methods acquire abundant information of entity interactions based on the graph structure. Furthermore, hypergraph-bere paaeed methods acquire high-order structure information, ultimately leading to better performance. In contrast, for the DPA dataset, non non-graph methods outperform graph-based methods, even demonstrating competitive performance compared to hypergraph-basse methods. This can be attributed to the fact that the association proportion within the DPA dataset is conspicuously low (around 0.002%). indicating a relatively sparse graph structure. Under such conditions, the non-graph methods are able to obtain models, which may struggle to capture meaningful patterns from the limited graph connectivity.
robust association information compared to graph-based based
Mooe details about the experiments can be found at Appendix B.

---

(a)
OC-A (176, #37, #128)
(A53, #41, A80)
(#132, #41, #9)
(#102, #38, #8)
(H227, #44, #80)
1-hop
2-hop
2-hop
2-hop
2-hop
1.03%
1.58%
1.57%
1.55%
0.001%
Drug #53
#53 Cefadroxil
#132 Hetacillin
#102 Econazole
#76 Chlorhexidine acetate
#227 Tobramycin
(b)
(#207, #54, H55)
(H100, #54, 147)
(#209, #54, H62)
(978, #28, 169)
(H160, M16, #110)
1-hop
2-hop
2-hop
2-hop
2-hop
2.98%
2.98%
2.98%
0.001%
Drug #207
W207 5-(2-Thienyl)- Shiieny)-LL-cysteine L-cysteine #100 Diphenyl Disulfide #209 Salicylic Acid #788 Chrysophanic Acid #160 Moxifloxacin
The interpretability cases N=100 association patterns related drug #53 and #207 in DMD dataset. The pattern commonality coefficients are represented in the form of percentage to indicate the contribution for visualization, with each pattern typically assigned a default value of 1%. Larger pattern commonality coefficients indicate more significant contribution to the target drugs, and these patterns frequently exhibit similar or even identical biological pathways. Conversely, smaller commonality coefficients suggest lack of relevance to these drugs.
Figure 3:
5.3 Association Pattern Interpretability
In order to investigate the potential relation among different association patterns, we visualize the pattern commonality coefficients Cefadroxil (drug #53) and 2-(-Thiienyll-Leeytteii (drug #207) in DMD dataset, as shown in Figure 3. Larger pattern commonality coefficients contribute more to the original drug pathway, whereas smaller coefficients often relate different biological mechanisms. It can be observed that:
For Cefadroxil (drug #53), the common patterns consisted by Chlorhexidine acetate (drug #76) [48], Hetacillin (drug #132) [49], and Econazole (drug #102) [50, 51] considered more similar as the mechanism [52] that actives on the cell wall and envelope leading the change of microbe physiological activities, thus treating the diseases. Instead, Tobramycin (drug #227) inhibits mRNA be translated into protein and thus promotes microbe cell death [53, 54].
For S-(2- $-(2-Thieny)J---cysteim (drug #207), the common patterns consisted by Diphenyl Disulfide (drug #100) [55, 56), Salicylic Acid (drug #209) [57], and Chrysophanie Acid (drug #78) [58, 59] are considered more similar as the mechanism that [60-62] inducts oxidative stress in bacteria and thus damage components of the microbe cell. Instead, Moxifloxacin (drug #160) inhibits DNA gyrase and topoisomerase IV enzymes to separate bacterial DNA, thereby inhibiting cell replication [63, 64].
In summary, the analyzed cases illustrate the sigh-oommnnaliyy patterns derived from Battern-BERi demonstrating how these small molecule drugs influence microbial activities to treat diseases, gener- ally following consistent physiological route. These cases indicate that our method can discover potential common patterns through the API block, and obtain larger pattern coefficients through the at- tention mechanism, which drives the nodes to learn these important common interactions and acquire more expressive representations. Details about additional cases can be found at Appendix B.3.2.

---

Table Ablation study results on DDC dataset with different module designs. "BFR" denotes Bind- relation Feature Reconstruction; "HNS" denotes Hard Negative Sampling; "HC" denotes Hypergraph Convolution block; "DE" denotes Distance Embedding; "API" denotes Association Pattern-aware Interaction block. All the presented scores are in %, and the best result is marked in bold.
w/o BFR w/o HNS w/o HC w/o DE w/o API hits@1
hits@3
hits@5
ndeg@1
ndeg@3
n
44.76
64.74
73.08
44.76
56.42
59.85
45.03
65.86
74.97
45.03
57.19
60.95
42.81
64.48
74.02
42.81
55.41
59.33
f
40.29
62.87
72.44
40.29
53.41
57.35
d
44.69
65.17
74.51
44.69
56.62
60.45
47.11
66,56
74.89
47.11
58.47
61.90
48.01
68.40
76.39
48.01
59.84
63.13
5.4 Ablation Study
To investigate the necessity of each component in PatternBERRP, we conduct several comparisons be- tween Pattern-BERP and its variants on the test set. As illustrated in Table 2, when basic components of Pattern-BERI have been removed, the performances of corresponding variants on DDC dataset significantly decline, indicating that these components all contribute to the performance. Besides, we have other observations: (1) when only DE module is removed, the performance is inferior to the one removing the entire APA module containing DE, demonstrating that inaccurate entity distance information has more detrimental impact the prediction performance; (2) eliminating HNS module results in a significant drop in the performance, highlighting its crucial role in enhancing the model's robustness and discrinination capability and thus indicating that the proposed negative sampling strategy contributes to efficient learning; (3) when removing HC module leads to a slight performance degradation, the impact is relatively limited which suggests that the capability of HC module in representing complex associations is relatively modest for the highly dense- dense-association associaaiionn DDC dataset, but it still provides some beneficial effects towards the final performance improvement.
To verify the affect of hyperparameter settings to model performance, we first conduct ablation experiments the three main parameters of the APF module, namely number of attention heads, number of max pattern distance, and number of sampled association patterns. As illustrated in Appendix Figure 5, when the three hyperparameters are increased, the prediction performance DDC dataset exhibits an overall upward trend, suggesting that increasing these hyperparameters helps the model better capture the complex association patterns, therehy improving the final performance. Additionally, we conduct experiments on the loss-balanced coefficient a and the bind-relation prediction probability threshold y, both adjusted from 1 to 0.9. Results in Appendix Figure 6 show that setting and to 0.5 yields the best performance. Specifically, for a, since the final prediction task involves predicting the associations among entity A. B, and C, the two tasks of A B and B + C are intuitively of equal importance, therefore the balanced coefficient a set to 0.5; for bind-relation prediction is fundamentally binary classification task, thus the threshold y is to 0.5. Details about ablation experiments are presented at Appendix B.3.3.
6 Conclusion
In this work, we propose a novel association pattern-aware fusion method Pattem-BERR for biological entity relationship prediction, which effectively combines the related association pattern information into entity representation learning. In addition, to enhance the missing information of the low-order message passing, we devise bind-relation module that considers the strong bind of low-order entity associations. The evaluation on three biological datasets quantitatively demonstrate that the proposed method consistently achieve superior performance over the competing baselines. Moreover, the interpretability explanations of association patterns reveal the intrinsic biological mechanisms and thus promote the method to be deployed in real-world scenarios.
Due to the domain-specific task, Satten--BER focuses on the fixed-length association patterns. Extending the approach to capture variable-length pathways could further enhance the representational power. Additionally, exploring the applicability the method in other domains beyond biology, such general knowledge graph completion, would help evaluate its broader generalizability.
10

---

Acknowledgments and Disclosure of Funding
This work supported by the Zhejiang Province High-Level Talents Special Support Program "Leading Talent of Technological Innovation of Ten-Thousands Talents Program" (No. 2022R52046), the Fundamental Research Funds for the Central Universities (226-2024001155), and the Scientific Research Fund of Zhejiang University (No. XY2023020).
References
[1] Shengbo Wu, Shujuan Yang, Manman Wang, Nan Song, Jie Feng, Hao Wu, Aidong Yang, Chunjiang Liu, Yanni Li, Fei Guo, et al. Quorum sensing-based interactions among drugs, microbes, and diseases. Science China Life Sciences, 66(1):137-151, 2023.
[2] Lei Wang, Yaqin Tan, Xiaoyu Yang, Linai Kuang, and Pengyao Ping. Review on predicting pairwise relationships between human microbes, drugs and diseases: from biological data to computational models. Briefings in Bioinformatics, 23(3):bbac080, 2022.
[3] Jia Jia, Feng Zhu, Xiaohua Ma, Zhiwei W Cao, Yixue X Li, and Yu Zong Chen. Mechanisms of drug combinations: interaction and network perspectives. Nature reviews Drug discovery, 8 (2):111-128, 2009.
[4] Mathew J Garnett, Elena J Edelman, Sonja Heidorn, Chris D Greenman, Anahita Dastur, King Wai Lau, Patricia Greninger, I Richard Thompson, Xi Luo, Jorge Soares, al. Systematic identification of genomic markers of drug sensitivity in cancer cells. Nature, 483(7391):570-577 2012.
[5] Lun Yang, Jian Chen, and Lin He. Harvesting candidate genes responsible for serious ad- verse drug reactions from a themical-protehi interactome. PLoS computational biology. 5(7): el e1000441, 2009.
[6] Lun Yang, Kejian Wang, Jian Chen, Anil G Jegga, Heng Luo, Leming Shi, Chunling Wan, Xizhi Guo, Shengying Qin, Guang He, et al. Exploring off-target and off-systems for adverse drug reactions via chemical- protein interactome clozapine- induced agranulocytosis as case study. PLoS computational biology, 703)x100 2011.
[7] Anna Gogleva, Dimitris Polychronopoulos, Matthias Pfeifer, Vladimir Poroshin, Michaël Ughetto, Matthew J Martin, Hannah Thorpe, Aurelie Bornot, Paul D Smith, Ben Sidders, et al. Knowledge graph-based recommendation framework identifies drivers resistance in egfr mutant non-small cell lung cancer. Nature communications, 13(1):1667, 2022.
[8] Yiyuan Pu, Daniel Beck, and Karin Verspoor. Graph embedding based link prediction for literature-based discovery in alzheimer's disease. Journal of Biomedical Informatics, 145: 104464, 2023.
[9] Ali Daowd, Samina Abidi, and Syed Sibte Raza Abidi. A knowledge graph completion method applied to literature-based discovery for predicting missing links targeting cancer drug repurposing. In International Conference on Artificial Intelligence in Medicine, pages 24-34. Springer, 2022.
[10] Fionn Murtagh. Multilayer perceptrons for classification and regression. Neurocomputing, 2(5): 183-197, 1991. ISSN 0925-2312.
[11] Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review, 51 (3):455-500, 2009.
[12] Hanpeng Liu, Yaguang Li, Michael Tsang, and Yan Liu. Costco: A neural tensor completion model for sparse tensors. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Minin pages 324-334, 2019.
[13] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference Learning Representations((CLR)) 2016.

---

[14] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems/Neurl(S), 30, 2017.
[15] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Represen- Iulions(ICRR), 2018.
[16] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representatiooss/CCLBR, 2019.
[17] Ke Tu, Peng Cui, Xiao Wang, Fei Wang, and Wenwu Zhu. Structural deep embedding for hyper-networks In Proceedings of the AAAI conference on artificial intelligence((AAAI) pages 2018.
[18] Jianwen Jiang, Yuxuan Wei, Yifan Feng, Jingxuan Cao, and Yue Gao. Dynamic hypergraph neu- ral networks. In Proceedings of International Joint Conferences on Artificial Intelligence(([CA)), 2019.
[19] Ruochi Zhang, Yuesong Zou, and Jian Ma. Hyper-sagnn: a self-attention based graph neural network for hypergraphs. In International Conference on Learning Representations (ICLR), 2020.
[20] Xuan Liu, Congzhi Song, Shichao Liu, Menglu Li, Xionghui Zhou, and Wen Zhang. Multi-way relation-enhaneed hypergraph representation learning for anti-cancer drug synergy prediction. Bioinformatics, 38(20)44782 4789, 2022.
[21] Luotao Liu, Feng Huang, Xuan Liu, Zhankun Xiong, Menglu Li. Congzhi Song, and Wen Zhang. Multi-view contrastive learning hypergraph neural network for drug- seng--mccobbe-ieeee association prediction. In Proceedings of the -ecooy International Joint Conference Artificial Intelligence(JCAAI),, pages 4837, 8 2023.
[22] Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence(AAAT)) pages 5558-3565, 2019.
[23] Huiyuan Chen and Jing Li. Modeling relational drag-target-disease interactions via tensor factorization with multiple web sources. In Proceedings of the 28th international conference on World Wide Web(WWW), pages 218-227,1 2019.
[24] Huiyuan Chen and Jing Li. Learning data-driven drug-target- -arget- interaction via neural tensor network. In International joint conference on artificial intelligence (IJCAI), 2020.
[25] Sergey Brin and Lawrence Page. The anatomy of a large- scale hypertextual web search engine. Computer networks and ISDN systems, 30(1-7)1007-117, 1998.
[26] Glen Jeh and Jennifer Widom. Simrank: measure of structural-context similarity. In Proceed- ings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining(KDD)
pages 538-543, 2002.
[27] Glen Jeh and Jennifer Widom. Scaling personalized web search. In Proceedings of the 12th international conference World Wide Web(WWW)), pages 27 1-19,7, 2003.
[28] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social repre- sentations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining(KDD) pages 701-710, 2014.
[29] Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S Yu, and Tianyi Wu. Pathsim: Meta path- based top-k similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment, 4(1 1):99210033 2011.
[30] Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining KDD), pages 135-144, 2017.
12

---

[31] Yuchen Wang. Jinghui Zhang, Zhengjie Huang, Weibin Li, Shikun Feng, Ziheng Ma, Yu Sun, Dianhai Yu, Fang Dong. Jiahui Jin, et Label information enhanced fraud detection against low homophily in graphs. In Proceedings of the ACM Web Conference 2023(WWW), pages 406-416, 2023.
[32] Xiaowen Chen, Hongho Shi, Feng Yang. Lei Yang, Yingli Lv, Shuyuan Wang, Enyu Dai, Dianjun Sun, and Wei Jiang- Large-scale identification of adverse drug reaction-related proteins through a random walk model. Scientific reports, 6(1):36325, 2016.
[33] Francois Fouss, Alain Pirotte, Jean-Michel Renders, and Marco Saerens. Random-walk compu- tation of similarities between nodes of graph with application to collaborative recommendation. IEEE Transactions on knowledge and data engineering(TKDE)) 19(3):355-365 369, 2007.
[34] Reid Andersen, Fan Chung, and Kevin Lang. Local graph partitioning using pagerank vectors. In 2006 47th Annual IEEE Symposium on Foundations of Computer Science(FOCS), pages 475 486. IEEE, 2006.
[35] Zihao Zhao, Jiawei Chen, Sheng Zhou, Xiangnan He, Xuezhi Cao, Fuzheng Zhang, and Wei Wu. Popularity bias is not always evil: Disentangling benign and harmful bias for recommendation. IEEE Trans actions on Knowledge and Data Engineering, 35(10);9920-99)
2022.
[36] Jiawei Chen, Junkang Wu, Jiancan Wu, Xuezhi Cao, Sheng Zhou, and Xiangnan He. Adap-T: Adaptively modulating embedding magnitude for recommendation. In Proceedings of the ACM Web Conference 2023, pages 1085-1096, 2023.
[37] Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng. Meng Wang, and Xiangnan He. Bias and debias in recommender system: A survey and future directions. ACM Transactions Information Systems, -1(3):1-39, 2023.
[38] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems(NeurIPS) 30, 2017.
[39] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[40] Yue Gao, Yifan Feng. Shuyi Ji, and Rongrong Ji. Hgnn+: General hypergraph neural networks. IEEE Trunsactions Putlern Analysis and Machine Intelligence(TPAMI), 450)3311111 2022.
[41] Ronald Eccles. Efficacy and safety of over-the-counter analgesics in the treatment of common cold and flu. Journal of clinical pharmacy and therapeutics, 31(4)3309-31, 2006.
[42] Michael D Kogan, Gregory Pappas, M Yu Stella, and Milton Kotelchuck. Over-the-counter medication use among us preschool-age children. Juma, 272(113) 1994.
[43] Chaoyang He, Tian Xie, Yu Rong, Wenbing Huang, Junzhou Huang, Xiang Ren, and Cyrus Shahabi. Cascade-bgnn: Toward efficient self-supervised representation learning on large-scale bipartite graphs. arXiv preprint arXiv: /906. / 1994, 2019,
[44] Li-Hong Huang, Qiu-Shun He, Ke Liu, Jiao Cheng, Min-Dong Zhong, Lin-Shan Chen, Li- Xia Yao, and Zhi-Liang Ji. Adrecs-target: target profiles for aiding drug safety research and application. Nucleic acids research, 46(D1):911-DD11, 2018
[45] Leo Breiman. Random forests. Machine learning, 45:5-32, 2001.
[46] Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. Improving sequential recommendation with ennnleege-enhanced memory networks. In The 41st inter- national ACM SIGIR conference on research & development information retrieval(SIGIR)) pages 505-514, 2018
13

---

[47] Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji-Rong Wen, and Daxiang Dong. Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In Proceedings of the twelfth ACM international conference on web search and data mining( mining(WSDM)) pages 573-581, 2019.
[48] Jerrold B. Leikin and Frank P. Paloucek. Chlorhexidine gluconate. In Jerrold B. Leikin and Frank P. Paloucek, editors, Poisoning and Toxicology Handbook, pages 183-184. Informa, 4th edition, 2008.
[49] Jed F Fisher, Samy O Merouch, and Shahriar Mobashery. Bacterial resistance to 3-lactam antibiotics: compelling opportunism, compelling opportunity. Chemical reviews, 105(2):395- 424, 2005.
[50] D Thienpont, J Van Cutsem, JM Van Nueten, CJ Niemegeers, and R Marsboom. Bilogical and toxicological properties of econazole, a broad-spectrum antimycotic. Arzneimiitel-forscuung, 25(2):224-230, 1975.
[51] Matthew C Fisher, Ana Eustruey-Izqqierdee Judith Berman, Tihana Bicanic, Elaine M Bignell, Paul Bowyer, Michael Bromley, Roger Brüggemann, Gary Garber, Oliver A Cornely, et al. Tack- ling the emerging threat of antifungal resistance to human health. Nature reviews microbiology. 20(9):557-571 2022.
[52] Donald J Tipper and Jack L Strominger. Mechanism of action of penicillins: proposal based on their structural similarity to slylld-alanyllddaaaninee. Proceedings of the National Academy of Sciences, 54(4):1133 1411, 1965.
[53] Grace Yang, Joanna Trylska, Yitzhak Tor, and J Andrew McCammon. Binding of aminogly- cosidic antibiotics to the oligonucleotide a-site model and 30s ribosomal subunit: Poisson- boltzmann model, thermal denaturation, and fluorescence studies. Journal of medicinal chem- 2006.
istry, (18))1947777888
[54] Jalal Haddad, Lakshmi P Kotra, Beatriz Llano-Sotelo, Choonkeun Kim, Eduardo F Azucena, Meizheng Liu, Sergei B Vakulenko, Christine S Chow, and Shahriar Mobashery. Design of novel antibiotics that bind to the ribosomal acyltransfer site. Journal of the American Chemical Society, 124(13):322 (34(13):32293233 3237, 2002.
[55] Munday. Toxicity of aromatic disulphides. ii intra-erythrocytic hydrogen peroxide formation and oxidative damage by aromatic disulphides. Journal of applied toxicology, 5(6):409-413, 1985.
[56] Victoria Osipova, Yulia Gracheva, Maria Polovinkina, Daria Burmistrova, and Nadezhda Berberova. Antioxidant activity and cytotoxicity of aromatic oligosulfides. Molecules, 27(12): 3961, 2022.
[57] Warren W Kaeding. Oxidation of aromatic acids, iv. decarboxylation of salicylic acids. The Journal of Organic Chemistry, 29(9):5556-255 1964.
[58] Chien-Hang Ni, Po- Yuan Chen, Hsu-Feng Lu, Jai-Sing Yang, Hui- Ying Huang, Shin-Hwar Wu, Siu-Wan Ip. Chin-Tung Wu, Su-Yi Yin Chiang, Jaung- Geng Lin, al. Chrysophamol-induus necrotic-like cell death through an impaired mitochondrial atp synthesis in hep3b human liver cancer cells. Archives of pharmacal research, 35:887-899 895, 2012.
[59] Longfei Lin, Yuling Liu, Ming Zhong, Tanggui Xie, Jian Ni, Hui Li, et al. Hepatotoxicity and mechanism study of 120: 109531, 2019.
in vitro. Biomedicine & Pharmucotherapy,
[60] Rajindar Sohal. Role of oxidative stress and protein oxidation in the aging process. Free Radical Biology and Medicine, (3(1)337-4,.002.
[61] Bernd Moosmann and Christian Behl. Mitochondrially encoded cysteine predicts animal lifespan. Aging cell, 7(1)322-46, 2008
[62] Candice E Paulsen and Kate S Carroll. Cysteine-mediated redox signaling: chemistry, biology, and tools for discovery. Chemical reviews, 113(7):4633-4679 2013.
14

---

[63] Edwin M Ory and Ellard M Yow. The use and abuse of the broad spectrum antibiotics. Jama, 185(4))273-77, 1963.
[64] Karl Drlica and Xilin Zhao. Dna gyrase, topoisomerase iv, and the 4-quinolones. Microbiology and molecular biology reviews, 61(3):377-33 377-392, 1997.
[65] Serkan Kiranyaz, Onur Avei, Osama Abdeljaber, Turker Ince, Moncef Gabbouj, and Daniel J Inman. 1d convolutional neural networks and applications: A survey, Mechanical systems and signal processing, 151:107398, 2021.
[66] James Z Wang, Zhidian Du, Rapeeporn Payattakool, Philip S Yu, and Chin-Fu Chen. A new method to measure the semantic similarity of go terms. Bioinformatics, 23(10):124411281, 2007.
[67] Susan L Holbeck, Richard Camalier, James A Crowell. Jeevan Prasaad Govindharajulu, Melinda Hollingshead,, Lawrence W Anderson, Eric Pollcy, Larry Rubinstein, Apurva Srivastava, Debo- rah Wilsker, et al. The national cancer institute almanac: comprehensive screening resource for the detection of anticancer drug pairs with enhanced therapeutic activity. Cancer research, 77(13):3564 (7(13):3564-3576 2017.
[68] Yuanjing Ma and Hongmei Jiang. Ninimhmda: neural integration of neighborhood information on multiplex heterogeneous network for multiple types of human microbe- disease association. Bioinformaticx, 36(24):5665-567)
[69] Yanli Wang. Jewen Xiao, Tugba Suzek, Jian Zhang, Jiyao Wang, and Stephen H Bryant. Pubchem: a public information system for analyzing bioactivities of small molecules. Nucleic acids research, (supppl 2):W623-WW0 2009.
[70] Emmanuel Boutet, Damien Lieberherr, Michael Tognolli, Michel Schneider, and Amos Bairoch. Uniprotkb/swiss-pra the manually annotated section of the uniprot knowledgebase. In Plant bioinformatics: methods and protocols, pages 89-112. Springer, 2007.
[71] Mei-Chun Cai, Quan Xu, Yan-Jing Pan, Wen Pan, Nan Ji, Yin-Bo Li, Hai-Jing Jin, Ke Liu, and Zhi Zhi-Liang Ji. Adrecs: an ontology database for aiding standardization and hierarchical classification of adverse drug reaction terms. Nucleic acids research, 439DD):D907-D913, 2015.
[72] Amit S Kalgutkar. Designing around structural alerts in drug discovery. Journal of Medicinal Chemistry, 63((2):6276 (22)))622666662. 2019.
[73] Sidney D Nelson. Metabolic activation and drug toxicity. Journal of medicinal chemistry. 25 (7):753-765, 1982.
[74] Allan B Bailey, Ronald Chanderbhan, Nancy Collazo-Braid Mitchell A Cheeseman, and Michelle L Twaroski. The use of structure structure-activiiy relationship analysis in the food contact notification program. Regulatory Toxicology and Pharmacology, 42(2):225-22) 235, 2005.
[75] Lingxiang Jia, Zunlei Feng, Haotian Zhang, Jie Song, Zipeng Zhong, Shaolun Yao, and Mingli Song. Explainable fragment-based molecular property attribution. Advanced Intelligent Systems, 4(10):2200104 2022.
[76] RP Elander. Industrial production of 3-lactam antibiotics. Applied microbiology and biotech- nology, 61(5):385-39) 2003.
[77] Md Badrul Islam, Md Inshaful Islam, Nikhil Nath, Talha Bin Emran, Md Rezaur Rahman, Rohit Sharma, Mohammed Mahbubul Matin, et al. Recent advances in pyridine scaffold: Focus on chemistry, synthesis, and antibacterial activities. BioMed research international, 2023, 2023.
[78] Rajib K Singh, Arvind Kumar, and Arun K Mishra. Chemistry and pharmacology of acetanilide derivatives: a mini review. Letters in Organic Chemistry, 16(1):6-15, 2019.
[79] George G Zhanel, James Karlowsky, Ethan Rubinstein, and Daryl Hoban. Tigecycline: novel glycylcycline antibiotic. Experi review of anti-infective therapy, 4(1):9-25, 2006.
[80] Pili Hu and Wing Cheong Lau. survey and taxonomy of graph sampling. arXiv preprint arXiv:1308.5865 2013.
15

---

Appendix
In this appendix, we provide a comprehensive elaboration of the methodologies. experimental details, and additional insights that support the findings presented in the main manuscript. The appendix structured into details of the proposed method, details of the experiments, and other discussion contents including limitation and extension. Furthermore, our data, code and all raw experimental results are provided in the Github repository attps://gitthnn httpa//....oyyylki/aaattteeeeaae
A Details of Pattern-BERP Method
A.1 Entity Attribute
Before constructing the graphs, the attributes of each entity are initialized node features on the graphs with the domain knowledge of bio-entities. Due to the varying types of entities being studied, multiple encoders are employed align the attribute embeddings for different types of entities. Hence, three specific types of encoders are considered as follows:
For drug entities D, their SMILES strings can be converted into molecular graphs Go (Sp, Ap), where Sp is the attribute matrix of all nodes representing the atoms and Ap the adjacency matrix of these nodes. Vallina GIN [16] is adopted to learn atom representations and then these atom representations are summarized into drug-level feature vector through a global max pooling (GMP), which is formulated as:
(1+1) MLP((+)) ((AD + (1 (1)))))X)), X
(14)
where I is the current epoch of GIN, I is the identity matrix, is fixed scalar, and X (0) Sp. After applying the GMP over all molecular graphs, the features of all drug entities can be compiled into XD E TR iP|xd where d represents the dimension of entity attribute vector.
For protein entities P, taken the one-dimensional protein sequence T as the input, we first convert the sequence string to an integer vector as the initialized embedding Sp E R Then, considering x (0) Sp. the ID CNN [65] model is used to extract the protein representation. The propagation mechanism of each CNN layer works as follows:
x((+1) = a (CNN(X
P (!) d!l) (0)) )))
(15)
where ( x((+1) are the hidden feature vectors of the 7h and (1 (1+ 1)th CNN layer, respec- tively; d del ()) are the number of channels in the input, number of channels produced by the convolution and the convolving kernel size of the h CNN layer; a(-) represents nonlinear
activation function, specially ReLU. After CNN layers, the d-dimensional feature vectors of all target proteins are denoted as X E R!P|xd
For other entities O (such as diseases), we compile similarity matrices to acquire the initial embedding So E [[O|x(0) (O) based on the methods provided by [32, 66-68] and followed by fully-connected networks to obtain the entity representation, denoted as Xo E kl More specifically, for microbe nodes, cell lines and disease nodes, we compile similarity matrices S. E {1, M) [M]MM Se [|4X and SN R [MM x based the methods provided by [66-68], which then are transformed into X. || Xx 4lxd and X IR (V xd by fully-connected networks. For ADRs nodes, we first use the co-occurrence of drugs to evaluate ADRs similarity. For two ADRs and j. the Jaccard score is calculated as follows:
[Di n,
Jaccard_score
(16)
D,
where D; and Dj denote drug sets that cause ADR i and ADR j. respectively. The ADR similarity matrix SR. E [||]||||| is constructed according to the Jaceard score, finally denoted as Xx= R|
Based on the above construction rules, we generate the corresponding initial attributes for entity datasets A. B, and C through the entity types. Finally, the entity attributes X consists of features X A X5 and Xc. When encountering specific datasets, the three entity types can be substituted into (A, B, C) respectively. For example, X consists of Xp. Xx and X in DPA dataset.
16

---

Table 3: Detailed information of three datasets of different biological entity association.
Datasets
Entity Types
#Nodes
#Associations
Ratio
DMD
Drugs
Microbes
Discases
270
58
167
2,763
0. 0.106%
DDC
synergistic Drugs
Drugs
Cell lines
87
87
55
2,044
0.491 %
DPA
Drugs
Proteins
ADRs
298
552
280
1,079
0.002%
A.2
Association Pattern Distance
Given the maximum number of hops U (i.e., the max positional distance for certain entity node), the distance matrix D E [[x]] is constructed based on Definition 1 as follows:
I-hop distance:
For each node and hyperedge if hyperedge j directly contains node i, then u,e represents the association of hyperedge j the 1-hop pattern.
2-hop distance: If hyperedge does not directly contain node but shares node with hyperedge that does, then De.e = 2 represents the association of hyperedge e the 2-hop pattern.
u-hop distance: Recursively calculate further distances up to u-hop, then Dz,e
Unreachable: If node v cannot reach hyperedge e within U'hoop, then De,e -00.
Hence, the distance tokens D, for node v are transformed by one position encoding layer, thereby yielding the positional embeddings for the self-attention mechanism of the subsequent API block.
A.3 Theory Analysis
As empirical method, the Matern-BERP method extracts common patterns or rules from large bio-associated networks, similar to the k-means clustering. Specifically, given a triplet-wise dataset
[11, 82, ... 8,,.-.,887 8T) and K common patterns, the optimization goal is minimize the total distance of sampled patterns to their respective pattern centers. First, initialize the pattern centers M {ml m2......... m...... mg]: Next, assign each sample the nearest pattern center by computing the center index with arg min; ||s, m&I for all the centers in M. Then, update the position of these pattern centers using m JC+ Ex,ec, St, where C& the of samples assigned to pattern & calculated as above. Iterate these assignment and update steps until the pattern centers converge. The objective function to minimize is J = Σk Σx, EC, ||s; m& 1|2 and by minimizing Pattern-EERP extracts representative biological association patterns effectively.
B
Details of the Experiments
B.1
Datasets
In this paper, three biological datasets are used evaluate the efficacy of the proposed method. Each dataset encompasses three different entities and their associations. Appendix Table 3 provides detailed presentation of the specific entities within each dataset, including the count of nodes per entity, the number of associations among these nodes, and the corresponding association ratio. All entity associations are structured into triplet scheme, such as <drug, protein, adr> for DPA dataset.
DPA dataset. DPA dataset is first constructed and preprocessed with the origin data from [44]. Specifically, only <drug, target, adr> triplets that have complete field information are retained, that is, drug Pubchem CID, target protein identifier UniProt ID, ADR term name. Then, the detail information obtained through these unique identifiers, respectively. For drugs, the PubChem database [69] is queried with the PubChem CID of each drug, and their Canonical SMILES are recorded. For proteins, the UniProtKB database [70] is queried with the UniProtKB of each protein, and their protein sequences are recorded. For ADRs, the incidence matrix about the reaction of specific ADRs to 1,,0000 drugs is obtained from publicly available datasets [71]. In this matrix, a value of indicates that particular drug causes specific ADR, while value of 0 denotes no relation between them. Furthermore, following the method described in Appendix A, the ADR similarity matrix can be constructed for subsequent feature generation fo ADR entities.
17

---

Table Detailed information of memory usage (GB) with varying the number of sampled patterns N across three datasets on a single NVIDIA A6000 GPU with Intel(R) Xeon CPU (24 cores).
Datasets
Number of Nodes
Number of Associations
N =
N
10
=0
N 50
N 100
DMD
495
2.763
10.52
10.65
10.93
11.77
13.30
DDC
229
2,044
4.35
4.40
4.58
4.96
5.64
DPA
1,130
1,079
9.68
10.01
10.62
12.55
16.01
B.2
Details of Implementation
B.2.1
Implementation of Baselines
The implementation of all baseline methods is conducted using their respective publicly accessible source codes. Optimal or default configurations for each method are employed ensure robustness. Specifically, for methods such as RF, MLP [10], CP and Tucker [11]. meticulous parameter tuning is engaged in elicit their peak performance levels. For GCN [13]. GAT [15], GraphSAGE [14] and GIN [16]. the original triple-wise associations are initially decomposed into two pair-wise associations. Subsequently, each pair-wise association independently modeled using the corresponding graph neural network model, and the prediction probabilities of the two pair-wise associations are multiplied to obtain the final prediction value for the triple-wise association. Additionally, for DHNE [17] and HyperSAGNN [19], biological embeddings are integrated with their original structural embeddings to ensure fairness. For CoSTCo [12], HGSynergy [20] and MCHNN [21], the parameter settings outlined in the original publications are followed. It is important to note that for the sake of equitable
comparison, all of these methods utilize negative sampling setting from MCHNN that consistent across implementations.
B.2.2 Implementation Settings
The initialized entity embedding size d is fixed to 128. The number of BGNN, APF layers are all fixed to 2. The training epoch is setting to 1,000 for DMD, DPA datasets and 2,000 for DDC dataset. The number of max hop in pattern sampling U is setting to 3. The number of sampled patterns N setting to 100. The number of attention heads is 32 for DMD dataset, 16 for DDC dataset, 4 for DPA dataset. The boss-balanced coefficient for bind-relation task is fixed to 0.5. The threshold for bind-relation prediction probability is fixed to 0.5. In addition, in line with the evaluation strategy of [21], 29 negative samples randomly generate for each test triplet in four scenarios for the evaluation of all methods, and thus display the average metrics over all test triplets. These scenario settings comprehensively evaluate the model ability to identify positive and negative samples under these stringent conditions.
Furthermore, all experiments are conducted on single NVIDIA A6000 Tensor Core GPU (48GB) and Intel(R) Xeon CPU with 24 cores and 500G memory. The whole training time for DMD, DDC, DPA datasets is about 8, 8, 4 hours, respectively. In addition, Appendix Table 4 exhibits the GPU memory usage with varying the number of sampled patterns N across three datasets, and Appendix Figure presents the inference time of Pattern-BERP in comparison with these advanced baselines for each 100 samples with milliseconds.
B.3 Details of Results
B.3.1 Additional Performance Comparison on the ndeg
As supplement to Table I, Appendix Table 5 incorporates additional metrics ndeg@) @ I, ndcg @3 and ndcg@5. Considering all these metrics, Pattern-BERh -BRR significantly outperforms previous SOTA baselines all three datasets, exhibiting the remarkable advantages of association pattern mining.
Furthermore, each result of these methods is from the average of 5-fold coos-validattion experiments with four scenarios. However, significant variation in prediction difficulty across different scenarios makes relatively unreasonable to provide an error bar for all 20 results. Instead, to demonstrate the statistical validity of Pattern-BERR we have provided the experimental results.
18

---

Table Performance comparison on three datasets of different biological entity associations. Each result of these methods is from the average of 5-fold cross-validation experiments with four scenarios. The best result for each dataset and metric marked bold. All the presented ndeg scores are in
DMD
DDC
DPA
Methods
ndog@1
ndeg@3
ndeg@5
ndeg @
ndog@3
ndeg@
ndeg@3
ndeg@5
RF
34.06
47.51
52.08
8.93
14.50
18.45
26.62
32.86
35,
ML.P
42.72
55.88
59.99
13.27
21.73
26.63
27.55
35.13
38.47
CP
44.73
57.50
61.51
13.71
22.31
27.37
31.30
39.53
43.05
Tucker
45,27
$7.64
61.92
13.24
22.03
26.61
28.80
37.10
40.23
CoSTCo
38.69
51.21
55.93
10.93
17.73
22.13
31.06
37.43
39.86
GCN
62.66
71.00
72.31
25.86
38.07
42.70
18.38
25.39
28.71
GraphSAGE
56,98
66.84
68.45
22.23
33.43
38.05
12.36
19.01
22.17
GAT
47.13
59.35
62.38
21.60
34.53
38.85
21.53
28.26
31.65
GIN
40.08
52.27
55.71
12.68
21.74
25.50
16.44
24.24
27.95
DHNE
81.86
88.88
89.86
43.42
54.54
58.40
32.64
41.45
45.00
HyperSAGNN
87.04
91.18
92.05
41.31
55,88
59.97
33.24
42.73
46.36
HGSynergy
88.68
90.82
91.28
41.07
51.51
55.19
28.19
35.21
38,
MCHNN
90.04
92.38
92.91
41.91
53.03
57.54
32.27
38.59
41.49
Pattern-BER BERP
93.94
96.10
96,39
48.01
59.84
63.13
43.52
51.67
54.36
A
+3.90
+3.72
+3.48
+4.59
+3.96
+3.16
+10.28
+8.94
+8.00
(A52, #41, #8)
(#52, #41, #26)
(#53, #41, #8)
621 #50, #9)
(#207, #2, #26)
(#266. #41, #S)
1-hop
1-hop
2-hop
2-hop
2-hop
2-hop
0.41%
0.65%
2.23%
2.05%
2.0%%
0.31%
Drug #52
Drug
Drug
Drug #52
Drug #207
Drug #266
Figure The interpretability case of ==100 association patterns related to drug #52 in DMD dataset. The pattern commonality coefficients represented in the form of percentage to indicate the contribution for visualization. Larger pattern commonality coefficients indicate a more significant contribution the target drug #52, and these patterns frequently exhibit similar even identical biological pathways. Conversely, smaller coefficients suggest a lack of relevance to drug #52.
B.3.2 Additional Interpretability Analysis
In order to investigate the potential relation among different association patterns, we visualize the pattern coefficients of drug #52 DMD dataset, as shown in Appendix Figure 4. It can observed that the patterns that make important contributions are not necessarily 1-hop patterns, with pattern coefficients 0.41% 1% and 0.65%. contrast, the 2-hop patterns can exhibit considerable relevance, due to the similar mechanisms of toxicity against mierobes exhibited by the corresponding drugs (#53, #162, #207) compared to the target drug #52 under investigation. Additionally, the weakening of drug #52 itself is because the original drug #52 is complex peptide structure with repetitive and redundant information, thus acquiring simpler and more straightforward representations through the aforementioned information interaction.
Furthermore, from the view of structural and functional groups mentioned by [72-75], we have several salient observations as follows: (1) As the drug in the most influenced pattern, drug #53
19

---

Table Ablation study results on DMD dataset with different module designs. "BFR" denotes Bind- relation Feature Reconstruction; "HNS" denotes Hard Negative Sampling; "HC" denotes Hypergraph Convolution block; "DE" denotes Distance Embedding; "API" denotes Association Pattern-aware
Interaction block. All the presented scores are in %.
w/o BFR w/o HNS w/o HC w/o DE w/o API hits@1
hits@5 hits@3
ndeg@)
ndeg@3
n
77.67
91.72
95.16
77.67
86.06
87.48
78.96
92.71
95.98
78.96
87.15
88.52
92.85
97.10
98.06
92.85
95.39
95.80
f
92.83
96.94
97.88
92.83
95.31
95.70
d
93.76
97.38
98.09
93.76
95.95
96.24
93.59
97.27
98.06
93.59
95.97
96.18
93.94
97.53
98.24
93.94
96.10
96.39
Table 7: Ablation study results on DPA dataset with different module designs. "BFR" denotes Bind- relation Feature Reconstruction; "HNS" denotes Hard Negative Sampling: "HC" denotes Hypergraph Convolution block; "DE" denotes Distance Embedding; "API" denotes Association Pattern-aware
Interaction block. All the presented scores are in %.
w/o BFR w/o HNS w/o HC w/o DE w/o API hits @
hits
hits@5
ndeg @
ndcg ndeg@3
@5
37.73
53.94
62.41
37.73
47.15
50.64
34.21
52.18
60.79
34.21
44.62
48.17
42.27
57.45
62.96
42.27
51.12
53.39
/
41.57
55.65
63.01
41.57
49.74
52.77
/
/
46.16
60.60
67.22
46.16
54.63
57.36
/
44.68
58.52
64.91
44.68
52.79
55.44
43.52
57.36
63.89
43.52
51.67
54.36
contains 3-lactam as shown in the red rectangle of Figure 3. 3-lactam is the crucial component of 3-lactam antibiotics [76], which is one of the most widely used classes of antibiotics available. In addition, Islam et al. [77] also prove that pyridine scaffold (aromatic ring with nitrogen) bearing poor basicity generally improves water solubility in pharmaceutically potential molecules and has led to the discovery of numerous broad-spectrum therapeutic agents. (2) Similar with drug #53, drug #162 has acetanilide structure, which serves the basis for antimicrobial activity and disease treatment [78]. (3) Drug #207 has the dimethylglyeine structure, which is crucial to Tigecycline [79]. Tigecycline binds the bacterial ribosome, blocking the binding of amino-acyl-RRNA to the acceptor site on the mRNA-ribosome complex, thereby inhibiting protein synthesis. (4) Unlike the previous three. drug #266 may interact with membrane lipids alter membrane fluidity and permeability, thereby exerting its effect on microorganisms. However, this mechanism of action is distinct from the specific functional group interactions of the previous three drugs.
As compound with a complex structure containing multiple functional groups, drug #52 similar to the previous three drugs due its complex structure and diverse functional groups. These characteristics enable it to exert toxicity through specific interactions with the critical pocket of microbes. Compared to drug #266, these complex drugs have more diverse and specific toxicity mechanisms, which is why they have received less attention.
B.3.3 Ablation Results
Network Module Designs. To investigate the necessity of each component in conduct several comparisons between Pattern-BER and variants on the independent test set. Table 2, 6, denotes the results of DDC, DMD, DPA dataset, respectively.
When basic components variants on all datasets exhibit significant declines, though the degree of performance degradation varies. This indicates that these components all contribute to the performance. Besides, we have other observations: (1) for the relatively sparse datasets, DMD and DPA, the performance decline after removing BFR is more significant compared to the DDC dataset. (2) for all datasets, eliminating HNS module results in drop in the performance, highlighting its effect in enhancing the model's 's robustness
Pattern-BERI have been removed, the performances corresponding
20

---

56
No. of Adtuntinn Hood
No. of Aduntin Hoad
No. Allntionn Hoad
32
15
60
32
66
Perfomence
Performnne
80
53
NANEED Mebri:
NSHH21
tekgoo
Metric
Metric
(a)
(b)
(c)
Hop
Pefformeroo 70
Prfformerr De
52
40
MIHEE
Neiego
tenego
55
No. of Sampled Parter
of Sampled Parter
No. Sampled Partem
60
600
50
10
100
100
10
100
20
20
Perfommmnc
Perfommenc
15
25
45
HR:g3 Matic
WHsg5
htsig3 Melli
NSA01
Hts@3 Malric
NXSQ5
(h)
(i)
Figure 5: Ablation study results with different hyperparameter settings. The three rows are for DMD,
DDC, DPA dataset, respectively.
and discrimination capability and thus indicating that the proposed sampling strategy contributes to efficient learning; (3) for all datasets, when only DE module removed, the performance is inferior to the one removing the entire API module containing DE, demonstrating that inaccurate entity distance information has more detrimental impact on the prediction performance; (4) when removing HC module leads to a slight performance degradation, the impact is relatively limited which suggests that the capability of HC module in representing complex associations relatively modest for the relatively dense-association DDC and DMD dataset, but it still provides towards the final performance improvement. But for highly sparse DPA datasets, removing HC module leads to performance improvement, demonstrating that the combined performance of HC and APA is instead constrained by the confusion introduced by the two aggregation modes on the extremely sparse structures.
beneficial effects
Hyperparamter Settings. To verify the ablation of hyperparameter settings, we conduct ablation experiments on the three main parameters, namely number of attention heads, number of max pattern distance, and number of sampled association patterns. As illustrated Appendix Figure 5, when the number of attention head are increased, the prediction performance on all datasets exhibits an
21

---

94.0
94.0
hit (%)
hit@ (%
93.5
93.5
93.0
92.5
93.0
92.0
92.5
91.5
0.1
0.3 0.4
0.7 0.8
0.1
0.2
0.3
0.4 0.5 0.6 0.7 0.8 0.9
Loss-balanced coefficient
Bind-relation threshold
(a)
(b)
Figure 6: Ablation study results on the loss-balanced coefficient and the probability threshold of bind-relation task both varying from 0.1 to 0.9. Note that the red cross in (b) indicates that valid negative samples cannot be generated when y 0.1, hence the predictions cannot be made.
Methods
12.5
RF
GAT
Time (ms)
10.0
ML.P
GIN
7.5
CP
DHINE
Tucker
HYperSAGNN
5.0
CaSTCo
HGGSyneryy
2.5
GCN
MCHNN
GraphSAGE
Pattem: BERP
0.0
DMD
DDC
DPA
Figure 7: Inference time comparison three datasets of different biological entity associations. Each result of these methods is for 100 triplet samples. All the presented scores are in milliseconds (ms). Note that the original implementation of CP, Tucker and MCHNN is CPU-only based.
overall upward trend, suggesting that increasing the attention heads helps the model better capture the complex association patterns, thereby improving the final performance. In addition, given the significant disparity in the association ratios across the three datasets, the performance of DDC datasct improves the max hop hyperparameter is increased. In contrast, for the relatively sparse DMD dataset, is advisable to restrict the max hop to the range from to 3, as the performance drops sharply when the parameter is to 4. Moreover, for the sparsest DPA dataset, selecting only 1-hop patterns appears to the optimal choice. Moreover, regarding the number of sampled patterns, a similar trend is observed across the datasets. For DMD and DPA datasets, which relatively sparse, increasing the number of sampled patterns is not necessarily beneficial for performance. In contrast, on DDC dataset, augmenting the number of sampled patterns continues yield performance.
Furthermore, we also conduct ablation experiments on another two parameters, namely loss-balanced coefficient and the bind-relation prediction probability threshold 7. Results in Appendix Figure 6 show that when a and are both set to 0.5 by our default, the performance surpasses those of other settings. Specifically, for a, since the final prediction task involves predicting the associations among entity A, B, and C, the two tasks of A + B and B - should be intuitively considered to be of equal importance, therefore the balanced coefficient set to 0.5; for y, bind-relation prediction is fundamentally binary classification task, thus the threshold y is set 0.5.
C Discussions
Computation Complexity Discussion. To facilitate applying across wider range of dataset types and particularly large datasets, it is imperative to rigorously address the issue of complexity explosion. Regarding computation complexity, the discussion in Section 4.5 indicates that the decisive factors are the number of entity nodes [V] and the number of sampled patterns N. Hence, when encountering larger datasets or more complex biological networks, we can employ following two strategies to avoid computation complexity explosion: (1) Graph Sampling. By sampling smaller subgraphs [80], computational resource consumption reduces significantly, boosting algorithm speed and efficiency: (2) High-confidence Pattern Selection. As shown in Appendix Figure 5 (g)-(i), reducing N from 100
22

---

to 5 across three datasets slightly decreases performance but still surpasses these baselines. Thus, adjusting the sampling quantity is acceptable effectively reduce time complexity.
Moreover, Appendix Table shows the GPU usage varying from the number of sampled pattern across three datasets with significant differences in the number of entities and associations. 'he inference time of each 100 samples with milliseconds compared to baselines are provided in Appendix Figure 7, It is evident that the current computational resources are sufficient handle existing datasets, and the difference in inference time for our method is constant and even faster than several baselines.
Potential Application in Real-world Scenarios. The preliminary evaluation results three bio- logical datasets quantitatively demonstrate that the proposed method consistently achieves superior performance over the competing baselines. Additionally, the learndd association patterns show potential in interpreting biological mechanisms. This finding provides hope for the future practical application of our approach, particularly in addressing the "black box" issue in the field of bioinformat- ics. As a computer-aided tool, our method holds the potential to exhibit a broad array of applications in real-world scenarios, contingent upon sufficient clinical tests validations. Consequently, it may significantly contribute to human health and well-being,
Limitations and Future Work. While the proposed Pattern-BERP has demonstrated strong per- formance on biological entity relationship prediction, there are several limitations and potential directions for future research. One limitation is that the current model only considers fixed-length association patterns. In real-world biological systems, relevant relationships may be expressed through variable-length pathways involving multiple intermediate entities and relations. Extending the model to capture and leverage such variable-length association patterns could further enhance its representational power and predictive accuracy. Additionally, the applicability of Pattem-BERR method has so far been explored only in the biological domain. Investigating the effectiveness of this approach in other domains, such knowledge graph completion for general entities relation extraction from text, would help evaluate its broader generalizability and potential to benefit other fields that rely structured knowledge representations. Adapting the model architecture and training strategies to accommodate the unique characteristics of different domains could lead to fruitful avenues for future research.
By addressing these limitations and exploring these future research directions, we believe Pattern- BERP can be further improved and extended have greater impact advancing understanding of complex biological systems and knowledge representation in general.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Section 1.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See "Limitations and Future Work" part in the Appendix C.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
See Section 4.2 "Association Pattern-aware Fusion" & Section 5.2 - "Perfor- mance Comparison with Advanced Baselines", Furthermore, the data and source code are also provided in the Github repository.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
25

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The data and source code provided httts:://ggiiiiii wttp:/tu....oom////........ PatternBERP.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips...
(ogllc/guides/Codeeee
onooonnlooilll icy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. cctpublic//aeccccccccccccccc /gwpi/eg///////eeeeee /Ce///eeeubbiiic sotesuubbisiioni ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section 5.1 "Implementation Details" Appendix B.2.2 "Implementa- tion Settings".
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We have provided the raw experimental results of all 5-fold cross-validation results across the four scenarios, resulting in total of 20 individual results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
26

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See Appendix B.2.2- n.22--"Imllementation Settings".
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:: //neurip/// cc httpp//utipscccpppliiilici/
caGuidelines:
Answer: [Yes]
Justification: This paper adheres to the NeurlPS Code Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Appendix C-"Application in Real-world Scenarios".
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
27

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The source code & citations in the paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi wwwode.hcoe...c. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
28

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The source code provided in our repository.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "0E8SInLvo4", "reviewer": "Reviewer_ugjG", "summary": "This work presents a novel approach - Pattern-BERP - for the prediction of biological entity relationships, it utilises entity association patterns as opposed to a lot of existing research that focuses primarily on entity-centric information mapping and aggregation. \n\nThe evaluation is done on three different biological datasets (DMD, DDC, DPA), and compared against a range of different approaches including non-graph, graph-based and hypergraph-based ones.\n\nThe results show state-of-the-art performance across the board (an increase from 2 to 10 points over the next best method). \n\nThe authors also do an ablation study, showing the importance of each component and demonstrating the relative improvement.", "questions": "Similar to the limitations field, I'd be very interested in an explanation of how the computation complexity of Pattern-BERP scales with the number of entities and associations. \n\nIs it possible to provide a bit more theory behind the success of your method compared to others? The justification for the association pattern-aware fusion to me seems mostly intuitive rather than theoretical.\n\nIs there a way to extend the method to capture variable-length pathways?", "limitations": "It would be great to see a longer discussion on limitations: my primary concerns are complexity explosion and very large datasets. Would be great to see a bit of a theoretical analysis (or even empirical) on how it scales for millions of entities or associations (which are not rare in the domain).\n\nThe interpretability section focuses on only drug #52, it would be great to see that expanded, as making conclusions from one example is bit of a stretch. \n\nReal-world applications might be a bit of an overstatement, while it is possible, nothing was really tested or proven.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "A novel idea presented in an easy-to-understand way. Manuscript is nicely written and clear, with a broad impact. An extensive evaluation concerning methods, state-of-the-art results and a detailed ablation study.", "weaknesses": "This is a very general method but it was tested on only one dataset type and relatively small datasets. Would be great to see it expanded in the future.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for all the comments. We believe we have addressed all the concerns and are happy to follow up in the discussion phase.\n\n> W: Possibility to test on more dataset types and relatively huge datasets?\n\nA: We appreciate your insight. Firstly, we clarify that the limited associations in this study result from current research constraints, despite large Cartesian products in Appendix Table 3. Previous works [15-17,19] also used similar dataset types. To test on larger, diverse datasets, our approach is refined to capture variable-length patterns and applied to large-scale molecular property datasets. As shown in Table R2 (response PDF), the association patterns significantly enhance atom (analogous to entity nodes in this study) representations and improve prediction performance.\n\n> Q1: How the computation complexity of Pattern-BERP scales with the number of entities and associations?\n\nA: Thanks for your concerns. The most critical module is undoubtedly the APF, which aggregates multiple patterns across different entity nodes. This module includes Multi-Head Attention (MHA) and FeedForward Network (FFN). For an entity node with $N$ sampled patterns from the APS module, the input and hidden features in the MHA layer are of dimension $d$, and hidden features in the FFN layer are $f$, and the query, key, and value matrices are derived from the same input sequence and share length $N$. In APF, the primary operations include scaled dot-product attention, multiplication of attention weights and values, MHA linear transformation, and FFN linear projection. The time complexity is expressed as $\\mathcal{O}(N^2 \\cdot d + N \\cdot d^2 + N \\cdot d \\cdot f)$. Hence, for the entire graph, the total complexity is $\\mathcal{O}\\left(|\\mathcal{V}| \\cdot (N^2 \\cdot d + N \\cdot d^2 + N \\cdot d \\cdot f)\\right)$, where $|\\mathcal{V}|$ is the number of entity nodes. Moreover, Table R1 (response PDF) shows the GPU usage of Pattern-BERP scaling with the number of entities and patterns.\n\n> Q2: Theory behind the success of the method.\n\nA: Thanks for highlighting this point. Pattern-BERP is regarded as an empirical method. It extracts common patterns or rules from large bio-associated datasets, similar to k-means clustering. Given a triplet dataset $\\mathcal{D}=\\\\{\\mathbf{x}\\_1,\\mathbf{x}\\_2,\\ldots,\\mathbf{x}\\_n\\\\}$ and $k$ common patterns, our goal is to minimize the total distance of samples to their respective pattern centers within the same pattern. First, initialize the pattern centers $\\mathbf{m}\\_1,\\mathbf{m}\\_2,\\ldots,\\mathbf{m}\\_k$. Next, assign each sample to the nearest pattern center by computing $j=\\arg\\min_k \\\\|\\mathbf{x}\\_i-\\mathbf{m}\\_k\\\\|$. Then, update the pattern centers using $\\mathbf{m}\\_j=\\frac{1}{|C_j|}\\sum_{\\mathbf{x}\\_i \\in C_j}\\mathbf{x}\\_i$, where $C_j$ is the set of samples assigned to pattern $j$. Iterate these assignment and update steps until the pattern centers converge. The objective function to minimize is $J=\\sum_{j=1}^{k}\\sum_{\\mathbf{x}\\_i \\in C_j} \\\\|\\mathbf{x}\\_i - \\mathbf{m}\\_j\\\\|^2$, and by minimizing $J$, Pattern-BERP extracts representative biological association patterns effectively. In the future, additional corresponding theoretical analysis will be provided in detail.\n\n> Q3: Method extension to capture variable-length pathways.\n\nA: Thanks for constructive advice. As discussed this limitation in Appendix Section C, we strived to extend the method to capture variable-length patterns after the paper submission. Since now, the proposed method has been optimized with variable-length settings and thus applied on the molecular property prediction tasks to enhance the atom (in line with entity in this paper) representation in the molecular graph. Table R2 (response PDF) demonstrates the effectiveness of variable-length association patterns.\n\n> L1: Discussion on very large datasets and complexity explosion.\n\nA:  We understand your primary concern. Regarding computation complexity, the discussion in Q1 indicates that the decisive factors are the number of entity nodes $|\\mathcal{V}|$ and the number of sampled patterns $N$. \n\nHence, when encountering larger datasets or more complex biological networks, we can employ the following two strategies to avoid a computation complexity explosion: (1) Graph Sampling. By sampling smaller subgraphs [A], computational resource consumption reduces significantly, boosting algorithm speed and efficiency; (2) High-confidence Pattern Selection. As shown in Appendix Fig. 4 (g)-(i), reducing $N$ from 100 to 5 across three datasets slightly decreases performance but still surpasses these baselines. Thus, adjusting the sampling quantity is acceptable to effectively reduce time complexity.\n\nMoreover, Table R1 (response PDF) shows the GPU usage varying from the number of sampled pattern across three datasets with significant differences in the number of entities and associations. The inference time compared to baselines can be found in W3.2 of reviewer 4Nwa.\n\n> L2: Interpretability analysis of more cases.\n\nA: Thanks for your insight. There are two additional interpretation cases in Fig. R1 (response PDF), which shows the common patterns closely linked to the original drug node reflect the mechanism that acts on the cell wall and envelope and thus influences microbe physiological activities, thus treating diseases. In contrast, the low-contribution pattern causes cell death by inhibiting protein synthesis.\n\n> L3: Restate real-world applications of the proposed method.\n\nA: Thanks for pointing this out. Actually in Appendix Section C, we would have intended to discuss the possibility of applying our proposed APF-BERP in real-world scenarios and the corresponding limitations. Due to limited resource and time, we cannot indeed validate it with enough real tests or proofs. We will restate the confusing description about the real-world applications. \n\n\n\n**References**\n\n[A] Hu and Lau, A survey and taxonomy of graph sampling. arXiv, 2013.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jyKO7h76yv", "reviewer": "Reviewer_m3DY", "summary": "The author proposed a deep learning method, termed the Pattern BERP method, designed to elucidate the potential associations among triple-wise biological entities. This method innovativly incorporates association pattern information into the entity representation learning. Moreover, it employs two additional bipartite graphs to enrich the binary relation information extracted from triple entities. Evaluated on three biological datasets, the proposed method demonstrates superior performance compared to non-graph, graph-based, and hypergraph-based methodologies. This work is noteworthy not only for its impressive predictive accuracy but also for its potential applicability in real-world scenarios.", "questions": "1. The abbreviation \"APF\" (association pattern fusion) is not defined in the text at line 219. It is essential to provide the full name when first introduced for clarity.\n\n2. In equation (11), the LBE is not updated every epoch, which contradicts the description in the text.\n\n3. The manuscript features two bipartite graphs in Figure 2, yet it remains unclear whether these graph features are integrated with hypergraph features. Additionally, the role of the blue dotted line arrow, which is not discussed in the text, needs clarification to enhance understanding of the graphical representations.\n\n4. Although the association pattern-aware fusion is novel and central to the paper's title, its contribution for model performance seems not to be impressive. Specifically, while Table 2 shows significant impairment in performance on the DDC dataset without the association pattern interaction (API), Tables 3 and 4 for the DMD and DPA datasets suggest that the Binding-relation Feature Reconstruction (BFR) for bipartite graphs predominantly enhances performance. It may be misleading to attribute the main contribution of the work solely to APF given these findings.\n\n5. The model interpretation in Figure 3 lacks persuasiveness, as Drug #52, a peptide with numerous sub-group structures, is not ideal for visualization comparison with other compound drugs. Furthermore, in equation (3), the attention mechanism is applied across different triple entities, not just one. Analyzing only the drugs without considering the other two entities in the figures is insufficient. At minimum, the roles of the other two entities should be elucidated to provide a more comprehensive analysis.\n\n6. Equation 2 describes the generation method of the distance matrix D and the distance token 𝐷𝑣. How is the specific position embedding done on line 167? Is the initial pattern 𝑃 directly concatenated from the three nodes?\n\n7. In the bind-relation enhancement, why is only the bipartite graph between AB and BC considered? Why not consider AC? Not all multi-entity relationships have a clear chain pattern from A to B to C.\n\n8. Following up on question 7, how are relationships defined between two entities in Equations 7 and 8?", "limitations": "The model lacks practical application scenarios. It primarily learns from the interaction network of existing entities such as drugs, diseases, and symptoms. Such methods generally struggle to predict the relationships between a completely new entity and the existing entities in the network. Moreover, the relationships within the network are already well-known. The more important task is to uncover new and potential relationships. The comprehensiveness and accuracy of the existing relationships are merely the tasks designed by the model itself.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.\tThe concept of k-hop neighbors on a hypergraph has been defined, incorporating pattern-similar triples information as neighbors into each hypernode.\n\n2.\tThe information of triples has been decomposed into pairwise relationships, and these features have been injected into the backbone network.\n\n3.\tA parameter-based negative sampling technique has been introduced, which is more efficient compared to random sampling.", "weaknesses": "1. The interpretation provided for the model is not entirely persuasive. The analysis of the relationship between peptide-based micro-molecular drugs and small-molecular chemical drugs is unconventional and lacks commonality. \n\n2. While the concept of association pattern-aware fusion is highly innovative and appropriately highlighted as the main focus of the paper, its contribution for model performance seems not to be impressive, especially in Table 5 and Table 6.\n\n3. The model is overly complex, with multiple stages of tasks and losses, which may lead to difficulties in optimization.\n\n4. The role of the triple information has been simplified to a single chain pattern from A to B to C.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for all the comments. We believe we have addressed all the concerns and are happy to follow up in the discussion phase.\n\n> W1: The provided interpretation study lacks absolute persuasiveness. \n\nA: Thanks for your insight. There are two additional interpretation cases in Fig. R1 (response PDF), which shows the common patterns closely linked to the original drug node reflect the mechanism that acts on the cell wall and envelope and thus influences microbe physiological activities, thus treating diseases. In contrast, the low-contribution pattern causes cell death by inhibiting protein synthesis. We will provide detailed text descriptions for interpretation cases in the future.\n\n> W2: The contribution of APF seems not to be impressive in Table 5&6. Which module contributes more to three different datasets?\n\nA: Thanks for your valuable feedback. The situation arises due to significant differences in the distribution and sparsity of these three datasets (see Appendix Table 3). Table 2 shows that APF's contribution stands out in the DDC. However, in the relatively sparse DMD and extremely sparse DPA, there are fewer high-confidence patterns associated with entity nodes (some nodes link to only one pattern), reducing the APF module's effectiveness. In the future, as biological association datasets grow larger and denser, APF's pattern exploration will yield more expressive representations, making its contribution more impressive.\n\n> W3: Overly complex model architecture with multiple stages of tasks and losses.\n\nA: Thanks for concern. The core components are two complementary modules: Association Pattern-aware Fusion (APF), Bind-relation Enhancement (BE) with their corresponding losses. Due to the limited response characters, please refer to W1 of reviewer UqdA for details. \n\n> W4: The triple information is simplified to a single chain pattern from A to B to C.\n\nA: Thanks for pointing this out. In this study, the triplet relationship we investigate is defined as the A→B→C pattern due to dataset limitations, making it challenging to collect substantial longer physiological pathway data. As noted in the Limitation section, we plan to extend to longer pattern pathways and have already obtained some preliminary results.\n\n> Q1: Full name of APF when first introduced.\n\nA: Thanks for careful reading. The abbreviation APF denotes Assocaition Pattern-aware Fusion, which will be introduced in Line 149.\n\n> Q2: Ambiguity for total loss function in Section 4.4.\n\nA: Thanks for carefulness. The total function is revised by the rule: BE module trains for the first 4 epochs of each 5-epoch cycle; APF module trains for the final epoch. Please refer to W1 of reviewer UqdA for details.\n\n> Q3: Unclear description for bipartite graph feature in Fig. 2 and its caption.\n\nA: Thanks for pointing it out. The bipartite graph features are integrated with hypergraph features, represented by the blue dotted line arrow. Please refer to W3.3 of reviewer UqdA for details.\n\n>  Q4: The peptide is not ideal for visualization comparison with other compound drugs.\n\nA: We understand your concern. Additional cases in Fig. R1 (response PDF) are both discussed with small molecule drugs.\n\n> Q5: The roles of the other two entities should be elucidated in the interpretation study.\n\nA: Thanks for highlighting this aspect. Due to insufficient dataset information, we cannot obtain identifiers for Microbe and Disease entities, which are directly processed into feature vectors in the original DMD dataset [17]. In the future, we will focus on intrinsic interpretation using more comprehensive datasets.\n\n> Q6: Details about how the position embedding works (line 167). Is the initial pattern $P$ directly concatenated from the three nodes?\n\nA: Thanks for pointing it out. The purpose of designing this position encoding is to enable the model to recognize the relative position of the selected entity node in relation to these patterns. Please refer to W3.2 of reviewer UqdA for details.\n\nIn this work, each sampled pattern consists of three different types of entity nodes representing a reaction pathway. These patterns may contain common knowledge (see Fig. 2), aiding in learning more expressive representations for entity nodes. Additionally, in the Limitation section, we propose exploring variable-length patterns to identify more representative pathways.\n\n> Q7: Why not consider A→C in the bind-relation enhancement?\n\nA: Thanks for your concern. Our focus is on the triplet relationships like Drug→Protein→ADR discussed in the paper. Typically, a drug interacts with a target protein, triggering signals that cause various activities, some manifesting as ADRs like vomiting or diarrhea. Therefore, we did not consider a direct A→C relationship, as there is no connection in this context. However, we acknowledge your perspective on broader multi-pathway relationships where a direct A→C could exist. We will consider it for future tasks.\n\n> Q8: The defined relationships between two entities in Equations 7 and 8.\n\nA: Thanks for highlighting this aspect. In this paper, A→B and B→C have clear biological associations, reflected in the data through labels. However, the A→C relationship is determined by modeling new negative samples A→B→C (Bottom right of Fig. 2, Line 211-217).\n\n> L: The model struggle to predict the relationships between a completely new entity and the existing entities in the network, thus lacking practical application scenarios.\n\nA: Thanks for guidance on future direction. Our core goal is to uncover common association patterns between biological entities, which remain unchanged with the addition of new nodes. Conversely, by linking new nodes to existing ones through common patterns, we aim to uncover new potential relationships. Of course, we highly appreciate your viewpoint. The current model design indeed lacks the ability to introduce new entity nodes. We will improve it in the future to facilitate the practical application.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E1YjuXigNG", "reviewer": "Reviewer_4Nwa", "summary": "This paper presents a novel approach to predicting relationships among biological entities by addressing limitations in current deep learning methods that focus solely on entity-centric information. The authors introduce an association pattern-aware fusion method that integrates association pattern information into entity representation learning. Additionally, they develop a bind-relation module to enhance low-order message passing by considering strong low-order entity associations. Extensive experiments on three biological datasets demonstrate significant improvements compared to advanced baselines. The paper also provides detailed explanations of the interpretability of association patterns, highlighting intrinsic biological mechanisms and potential real-world applications.", "questions": "Please see the weaknesses.", "limitations": "They provide a limitation section in the appendix.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The introduction of an association pattern-aware fusion method is innovative. It effectively integrates association pattern information into entity representation learning, which is a novel approach compared to traditional entity-centric methods.\n- The method includes several interconnected modules such as association pattern sampling, pattern-aware interaction, and bind-relation enhancement. Extensive experiments on three biological datasets demonstrate significant improvements compared to advanced baselines.\n- The method provides detailed explanations of the interpretability of association patterns. This helps in understanding the intrinsic biological mechanisms, making the results more transparent and useful for real-world applications.", "weaknesses": "- The proposed model involves multiple modules and several hyperparemeters. Some hyperparameters (such as the number of attention head) are vary on different datasets. The authors may aslo need to add hyperparameter study for loss-balanced coefficient and the threshold for bind-relation prediction probability.\n- The approach may not scale well with increasing dataset sizes or more complex biological networks due to its computational requirements and the need for extensive pattern sampling and interaction modeling.\n- The proposed model involves multiple components such as hypergraph convolution, pattern-aware fusion, and bind-relation enhancement, which can be computationally intensive. This might limit its applicability for large-scale real-world datasets or in environments with limited computational resources. The author may also need to provide inference speed study with some of the baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for all the comments. We believe we have addressed all the concerns and are happy to follow up in the discussion phase.\n\n> W1.1: Some hyperparameters (such as the number of attention heads) vary on different datasets. \n\nA: We appreciate your valuable feedback. Actually in the machine learning field, performing grid search for hyperparameters is a common practice across different datasets or tasks to achieve optimal results. Therefore, given the significant differences in distribution and association sparsity among datasets, it is entirely reasonable for us to obtain distinct optimal hyperparameter sets, including the number of attention heads mentioned by the reviewer.\n\n> W1.2: More hyperparameter studies for loss-balanced coefficient and the threshold for bind-relation prediction probability.\n\nA: Thanks for your careful reading. We provide additional hyperparameter studies as follows:\n\n* Loss-balanced coefficient $\\alpha$. Since the final prediction task involves predicting the associations among entity A, B, and C, we intuitively consider the two tasks of A→B and B→C to be of equal importance and thus the $\\alpha$ is set to 0.5 by default. Here, we attempt to adjust $\\alpha$ from 0.1 to 0.9, and the results are shown below (the hits@1 with default $\\alpha$=0.5 is 93.94). The hits@1 performance of the default setting surpasses other ablation settings.\n| coefficient $\\alpha$ | hits@1 | coefficient $\\alpha$ | hits@1 |\n| -------------------- | ------ | -------------------- | ------ |\n| 0.1                  | 92.91  | 0.6                  | 92.79  |\n| 0.2                  | 93.87  | 0.7                  | 93.25  |\n| 0.3                  | 93.86  | 0.8                  | 93.66  |\n| 0.4                  | 93.32  | 0.9                  | 93.23  |\n\n* Threshold $\\gamma$ for bind-relation prediction probability. Bind-relation prediction is fundamentally a binary classification task, so setting the threshold to 0.5 by default is quite common. Here, we also attempt to adjust the threshold $\\gamma$ from 0.1 to 0.9, and the results are shown below (the hits@1 with default $\\gamma$=0.5 is 93.94). The hits@1 performance of the default setting surpasses other ablation settings. It is noteworthy that when $\\gamma$=0.1, the generation of negative samples is extremely hard as defined in Eq. 9, and thus the BE module cannot provide valid hard samples. \n| threshold $\\gamma$ | hits@1           | threshold $\\gamma$ | hits@1 |\n| ------------------ | ---------------- | ------------------ | ------ |\n| 0.1| No Valid Samples | 0.6| 92.17  |\n| 0.2| 91.88 | 0.7| 93.67  |\n| 0.3| 92.91 | 0.8| 93.88  |\n| 0.4| 93.48 | 0.9 | 93.81  |\n\n> W2: Computation complexity explosion for huger dataset or more complex biological networks.\n\nA: Thanks for your valuable insight. As discussed in Q1 by reviewer ugjG, the total complexity of the entire graph is denoted as $ \\mathcal{O}\\left(|\\mathcal{V}| \\cdot (N^2 \\cdot d + N \\cdot d^2 + N \\cdot d \\cdot f) \\right)$, where $|\\mathcal{V}|$ represents the number of entity nodes, $N$ represents the number of sampled patterns,  the dimensions of all input pattern features and hidden features in MHA layer are $d$ and the hidden features in FFN layer is $f$. It is evident that computational complexity is primarily determined by the number of entity nodes $|\\mathcal{V}|$ and the number of sampled patterns $N$. \n\nHence, when encountering larger datasets or more complex biological networks, we can employ the following two strategies to avoid a computation complexity explosion: (1) Graph Sampling. Processing large graph data requires significant computational resources. By sampling smaller subgraphs [A], computational resource consumption can be significantly reduced, improving the speed and efficiency of algorithms; (2) High-confidence Pattern Selection. As shown in Fig. 4 (g)-(i), reducing $N$ from 100 to 5 across three datasets slightly decreases performance but still surpasses these baselines. Thus, adjusting the sampling quantity is acceptable to effectively reduce time complexity.\n\nFurthermore, to test on larger and diverse datasets, our approach is refined to capture variable-length patterns and applied to large-scale molecular property datasets. As shown in Table R2 (response PDF), the association patterns significantly enhance atom (analogous to entity nodes in this study) representations and improve prediction performance.\n\n> W3.1: Limitation of applying in large-scale real-world datasets or environments with limited computational resources.\n\nA: Thanks for your further concern regarding the real-world scenarios. As discussed in W2, the computation complexity can be controlled through some reasonable strategies, such as graph sampling and pattern selection. Moreover, substantial computational resources are typically deployed on central servers in the field of bioinformatics, unlike the field of computer vision, which often requires deployment on resource-constrained devices.\n\n> W3.2: Inference speed study compared to baselines.\n\nA: Thanks for highlighting this aspect. We provide the inference speed comparison with other methods as follows and present the inference time of each 100 samples with milliseconds. Compared to the baseline methods, the difference in inference time for our method is constant and even faster than several baselines.\n\n| Method | DMD (ms) | DDC (ms) | DPA (ms) |\n| ---------------- | -------- | -------- | -------- |\n| RF | 0.36|0.66| 0.57 |\n| MLP | 0.18| 0.21 | 0.26 |\n| CP| 8.98| 9.19 | 10.25|\n| Tucker| 7.92 | 8.84 | 7.96|\n| CoSTCo| 0.75 | 1.51| 3.15|\n| GCN | 0.80 | 0.99| 1.95|\n| GraphSAGE | 0.81 | 0.95| 1.92|\n| GAT | 1.18 | 1.61 | 3.11 |\n| GIN | 0.72| 1.14 | 1.86|\n| DHNE  | 0.29 | 0.29 | 0.39|\n| HyperSAGNN | 0.63 | 0.69| 0.56|\n| HGSynergy | 0.64 | 0.89| 1.55 |\n| MCHNN (CPU-only) | 8.65 | 9.54 | 13.72 |\n| Pattern-BERP | 1.28 | 1.24| 2.96|\n\n\n\n**References**\n\n[A] Hu and Lau, A survey and taxonomy of graph sampling. arXiv, 2013.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9TebpFuMBs", "reviewer": "Reviewer_UqdA", "summary": "The paper introduces an innovative perspective, namely association pattern, for biological entity relationship prediction (e.g., drug-target protein-adverse reaction). The authors first summarize the characteristics of existing perspectives and emphasize the importance of the association pattern. Next, the association patterns of certain entity nodes are sampled in the constructed graph based on the defined distance and then serve as tokens for self-attention to fuse similar semantic information. Additionally, the bind-relation enhancement module can effectively reconstruct the missing feature and thus generate hard negative samples for further improving the performance. Extensive experiments on biological datasets demonstrate the effectiveness of the proposed method.", "questions": "Q1: The paper does not give a specific explanation of $L_{BE}$ (see Line 224). Please give a specific explanation of $L_{BE}$.\n\nQ2: In Table 1, the authors appear to have not reported the statistical measures like standard error.", "limitations": "The authors have adequately discussed the limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The perspective of triple-wise association patterns, which takes the actual meaning of each entity and their connections into account, is novel and explainable.\n2. The paper comprehensively discusses the advantages of the proposed method compared with existing non-graph, graph, and hypergraph perspectives. Moreover, the related works on biological entity relationship prediction and path-based graph mining are elaborated.\n3. The paper provides a clear and detailed description of each part of the method, and the formulas used in the paper seem correct. The overall design of the method is rather reasonable.\n4. Extensive experiments on biological datasets demonstrate the effectiveness of the proposed method. The reveal of the intrinsic biological mechanism can indeed promote the deployment of the method in real-world scenarios.\n5. The association pattern-aware perspective appears to apply to commonly used graph benchmarks or tasks, which can be leveraged to enhance existing graph-based methods.", "weaknesses": "1. The bind-relation feature reconstruction appears to be a standalone component that requires extra separate training, resulting in a two-stage process for the method. Additionally, such a design may influence the entire optimization process and distract the training flow, how to alleviate that?\n\n2. Generalization and extension issues have not been discussed. The pattern's length is fixed to 3, which may make the method not work when facing longer-range relationships. Are there any feasible solutions for sampling patterns with different lengths?\n\n3. Complexity has not been discussed. Some details are also not clear enough, such as in Line 167, could the authors provide details on how the position encoding is designed? In Fig. 2, the meaning of the dotted line \"Bind-relation Feature\" needs more explanation. The \"add\" operation is not reflected in Section 4.3. Please explain the meaning of the dotted line \"Bind-relation Feature\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for all the comments. We believe we have addressed all the concerns and are happy to follow up in the discussion phase.\n\n> W1: The bind-relation feature enhancement module needs extra separate training?\n\nA: Thanks for pointing this out. Our description may not be entirely clear, and the overall architecture is actually quite straightforward. The core components are two complementary modules: Association Pattern-aware Fusion (APF, Section 4.2), Bind-relation Enhancement (BE, Section 4.3) with their corresponding losses. \n\n* Training strategy. Actually, the method does not adopt two completely independent training stages, but rather achieves it through an alternating training strategy. Specifically, the BE module is trained for the first 4 epochs of every 5-epoch cycle, followed by the APF module, which is trained for the final epoch of each cycle. During the training of each module, the parameters of the other one are frozen. Moreover, this design satisfies the need of generating hard negative samples for the APF module, hence the BE module requires initial and additional training.\n* Loss funtions. The BE losses (Eq. 7 & Eq. 8) are utilized to supervise the reconstruction of missing pairwise feature and the generatation of hard negative samples. Consequently, the BE module necessitates additional initial training to prevent low-confidence negative samples for the APF module. Following this, the supervised loss $\\mathcal{L}_{APF}$ (Eq. 10) is utilized to determine the positive triplet samples and these negative samples generated from the BE module.\n\nHence, the total loss function is formulated as  $\\mathcal{L}(o) = \\mathcal{L}\\_{\\text{BE}}(o) \\cdot (1 - \\left\\lfloor \\frac{o \\mod 5}{4} \\right\\rfloor) + \\mathcal{L}\\_{\\text{APF}}(o) \\cdot \\left\\lfloor \\frac{o \\mod 5}{4} \\right\\rfloor$, where $o$ represents the current epoch; $\\lfloor \\cdot \\rfloor$ denotes the floor function; $\\text{mod} $ denotes the modulo operation. We will make the description of the entire architecture more clear for understanding.\n\n> W2: Generalization issues about sampling patterns with different lengths.\n\nA: Thanks for advice. As shown in Table R2 (response PDF), the proposed method has been extended with variable-length settings and thus applied on the molecular property prediction tasks. See Q3 of reviewer ugjG for details.\n\n> W3.1: Computation complexity.\n\nA: Thanks for concern. The computation complexity depends on the number of entity nodes and sampled patterns, and we discuss the solution for complexity explosion. Please refer to Q1 & L1 of reviewer ugjG for details.\n\n> W3.2: Details about how the position encoding designs.\n\nA: Thanks for your feedback. First, we would like to clarify that the purpose of designing this position encoding is to enable the model to recognize the relative position of the selected entity node in relation to these patterns. This function is analogous to how the Transformer [A] and ViT [B] architectures handle the relationship between \"tokens/patches and their positions.\"\n\nSpecifically, in Association Pattern Sampling (APS) block, the $N$ sampled patterns are assigned to certain node $v$ according to the defined distance. On the one hand, each sampled pattern consists of three entity nodes with $F$-dimension feature embedding, and thus the feature embedding of each pattern is denoted as $\\mathbb{R}^{3F}$, which represents the concatenation of the original node feature embeddings, thereby obtaining the feature vector $\\mathbf{P}_v \\in \\mathbb{R}^{N \\times(3F)}$ of all the $N$ sampled patterns; on the other hand, each sampled pattern has a distance label defined in Line 154-158 with node $v$. To connect the pattern feature embeddings with the corresponding position information $\\mathbf{D}_v \\in \\mathbb{R}^N$, we provide a position encoding layer $\\text{POS}(\\cdot)$ which represents a map function from $\\mathbb{R}^N$ → $\\mathbb{R}^{N \\times(3F)}$, and finally integrate $\\mathbf{P}_v$ and $\\text{POS}(\\mathbf{D}_v)$, thereby outputting the enhanced embedding for the next Association Pattern-aware Interaction (API) block.\n\n> W3.3: Explanations for \"Bind-relation Feature\" and \"add\" operation in Fig. 2.\n\nA: Thanks for detailed observation. Line 194-199 shows two main functions of the BE module: (1) to reconstruct the missing pairwise feature; (2) to generate hard negative samples. The \"Bind-relation Feature\" in Fig. 2 actually denotes the above missing pairwise future. Hence, the blue dotted line \"delivers\" the BE pairwise feature to the APF module, and the icon $\\oplus$ \"integrates\" the two feature embeddings from the BE module and the APF module. Finally, the enhanced feature embedding is fed into the predictor. The above integrating process has been mentioned in Line 219-220, and we will highlight the description with a more separate and clear paragraph.\n\n> Q1: Specific explanation of $\\mathcal{L}_{BE}$ (Line 224).\n\nA: Thanks for careful reading and feedback. The $\\mathcal{L}_{BE}$ actually denotes the supervised loss of the BE module. The calculation is defined in Line 209, which is equal to the $\\mathcal{L}_2$. We will modify $\\mathcal{L}_2$ in Line 209  to $\\mathcal{L}\\_{BE}$.\n\n> Q2: Standard error in Table 1?\n\nA: Thanks for the concern regarding statistical measures. As mentioned in the Appendix B.3.1 and Checklist Q7, each result of these methods in Table 1 is calculated from the average of 5-fold cross-validation experiments with four different scenarios. However, the significant variation in prediction difficulty across different scenarios makes it relatively unreasonable to provide an error bar for all 20 results. Hence, to demonstrate the statistical validity of Pattern-BERP, we have provided all the raw experimental results in Supplementary Materials.\n\n\n\n**References**\n\n[A] Vaswani et al., Attention is all you need. NeurIPS, 2017.\n\n[B] Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR, 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for the time spent reviewing the paper and recognizing the **significance** ("with a broad impact" – ugjG, "noteworthy" & "impressive predictive accuracy" & "more efficient" – m3DY, "significant improvements" – 4Nwa), **novelty** ("novel idea" – ugjG, "highly innovative" – m3DY, "innovative" & "a novel approach" – 4Nwa, "novel and explainable" & "reasonable" – UqdA) of our research direction, the **quality of the presentation** ("nicely written and clear" & "presented in an easy-to-understand way" – ugjG, "clear and detailed" & "comprehensively discusses" – UqdA), and **applicability** ("potential applicability in real-world scenarios" – m3DY, "more transparent and useful for real-world applications" – 4Nwa).

We have endeavored to consider the feedback as comprehensively as possible, leading to a revision process that significantly honed the paper. We have addressed every point in our responses and are happy to follow up on any aspect during the discussion phase. Specifically, we have tackled stated weaknesses (W), questions (Q), and limitations (L) with detailed answers (A). Additionally, we have included a single-page PDF containing extra referenced figures and tables to further clarify our points.

Furthermore, we have strived to balance making all necessary changes within the given space constraints. However, due to the response length limitations, our answers to some common but important problems that require detailed explanation may refer to the response provided to another reviewer, indicated by the statement "please refer to the response for another reviewer." We sincerely apologize for any inconvenience this may cause.

Finally, we would like to express our appreciation once again for the reviewers' constructive comments and careful reading, which undoubtedly lead to enhancing the quality of our work.

Meta-review: This work seeks to predict relationships between biological entities,  via an association pattern-aware fusion method that integrates association pattern information into entity representation learning.  It is a solid piece of work - the    association pattern-aware fusion method is creative.
Decision: Accept (poster)