Association Pattern-aware Fusion for Biological Entity Relationship Prediction

Authors: lingxiang  jia
yuchen  ying
zunlei  feng
zipeng  zhong
shaolun  yao
jiacong  hu
mingjiang  duan
xingen  wang
jie  song
mingli  song
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 08:48:49

Abstract: Deep learning-based methods significantly advance the exploration of associations among triple-wise biological entities (e.g., drug-target protein-adverse reaction), thereby facilitating drug discovery and safeguarding human health. However, existing researches only focus on entity-centric information mapping and aggregation, neglecting the crucial role of potential association patterns among different entities. To address the above limitation, we propose a novel association pattern-aware fusion method for biological entity relationship prediction, which effectively integrates the related association pattern information into entity representation learning. Additionally, to enhance the missing information of the low-order message passing, we devise a bind-relation module that considers the strong bind of low-order entity associations. Extensive experiments conducted on three biological datasets quantitatively demonstrate that the proposed method achieves about 4%-23% hit@1 improvements compared with state-of-the-art baselines. Furthermore, the interpretability of association patterns is elucidated in detail, thus revealing the intrinsic biological mechanisms and promoting it to be deployed in real-world scenarios. Our data and code are available at https://github.com/hry98kki/PatternBERP.

Main paper: 
 introduction: 
 Exploring potential associations among triple-wise biological entities (e.g., drug-target proteinadverse reaction) [1] [2] [3] [4] [5] [6] holds significant implications for elucidating underlying biological mechanisms and advancing personalized therapies [7] [8] [9] , thus promoting pharmaceutical innovation and ensuring human health. Recent deep learning-based methods have propelled auxiliary prediction tasks concerning biological entity relationship, with most focusing on binary associations (e.g., drug-target protein), while only a few methods offer insights for more complex triple-wise associations. Existing solutions for the association prediction task can be broadly categorized into three types: (1) non-graph methods [10] [11] [12] ; (2) graph-based methods [13] [14] [15] [16] ; (3) hypergraph-based methods [17] [18] [19] [20] [21] [22] .As illustrated in Figure 1 , non-graph methods typically concatenate the features of different entities, which are independently mapped by their respective entity encoders, to serve as representations. Graph-based methods adopt nodes and edges of the graph to represent entities and their relationships, and leverage the graph structure for feature propagation, thereby achieving effective representation learning of entity nodes. Similar yet distinct, hypergraph-based methods employ the hypergraph structure to obtain entity representations using the complex feature aggregation strategy. However, none of the aforementioned methods consider the significance of path patterns in the graph structure, which contain a vast amount of crucial information including hidden context and co-occurrence. hypergraph-based methods, and the proposed association pattern-aware method. Unlike existing methods that map or aggregate node features, the proposed method mines and then fuses association patterns for each target entity node in the graph to enhance the model's representative ability.To this end, we introduce a novel association pattern-aware message propagation strategy as shown in Figure 1(d) . The module leverages the potential relationships, such as commonality and diversity, of association patterns as the rule for facilitating message passing among entity features, which can efficiently expand the potential for representing complex interactions within the perspective of both basic graph structure and intrinsic biological mechanisms. Specifically, the related patterns within the graph structure are sampled through the pre-defined distance relation for each entity node. The message passing is driven by the interaction of its assigned patterns, i.e., the feature of the entity node is subsequently updated through feature fusion using adaptive coefficients that capture pattern commonality, generated during the pattern interaction stage. This process is followed by the acquisition of potential common patterns with genuine biological significance for various entities.In this paper, we propose a novel Association Pattern-aware Fusion method for Biological Entity Relationship Prediction, namely Pattern-BERP. First, we devise an association pattern-aware strategy to solve the limitation caused by entity-centric feature mapping and aggregation. The strategy utilizes the association patterns related to each entity node within the graph to extract the common feature based on the attention mechanism for these patterns, thus expanding the ability to represent hidden complex interactions. In addition, to preserve the information interaction of different entities, a hypergraph-based block is incorporated with the association pattern-aware fusion module, thereby enhancing the model ability to capture relationships among various types of entities. Furthermore, to explore low-order associations of biological bind relations, we introduce a bind-relation enhancement module which can reconstruct missing feature of bind-relation entities and thus generate harder negative sample than random selection. Experimental results conducted on different biological datasets show that the proposed method achieves superior performance compared to advanced baselines, demonstrating its effectiveness and robustness in handling various biological entity relationships. More importantly, the obtained association patterns for the relationship of drug, microbe, and disease are quantitatively visualized with the following biological verification in detail.Our main contributions are summarized as follow:• We propose a novel association pattern-aware fusion method for biological entity relationship prediction. The introduce of association pattern-aware strategy can enhance the representation of complex interactions by aggregate features with potential association patterns.• A bind-relation enhancement module is devised to acquire low-order associations that reveal the biological bind relations, which is essential for reconstructing missing bind-relation entity features and generating challenging negative triplets to enhance the model training.• Extensive experiments are conducted to verify the superiority of Pattern-BERP, demonstrating its robustness for various biological scenarios. Uniquely compared to the other methods, the interpretability of association patterns is explained to reveal intrinsic biological mechanisms. 
related work: 
 In this section, we elaborate on the related work from two distinct yet interconnected perspectives: Biological Entity Relationship Prediction and Network Search and Mining. Each perceptive represents a fundamental aspect of our research, addressing specific challenges and methodologies in applying machine learning techniques to the prediction task.Biological Entity Relationship Prediction. The latest advancements in artificial intelligence have motivated researchers to employ deep learning methodologies for predicting triple-wise biological entity associations. Hypergraph neural network (HGNN)-based methods [17] [18] [19] [20] [21] [22] have become the mainstream research direction in this field. Tu et al. [17] proposed a deep hyper-network embedding model to preserve both local and global proximities in the embedding space. Building upon this, Jiang et al. [18] incorporated a dynamic hypergraph construction strategy to capture the hidden and important relations in data structures. Zhang et al. [19] developed a self-attention-based graph neural network applicable to homogeneous and heterogeneous hypergraphs with variable hyperedge sizes. Liu et al. [20] proposed a multi-way relation-enhanced hypergraph representation learning method to predict anti-cancer drug synergy. Liu et al. [21] proposed a multi-view contrastive learning-enhanced hypergraph model for drug-microbe-disease association prediction. In addition, Chen and Li [23] attempted to adopt the tensor decomposition strategy to predict which target a drug binds to when administered to a disease, and further proposed a neural tensor network model [24] that seamlessly combines tensor algebra and deep neural networks to effectively capture the complex nonlinear dependencies among drugs, targets, and diseases.Network Search and Mining. Network search and mining techniques, particularly those utilizing path information including random walks [25] [26] [27] [28] and meta-path [29] [30] [31] [32] , have been widely employed to extract local structural information from networks. These methods have found applications in areas such as content recommendation and community detection [33] [34] [35] [36] [37] . Brin and Page [25] introduced a classic ranking algorithm PageRank to determine the importance of web pages based on their link structure. Jeh and Widom [26] adopted a similarity measure based on pairwise random walk, which can capture the structural similarity between nodes, and further extended PageRank with a personalized version [27] . Perozzi et al. [28] proposed Deepwalk that leverages local random walk information to learn vertex latent representations based on deep learning techniques. These above methods are applicable to homogeneous networks and cannot fully utilize the rich semantic information in heterogeneous networks. To address the limitation, Sun et al. [29] introduced a meta path-based similarity framework for heterogeneous information networks, which can capture the subtle semantics of similarity among objects of the same type. Dong et al. [30] proposed a deep learning-based heterogeneous network representation learning method that automatically learns hidden meta-path semantics, generating general node embedding representations. Wang et al. [31] introduced a graph-based fraud detection method that addresses the issue of low homophily by integrating label information to generate distinguishable neighborhood information. Furthermore within the bioinformatics field, Chen et al. [32] proposed a computational algorithm that performs random walks on an integrated network to infer potential relations between proteins and ADRs.In line with the above methods, Pattern-BERP utilizes path information for triple-wise heterogeneous biological network mining. By leveraging the fusion of association patterns, it facilitates message passing among various biological entities, which will be described comprehensively in Section 4.2.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: See "Limitations and Future Work" part in the Appendix C.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
problem formulation: 
 Given three distinct entity types in biological networks, termed asA = {a 1 , a 2 , • • • , a i , • • • , a |A| }, B = {b 1 , b 2 , • • • , b j , • • • , b |B| }, and C = {c 1 , c 2 , • • • , c m , • • • , c |C| }, their Cartesian product S = A × B × C is a setof all possible triple-wise biological entity associations. For simplicity, the relation A-B-C is used to represent complex relational semantics for biological entities, such as "drug-microbe-disease", "synergistic drug-drug-cell line" or "drug-target protein-adverse reaction".For each triplet (a i , b j , c m ) ∈ S, we assign a label p ∈ {0, 1}. A label of p = 1 indicates that the existence of certain association has been confirmed, while p = 0 represents an unknown association which denotes that the association is not yet known and could potentially exist. The objective is to develop a credible model that can predict these potential associations from unknown ones. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
pattern-berp: 
 To enhance the ability to represent complex interactions, we propose the first association patternaware method, termed as Pattern-BERP to extract the rich semantic information embedded within the intricate structures of biological networks. As illustrated in Figure 2 , the section is divided into four parts: First, these entity relationships are represented in hypergraph and bipartite graph structures for subsequent module input. Next, the association pattern-aware fusion module is proposed to update entity node feature through association pattern-aware interaction. In addition, the bind-relation enhancement module is introduced to reconstruct bind-relation feature and thus generate hard negative samples. Finally, the detailed summary of loss function and complexity analysis is provided.Given the adjacency relationships among entity nodes, the hypergraph and bipartite graph structures are constructed to facilitate the subsequent extraction of structural information and relational association patterns within the respective graphs, which serve for association pattern-aware fusion module and bind-relation enhancement module.The attributes of each entity are initialized as node features on the graphs with the domain knowledge of bio-entities through its own type. Finally, the initialized node attributes of X ∈ R |V|×d consist of features X A , X B and X C , where d denotes the feature dimensional of initialized entity attributes. Details about entity attribute are provided at Appendix A.1.Hypergraph Construction. Triple-wise biological entity associations can be modeled as a hypergraph G = (V, E), which includes a vertex set V = A ∪ B ∪ C and a hyperedge set E ⊂ S that represents all known associations. Technically, G is further formulated as an attributed hypergraph with node attributes X ∈ R |V|×d and an incidence matrix H ∈ {0, 1} |V|×|E| , which is defined as:EQUATIONBipartite Graph Construction. The premise of constructing the bind-relation module is to decompose the original triple-wise associations to separately obtain pairwise relations of different types of entities. Since the original triplet is in the form of (a i , b j , c m ), and generally the interaction relationships between the three entities are hierarchical (e.g. drug a i acts on protein b j , and the activated b j then leads to certain adverse reaction c m ). Hence in this paper, we construct two bipartite graphs: G A,B1 for entity A → entity B 12 , G B2,C for entity B 2 → entity C. Then the edge sets of the bipartite graphs can be formulated as:EQUATIONNote that, the relationship of A → C is not under consideration due to there is no direct connection in this context. The constructed bipartite graphs G A,B1 , G B2,C serve as input for bind-relation module.To comprehensively account for the feature interactions within association patterns, the proposed Association Pattern-aware Fusion (APF) method comprises two fundamental components: Firstly, Association Pattern Sampling (APS) block is designed to sample association patterns by utilizing the distance tokens relevant to target nodes. Secondly, Association Pattern-aware Interaction (API) block is introduced to update node features by message interaction within the sampled association patterns and mine the intrinsic pattern commonality with biological support.Definition 1. Given certain entity node, the distance token between the node and one hyperedge or association pattern is defined as u-hop. Here, 1-hop patterns represent the hyperedges directly covering the node, 2-hop patterns represent the hyperedges directly covering all the 1-hop neighbor nodes of the node, and the relation continues for higher hop counts. If the node is unreachable when u reaches the max step U, we define these patterns as no-relation and set the distance to -∞.Based on the above definition 3 , we generate the distance tokens between each node and all hyperedges, ultimately obtaining a distance matrix D ∈ R |V|×|E| , defined as follows:EQUATIONwhere d(v, e) represents the defined distance from node v to pattern e; u indicates the number of hops from node v to pattern e.To represent an entity node with association patterns, the general principle is to prioritize and retain patterns that are closer for each node based on the distance tokens, with a total of N patterns sampled. Formally, given an entity node v, let D v ∈ R N and P v ∈ R N ×3d represent the distance tokens and feature vectors of the selected sampled patterns, where each sampled pattern consists of three entity nodes and the d-dimensional feature embeddings of each node are defined within the initial node embeddings X. Then, the output pattern feature z v for node v considers the relative position of these related patterns, which is thus formulated as z v = P v + POS(D v ), where POS(•) denotes the position encoding layer that maps from R N → R N ×(3d) . Finally, the integrated embedding z v ∈ R N ×3d with distance information is produced for use in the subsequent API block.Before put into the API block, we adopt a hypergraph convolution layer implemented by [40] on the constructed hypergraph G to facilitate neighbor-based information propagation among different entities. Then the updated node feature X * after hypergraph convolutions is used to construct the feature of patterns mentioned in the APS block, thereby yielding z v for the information interaction of these sampled patterns. Details of hypergraph convolution are provided at Appendix.Pattern Interaction. The API block is designed to search for and extract commonalities among different sampled patterns related to a specific entity node, and thus consists of a composition of Transformer layers [38] . Each Transformer layer has two modules: a multi-head self-attention mechanism (MHA) and a position-wise feed-forward network (FFN). For simplicity, we consider the single-head setting, and the extension to multi-head attention is standard and straightforward. Specifically, letEQUATION|V| ] ∈ R |V|×N ×(3d) denotes the input of the self-attention module where z v ∈ R N ×(3d) is the representation for node v. The input z v is projected by three matrices W Q ∈ R (3d)×d K , W K ∈ R (3d)×d K , and W V ∈ R (3d)×d V to obtain the corresponding query, key, and value representations Q v , K v , and V v for node v:EQUATIONwhere A v ∈ R N ×N is a matrix capturing the similarity between queries and keys; d K , d V denotes the feature dimensional of the key representations K v and the value representations V v4 . Then we will get the output of the self-attention module z ′ v ∈ R N ×(3d) . To summarize the process of transformer layer, the output of association pattern-aware attention block is computed as:EQUATIONwhere LN denotes layer normalization, and Z (l) is the output of the current transformer layer. After L 1 -layer transformers, we get the encoding output 3d) , and then apply the mean function to similar entities in N association patterns for each node v to obtain the learned embeddingZ (L1) = [z (L1) 1 , . . . , z (L1) |V| ] ∈ R |V|×N ×(z v ∈ R d .Pattern Commonality. To mine the commonality of N sampled patterns, a score is defined to represent the quantitative relation of these association patterns, termed by Pattern Commonality Coefficient, based on the attention scores A v of trained API block, which is formulated as follows:EQUATIONwhereC v ∈ R N and C v [n] ∈ (0, 1)indicates the commonality coefficient of the n-th pattern. Patterns with relatively high commonality coefficients tend to share the same or similar pathways, while showing significant differences in response compared to low commonality patterns. Corresponding biological validations are presented in the Section 5.3.In the context of multi-entity relationships, there often exist strong pairwise bind relation between entities, such as drug Aspirin to treat common cold [41, 42] . To mitigate the weakening or overlooking of low-order bind relation, a Bind-relation Enhancement (BE) module is designed to effectively reconstruct the missing feature by capturing these important pairwise associations. Additionally, the module can generate confident and challenging negative samples to aid the training.Bind-relation Feature Reconstruction. To efficiently learn entity representations in pairwise bind relations, we introduce an edge prediction classification task on the bipartite graphs E A,B , E B,C with the initial embeddingsEQUATIONEQUATIONC = X C . Specifically, a L 2 -layer selfsupervised BGNN model [43] is employed to learn node features on the bipartite graphs, followed by the Multi-layer Perception (MLP) [10] layer to output association probabilities, which is defined as:EQUATIONwhereEQUATIONC represent the node features at layer l; x(L2) a , x (L2) b1 , x (L2) b2 , x (L2) cdenote the final learned representations of entity a, b 1 , b 2 , c respectively; p(a,b1) , p(b2,c) represent the estimated probability of association between entities a, b 1 and entities b 2 , c respectively. After that, the loss of the supervised prediction task can be formulated as:EQUATIONThe loss L BE of bind-relation task is defined asL BE = αL A,B1 + (1 -α)L B2,C, where α is the balancing coefficient for the two losses.Hard Negative Sampling. Based on the above bind-relation task, negative samples are adaptively generated for triple-wise associations, instead of randomly selecting from the vast sample space. Moreover, the generated negative samples are challenging, which contributes to efficient learning. As illustrated in Figure 2 , three kinds of negative samples are considered as follows:EQUATIONwhere b ′ ∈ B represents another entity with random selection that differs from entity b in the original triplet to form the negative sample (a, b ′ , c); E ⊗ is the set of generated negative triplets; γ represents the threshold for the prediction probability to determine whether the association exists.Base on the above modules, z v of node v learned by the APF module is updated with reconstructed features x v from the BE module according to the entity type, thereby generate the enhanced embedding of v with z * v = z v + x v for the association predictor network. Hence, for the triple-wise association prediction, we utilize the learned embeddings z * a , z * b , and z * c of entity a, b, c to output the probability of the association p through a scoring function p(a,b,c) = MLP(z * a ∥ z * b ∥ z * c ).The loss of the association prediction task for true sample set E and negative sample set E ⊗ can be formulated as:EQUATIONHence, the total loss L is computed through an alternating training strategy of the two modules, where the BE module is trained for the first 4 epochs of every 5-epoch cycle, followed by the APF module, which is trained for the final epoch of each cycle. During the training of each module, the parameters of the other one are frozen. The equation for L is defined as follows:EQUATIONwhere o represents the current epoch; ⌊•⌋ denotes floor function; mod denotes modulo operation.Considering the significantly higher complexity of the APF module in comparison to other network components, we only consider APF which aggregates multiple patterns across various entity nodes. Specifically, for one entity node with N sampled patterns from the APS module, the input and hidden features in the MHA layers are of dimension f M , and hidden features in the FFN layers are f F . In APF, the query, key, and value matrices are derived from the same input sequence and share length N . The primary operations for APF include scaled dot-product attention, multiplication of attention weights and values, MHA linear transformation, and FFN linear projection. The time complexity is expressed asO(N 2 • f M + N • f 2 M + N • f M • f F ). Hence, for the entire graph, the total complexity is O |V| • (N 2 • f M + N • f 2 M + N • f M • f F ), where |V| is the number of entity nodes. 
experimental settings: 
 Datasets. In this paper, we adopt three biological entity association datasets with significant biological meaning, namely DMD (Drug-Microbe-Disease), DDC (synergistic Drug-Drug-Cell line) and DPA (Drug-target Protein-Adverse reaction), among which DPA dataset is directly constructed. In line with DMD and DDC, we utilize preprocessing tools provided by [21] to deal with the original data from ADReCS-Target [44] , and collect a total of 1,079 triplets that are structured into the data schema <drug, protein, adr>. Appendix Table 3 presents the statistics and characteristics of these datasets.Baselines. To verify the effectiveness of Pattern-BERP, we compare it with three kinds of methods:(1) Non-graph methods. Following the work from [21] , five non-graph methods are adopted including Random Forest (RF) [45] , MLP [10] , CP [11] , Tucker [11] , CoSTCo [12] ; (2) Graph-based methods. We select four classical architectures of graph neural network, including GCN [13] , GraphSAGE [14] , GAT [15] , GIN [16] ; (3) Hypergraph-based methods. Recent hypergraph learning methods to address triple-wise biological entities associations or similar tasks are considered as the baselines, including DHNE [17] , HyperSAGNN [19] , HGSynergy [20] , MCHNN [21] .Table 1 : Performance comparison on three datasets of different biological entity associations. Each result of these methods is from the average of 5-fold cross-validation experiments with four scenarios. The best result for each dataset and metric is marked in bold. All the presented hits scores are in %.Methods hits@1 hits@3 hits@5 hits@1 hits@3 hits@5 hits@1 hits@3 hits@5 Implementation Details. To accurately evaluate model performance and prevent overfitting, 5-fold cross-validation is used to evaluate the performance. Specifically, we randomly split the dataset into a 90% cross-validation set and a 10% independent test set. On the cross-validation set, the 5-fold cross-validation is implemented. Moreover, the independent testing, in which the model is trained on the cross-validation set and tested on the independent test set, is conducted to obtain the prediction results. In the training stage, Binary Cross Entropy loss is adopted to measure model performance and Adam optimizer is adopted to optimize all of model parameters with a learning rate of 0.001 5 .RFEvaluations. To evaluate the prediction performance on the triplet associations, hit ratio (hit@n) and normalized discounted cumulative gain (ndcg@n), which are widely used in recommendation tasks [46, 47] , are employed for model ability to provide a comprehensive assessment for the task. In order to investigate the potential relation among different association patterns, we visualize the pattern commonality coefficients of Cefadroxil (drug #53) and S-(2-Thienyl)-L-cysteine (drug #207) in DMD dataset, as shown in Figure 3 . Larger pattern commonality coefficients contribute more to the original drug pathway, whereas smaller coefficients often relate to different biological mechanisms. It can be observed that:• For Cefadroxil (drug #53), the common patterns consisted by Chlorhexidine acetate (drug #76) [48] , Hetacillin (drug #132) [49] , and Econazole (drug #102) [50, 51] are considered more similar as the mechanism [52] that actives on the cell wall and envelope leading the change of microbe physiological activities, thus treating the diseases. Instead, Tobramycin (drug #227) inhibits mRNA be translated into protein and thus promotes microbe cell death [53, 54] .• For S-(2-Thienyl)-L-cysteine (drug #207), the common patterns consisted by Diphenyl Disulfide (drug #100) [55, 56] , Salicylic Acid (drug #209) [57] , and Chrysophanic Acid (drug #78) [58, 59] are considered more similar as the mechanism that [60] [61] [62] inducts oxidative stress in bacteria and thus damage all components of the microbe cell. Instead, Moxifloxacin (drug #160) inhibits DNA gyrase and topoisomerase IV enzymes to separate bacterial DNA, thereby inhibiting cell replication [63, 64] .In summary, the analyzed cases illustrate the high-commonality patterns derived from Pattern-BERP, demonstrating how these small molecule drugs influence microbial activities to treat diseases, generally following a consistent physiological route. These cases indicate that our method can discover potential common patterns through the API block, and obtain larger pattern coefficients through the attention mechanism, which drives the nodes to learn these important common interactions and acquire more expressive representations. Details about additional cases can be found at Appendix B.3.2. To investigate the necessity of each component in Pattern-BERP, we conduct several comparisons between Pattern-BERP and its variants on the test set. As illustrated in Table 2 , when basic components of Pattern-BERP have been removed, the performances of corresponding variants on DDC dataset significantly decline, indicating that these components all contribute to the performance. Besides, we have other observations: (1) when only DE module is removed, the performance is inferior to the one removing the entire APA module containing DE, demonstrating that inaccurate entity distance information has a more detrimental impact on the prediction performance; (2) eliminating HNS module results in a significant drop in the performance, highlighting its crucial role in enhancing the model's robustness and discrimination capability and thus indicating that the proposed negative sampling strategy contributes to efficient learning; (3) when removing HC module leads to a slight performance degradation, the impact is relatively limited which suggests that the capability of HC module in representing complex associations is relatively modest for the highly dense-association DDC dataset, but it still provides some beneficial effects towards the final performance improvement.To verify the affect of hyperparameter settings to model performance, we first conduct ablation experiments on the three main parameters of the APF module, namely number of attention heads, number of max pattern distance, and number of sampled association patterns. As illustrated in Appendix Figure 5 , when the three hyperparameters are increased, the prediction performance on DDC dataset exhibits an overall upward trend, suggesting that increasing these hyperparameters helps the model better capture the complex association patterns, thereby improving the final performance.Additionally, we conduct experiments on the loss-balanced coefficient α and the bind-relation prediction probability threshold γ, both adjusted from 0.1 to 0.9. Results in Appendix Figure 6 show that setting α and γ to 0.5 yields the best performance. Specifically, for α, since the final prediction task involves predicting the associations among entity A, B, and C, the two tasks of A → B and B → C are intuitively of equal importance, therefore the balanced coefficient α set to 0.5; for γ, bind-relation prediction is fundamentally a binary classification task, thus the threshold γ is set to 0.5. Details about ablation experiments are presented at Appendix B.3.3. 
conclusion: 
 In this work, we propose a novel association pattern-aware fusion method Pattern-BERP for biological entity relationship prediction, which effectively combines the related association pattern information into entity representation learning. In addition, to enhance the missing information of the low-order message passing, we devise a bind-relation module that considers the strong bind of low-order entity associations. The evaluation on three biological datasets quantitatively demonstrate that the proposed method consistently achieve superior performance over the competing baselines. Moreover, the interpretability explanations of association patterns reveal the intrinsic biological mechanisms and thus promote the method to be deployed in real-world scenarios.Due to the domain-specific task, Pattern-BERP focuses on the fixed-length association patterns.Extending the approach to capture variable-length pathways could further enhance the representational power. Additionally, exploring the applicability of the method in other domains beyond biology, such as general knowledge graph completion, would help evaluate its broader generalizability. Given the maximum number of hops U (i.e., the max positional distance for certain entity node), the distance matrix D ∈ R |V|×|E| is constructed based on Definition 1 as follows:• 1-hop distance: For each node v and hyperedge e, if hyperedge j directly contains node i, then D v,e = 1 represents the association of hyperedge j is the 1-hop pattern. • 2-hop distance: If hyperedge e does not directly contain node v but shares a node with a hyperedge that does, then D v,e = 2 represents the association of hyperedge e is the 2-hop pattern. • u-hop distance: Recursively calculate further distances up to u-hop, then D v,e = u.• Unreachable: If node v cannot reach hyperedge e within U -hop, then D v,e = -∞.Hence, the distance tokens D v for node v are transformed by one position encoding layer, thereby yielding the positional embeddings for the self-attention mechanism of the subsequent API block. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We have provided the raw experimental results of all 5-fold cross-validation results across the four scenarios, resulting in a total of 20 individual results.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: The source code & citations in the paper. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: As an empirical method, the Pattern-BERP method extracts common patterns or rules from large bio-associated networks, similar to the k-means clustering. Specifically, given a triplet-wise dataset S = {s 1 , s 2 , . . . , s t , . . . , s T } and K common patterns, the optimization goal is to minimize the total distance of sampled patterns to their respective pattern centers. First, initialize the pattern centers M = {m 1 , m 2 , . . . , m k , . . . , m K }. Next, assign each sample to the nearest pattern center by computing the center index with arg min k ∥s i -m k ∥ for all the centers in M. Then, update the position of these pattern centers using m k = 1 |C k | st∈C k s t , where C k is the set of samples assigned to pattern k calculated as above. Iterate these assignment and update steps until the pattern centers converge. The objective function to minimize is J = K k=1 st∈C k ∥s t -m k ∥ 2 , and by minimizing J, Pattern-BERP extracts representative biological association patterns effectively.In this paper, three biological datasets are used to evaluate the efficacy of the proposed method. Each dataset encompasses three different entities and their associations. Appendix Table 3 provides a detailed presentation of the specific entities within each dataset, including the count of nodes per entity, the number of associations among these nodes, and the corresponding association ratio. All entity associations are structured into triplet scheme, such as <drug, protein, adr> for DPA dataset. DPA dataset. DPA dataset is first constructed and preprocessed with the origin data from [44] . Specifically, only <drug, target, adr> triplets that have complete field information are retained, that is, drug Pubchem CID, target protein identifier UniProt ID, ADR term name. Then, the detail information is obtained through these unique identifiers, respectively. For drugs, the PubChem database [69] is queried with the PubChem CID of each drug, and their Canonical SMILES are recorded. For proteins, the UniProtKB database [70] is queried with the UniProtKB ID of each protein, and their protein sequences are recorded. For ADRs, the incidence matrix about the reaction of specific ADRs to ∼15,000 drugs is obtained from publicly available datasets [71] . In this matrix, a value of 1 indicates that a particular drug causes a specific ADR, while a value of 0 denotes no relation between them. Furthermore, following the method described in Appendix A.1, the ADR similarity matrix can be constructed for subsequent feature generation fo ADR entities. The implementation of all baseline methods is conducted using their respective publicly accessible source codes. Optimal or default configurations for each method are employed to ensure robustness. Specifically, for methods such as RF, MLP [10] , CP and Tucker [11] , meticulous parameter tuning is engaged in to elicit their peak performance levels. For GCN [13] , GAT [15] , GraphSAGE [14] and GIN [16] , the original triple-wise associations are initially decomposed into two pair-wise associations. Subsequently, each pair-wise association is independently modeled using the corresponding graph neural network model, and the prediction probabilities of the two pair-wise associations are multiplied to obtain the final prediction value for the triple-wise association. Additionally, for DHNE [17] and HyperSAGNN [19] , biological embeddings are integrated with their original structural embeddings to ensure fairness. For CoSTCo [12] , HGSynergy [20] and MCHNN [21] , the parameter settings outlined in the original publications are followed. It is important to note that for the sake of equitable comparison, all of these methods utilize negative sampling setting from MCHNN that is consistent across implementations.The initialized entity embedding size d is fixed to 128. The number of BGNN, APF layers are all fixed to 2. The training epoch is setting to 1,000 for DMD, DPA datasets and 2,000 for DDC dataset.The number of max hop in pattern sampling U is setting to 3. The number of sampled patterns N is setting to 100. The number of attention heads is 32 for DMD dataset, 16 for DDC dataset, 4 for DPA dataset. The loss-balanced coefficient for bind-relation task α is fixed to 0.5. The threshold for bind-relation prediction probability γ is fixed to 0.5. In addition, in line with the evaluation strategy of [21] , 29 negative samples are randomly generate for each test triplet in four scenarios for the evaluation of all methods, and thus display the average metrics over all test triplets. These scenario settings can comprehensively evaluate the model ability to identify positive and negative samples under these stringent conditions.Furthermore, all experiments are conducted on a single NVIDIA A6000 Tensor Core GPU (48GB) and Intel(R) Xeon CPU with 24 cores and 500G memory. The whole training time for DMD, DDC, DPA datasets is about 8, 8, 4 hours, respectively. In addition, Appendix Table 4 exhibits the GPU memory usage with varying the number of sampled patterns N across three datasets, and Appendix Figure 7 presents the inference time of Pattern-BERP in comparison with these advanced baselines for each 100 samples with milliseconds.As a supplement to Table 1 , Appendix Table 5 incorporates additional metrics ndcg@1, ndcg@3 and ndcg@5. Considering all these metrics, Pattern-BERP significantly outperforms previous SOTA baselines on all three datasets, exhibiting the remarkable advantages of association pattern mining.Furthermore, each result of these methods is from the average of 5-fold cross-validation experiments with four scenarios. However, significant variation in prediction difficulty across different scenarios makes it relatively unreasonable to provide an error bar for all 20 results. Instead, to demonstrate the statistical validity of Pattern-BERP, we have provided the raw experimental results.Table 5 : Performance comparison on three datasets of different biological entity associations. Each result of these methods is from the average of 5-fold cross-validation experiments with four scenarios. The best result for each dataset and metric is marked in bold. All the presented ndcg scores are in %.Methods ndcg@1 ndcg@3 ndcg@5 ndcg@1 ndcg@3 ndcg@5 ndcg@1 ndcg@ In order to investigate the potential relation among different association patterns, we visualize the pattern coefficients of drug #52 in DMD dataset, as shown in Appendix Figure 4 . It can be observed that the patterns that make important contributions are not necessarily 1-hop patterns, with pattern coefficients of 0.41% and 0.65%. In contrast, the 2-hop patterns can exhibit considerable relevance, due to the similar mechanisms of toxicity against microbes exhibited by the corresponding drugs (#53, #162, #207) compared to the target drug #52 under investigation. Additionally, the weakening of drug #52 itself is because the original drug #52 is a complex peptide structure with repetitive and redundant information, thus acquiring simpler and more straightforward representations through the aforementioned information interaction.Furthermore, from the view of structural and functional groups mentioned by [72] [73] [74] [75] , we have several salient observations as follows: (1) As the drug in the most influenced pattern, drug #53 contains β-lactam as shown in the red rectangle of Figure 3 . β-lactam is the crucial component of β-lactam antibiotics [76] , which is one of the most widely used classes of antibiotics available. In addition, Islam et al. [77] also prove that pyridine scaffold (aromatic ring with nitrogen) bearing poor basicity generally improves water solubility in pharmaceutically potential molecules and has led to the discovery of numerous broad-spectrum therapeutic agents. (2) Similar with drug #53, drug #162 has acetanilide structure, which serves as the basis for antimicrobial activity and disease treatment [78] . (3) Drug #207 has the dimethylglycine structure, which is crucial to Tigecycline [79] . Tigecycline binds to the bacterial ribosome, blocking the binding of amino-acyl-tRNA to the acceptor site on the mRNA-ribosome complex, thereby inhibiting protein synthesis. (4) Unlike the previous three, drug #266 may interact with membrane lipids to alter membrane fluidity and permeability, thereby exerting its effect on microorganisms. However, this mechanism of action is distinct from the specific functional group interactions of the previous three drugs.As a compound with a complex structure containing multiple functional groups, drug #52 is similar to the previous three drugs due to its complex structure and diverse functional groups. These characteristics enable it to exert toxicity through specific interactions with the critical pocket of microbes. Compared to drug #266, these complex drugs have more diverse and specific toxicity mechanisms, which is why they have received less attention.Network Module Designs. To investigate the necessity of each component in Pattern-BERP, we conduct several comparisons between Pattern-BERP and its variants on the independent test set. Table 2 , 6, 7 denotes the results of DDC, DMD, DPA dataset, respectively.When basic components of Pattern-BERP have been removed, the performances of corresponding variants on all datasets exhibit significant declines, though the degree of performance degradation varies. This indicates that these components all contribute to the performance. Besides, we have other observations: (1) for the relatively sparse datasets, DMD and DPA, the performance decline after removing BFR is more significant compared to the DDC dataset. ( 2) for all datasets, eliminating HNS module results in a drop in the performance, highlighting its effect in enhancing the model's robustness (a)hits@1 hits@3 hits@5 hits@1 hits@3 hits@5 Metric (e)hits@1 hits@3 hits@5 Metric and discrimination capability and thus indicating that the proposed sampling strategy contributes to efficient learning; (3) for all datasets, when only DE module is removed, the performance is inferior to the one removing the entire API module containing DE, demonstrating that inaccurate entity distance information has a more detrimental impact on the prediction performance; (4) when removing HC module leads to a slight performance degradation, the impact is relatively limited which suggests that the capability of HC module in representing complex associations is relatively modest for the relatively dense-association DDC and DMD dataset, but it still provides some beneficial effects towards the final performance improvement. But for highly sparse DPA datasets, removing HC module leads to performance improvement, demonstrating that the combined performance of HC and APA is instead constrained by the confusion introduced by the two aggregation modes on the extremely sparse structures.Hyperparamter Settings. To verify the ablation of hyperparameter settings, we conduct ablation experiments on the three main parameters, namely number of attention heads, number of max pattern distance, and number of sampled association patterns. As illustrated in Appendix Figure 5 , when the number of attention head are increased, the prediction performance on all datasets exhibits an 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Loss-balanced coefficient overall upward trend, suggesting that increasing the attention heads helps the model better capture the complex association patterns, thereby improving the final performance. In addition, given the significant disparity in the association ratios across the three datasets, the performance of DDC dataset improves as the max hop hyperparameter is increased. In contrast, for the relatively sparse DMD dataset, it is advisable to restrict the max hop to the range from 1 to 3, as the performance drops sharply when the parameter is set to 4. Moreover, for the sparsest DPA dataset, selecting only 1-hop patterns appears to be the optimal choice. Moreover, regarding the number of sampled patterns, a similar trend is observed across the datasets. For DMD and DPA datasets, which are relatively sparse, increasing the number of sampled patterns is not necessarily beneficial for performance. In contrast, on DDC dataset, augmenting the number of sampled patterns continues to yield performance. Furthermore, we also conduct ablation experiments on another two parameters, namely loss-balanced coefficient α and the bind-relation prediction probability threshold γ. Results in Appendix Figure 6 show that when α and γ are both set to 0.5 by our default, the performance surpasses those of other settings. Specifically, for α, since the final prediction task involves predicting the associations among entity A, B, and C, the two tasks of A → B and B → C should be intuitively considered to be of equal importance, therefore the balanced coefficient α set to 0.5; for γ, bind-relation prediction is fundamentally a binary classification task, thus the threshold γ is set to 0.5.Computation Complexity Discussion. To facilitate applying across a wider range of dataset types and particularly large datasets, it is imperative to rigorously address the issue of complexity explosion. Regarding computation complexity, the discussion in Section 4.5 indicates that the decisive factors are the number of entity nodes |V| and the number of sampled patterns N . Hence, when encountering larger datasets or more complex biological networks, we can employ the following two strategies to avoid a computation complexity explosion: (1) Graph Sampling. By sampling smaller subgraphs [80] , computational resource consumption reduces significantly, boosting algorithm speed and efficiency;(2) High-confidence Pattern Selection. As shown in Appendix Figure 5 (g)-(i), reducing N from 100 to 5 across three datasets slightly decreases performance but still surpasses these baselines. Thus, adjusting the sampling quantity is acceptable to effectively reduce time complexity. Moreover, Appendix Table 4 shows the GPU usage varying from the number of sampled pattern across three datasets with significant differences in the number of entities and associations. The inference time of each 100 samples with milliseconds compared to baselines are provided in Appendix Figure 7 . It is evident that the current computational resources are sufficient to handle existing datasets, and the difference in inference time for our method is constant and even faster than several baselines.Potential Application in Real-world Scenarios. The preliminary evaluation results on three biological datasets quantitatively demonstrate that the proposed method consistently achieves superior performance over the competing baselines. Additionally, the learned association patterns show potential in interpreting biological mechanisms. This finding provides hope for the future practical application of our approach, particularly in addressing the "black box" issue in the field of bioinformatics. As a computer-aided tool, our method holds the potential to exhibit a broad array of applications in real-world scenarios, contingent upon sufficient clinical tests or validations. Consequently, it may significantly contribute to human health and well-being.Limitations and Future Work. While the proposed Pattern-BERP has demonstrated strong performance on biological entity relationship prediction, there are several limitations and potential directions for future research. One limitation is that the current model only considers fixed-length association patterns. In real-world biological systems, relevant relationships may be expressed through variable-length pathways involving multiple intermediate entities and relations. Extending the model to capture and leverage such variable-length association patterns could further enhance its representational power and predictive accuracy. Additionally, the applicability of Pattern-BERP method has so far been explored only in the biological domain. Investigating the effectiveness of this approach in other domains, such as knowledge graph completion for general entities or relation extraction from text, would help evaluate its broader generalizability and potential to benefit other fields that rely on structured knowledge representations. Adapting the model architecture and training strategies to accommodate the unique characteristics of different domains could lead to fruitful avenues for future research.By addressing these limitations and exploring these future research directions, we believe Pattern-BERP can be further improved and extended to have an even greater impact on advancing our understanding of complex biological systems and knowledge representation in general. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.To facilitate distinction, we set the entity term B1 = B2 = B.The definition is analogous to positional embedding in Transformer[38] and ViT[39].In the single-head setting, dK = dV = 3d.More details about the experiments can be found at Appendix B. 


Decision: This work seeks to predict relationships between biological entities,  via an association pattern-aware fusion method that integrates association pattern information into entity representation learning.  It is a solid piece of work - the    association pattern-aware fusion method is creative.

Reviews: review: paper_summary: paper_summary: This work presents a novel approach - Pattern-BERP - for the prediction of biological entity relationships, it utilises entity association patterns as opposed to a lot of existing research that focuses primarily on entity-centric information mapping and aggregation. 

The evaluation is done on three different biological datasets (DMD, DDC, DPA), and compared against a range of different approaches including non-graph, graph-based and hypergraph-based ones.

The results show state-of-the-art performance across the board (an increase from 2 to 10 points over the next best method). 

The authors also do an ablation study, showing the importance of each component and demonstrating the relative improvement.
strength_weakness: strength_weakness: This is a very general method but it was tested on only one dataset type and relatively small datasets. Would be great to see it expanded in the future.
questions: questions: Similar to the limitations field, I'd be very interested in an explanation of how the computation complexity of Pattern-BERP scales with the number of entities and associations. 

Is it possible to provide a bit more theory behind the success of your method compared to others? The justification for the association pattern-aware fusion to me seems mostly intuitive rather than theoretical.

Is there a way to extend the method to capture variable-length pathways?
limitations: limitations: It would be great to see a longer discussion on limitations: my primary concerns are complexity explosion and very large datasets. Would be great to see a bit of a theoretical analysis (or even empirical) on how it scales for millions of entities or associations (which are not rare in the domain).

The interpretability section focuses on only drug #52, it would be great to see that expanded, as making conclusions from one example is bit of a stretch. 

Real-world applications might be a bit of an overstatement, while it is possible, nothing was really tested or proven.
paper_summary: paper_summary: The author proposed a deep learning method, termed the Pattern BERP method, designed to elucidate the potential associations among triple-wise biological entities. This method innovativly incorporates association pattern information into the entity representation learning. Moreover, it employs two additional bipartite graphs to enrich the binary relation information extracted from triple entities. Evaluated on three biological datasets, the proposed method demonstrates superior performance compared to non-graph, graph-based, and hypergraph-based methodologies. This work is noteworthy not only for its impressive predictive accuracy but also for its potential applicability in real-world scenarios.
strength_weakness: strength_weakness: 1. The interpretation provided for the model is not entirely persuasive. The analysis of the relationship between peptide-based micro-molecular drugs and small-molecular chemical drugs is unconventional and lacks commonality. 

2. While the concept of association pattern-aware fusion is highly innovative and appropriately highlighted as the main focus of the paper, its contribution for model performance seems not to be impressive, especially in Table 5 and Table 6.

3. The model is overly complex, with multiple stages of tasks and losses, which may lead to difficulties in optimization.

4. The role of the triple information has been simplified to a single chain pattern from A to B to C.
questions: questions: 1. The abbreviation "APF" (association pattern fusion) is not defined in the text at line 219. It is essential to provide the full name when first introduced for clarity.

2. In equation (11), the LBE is not updated every epoch, which contradicts the description in the text.

3. The manuscript features two bipartite graphs in Figure 2, yet it remains unclear whether these graph features are integrated with hypergraph features. Additionally, the role of the blue dotted line arrow, which is not discussed in the text, needs clarification to enhance understanding of the graphical representations.

4. Although the association pattern-aware fusion is novel and central to the paper's title, its contribution for model performance seems not to be impressive. Specifically, while Table 2 shows significant impairment in performance on the DDC dataset without the association pattern interaction (API), Tables 3 and 4 for the DMD and DPA datasets suggest that the Binding-relation Feature Reconstruction (BFR) for bipartite graphs predominantly enhances performance. It may be misleading to attribute the main contribution of the work solely to APF given these findings.

5. The model interpretation in Figure 3 lacks persuasiveness, as Drug #52, a peptide with numerous sub-group structures, is not ideal for visualization comparison with other compound drugs. Furthermore, in equation (3), the attention mechanism is applied across different triple entities, not just one. Analyzing only the drugs without considering the other two entities in the figures is insufficient. At minimum, the roles of the other two entities should be elucidated to provide a more comprehensive analysis.

6. Equation 2 describes the generation method of the distance matrix D and the distance token 𝐷𝑣. How is the specific position embedding done on line 167? Is the initial pattern 𝑃 directly concatenated from the three nodes?

7. In the bind-relation enhancement, why is only the bipartite graph between AB and BC considered? Why not consider AC? Not all multi-entity relationships have a clear chain pattern from A to B to C.

8. Following up on question 7, how are relationships defined between two entities in Equations 7 and 8?
limitations: limitations: The model lacks practical application scenarios. It primarily learns from the interaction network of existing entities such as drugs, diseases, and symptoms. Such methods generally struggle to predict the relationships between a completely new entity and the existing entities in the network. Moreover, the relationships within the network are already well-known. The more important task is to uncover new and potential relationships. The comprehensiveness and accuracy of the existing relationships are merely the tasks designed by the model itself.
paper_summary: paper_summary: This paper presents a novel approach to predicting relationships among biological entities by addressing limitations in current deep learning methods that focus solely on entity-centric information. The authors introduce an association pattern-aware fusion method that integrates association pattern information into entity representation learning. Additionally, they develop a bind-relation module to enhance low-order message passing by considering strong low-order entity associations. Extensive experiments on three biological datasets demonstrate significant improvements compared to advanced baselines. The paper also provides detailed explanations of the interpretability of association patterns, highlighting intrinsic biological mechanisms and potential real-world applications.
strength_weakness: strength_weakness: - The proposed model involves multiple modules and several hyperparemeters. Some hyperparameters (such as the number of attention head) are vary on different datasets. The authors may aslo need to add hyperparameter study for loss-balanced coefficient and the threshold for bind-relation prediction probability.
- The approach may not scale well with increasing dataset sizes or more complex biological networks due to its computational requirements and the need for extensive pattern sampling and interaction modeling.
- The proposed model involves multiple components such as hypergraph convolution, pattern-aware fusion, and bind-relation enhancement, which can be computationally intensive. This might limit its applicability for large-scale real-world datasets or in environments with limited computational resources. The author may also need to provide inference speed study with some of the baselines.
questions: questions: Please see the weaknesses.
limitations: limitations: They provide a limitation section in the appendix.
paper_summary: paper_summary: The paper introduces an innovative perspective, namely association pattern, for biological entity relationship prediction (e.g., drug-target protein-adverse reaction). The authors first summarize the characteristics of existing perspectives and emphasize the importance of the association pattern. Next, the association patterns of certain entity nodes are sampled in the constructed graph based on the defined distance and then serve as tokens for self-attention to fuse similar semantic information. Additionally, the bind-relation enhancement module can effectively reconstruct the missing feature and thus generate hard negative samples for further improving the performance. Extensive experiments on biological datasets demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. The bind-relation feature reconstruction appears to be a standalone component that requires extra separate training, resulting in a two-stage process for the method. Additionally, such a design may influence the entire optimization process and distract the training flow, how to alleviate that?

2. Generalization and extension issues have not been discussed. The pattern's length is fixed to 3, which may make the method not work when facing longer-range relationships. Are there any feasible solutions for sampling patterns with different lengths?

3. Complexity has not been discussed. Some details are also not clear enough, such as in Line 167, could the authors provide details on how the position encoding is designed? In Fig. 2, the meaning of the dotted line "Bind-relation Feature" needs more explanation. The "add" operation is not reflected in Section 4.3. Please explain the meaning of the dotted line "Bind-relation Feature".
questions: questions: Q1: The paper does not give a specific explanation of $L_{BE}$ (see Line 224). Please give a specific explanation of $L_{BE}$.

Q2: In Table 1, the authors appear to have not reported the statistical measures like standard error.
limitations: limitations: The authors have adequately discussed the limitations.

Comments: title: All clear
To Authors
To Authors
comment: We appreciate your effort on our paper and the recognition of our responses. Your unique insights and detailed review have significantly enriched our work, allowing us to further refine this manuscript. Thank you once again for your valuable feedback.
Thanks for detailed response. I have raised the score.
Thank you for your time and effort on our paper. We are delighted to hear that our responses are clear and satisfactory. Your thorough review and constructive feedback have significantly contributed to the refinement of our manuscript.
Thank you for the answers, all is clear.
We appreciate your acknowledgment of our responses in addressing your concerns. The relevant discussions regarding the extension of variable-length patterns will be elaborated in the final version. Your insights are crucial in enhancing the quality of our work. Thank you once again for the thoughtful review.
I thank the clear and comprehensive response, and my concerns have been mainly addressed especially on the generalization issue, so I raise my final voting to 7. Please provide such discussions in camera-ready.
Thanks for your efforts and the considerable recognition of our paper. The quality of the manuscript will be greatly improved in accordance with your valuable comments.
Thanks for your responses. I think my rating is reasonable and fair.