Title: Association Pattern-aware Fusion for Biological Entity Relationship Prediction
Venue: NeurIPS 2024
Accepted: yes
Keywords: association pattern

deep learning

interpretable machine learning

biological network
Abstract: Deep learning-based methods significantly advance the exploration of associations among triple-wise biological entities (e.g., drug-target protein-adverse reaction), thereby facilitating drug discovery and safeguarding human health. However, existing researches only focus on entity-centric information mapping and aggregation, neglecting the crucial role of potential association patterns among different entities. To address the above limitation, we propose a novel association pattern-aware fusion method for biological entity relationship prediction, which effectively integrates the related association pattern information into entity representation learning. Additionally, to enhance the missing information of the low-order message passing, we devise a bind-relation module that considers the strong bind of low-order entity associations. Extensive experiments conducted on three biological datasets quantitatively demonstrate that the proposed method achieves about 4%-23% hit@1 improvements compared with state-of-the-art baselines. Furthermore, the interpretability of association patterns is elucidated in detail, thus revealing the intrinsic biological mechanisms and promoting it to be deployed in real-world scenarios. Our data and code are available at https://github.com/hry98kki/PatternBERP.

Association Pattern-aware Fusion for Biological Entity Relationship Prediction

Abstract

Deep learning-based methods significantly advance the exploration of associations among triple-wise biological entities (e.g., drug-target protein-adverse reaction), thereby facilitating drug discovery and safeguarding human health. However, existing researches only focus on entity-centric information mapping and aggregation, neglecting the crucial role of potential association patterns among different entities. To address the above limitation, we propose a novel association pattern-aware fusion method for biological entity relationship prediction, which effectively integrates the related association pattern information into entity representation learning. Additionally, to enhance the missing information of the low-order message passing, we devise a bind-relation module that considers the strong bind of low-order entity associations. Extensive experiments conducted on three biological datasets quantitatively demonstrate that the proposed method achieves about 4%-23% hit@1 improvements compared with state-of-the-art baselines. Furthermore, the interpretability of association patterns is elucidated in detail, thus revealing the intrinsic biological mechanisms and promoting it to be deployed in real-world scenarios. Our data and code are available at https://github.com/hry98kki/PatternBERP .

1 Introduction

Exploring potential associations among triple-wise biological entities (e.g., drug-target proteinadverse reaction) [1-6] holds significant implications for elucidating underlying biological mechanisms and advancing personalized therapies [7-9], thus promoting pharmaceutical innovation and ensuring human health. Recent deep learning-based methods have propelled auxiliary prediction tasks concerning biological entity relationship, with most focusing on binary associations (e.g., drug-target protein), while only a few methods offer insights for more complex triple-wise associations. Existing solutions for the association prediction task can be broadly categorized into three types: (1) non-graph methods [10-12]; (2) graph-based methods [13-16]; (3) hypergraph-based methods [17-22].

As illustrated in Figure 1, non-graph methods typically concatenate the features of different entities, which are independently mapped by their respective entity encoders, to serve as representations. Graph-based methods adopt nodes and edges of the graph to represent entities and their relationships, and leverage the graph structure for feature propagation, thereby achieving effective representation learning of entity nodes. Similar yet distinct, hypergraph-based methods employ the hypergraph structure to obtain entity representations using the complex feature aggregation strategy. However, none of the aforementioned methods consider the significance of path patterns in the graph structure, which contain a vast amount of crucial information including hidden context and co-occurrence.

(a) Non-graph Methods

(c) Hypergraph-based Methods

(d) Association Pattern-aware Method

(b) Graph-based Methods

Encoder

Encoder

Encoder

1-hop

2-hop

3-hop

4-hop

Entity

Entity

Entity

Target Entity

To this end, we introduce a novel association pattern-aware message propagation strategy as shown in Figure 1(d). The module leverages the potential relationships, such as commonality and diversity, of association patterns as the rule for facilitating message passing among entity features, which can efficiently expand the potential for representing complex interactions within the perspective of both basic graph structure and intrinsic biological mechanisms. Specifically, the related patterns within the graph structure are sampled through the pre-defined distance relation for each entity node. The message passing is driven by the interaction of its assigned patterns, i.e., the feature of the entity node is subsequently updated through feature fusion using adaptive coefficients that capture pattern commonality, generated during the pattern interaction stage. This process is followed by the acquisition of potential common patterns with genuine biological significance for various entities.

In this paper, we propose a novel Association Pattern -aware Fusion method for B iological E ntity R elationship P rediction, namely Pattern-BERP . First, we devise an association pattern-aware strategy to solve the limitation caused by entity-centric feature mapping and aggregation. The strategy utilizes the association patterns related to each entity node within the graph to extract the common feature based on the attention mechanism for these patterns, thus expanding the ability to represent hidden complex interactions. In addition, to preserve the information interaction of different entities, a hypergraph-based block is incorporated with the association pattern-aware fusion module, thereby enhancing the model ability to capture relationships among various types of entities. Furthermore, to explore low-order associations of biological bind relations, we introduce a bind-relation enhancement module which can reconstruct missing feature of bind-relation entities and thus generate harder negative sample than random selection. Experimental results conducted on different biological datasets show that the proposed method achieves superior performance compared to advanced baselines, demonstrating its effectiveness and robustness in handling various biological entity relationships. More importantly, the obtained association patterns for the relationship of drug, microbe, and disease are quantitatively visualized with the following biological verification in detail.

Our main contributions are summarized as follow:

We propose a novel association pattern-aware fusion method for biological entity relationship prediction. The introduce of association pattern-aware strategy can enhance the representation of complex interactions by aggregate features with potential association patterns.

A bind-relation enhancement module is devised to acquire low-order associations that reveal the biological bind relations, which is essential for reconstructing missing bind-relation entity features and generating challenging negative triplets to enhance the model training.

Extensive experiments are conducted to verify the superiority of Pattern-BERP, demonstrating its robustness for various biological scenarios. Uniquely compared to the other methods, the interpretability of association patterns is explained to reveal intrinsic biological mechanisms.

2 Related Work

In this section, we elaborate on the related work from two distinct yet interconnected perspectives: Biological Entity Relationship Prediction and Network Search and Mining. Each perceptive represents a fundamental aspect of our research, addressing specific challenges and methodologies in applying machine learning techniques to the prediction task.

Biological Entity Relationship Prediction. The latest advancements in artificial intelligence have motivated researchers to employ deep learning methodologies for predicting triple-wise biological entity associations. Hypergraph neural network (HGNN)-based methods [17-22] have become the mainstream research direction in this field. Tu et al. [17] proposed a deep hyper-network embedding model to preserve both local and global proximities in the embedding space. Building upon this, Jiang et al. [18] incorporated a dynamic hypergraph construction strategy to capture the hidden and important relations in data structures. Zhang et al. [19] developed a self-attention-based graph neural network applicable to homogeneous and heterogeneous hypergraphs with variable hyperedge sizes. Liu et al. [20] proposed a multi-way relation-enhanced hypergraph representation learning method to predict anti-cancer drug synergy. Liu et al. [21] proposed a multi-view contrastive learning-enhanced hypergraph model for drug-microbe-disease association prediction. In addition, Chen and Li [23] attempted to adopt the tensor decomposition strategy to predict which target a drug binds to when administered to a disease, and further proposed a neural tensor network model [24] that seamlessly combines tensor algebra and deep neural networks to effectively capture the complex nonlinear dependencies among drugs, targets, and diseases.

Network Search and Mining. Network search and mining techniques, particularly those utilizing path information including random walks [25-28] and meta-path [29-32], have been widely employed to extract local structural information from networks. These methods have found applications in areas such as content recommendation and community detection [33-37]. Brin and Page [25] introduced a classic ranking algorithm PageRank to determine the importance of web pages based on their link structure. Jeh and Widom [26] adopted a similarity measure based on pairwise random walk, which can capture the structural similarity between nodes, and further extended PageRank with a personalized version [27]. Perozzi et al. [28] proposed Deepwalk that leverages local random walk information to learn vertex latent representations based on deep learning techniques. These above methods are applicable to homogeneous networks and cannot fully utilize the rich semantic information in heterogeneous networks. To address the limitation, Sun et al. [29] introduced a meta path-based similarity framework for heterogeneous information networks, which can capture the subtle semantics of similarity among objects of the same type. Dong et al. [30] proposed a deep learning-based heterogeneous network representation learning method that automatically learns hidden meta-path semantics, generating general node embedding representations. Wang et al. [31] introduced a graph-based fraud detection method that addresses the issue of low homophily by integrating label information to generate distinguishable neighborhood information. Furthermore within the bioinformatics field, Chen et al. [32] proposed a computational algorithm that performs random walks on an integrated network to infer potential relations between proteins and ADRs.

In line with the above methods, Pattern-BERP utilizes path information for triple-wise heterogeneous biological network mining. By leveraging the fusion of association patterns, it facilitates message passing among various biological entities, which will be described comprehensively in Section 4.2.

3 Problem Formulation

Given three distinct entity types in biological networks, termed as A = { a 1 , a 2 , · · · , a i , · · · , a |A| } , B = { b 1 , b 2 , · · · , b j , · · · , b |B| } , and C = { c 1 , c 2 , · · · , c m , · · · , c |C| } , their Cartesian product S = A × B × C is a set of all possible triple-wise biological entity associations. For simplicity, the relation A -B -C is used to represent complex relational semantics for biological entities, such as 'drug-microbe-disease', 'synergistic drug-drug-cell line' or 'drug-target protein-adverse reaction'.

For each triplet ( a i , b j , c m ) ∈ S , we assign a label p ∈ { 0 , 1 } . A label of p = 1 indicates that the existence of certain association has been confirmed, while p = 0 represents an unknown association which denotes that the association is not yet known and could potentially exist. The objective is to develop a credible model that can predict these potential associations from unknown ones.

Bio-encoding

Bind-relation Enhancement

Entity

Entity

Entity

Negative Sample

Bind-relation Feature

Negative Sampling

Association Pattern-aware Fusion

......

......

Association Predictor

Target Entity

Negative

Positive

1-hop

2-hop

3-hop

4-hop

Pattern Commonality

Pattern Interaction

HGNN

Element Add

...

= (

,

)

4 Pattern-BERP

To enhance the ability to represent complex interactions, we propose the first association patternaware method, termed as Pattern-BERP to extract the rich semantic information embedded within the intricate structures of biological networks. As illustrated in Figure 2, the section is divided into four parts: First, these entity relationships are represented in hypergraph and bipartite graph structures for subsequent module input. Next, the association pattern-aware fusion module is proposed to update entity node feature through association pattern-aware interaction. In addition, the bind-relation enhancement module is introduced to reconstruct bind-relation feature and thus generate hard negative samples. Finally, the detailed summary of loss function and complexity analysis is provided.

4.1 Graph Construction

Given the adjacency relationships among entity nodes, the hypergraph and bipartite graph structures are constructed to facilitate the subsequent extraction of structural information and relational association patterns within the respective graphs, which serve for association pattern-aware fusion module and bind-relation enhancement module.

The attributes of each entity are initialized as node features on the graphs with the domain knowledge of bio-entities through its own type. Finally, the initialized node attributes of X ∈ R |V|× d consist of features X A , X B and X C , where d denotes the feature dimensional of initialized entity attributes. Details about entity attribute are provided at Appendix A.1.

Hypergraph Construction. Triple-wise biological entity associations can be modeled as a hypergraph G = ( V , E ) , which includes a vertex set V = A ∪ B ∪ C and a hyperedge set E ⊂ S that represents all known associations. Technically, G is further formulated as an attributed hypergraph with node attributes X ∈ R |V|× d and an incidence matrix H ∈ { 0 , 1 } |V|×|E| , which is defined as:

Bipartite Graph Construction. The premise of constructing the bind-relation module is to decompose the original triple-wise associations to separately obtain pairwise relations of different types of entities. Since the original triplet is in the form of ( a i , b j , c m ) , and generally the interaction relationships between the three entities are hierarchical (e.g. drug a i acts on protein b j , and the activated b j then leads to certain adverse reaction c m ). Hence in this paper, we construct two bipartite

graphs: G A , B 1 for entity A → entity B 1 2 , G B 2 , C for entity B 2 → entity C . Then the edge sets of the bipartite graphs can be formulated as:

Note that, the relationship of A → C is not under consideration due to there is no direct connection in this context. The constructed bipartite graphs G A , B 1 , G B 2 , C serve as input for bind-relation module.

4.2 Association Pattern-aware Fusion

To comprehensively account for the feature interactions within association patterns, the proposed Association Pattern-aware Fusion (APF) method comprises two fundamental components: Firstly, Association Pattern Sampling (APS) block is designed to sample association patterns by utilizing the distance tokens relevant to target nodes. Secondly, Association Pattern-aware Interaction (API) block is introduced to update node features by message interaction within the sampled association patterns and mine the intrinsic pattern commonality with biological support.

4.2.1 Association Pattern Sampling

Definition 1. Given certain entity node, the distance token between the node and one hyperedge or association pattern is defined as u-hop . Here, 1-hop patterns represent the hyperedges directly covering the node, 2-hop patterns represent the hyperedges directly covering all the 1-hop neighbor nodes of the node, and the relation continues for higher hop counts. If the node is unreachable when u reaches the max step U , we define these patterns as no-relation and set the distance to -∞ .

Based on the above definition 3 , we generate the distance tokens between each node and all hyperedges, ultimately obtaining a distance matrix D ∈ R |V|×|E| , defined as follows:

where d ( v, e ) represents the defined distance from node v to pattern e ; u indicates the number of hops from node v to pattern e .

To represent an entity node with association patterns, the general principle is to prioritize and retain patterns that are closer for each node based on the distance tokens, with a total of N patterns sampled. Formally, given an entity node v , let D v ∈ R N and P v ∈ R N × 3 d represent the distance tokens and feature vectors of the selected sampled patterns, where each sampled pattern consists of three entity nodes and the d -dimensional feature embeddings of each node are defined within the initial node embeddings X . Then, the output pattern feature z v for node v considers the relative position of these related patterns, which is thus formulated as z v = P v +POS( D v ) , where POS( · ) denotes the position encoding layer that maps from R N → R N × (3 d ) . Finally, the integrated embedding z v ∈ R N × 3 d with distance information is produced for use in the subsequent API block.

4.2.2 Association Pattern-aware Interaction

Before put into the API block, we adopt a hypergraph convolution layer implemented by [40] on the constructed hypergraph G to facilitate neighbor-based information propagation among different entities. Then the updated node feature X ∗ after hypergraph convolutions is used to construct the feature of patterns mentioned in the APS block, thereby yielding z v for the information interaction of these sampled patterns. Details of hypergraph convolution are provided at Appendix.

Pattern Interaction. The API block is designed to search for and extract commonalities among different sampled patterns related to a specific entity node, and thus consists of a composition of Transformer layers [38]. Each Transformer layer has two modules: a multi-head self-attention mechanism (MHA) and a position-wise feed-forward network (FFN). For simplicity, we consider the single-head setting, and the extension to multi-head attention is standard and straightforward. Specifically, let Z (0) = [ z (0) 1 , . . . , z (0) |V| ] ∈ R |V|× N × (3 d ) denotes the input of the self-attention module

where z v ∈ R N × (3 d ) is the representation for node v . The input z v is projected by three matrices W Q ∈ R (3 d ) × d K , W K ∈ R (3 d ) × d K , and W V ∈ R (3 d ) × d V to obtain the corresponding query, key, and value representations Q v , K v , and V v for node v :

where A v ∈ R N × N is a matrix capturing the similarity between queries and keys; d K , d V denotes the feature dimensional of the key representations K v and the value representations V v 4 . Then we will get the output of the self-attention module z ′ v ∈ R N × (3 d ) . To summarize the process of transformer layer, the output of association pattern-aware attention block is computed as:

where LN denotes layer normalization, and Z ( l ) is the output of the current transformer layer. After L 1 -layer transformers, we get the encoding output Z ( L 1 ) = [ z ( L 1 ) 1 , . . . , z ( L 1 ) |V| ] ∈ R |V|× N × (3 d ) , and then apply the mean function to similar entities in N association patterns for each node v to obtain the learned embedding z v ∈ R d .

Pattern Commonality. To mine the commonality of N sampled patterns, a score is defined to represent the quantitative relation of these association patterns, termed by Pattern Commonality Coefficient , based on the attention scores A v of trained API block, which is formulated as follows:

where C v ∈ R N and C v [ n ] ∈ (0 , 1) indicates the commonality coefficient of the n -th pattern. Patterns with relatively high commonality coefficients tend to share the same or similar pathways, while showing significant differences in response compared to low commonality patterns. Corresponding biological validations are presented in the Section 5.3.

4.3 Bind-relation Enhancement

In the context of multi-entity relationships, there often exist strong pairwise bind relation between entities, such as drug Aspirin to treat common cold [41, 42]. To mitigate the weakening or overlooking of low-order bind relation, a Bind-relation Enhancement (BE) module is designed to effectively reconstruct the missing feature by capturing these important pairwise associations. Additionally, the module can generate confident and challenging negative samples to aid the training.

Bind-relation Feature Reconstruction. To efficiently learn entity representations in pairwise bind relations, we introduce an edge prediction classification task on the bipartite graphs E A , B , E B , C with the initial embeddings X (0) A = X A , X (0) B 1 = X (0) B 2 = X B , X (0) C = X C . Specifically, a L 2 -layer selfsupervised BGNN model [43] is employed to learn node features on the bipartite graphs, followed by the Multi-layer Perception (MLP) [10] layer to output association probabilities, which is defined as:

where X ( l ) A , X ( l ) B 1 , X ( l ) B 2 , X ( l ) C represent the node features at layer l ; x ( L 2 ) a , x ( L 2 ) b 1 , x ( L 2 ) b 2 , x ( L 2 ) c denote the final learned representations of entity a, b 1 , b 2 , c respectively; ˆ p ( a,b 1 ) , ˆ p ( b 2 ,c ) represent the estimated probability of association between entities a, b 1 and entities b 2 , c respectively. After that, the loss of the supervised prediction task can be formulated as:

The loss L BE of bind-relation task is defined as L BE = α L A , B 1 +(1 -α ) L B 2 , C , where α is the balancing coefficient for the two losses.

Hard Negative Sampling. Based on the above bind-relation task, negative samples are adaptively generated for triple-wise associations, instead of randomly selecting from the vast sample space. Moreover, the generated negative samples are challenging, which contributes to efficient learning. As illustrated in Figure 2, three kinds of negative samples are considered as follows:

where b ′ ∈ B represents another entity with random selection that differs from entity b in the original triplet to form the negative sample ( a, b ′ , c ) ; E ⊗ is the set of generated negative triplets; γ represents the threshold for the prediction probability to determine whether the association exists.

4.4 Total Loss

Base on the above modules, z v of node v learned by the APF module is updated with reconstructed features x v from the BE module according to the entity type, thereby generate the enhanced embedding of v with z ∗ v = z v + x v for the association predictor network. Hence, for the triple-wise association prediction, we utilize the learned embeddings z ∗ a , z ∗ b , and z ∗ c of entity a , b , c to output the probability of the association ˆ p through a scoring function ˆ p ( a,b,c ) = MLP( z ∗ a ∥ z ∗ b ∥ z ∗ c ) . The loss of the association prediction task for true sample set E and negative sample set E ⊗ can be formulated as:

Hence, the total loss L is computed through an alternating training strategy of the two modules, where the BE module is trained for the first 4 epochs of every 5-epoch cycle, followed by the APF module, which is trained for the final epoch of each cycle. During the training of each module, the parameters of the other one are frozen. The equation for L is defined as follows:

where o represents the current epoch; ⌊·⌋ denotes floor function; mod denotes modulo operation.

4.5 Complexity Analysis

Considering the significantly higher complexity of the APF module in comparison to other network components, we only consider APF which aggregates multiple patterns across various entity nodes. Specifically, for one entity node with N sampled patterns from the APS module, the input and hidden features in the MHA layers are of dimension f M , and hidden features in the FFN layers are f F . In APF, the query, key, and value matrices are derived from the same input sequence and share length N . The primary operations for APF include scaled dot-product attention, multiplication of attention weights and values, MHA linear transformation, and FFN linear projection. The time complexity is expressed as O ( N 2 · f M + N · f 2 M + N · f M · f F ) . Hence, for the entire graph, the total complexity is O ( |V| · ( N 2 · f M + N · f 2 M + N · f M · f F ) ) , where |V| is the number of entity nodes.

5 Experiment

5.1 Experimental Settings

Datasets. In this paper, we adopt three biological entity association datasets with significant biological meaning, namely DMD (Drug-Microbe-Disease), DDC (synergistic Drug-Drug-Cell line) and DPA (Drug-target Protein-Adverse reaction), among which DPA dataset is directly constructed. In line with DMD and DDC, we utilize preprocessing tools provided by [21] to deal with the original data from ADReCS-Target [44], and collect a total of 1,079 triplets that are structured into the data schema <drug, protein, adr>. Appendix Table 3 presents the statistics and characteristics of these datasets.

Baselines. To verify the effectiveness of Pattern-BERP, we compare it with three kinds of methods: (1) Non-graph methods. Following the work from [21], five non-graph methods are adopted including Random Forest ( RF ) [45], MLP [10], CP [11], Tucker [11], CoSTCo [12]; (2) Graph-based methods. We select four classical architectures of graph neural network, including GCN [13], GraphSAGE [14], GAT [15], GIN [16]; (3) Hypergraph-based methods. Recent hypergraph learning methods to address triple-wise biological entities associations or similar tasks are considered as the baselines, including DHNE [17], HyperSAGNN [19], HGSynergy [20], MCHNN [21].

Implementation Details. To accurately evaluate model performance and prevent overfitting, 5-fold cross-validation is used to evaluate the performance. Specifically, we randomly split the dataset into a 90% cross-validation set and a 10% independent test set. On the cross-validation set, the 5-fold cross-validation is implemented. Moreover, the independent testing, in which the model is trained on the cross-validation set and tested on the independent test set, is conducted to obtain the prediction results. In the training stage, Binary Cross Entropy loss is adopted to measure model performance and Adam optimizer is adopted to optimize all of model parameters with a learning rate of 0.001 5 .

Evaluations. To evaluate the prediction performance on the triplet associations, hit ratio (hit@n) and normalized discounted cumulative gain (ndcg@n), which are widely used in recommendation tasks [46, 47], are employed for model ability to provide a comprehensive assessment for the task.

5.2 Performance Comparison with Advanced Baselines

Table 1 summarizes the prediction performances of Pattern-BERP in comparison with other baselines across the DMD, DDC and DPA datasets. It is evident that Pattern-BERP significantly surpasses the previous state-of-the-art baselines, including non-graph, graph-based, and hypergraph-based methods, across all three datasets by a large margin, with a particularly notable hit@1 improvement of approximately 23.6% on the DPA dataset (from 33.24 to 43.52). The results underscore the broad accuracy and applicability of the proposed method in various biological association scenarios.

A salient observation is that hypergraph-based methods achieve superior performance than those of the other two types. This phenomenon empirically demonstrates the advantages of utilizing high-order structure information over the other methods that we compare. Additionally, we observe that graphbased methods exceed non-graph methods for DMD and DDC datasets. The relatively large number of associations and proportion of DMD and DDC datasets, as shown in Appendix Table 3, indicate a higher density and stronger interconnectivity within the underlying entity relationships. Hence, graph-based methods acquire abundant information of entity interactions based on the graph structure. Furthermore, hypergraph-based methods acquire high-order structure information, ultimately leading to better performance. In contrast, for the DPA dataset, non-graph methods outperform graph-based methods, even demonstrating competitive performance compared to hypergraph-based methods. This can be attributed to the fact that the association proportion within the DPA dataset is conspicuously low (around 0.002%), indicating a relatively sparse graph structure. Under such conditions, the non-graph methods are able to obtain more robust association information compared to graph-based models, which may struggle to capture meaningful patterns from the limited graph connectivity.

(#53, #41, #80)

(#76, #37, #128)

(#132, #41, #9)

(#102, #38, #8)

2-hop

2-hop

2-hop

(#227, #44, #80)

2-hop

#102 Econazole

#227 Tobramycin

#132 Hetacillin

#53 Cefadroxil

#76 Chlorhexidine acetate

Drug #53

1-hop

1.03%

1.58%

1.57%

1.55%

0.001%

...

...

(a)

(#207, #54, #55)

(#100, #54, #47)

(#209, #54, #62)

(#78, #28, #69)

2-hop

2-hop

2-hop

(#160, #16, #110)

2-hop

#209 Salicylic Acid

#160 Moxifloxacin

#207 S-(2-Thienyl)-L-cysteine

#78 Chrysophanic Acid

Drug #207

1-hop

1.37%

2.98%

2.98%

2.98%

0.001%

...

...

(b)

HO

NH

HO

HN

HO

OH

HO

H,N

HO

NH

OH

OH

OH

HO

HO

5.3 Association Pattern Interpretability

In order to investigate the potential relation among different association patterns, we visualize the pattern commonality coefficients of Cefadroxil (drug #53) and S-(2-Thienyl)-L-cysteine (drug #207) in DMD dataset, as shown in Figure 3. Larger pattern commonality coefficients contribute more to the original drug pathway, whereas smaller coefficients often relate to different biological mechanisms. It can be observed that:

For Cefadroxil (drug #53), the common patterns consisted by Chlorhexidine acetate (drug #76) [48], Hetacillin (drug #132) [49], and Econazole (drug #102) [50, 51] are considered more similar as the mechanism [52] that actives on the cell wall and envelope leading the change of microbe physiological activities, thus treating the diseases. Instead, Tobramycin (drug #227) inhibits mRNA be translated into protein and thus promotes microbe cell death [53, 54].

For S-(2-Thienyl)-L-cysteine (drug #207), the common patterns consisted by Diphenyl Disulfide (drug #100) [55, 56], Salicylic Acid (drug #209) [57], and Chrysophanic Acid (drug #78) [58, 59] are considered more similar as the mechanism that [60-62] inducts oxidative stress in bacteria and thus damage all components of the microbe cell. Instead, Moxifloxacin (drug #160) inhibits DNA gyrase and topoisomerase IV enzymes to separate bacterial DNA, thereby inhibiting cell replication [63, 64].

In summary, the analyzed cases illustrate the high-commonality patterns derived from Pattern-BERP, demonstrating how these small molecule drugs influence microbial activities to treat diseases, generally following a consistent physiological route. These cases indicate that our method can discover potential common patterns through the API block, and obtain larger pattern coefficients through the attention mechanism, which drives the nodes to learn these important common interactions and acquire more expressive representations. Details about additional cases can be found at Appendix B.3.2.

5.4 Ablation Study

To investigate the necessity of each component in Pattern-BERP, we conduct several comparisons between Pattern-BERP and its variants on the test set. As illustrated in Table 2, when basic components of Pattern-BERP have been removed, the performances of corresponding variants on DDC dataset significantly decline, indicating that these components all contribute to the performance. Besides, we have other observations: (1) when only DE module is removed, the performance is inferior to the one removing the entire APA module containing DE, demonstrating that inaccurate entity distance information has a more detrimental impact on the prediction performance; (2) eliminating HNS module results in a significant drop in the performance, highlighting its crucial role in enhancing the model's robustness and discrimination capability and thus indicating that the proposed negative sampling strategy contributes to efficient learning; (3) when removing HC module leads to a slight performance degradation, the impact is relatively limited which suggests that the capability of HC module in representing complex associations is relatively modest for the highly dense-association DDC dataset, but it still provides some beneficial effects towards the final performance improvement.

To verify the affect of hyperparameter settings to model performance, we first conduct ablation experiments on the three main parameters of the APF module, namely number of attention heads, number of max pattern distance, and number of sampled association patterns. As illustrated in Appendix Figure 5, when the three hyperparameters are increased, the prediction performance on DDC dataset exhibits an overall upward trend, suggesting that increasing these hyperparameters helps the model better capture the complex association patterns, thereby improving the final performance. Additionally, we conduct experiments on the loss-balanced coefficient α and the bind-relation prediction probability threshold γ , both adjusted from 0.1 to 0.9. Results in Appendix Figure 6 show that setting α and γ to 0.5 yields the best performance. Specifically, for α , since the final prediction task involves predicting the associations among entity A , B , and C , the two tasks of A → B and B → C are intuitively of equal importance, therefore the balanced coefficient α set to 0.5; for γ , bind-relation prediction is fundamentally a binary classification task, thus the threshold γ is set to 0.5. Details about ablation experiments are presented at Appendix B.3.3.

6 Conclusion

In this work, we propose a novel association pattern-aware fusion method Pattern-BERP for biological entity relationship prediction, which effectively combines the related association pattern information into entity representation learning. In addition, to enhance the missing information of the low-order message passing, we devise a bind-relation module that considers the strong bind of low-order entity associations. The evaluation on three biological datasets quantitatively demonstrate that the proposed method consistently achieve superior performance over the competing baselines. Moreover, the interpretability explanations of association patterns reveal the intrinsic biological mechanisms and thus promote the method to be deployed in real-world scenarios.

Due to the domain-specific task, Pattern-BERP focuses on the fixed-length association patterns. Extending the approach to capture variable-length pathways could further enhance the representational power. Additionally, exploring the applicability of the method in other domains beyond biology, such as general knowledge graph completion, would help evaluate its broader generalizability.

Acknowledgments and Disclosure of Funding

This work is supported by the Zhejiang Province High-Level Talents Special Support Program 'Leading Talent of Technological Innovation of Ten-Thousands Talents Program' (No. 2022R52046), the Fundamental Research Funds for the Central Universities (226-2024-00145), and the Scientific Research Fund of Zhejiang University (No. XY2023020).

References

Shengbo Wu, Shujuan Yang, Manman Wang, Nan Song, Jie Feng, Hao Wu, Aidong Yang, Chunjiang Liu, Yanni Li, Fei Guo, et al. Quorum sensing-based interactions among drugs, microbes, and diseases. Science China Life Sciences , 66(1):137-151, 2023.

Lei Wang, Yaqin Tan, Xiaoyu Yang, Linai Kuang, and Pengyao Ping. Review on predicting pairwise relationships between human microbes, drugs and diseases: from biological data to computational models. Briefings in Bioinformatics , 23(3):bbac080, 2022.

Jia Jia, Feng Zhu, Xiaohua Ma, Zhiwei W Cao, Yixue X Li, and Yu Zong Chen. Mechanisms of drug combinations: interaction and network perspectives. Nature reviews Drug discovery , 8 (2):111-128, 2009.

Mathew J Garnett, Elena J Edelman, Sonja J Heidorn, Chris D Greenman, Anahita Dastur, King Wai Lau, Patricia Greninger, I Richard Thompson, Xi Luo, Jorge Soares, et al. Systematic identification of genomic markers of drug sensitivity in cancer cells. Nature , 483(7391):570-575, 2012.

Lun Yang, Jian Chen, and Lin He. Harvesting candidate genes responsible for serious adverse drug reactions from a chemical-protein interactome. PLoS computational biology , 5(7): e1000441, 2009.

Lun Yang, Kejian Wang, Jian Chen, Anil G Jegga, Heng Luo, Leming Shi, Chunling Wan, Xizhi Guo, Shengying Qin, Guang He, et al. Exploring off-targets and off-systems for adverse drug reactions via chemical-protein interactome-clozapine-induced agranulocytosis as a case study. PLoS computational biology , 7(3):e1002016, 2011.

Anna Gogleva, Dimitris Polychronopoulos, Matthias Pfeifer, Vladimir Poroshin, Michaël Ughetto, Matthew J Martin, Hannah Thorpe, Aurelie Bornot, Paul D Smith, Ben Sidders, et al. Knowledge graph-based recommendation framework identifies drivers of resistance in egfr mutant non-small cell lung cancer. Nature communications , 13(1):1667, 2022.

Yiyuan Pu, Daniel Beck, and Karin Verspoor. Graph embedding-based link prediction for literature-based discovery in alzheimer's disease. Journal of Biomedical Informatics , 145: 104464, 2023.

Ali Daowd, Samina Abidi, and Syed Sibte Raza Abidi. A knowledge graph completion method applied to literature-based discovery for predicting missing links targeting cancer drug repurposing. In International Conference on Artificial Intelligence in Medicine , pages 24-34. Springer, 2022.

Fionn Murtagh. Multilayer perceptrons for classification and regression. Neurocomputing , 2(5): 183-197, 1991. ISSN 0925-2312.

Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review , 51 (3):455-500, 2009.

Hanpeng Liu, Yaguang Li, Michael Tsang, and Yan Liu. Costco: A neural tensor completion model for sparse tensors. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining(KDD) , pages 324-334, 2019.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations(ICLR) , 2016.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems(NeurIPS) , 30, 2017.

Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations(ICLR) , 2018.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations(ICLR) , 2019.

Ke Tu, Peng Cui, Xiao Wang, Fei Wang, and Wenwu Zhu. Structural deep embedding for hyper-networks. In Proceedings of the AAAI conference on artificial intelligence(AAAI) , pages 426-433, 2018.

Jianwen Jiang, Yuxuan Wei, Yifan Feng, Jingxuan Cao, and Yue Gao. Dynamic hypergraph neural networks. In Proceedings of International Joint Conferences on Artificial Intelligence(IJCAI) , 2019.

Ruochi Zhang, Yuesong Zou, and Jian Ma. Hyper-sagnn: a self-attention based graph neural network for hypergraphs. In International Conference on Learning Representations (ICLR) , 2020.

Xuan Liu, Congzhi Song, Shichao Liu, Menglu Li, Xionghui Zhou, and Wen Zhang. Multi-way relation-enhanced hypergraph representation learning for anti-cancer drug synergy prediction. Bioinformatics , 38(20):4782-4789, 2022.

Luotao Liu, Feng Huang, Xuan Liu, Zhankun Xiong, Menglu Li, Congzhi Song, and Wen Zhang. Multi-view contrastive learning hypergraph neural network for drug-microbe-disease association prediction. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence(IJCAI) , pages 4829-4837, 8 2023.

Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence(AAAI) , pages 3558-3565, 2019.

Huiyuan Chen and Jing Li. Modeling relational drug-target-disease interactions via tensor factorization with multiple web sources. In Proceedings of the 28th international conference on World Wide Web(WWW) , pages 218-227, 2019.

Huiyuan Chen and Jing Li. Learning data-driven drug-target-disease interaction via neural tensor network. In International joint conference on artificial intelligence (IJCAI) , 2020.

Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems , 30(1-7):107-117, 1998.

Glen Jeh and Jennifer Widom. Simrank: a measure of structural-context similarity. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining(KDD) , pages 538-543, 2002.

Glen Jeh and Jennifer Widom. Scaling personalized web search. In Proceedings of the 12th international conference on World Wide Web(WWW) , pages 271-279, 2003.

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining(KDD) , pages 701-710, 2014.

Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S Yu, and Tianyi Wu. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment , 4(11):992-1003, 2011.

Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining(KDD) , pages 135-144, 2017.

Yuchen Wang, Jinghui Zhang, Zhengjie Huang, Weibin Li, Shikun Feng, Ziheng Ma, Yu Sun, Dianhai Yu, Fang Dong, Jiahui Jin, et al. Label information enhanced fraud detection against low homophily in graphs. In Proceedings of the ACM Web Conference 2023(WWW) , pages 406-416, 2023.

Xiaowen Chen, Hongbo Shi, Feng Yang, Lei Yang, Yingli Lv, Shuyuan Wang, Enyu Dai, Dianjun Sun, and Wei Jiang. Large-scale identification of adverse drug reaction-related proteins through a random walk model. Scientific reports , 6(1):36325, 2016.

Francois Fouss, Alain Pirotte, Jean-Michel Renders, and Marco Saerens. Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Transactions on knowledge and data engineering(TKDE) , 19(3):355-369, 2007.

Reid Andersen, Fan Chung, and Kevin Lang. Local graph partitioning using pagerank vectors. In 2006 47th Annual IEEE Symposium on Foundations of Computer Science(FOCS) , pages 475-486. IEEE, 2006.

Zihao Zhao, Jiawei Chen, Sheng Zhou, Xiangnan He, Xuezhi Cao, Fuzheng Zhang, and Wei Wu. Popularity bias is not always evil: Disentangling benign and harmful bias for recommendation. IEEE Trans actions on Knowledge and Data Engineering , 35(10):9920-9931, 2022.

Jiawei Chen, Junkang Wu, Jiancan Wu, Xuezhi Cao, Sheng Zhou, and Xiangnan He. Adapτ : Adaptively modulating embedding magnitude for recommendation. In Proceedings of the ACM Web Conference 2023 , pages 1085-1096, 2023.

Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems , 41(3):1-39, 2023.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems(NeurIPS) , 30, 2017.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations , 2021.

Yue Gao, Yifan Feng, Shuyi Ji, and Rongrong Ji. Hgnn+: General hypergraph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence(TPAMI) , 45(3):3181-3199, 2022.

Ronald Eccles. Efficacy and safety of over-the-counter analgesics in the treatment of common cold and flu. Journal of clinical pharmacy and therapeutics , 31(4):309-319, 2006.

Michael D Kogan, Gregory Pappas, M Yu Stella, and Milton Kotelchuck. Over-the-counter medication use among us preschool-age children. Jama , 272(13):1025-1030, 1994.

Chaoyang He, Tian Xie, Yu Rong, Wenbing Huang, Junzhou Huang, Xiang Ren, and Cyrus Shahabi. Cascade-bgnn: Toward efficient self-supervised representation learning on large-scale bipartite graphs. arXiv preprint arXiv:1906.11994 , 2019.

Li-Hong Huang, Qiu-Shun He, Ke Liu, Jiao Cheng, Min-Dong Zhong, Lin-Shan Chen, LiXia Yao, and Zhi-Liang Ji. Adrecs-target: target profiles for aiding drug safety research and application. Nucleic acids research , 46(D1):D911-D917, 2018.

Leo Breiman. Random forests. Machine learning , 45:5-32, 2001.

Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. Improving sequential recommendation with knowledge-enhanced memory networks. In The 41st international ACM SIGIR conference on research & development in information retrieval(SIGIR) , pages 505-514, 2018.

Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji-Rong Wen, and Daxiang Dong. Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In Proceedings of the twelfth ACM international conference on web search and data mining(WSDM) , pages 573-581, 2019.

Jerrold B. Leikin and Frank P. Paloucek. Chlorhexidine gluconate. In Jerrold B. Leikin and Frank P. Paloucek, editors, Poisoning and Toxicology Handbook , pages 183-184. Informa, 4th edition, 2008.

Jed F Fisher, Samy O Meroueh, and Shahriar Mobashery. Bacterial resistance to β -lactam antibiotics: compelling opportunism, compelling opportunity. Chemical reviews , 105(2):395424, 2005.

D Thienpont, J Van Cutsem, JM Van Nueten, CJ Niemegeers, and R Marsboom. Bilogical and toxicological properties of econazole, a broad-spectrum antimycotic. Arzneimittel-forschung , 25(2):224-230, 1975.

Matthew C Fisher, Ana Alastruey-Izquierdo, Judith Berman, Tihana Bicanic, Elaine M Bignell, Paul Bowyer, Michael Bromley, Roger Brüggemann, Gary Garber, Oliver A Cornely, et al. Tackling the emerging threat of antifungal resistance to human health. Nature reviews microbiology , 20(9):557-571, 2022.

Donald J Tipper and Jack L Strominger. Mechanism of action of penicillins: a proposal based on their structural similarity to acyl-d-alanyl-d-alanine. Proceedings of the National Academy of Sciences , 54(4):1133-1141, 1965.

Grace Yang, Joanna Trylska, Yitzhak Tor, and J Andrew McCammon. Binding of aminoglycosidic antibiotics to the oligonucleotide a-site model and 30s ribosomal subunit: Poissonboltzmann model, thermal denaturation, and fluorescence studies. Journal of medicinal chemistry , 49(18):5478-5490, 2006.

Jalal Haddad, Lakshmi P Kotra, Beatriz Llano-Sotelo, Choonkeun Kim, Eduardo F Azucena, Meizheng Liu, Sergei B Vakulenko, Christine S Chow, and Shahriar Mobashery. Design of novel antibiotics that bind to the ribosomal acyltransfer site. Journal of the American Chemical Society , 124(13):3229-3237, 2002.

R Munday. Toxicity of aromatic disulphides. ii. intra-erythrocytic hydrogen peroxide formation and oxidative damage by aromatic disulphides. Journal of applied toxicology , 5(6):409-413, 1985.

Victoria Osipova, Yulia Gracheva, Maria Polovinkina, Daria Burmistrova, and Nadezhda Berberova. Antioxidant activity and cytotoxicity of aromatic oligosulfides. Molecules , 27(12): 3961, 2022.

Warren W Kaeding. Oxidation of aromatic acids. iv. decarboxylation of salicylic acids. The Journal of Organic Chemistry , 29(9):2556-2559, 1964.

Chien-Hang Ni, Po-Yuan Chen, Hsu-Feng Lu, Jai-Sing Yang, Hui-Ying Huang, Shin-Hwar Wu, Siu-Wan Ip, Chin-Tung Wu, Su-Yin Chiang, Jaung-Geng Lin, et al. Chrysophanol-induced necrotic-like cell death through an impaired mitochondrial atp synthesis in hep3b human liver cancer cells. Archives of pharmacal research , 35:887-895, 2012.

Longfei Lin, Yuling Liu, Ming Zhong, Tanggui Xie, Jian Ni, Hui Li, et al. Hepatotoxicity and mechanism study of chrysophanol-8-o-glucoside in vitro. Biomedicine & Pharmacotherapy , 120:109531, 2019.

Rajindar S Sohal. Role of oxidative stress and protein oxidation in the aging process. Free Radical Biology and Medicine , 33(1):37-44, 2002.

Bernd Moosmann and Christian Behl. Mitochondrially encoded cysteine predicts animal lifespan. Aging cell , 7(1):32-46, 2008.

Candice E Paulsen and Kate S Carroll. Cysteine-mediated redox signaling: chemistry, biology, and tools for discovery. Chemical reviews , 113(7):4633-4679, 2013.

Edwin M Ory and Ellard M Yow. The use and abuse of the broad spectrum antibiotics. Jama , 185(4):273-279, 1963.

Karl Drlica and Xilin Zhao. Dna gyrase, topoisomerase iv, and the 4-quinolones. Microbiology and molecular biology reviews , 61(3):377-392, 1997.

Serkan Kiranyaz, Onur Avci, Osama Abdeljaber, Turker Ince, Moncef Gabbouj, and Daniel J Inman. 1d convolutional neural networks and applications: A survey. Mechanical systems and signal processing , 151:107398, 2021.

James Z Wang, Zhidian Du, Rapeeporn Payattakool, Philip S Yu, and Chin-Fu Chen. A new method to measure the semantic similarity of go terms. Bioinformatics , 23(10):1274-1281, 2007.

Susan L Holbeck, Richard Camalier, James A Crowell, Jeevan Prasaad Govindharajulu, Melinda Hollingshead, Lawrence W Anderson, Eric Polley, Larry Rubinstein, Apurva Srivastava, Deborah Wilsker, et al. The national cancer institute almanac: a comprehensive screening resource for the detection of anticancer drug pairs with enhanced therapeutic activity. Cancer research , 77(13):3564-3576, 2017.

Yuanjing Ma and Hongmei Jiang. Ninimhmda: neural integration of neighborhood information on a multiplex heterogeneous network for multiple types of human microbe-disease association. Bioinformatics , 36(24):5665-5671, 2020.

Yanli Wang, Jewen Xiao, Tugba O Suzek, Jian Zhang, Jiyao Wang, and Stephen H Bryant. Pubchem: a public information system for analyzing bioactivities of small molecules. Nucleic acids research , 37(suppl_2):W623-W633, 2009.

Emmanuel Boutet, Damien Lieberherr, Michael Tognolli, Michel Schneider, and Amos Bairoch. Uniprotkb/swiss-prot: the manually annotated section of the uniprot knowledgebase. In Plant bioinformatics: methods and protocols , pages 89-112. Springer, 2007.

Mei-Chun Cai, Quan Xu, Yan-Jing Pan, Wen Pan, Nan Ji, Yin-Bo Li, Hai-Jing Jin, Ke Liu, and Zhi-Liang Ji. Adrecs: an ontology database for aiding standardization and hierarchical classification of adverse drug reaction terms. Nucleic acids research , 43(D1):D907-D913, 2015.

Amit S Kalgutkar. Designing around structural alerts in drug discovery. Journal of Medicinal Chemistry , 63(12):6276-6302, 2019.

Sidney D Nelson. Metabolic activation and drug toxicity. Journal of medicinal chemistry , 25 (7):753-765, 1982.

Allan B Bailey, Ronald Chanderbhan, Nancy Collazo-Braier, Mitchell A Cheeseman, and Michelle L Twaroski. The use of structure-activity relationship analysis in the food contact notification program. Regulatory Toxicology and Pharmacology , 42(2):225-235, 2005.

Lingxiang Jia, Zunlei Feng, Haotian Zhang, Jie Song, Zipeng Zhong, Shaolun Yao, and Mingli Song. Explainable fragment-based molecular property attribution. Advanced Intelligent Systems , 4(10):2200104, 2022.

RP Elander. Industrial production of β -lactam antibiotics. Applied microbiology and biotechnology , 61(5):385-392, 2003.

Md Badrul Islam, Md Inshaful Islam, Nikhil Nath, Talha Bin Emran, Md Rezaur Rahman, Rohit Sharma, Mohammed Mahbubul Matin, et al. Recent advances in pyridine scaffold: Focus on chemistry, synthesis, and antibacterial activities. BioMed research international , 2023, 2023.

Rajib K Singh, Arvind Kumar, and Arun K Mishra. Chemistry and pharmacology of acetanilide derivatives: a mini review. Letters in Organic Chemistry , 16(1):6-15, 2019.

George G Zhanel, James A Karlowsky, Ethan Rubinstein, and Daryl J Hoban. Tigecycline: a novel glycylcycline antibiotic. Expert review of anti-infective therapy , 4(1):9-25, 2006.

Pili Hu and Wing Cheong Lau. A survey and taxonomy of graph sampling. arXiv preprint arXiv:1308.5865 , 2013.

Appendix

In this appendix, we provide a comprehensive elaboration of the methodologies, experimental details, and additional insights that support the findings presented in the main manuscript. The appendix is structured into details of the proposed method, details of the experiments, and other discussion contents including limitation and extension. Furthermore, our data, code and all raw experimental results are provided in the Github repository https://github.com/hry98kki/PatternBERP .

A Details of Pattern-BERP Method

A.1 Entity Attribute

Before constructing the graphs, the attributes of each entity are initialized as node features on the graphs with the domain knowledge of bio-entities. Due to the varying types of entities being studied, multiple encoders are employed to align the attribute embeddings for different types of entities. Hence, three specific types of encoders are considered as follows:

For drug entities D , their SMILES strings can be converted into molecular graphs G D = ( S D , A D ) , where S D is the attribute matrix of all nodes representing the atoms and A D is the adjacency matrix of these nodes. Vallina GIN [16] is adopted to learn atom representations and then these atom representations are summarized into a drug-level feature vector through a global max pooling (GMP), which is formulated as:

where l is the current epoch of GIN, I is the identity matrix, ϵ is a fixed scalar, and X (0) D = S D . After applying the GMP over all molecular graphs, the features of all drug entities can be compiled into X D ∈ R |D|× d , where d represents the dimension of entity attribute vector.

For protein entities P , taken the one-dimensional protein sequence T as the input, we first convert the sequence string to an integer vector as the initialized embedding S P ∈ R T . Then, considering X (0) P = S P , the 1D CNN [65] model is used to extract the protein representation. The propagation mechanism of each CNN layer works as follows:

where X ( l ) P , X ( l +1) P are the hidden feature vectors of the l th and ( l +1) th CNN layer, respectively; d ( l ) in , d ( l ) out , ks ( l ) are the number of channels in the input, number of channels produced by the convolution and the convolving kernel size of the l th CNN layer; σ ( · ) represents nonlinear activation function, specially ReLU. After CNN layers, the d -dimensional feature vectors of all target proteins are denoted as X P ∈ R |P|× d .

For other entities O (such as diseases), we compile similarity matrices to acquire the initial embedding S O ∈ R |O|×|O| based on the methods provided by [32, 66-68] and followed by fully-connected networks to obtain the entity representation, denoted as X O ∈ R |O|× d . More specifically, for microbe nodes, cell lines and disease nodes, we compile similarity matrices S M ∈ { 1 , 0 } |M|×|M| , S L ∈ R |L|×|L| and S N ∈ R |N|×|N| based on the methods provided by [66-68], which then are transformed into X M ∈ R |M|× d , X L ∈ R |L|× d and X N ∈ R |N|× d by fully-connected networks. For ADRs nodes, we first use the co-occurrence of drugs to evaluate ADRs similarity. For two ADRs i and j , the Jaccard score is calculated as follows:

where D i and D j denote drug sets that cause ADR i and ADR j , respectively. The ADR similarity matrix S R ∈ R |R|×|R| is constructed according to the Jaccard score, finally denoted as X R ∈ R |R|× d .

Based on the above construction rules, we generate the corresponding initial attributes for entity datasets A , B , and C through the entity types. Finally, the entity attributes X consists of features X A , X B and X C . When encountering specific datasets, the three entity types can be substituted into ( A , B , C ) respectively. For example, X consists of X D , X P and X R in DPA dataset.

A.2 Association Pattern Distance

Given the maximum number of hops U (i.e., the max positional distance for certain entity node), the distance matrix D ∈ R |V|×|E| is constructed based on Definition 1 as follows:

1-hop distance: For each node v and hyperedge e , if hyperedge j directly contains node i , then D v,e = 1 represents the association of hyperedge j is the 1-hop pattern.

2-hop distance: If hyperedge e does not directly contain node v but shares a node with a hyperedge that does, then D v,e = 2 represents the association of hyperedge e is the 2-hop pattern.

u -hop distance: Recursively calculate further distances up to u -hop, then D v,e = u .

Unreachable: If node v cannot reach hyperedge e within U -hop, then D v,e = -∞ .

Hence, the distance tokens D v for node v are transformed by one position encoding layer, thereby yielding the positional embeddings for the self-attention mechanism of the subsequent API block.

A.3 Theory Analysis

As an empirical method, the Pattern-BERP method extracts common patterns or rules from large bio-associated networks, similar to the k-means clustering. Specifically, given a triplet-wise dataset S = { s 1 , s 2 , . . . , s t , . . . , s T } and K common patterns, the optimization goal is to minimize the total distance of sampled patterns to their respective pattern centers. First, initialize the pattern centers M = { m 1 , m 2 , . . . , m k , . . . , m K } . Next, assign each sample to the nearest pattern center by computing the center index with arg min k ∥ s i -m k ∥ for all the centers in M . Then, update the position of these pattern centers using m k = 1 | C k | ∑ s t ∈ C k s t , where C k is the set of samples assigned to pattern k calculated as above. Iterate these assignment and update steps until the pattern centers converge. The objective function to minimize is J = ∑ K k =1 ∑ s t ∈ C k ∥ s t -m k ∥ 2 , and by minimizing J , Pattern-BERP extracts representative biological association patterns effectively.

B Details of the Experiments

B.1 Datasets

In this paper, three biological datasets are used to evaluate the efficacy of the proposed method. Each dataset encompasses three different entities and their associations. Appendix Table 3 provides a detailed presentation of the specific entities within each dataset, including the count of nodes per entity, the number of associations among these nodes, and the corresponding association ratio. All entity associations are structured into triplet scheme, such as <drug, protein, adr> for DPA dataset.

DPA dataset. DPA dataset is first constructed and preprocessed with the origin data from [44]. Specifically, only <drug, target, adr> triplets that have complete field information are retained, that is, drug Pubchem CID, target protein identifier UniProt ID, ADR term name. Then, the detail information is obtained through these unique identifiers, respectively. For drugs, the PubChem database [69] is queried with the PubChem CID of each drug, and their Canonical SMILES are recorded. For proteins, the UniProtKB database [70] is queried with the UniProtKB ID of each protein, and their protein sequences are recorded. For ADRs, the incidence matrix about the reaction of specific ADRs to ∼ 15,000 drugs is obtained from publicly available datasets [71]. In this matrix, a value of 1 indicates that a particular drug causes a specific ADR, while a value of 0 denotes no relation between them. Furthermore, following the method described in Appendix A.1, the ADR similarity matrix can be constructed for subsequent feature generation fo ADR entities.

B.2 Details of Implementation

B.2.1 Implementation of Baselines

The implementation of all baseline methods is conducted using their respective publicly accessible source codes. Optimal or default configurations for each method are employed to ensure robustness. Specifically, for methods such as RF, MLP [10], CP and Tucker [11], meticulous parameter tuning is engaged in to elicit their peak performance levels. For GCN [13], GAT [15], GraphSAGE [14] and GIN [16], the original triple-wise associations are initially decomposed into two pair-wise associations. Subsequently, each pair-wise association is independently modeled using the corresponding graph neural network model, and the prediction probabilities of the two pair-wise associations are multiplied to obtain the final prediction value for the triple-wise association. Additionally, for DHNE [17] and HyperSAGNN [19], biological embeddings are integrated with their original structural embeddings to ensure fairness. For CoSTCo [12], HGSynergy [20] and MCHNN [21], the parameter settings outlined in the original publications are followed. It is important to note that for the sake of equitable comparison, all of these methods utilize negative sampling setting from MCHNN that is consistent across implementations.

B.2.2 Implementation Settings

The initialized entity embedding size d is fixed to 128. The number of BGNN, APF layers are all fixed to 2. The training epoch is setting to 1,000 for DMD, DPA datasets and 2,000 for DDC dataset. The number of max hop in pattern sampling U is setting to 3. The number of sampled patterns N is setting to 100. The number of attention heads is 32 for DMD dataset, 16 for DDC dataset, 4 for DPA dataset. The loss-balanced coefficient for bind-relation task α is fixed to 0.5. The threshold for bind-relation prediction probability γ is fixed to 0.5. In addition, in line with the evaluation strategy of [21], 29 negative samples are randomly generate for each test triplet in four scenarios for the evaluation of all methods, and thus display the average metrics over all test triplets. These scenario settings can comprehensively evaluate the model ability to identify positive and negative samples under these stringent conditions.

Furthermore, all experiments are conducted on a single NVIDIA A6000 Tensor Core GPU (48GB) and Intel(R) Xeon CPU with 24 cores and 500G memory. The whole training time for DMD, DDC, DPA datasets is about 8, 8, 4 hours, respectively. In addition, Appendix Table 4 exhibits the GPU memory usage with varying the number of sampled patterns N across three datasets, and Appendix Figure 7 presents the inference time of Pattern-BERP in comparison with these advanced baselines for each 100 samples with milliseconds.

B.3 Details of Results

B.3.1 Additional Performance Comparison on the ndcg

As a supplement to Table 1, Appendix Table 5 incorporates additional metrics ndcg@1, ndcg@3 and ndcg@5. Considering all these metrics, Pattern-BERP significantly outperforms previous SOTA baselines on all three datasets, exhibiting the remarkable advantages of association pattern mining.

Furthermore, each result of these methods is from the average of 5-fold cross-validation experiments with four scenarios. However, significant variation in prediction difficulty across different scenarios makes it relatively unreasonable to provide an error bar for all 20 results. Instead, to demonstrate the statistical validity of Pattern-BERP, we have provided the raw experimental results.

(#52, #41, #8)

(#52, #41, #26)

(#53, #41, #8)

(#162, #50, #9)

(#207, #2, #26)

1-hop

2-hop

2-hop

2-hop

(#266, #41, #8)

2-hop

Drug #207

Drug #266

Drug #162

Drug #53

Drug #52

Drug #52

1-hop

0.41%

0.65%

2.23%

2.05%

2.03%

0.31%

...

...

NH.

HO

OH

HO

OH

NH

B.3.2 Additional Interpretability Analysis

In order to investigate the potential relation among different association patterns, we visualize the pattern coefficients of drug #52 in DMD dataset, as shown in Appendix Figure 4. It can be observed that the patterns that make important contributions are not necessarily 1-hop patterns, with pattern coefficients of 0.41% and 0.65%. In contrast, the 2-hop patterns can exhibit considerable relevance, due to the similar mechanisms of toxicity against microbes exhibited by the corresponding drugs (#53, #162, #207) compared to the target drug #52 under investigation. Additionally, the weakening of drug #52 itself is because the original drug #52 is a complex peptide structure with repetitive and redundant information, thus acquiring simpler and more straightforward representations through the aforementioned information interaction.

Furthermore, from the view of structural and functional groups mentioned by [72-75], we have several salient observations as follows: (1) As the drug in the most influenced pattern, drug #53

contains β -lactam as shown in the red rectangle of Figure 3. β -lactam is the crucial component of β -lactam antibiotics [76], which is one of the most widely used classes of antibiotics available. In addition, Islam et al. [77] also prove that pyridine scaffold (aromatic ring with nitrogen) bearing poor basicity generally improves water solubility in pharmaceutically potential molecules and has led to the discovery of numerous broad-spectrum therapeutic agents. (2) Similar with drug #53, drug #162 has acetanilide structure, which serves as the basis for antimicrobial activity and disease treatment [78]. (3) Drug #207 has the dimethylglycine structure, which is crucial to Tigecycline [79]. Tigecycline binds to the bacterial ribosome, blocking the binding of amino-acyl-tRNA to the acceptor site on the mRNA-ribosome complex, thereby inhibiting protein synthesis. (4) Unlike the previous three, drug #266 may interact with membrane lipids to alter membrane fluidity and permeability, thereby exerting its effect on microorganisms. However, this mechanism of action is distinct from the specific functional group interactions of the previous three drugs.

As a compound with a complex structure containing multiple functional groups, drug #52 is similar to the previous three drugs due to its complex structure and diverse functional groups. These characteristics enable it to exert toxicity through specific interactions with the critical pocket of microbes. Compared to drug #266, these complex drugs have more diverse and specific toxicity mechanisms, which is why they have received less attention.

B.3.3 Ablation Results

Network Module Designs. To investigate the necessity of each component in Pattern-BERP, we conduct several comparisons between Pattern-BERP and its variants on the independent test set. Table 2, 6, 7 denotes the results of DDC, DMD, DPA dataset, respectively.

When basic components of Pattern-BERP have been removed, the performances of corresponding variants on all datasets exhibit significant declines, though the degree of performance degradation varies. This indicates that these components all contribute to the performance. Besides, we have other observations: (1) for the relatively sparse datasets, DMD and DPA, the performance decline after removing BFR is more significant compared to the DDC dataset. (2) for all datasets, eliminating HNS module results in a drop in the performance, highlighting its effect in enhancing the model's robustness

hits@1

hits@3

hits@5

Metric

91

92

93

94

95

96

97

98

99

Performance

No. of Attention Head

1

2

4

8

16

32

(a)

hits@1

hits@3

hits@5

Metric

40

45

50

55

60

65

70

75

80

Performance

No. of Attention Head

1

2

4

8

16

32

(b)

hits@1

hits@3

hits@5

Metric

40

45

50

55

60

65

Performance

No. of Attention Head

1

2

4

8

16

32

(c)

hits@1

hits@3

hits@5

Metric

92

93

94

95

96

97

98

99

Performance

1

2

3

4

(d)

hits@1

hits@3

hits@5

Metric

45

50

55

60

65

70

75

80

Performance

1

2

3

4

(e)

hits@1

hits@3

hits@5

Metric

40

45

50

55

60

65

Performance

1

2

3

4

(f)

hits@1

hits@3

hits@5

Metric

93

94

95

96

97

98

99

Performance

5

10

20

50

100

(g)

hits@1

hits@3

hits@5

Metric

45

50

55

60

65

70

75

80

Performance

5

10

20

50

100

(h)

hits@1

hits@3

hits@5

Metric

40

45

50

55

60

65

Performance

5

10

20

50

100

(i)

and discrimination capability and thus indicating that the proposed sampling strategy contributes to efficient learning; (3) for all datasets, when only DE module is removed, the performance is inferior to the one removing the entire API module containing DE, demonstrating that inaccurate entity distance information has a more detrimental impact on the prediction performance; (4) when removing HC module leads to a slight performance degradation, the impact is relatively limited which suggests that the capability of HC module in representing complex associations is relatively modest for the relatively dense-association DDC and DMD dataset, but it still provides some beneficial effects towards the final performance improvement. But for highly sparse DPA datasets, removing HC module leads to performance improvement, demonstrating that the combined performance of HC and APA is instead constrained by the confusion introduced by the two aggregation modes on the extremely sparse structures.

Hyperparamter Settings. To verify the ablation of hyperparameter settings, we conduct ablation experiments on the three main parameters, namely number of attention heads, number of max pattern distance, and number of sampled association patterns. As illustrated in Appendix Figure 5, when the number of attention head are increased, the prediction performance on all datasets exhibits an

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Loss-balanced coefficient

92.5

93.0

93.5

94.0

hit@1 (%)

(a)

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Bind-relation threshold

91.5

92.0

92.5

93.0

93.5

94.0

hit@1 (%)

(b)

DMD

DDC

DPA

0.0

2.5

5.0

7.5

10.0

12.5

Time (ms)

Methods

RF

MLP

CP

CoSTCo

GCN

GraphSAGE

GAT

GIN

DHNE

HyperSAGNN

HGSynergy

MCHNN

Pattern-BERP

overall upward trend, suggesting that increasing the attention heads helps the model better capture the complex association patterns, thereby improving the final performance. In addition, given the significant disparity in the association ratios across the three datasets, the performance of DDC dataset improves as the max hop hyperparameter is increased. In contrast, for the relatively sparse DMDdataset, it is advisable to restrict the max hop to the range from 1 to 3, as the performance drops sharply when the parameter is set to 4. Moreover, for the sparsest DPA dataset, selecting only 1-hop patterns appears to be the optimal choice. Moreover, regarding the number of sampled patterns, a similar trend is observed across the datasets. For DMD and DPA datasets, which are relatively sparse, increasing the number of sampled patterns is not necessarily beneficial for performance. In contrast, on DDC dataset, augmenting the number of sampled patterns continues to yield performance.

Furthermore, we also conduct ablation experiments on another two parameters, namely loss-balanced coefficient α and the bind-relation prediction probability threshold γ . Results in Appendix Figure 6 show that when α and γ are both set to 0.5 by our default, the performance surpasses those of other settings. Specifically, for α , since the final prediction task involves predicting the associations among entity A , B , and C , the two tasks of A → B and B → C should be intuitively considered to be of equal importance, therefore the balanced coefficient α set to 0.5; for γ , bind-relation prediction is fundamentally a binary classification task, thus the threshold γ is set to 0.5.

C Discussions

Computation Complexity Discussion. To facilitate applying across a wider range of dataset types and particularly large datasets, it is imperative to rigorously address the issue of complexity explosion. Regarding computation complexity, the discussion in Section 4.5 indicates that the decisive factors are the number of entity nodes |V| and the number of sampled patterns N . Hence, when encountering larger datasets or more complex biological networks, we can employ the following two strategies to avoid a computation complexity explosion: (1) Graph Sampling. By sampling smaller subgraphs [80], computational resource consumption reduces significantly, boosting algorithm speed and efficiency; (2) High-confidence Pattern Selection. As shown in Appendix Figure 5 (g)-(i), reducing N from 100

to 5 across three datasets slightly decreases performance but still surpasses these baselines. Thus, adjusting the sampling quantity is acceptable to effectively reduce time complexity.

Moreover, Appendix Table 4 shows the GPU usage varying from the number of sampled pattern across three datasets with significant differences in the number of entities and associations. The inference time of each 100 samples with milliseconds compared to baselines are provided in Appendix Figure 7. It is evident that the current computational resources are sufficient to handle existing datasets, and the difference in inference time for our method is constant and even faster than several baselines.

Potential Application in Real-world Scenarios. The preliminary evaluation results on three biological datasets quantitatively demonstrate that the proposed method consistently achieves superior performance over the competing baselines. Additionally, the learned association patterns show potential in interpreting biological mechanisms. This finding provides hope for the future practical application of our approach, particularly in addressing the 'black box' issue in the field of bioinformatics. As a computer-aided tool, our method holds the potential to exhibit a broad array of applications in real-world scenarios, contingent upon sufficient clinical tests or validations. Consequently, it may significantly contribute to human health and well-being.

Limitations and Future Work. While the proposed Pattern-BERP has demonstrated strong performance on biological entity relationship prediction, there are several limitations and potential directions for future research. One limitation is that the current model only considers fixed-length association patterns. In real-world biological systems, relevant relationships may be expressed through variable-length pathways involving multiple intermediate entities and relations. Extending the model to capture and leverage such variable-length association patterns could further enhance its representational power and predictive accuracy. Additionally, the applicability of Pattern-BERP method has so far been explored only in the biological domain. Investigating the effectiveness of this approach in other domains, such as knowledge graph completion for general entities or relation extraction from text, would help evaluate its broader generalizability and potential to benefit other fields that rely on structured knowledge representations. Adapting the model architecture and training strategies to accommodate the unique characteristics of different domains could lead to fruitful avenues for future research.

By addressing these limitations and exploring these future research directions, we believe PatternBERP can be further improved and extended to have an even greater impact on advancing our understanding of complex biological systems and knowledge representation in general.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: See Section 1.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: See 'Limitations and Future Work' part in the Appendix C.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: See Section 4.2 - 'Association Pattern-aware Fusion' & Section 5.2 - 'Performance Comparison with Advanced Baselines'. Furthermore, the data and source code are also provided in the Github repository.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The data and source code are provided at https://github.com/hry98kki/ PatternBERP .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: See Section 5.1 - 'Implementation Details' & Appendix B.2.2 - 'Implementation Settings'.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We have provided the raw experimental results of all 5-fold cross-validation results across the four scenarios, resulting in a total of 20 individual results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: See Appendix B.2.2-'Implementation Settings'.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: This paper adheres to the NeurIPS Code of Ethics in every respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: See Appendix C-'Application in Real-world Scenarios'.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The source code & citations in the paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The source code provided in our own repository.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an innovative perspective, namely association pattern, for biological entity relationship prediction (e.g., drug-target protein-adverse reaction). The authors first summarize the characteristics of existing perspectives and emphasize the importance of the association pattern. Next, the association patterns of certain entity nodes are sampled in the constructed graph based on the defined distance and then serve as tokens for self-attention to fuse similar semantic information. Additionally, the bind-relation enhancement module can effectively reconstruct the missing feature and thus generate hard negative samples for further improving the performance. Extensive experiments on biological datasets demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS:\n1. The perspective of triple-wise association patterns, which takes the actual meaning of each entity and their connections into account, is novel and explainable.\n2. The paper comprehensively discusses the advantages of the proposed method compared with existing non-graph, graph, and hypergraph perspectives. Moreover, the related works on biological entity relationship prediction and path-based graph mining are elaborated.\n3. The paper provides a clear and detailed description of each part of the method, and the formulas used in the paper seem correct. The overall design of the method is rather reasonable.\n4. Extensive experiments on biological datasets demonstrate the effectiveness of the proposed method. The reveal of the intrinsic biological mechanism can indeed promote the deployment of the method in real-world scenarios.\n5. The association pattern-aware perspective appears to apply to commonly used graph benchmarks or tasks, which can be leveraged to enhance existing graph-based methods.\n\nWEAKNESSES:\n1. The bind-relation feature reconstruction appears to be a standalone component that requires extra separate training, resulting in a two-stage process for the method. Additionally, such a design may influence the entire optimization process and distract the training flow, how to alleviate that?\n\n2. Generalization and extension issues have not been discussed. The pattern's length is fixed to 3, which may make the method not work when facing longer-range relationships. Are there any feasible solutions for sampling patterns with different lengths?\n\n3. Complexity has not been discussed. Some details are also not clear enough, such as in Line 167, could the authors provide details on how the position encoding is designed? In Fig. 2, the meaning of the dotted line \"Bind-relation Feature\" needs more explanation. The \"add\" operation is not reflected in Section 4.3. Please explain the meaning of the dotted line \"Bind-relation Feature\".\n\nQUESTIONS:\nQ1: The paper does not give a specific explanation of $L_{BE}$ (see Line 224). Please give a specific explanation of $L_{BE}$.\n\nQ2: In Table 1, the authors appear to have not reported the statistical measures like standard error.\n\nLIMITATIONS:\nThe authors have adequately discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel approach to predicting relationships among biological entities by addressing limitations in current deep learning methods that focus solely on entity-centric information. The authors introduce an association pattern-aware fusion method that integrates association pattern information into entity representation learning. Additionally, they develop a bind-relation module to enhance low-order message passing by considering strong low-order entity associations. Extensive experiments on three biological datasets demonstrate significant improvements compared to advanced baselines. The paper also provides detailed explanations of the interpretability of association patterns, highlighting intrinsic biological mechanisms and potential real-world applications.\n\nSTRENGTHS:\n- The introduction of an association pattern-aware fusion method is innovative. It effectively integrates association pattern information into entity representation learning, which is a novel approach compared to traditional entity-centric methods.\n- The method includes several interconnected modules such as association pattern sampling, pattern-aware interaction, and bind-relation enhancement. Extensive experiments on three biological datasets demonstrate significant improvements compared to advanced baselines.\n- The method provides detailed explanations of the interpretability of association patterns. This helps in understanding the intrinsic biological mechanisms, making the results more transparent and useful for real-world applications.\n\nWEAKNESSES:\n- The proposed model involves multiple modules and several hyperparemeters. Some hyperparameters (such as the number of attention head) are vary on different datasets. The authors may aslo need to add hyperparameter study for loss-balanced coefficient and the threshold for bind-relation prediction probability.\n- The approach may not scale well with increasing dataset sizes or more complex biological networks due to its computational requirements and the need for extensive pattern sampling and interaction modeling.\n- The proposed model involves multiple components such as hypergraph convolution, pattern-aware fusion, and bind-relation enhancement, which can be computationally intensive. This might limit its applicability for large-scale real-world datasets or in environments with limited computational resources. The author may also need to provide inference speed study with some of the baselines.\n\nQUESTIONS:\nPlease see the weaknesses.\n\nLIMITATIONS:\nThey provide a limitation section in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author proposed a deep learning method, termed the Pattern BERP method, designed to elucidate the potential associations among triple-wise biological entities. This method innovativly incorporates association pattern information into the entity representation learning. Moreover, it employs two additional bipartite graphs to enrich the binary relation information extracted from triple entities. Evaluated on three biological datasets, the proposed method demonstrates superior performance compared to non-graph, graph-based, and hypergraph-based methodologies. This work is noteworthy not only for its impressive predictive accuracy but also for its potential applicability in real-world scenarios.\n\nSTRENGTHS:\n1.\tThe concept of k-hop neighbors on a hypergraph has been defined, incorporating pattern-similar triples information as neighbors into each hypernode.\n\n2.\tThe information of triples has been decomposed into pairwise relationships, and these features have been injected into the backbone network.\n\n3.\tA parameter-based negative sampling technique has been introduced, which is more efficient compared to random sampling.\n\nWEAKNESSES:\n1. The interpretation provided for the model is not entirely persuasive. The analysis of the relationship between peptide-based micro-molecular drugs and small-molecular chemical drugs is unconventional and lacks commonality. \n\n2. While the concept of association pattern-aware fusion is highly innovative and appropriately highlighted as the main focus of the paper, its contribution for model performance seems not to be impressive, especially in Table 5 and Table 6.\n\n3. The model is overly complex, with multiple stages of tasks and losses, which may lead to difficulties in optimization.\n\n4. The role of the triple information has been simplified to a single chain pattern from A to B to C.\n\nQUESTIONS:\n1. The abbreviation \"APF\" (association pattern fusion) is not defined in the text at line 219. It is essential to provide the full name when first introduced for clarity.\n\n2. In equation (11), the LBE is not updated every epoch, which contradicts the description in the text.\n\n3. The manuscript features two bipartite graphs in Figure 2, yet it remains unclear whether these graph features are integrated with hypergraph features. Additionally, the role of the blue dotted line arrow, which is not discussed in the text, needs clarification to enhance understanding of the graphical representations.\n\n4. Although the association pattern-aware fusion is novel and central to the paper's title, its contribution for model performance seems not to be impressive. Specifically, while Table 2 shows significant impairment in performance on the DDC dataset without the association pattern interaction (API), Tables 3 and 4 for the DMD and DPA datasets suggest that the Binding-relation Feature Reconstruction (BFR) for bipartite graphs predominantly enhances performance. It may be misleading to attribute the main contribution of the work solely to APF given these findings.\n\n5. The model interpretation in Figure 3 lacks persuasiveness, as Drug #52, a peptide with numerous sub-group structures, is not ideal for visualization comparison with other compound drugs. Furthermore, in equation (3), the attention mechanism is applied across different triple entities, not just one. Analyzing only the drugs without considering the other two entities in the figures is insufficient. At minimum, the roles of the other two entities should be elucidated to provide a more comprehensive analysis.\n\n6. Equation 2 describes the generation method of the distance matrix D and the distance token 𝐷𝑣. How is the specific position embedding done on line 167? Is the initial pattern 𝑃 directly concatenated from the three nodes?\n\n7. In the bind-relation enhancement, why is only the bipartite graph between AB and BC considered? Why not consider AC? Not all multi-entity relationships have a clear chain pattern from A to B to C.\n\n8. Following up on question 7, how are relationships defined between two entities in Equations 7 and 8?\n\nLIMITATIONS:\nThe model lacks practical application scenarios. It primarily learns from the interaction network of existing entities such as drugs, diseases, and symptoms. Such methods generally struggle to predict the relationships between a completely new entity and the existing entities in the network. Moreover, the relationships within the network are already well-known. The more important task is to uncover new and potential relationships. The comprehensiveness and accuracy of the existing relationships are merely the tasks designed by the model itself.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work presents a novel approach - Pattern-BERP - for the prediction of biological entity relationships, it utilises entity association patterns as opposed to a lot of existing research that focuses primarily on entity-centric information mapping and aggregation. \n\nThe evaluation is done on three different biological datasets (DMD, DDC, DPA), and compared against a range of different approaches including non-graph, graph-based and hypergraph-based ones.\n\nThe results show state-of-the-art performance across the board (an increase from 2 to 10 points over the next best method). \n\nThe authors also do an ablation study, showing the importance of each component and demonstrating the relative improvement.\n\nSTRENGTHS:\nA novel idea presented in an easy-to-understand way. Manuscript is nicely written and clear, with a broad impact. An extensive evaluation concerning methods, state-of-the-art results and a detailed ablation study.\n\nWEAKNESSES:\nThis is a very general method but it was tested on only one dataset type and relatively small datasets. Would be great to see it expanded in the future.\n\nQUESTIONS:\nSimilar to the limitations field, I'd be very interested in an explanation of how the computation complexity of Pattern-BERP scales with the number of entities and associations. \n\nIs it possible to provide a bit more theory behind the success of your method compared to others? The justification for the association pattern-aware fusion to me seems mostly intuitive rather than theoretical.\n\nIs there a way to extend the method to capture variable-length pathways?\n\nLIMITATIONS:\nIt would be great to see a longer discussion on limitations: my primary concerns are complexity explosion and very large datasets. Would be great to see a bit of a theoretical analysis (or even empirical) on how it scales for millions of entities or associations (which are not rare in the domain).\n\nThe interpretability section focuses on only drug #52, it would be great to see that expanded, as making conclusions from one example is bit of a stretch. \n\nReal-world applications might be a bit of an overstatement, while it is possible, nothing was really tested or proven.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}