Original: Flashlight : Scalable Link Prediction with Effective Decoders
Revised: Flashlight : Scalable Link Prediction with Effective Decoders

Original: Abstract
Revised: Abstract

Original: Link prediction (LP) has been recognized as an important task in graph learning with its broad practical applications.
Revised: Link prediction (LP) has been recognized as an important task in graph learning with its broad practical applications.

Original: A typical application of LP is to retrieve the top scoring neighbors for a given source node, such as the friend recommendation.
Revised: A typical application of LP is to retrieve the top scoring neighbors for a given source node, such as the friend recommendation.

Original: These services desire the high inference scalability to find the top scoring neighbors from many candidate nodes at low latencies.
Revised: These services desire the high inference scalability to find the top scoring neighbors from many candidate nodes at low latencies.

Original: There are two popular decoders that the recent LP models mainly use to compute the edge scores from node embeddings: the HadamardMLP and Dot Product decoders.
Revised: There are two popular decoders that the recent LP models mainly use to compute the edge scores from node embeddings: the HadamardMLP and Dot Product decoders.

Original: After theoretical and empirical analysis, we find that the HadamardMLP decoders are generally more effective for LP.
Revised: After theoretical and empirical analysis, we find that the HadamardMLP decoders are generally more effective for LP.

Original: However, HadamardMLP lacks the scalability for retrieving top scoring neighbors on large graphs, since to the best of our knowledge, there does not exist an algorithm to retrieve the top scoring neighbors for HadamardMLP decoders in sublinear complexity.
Revised: However, HadamardMLP lacks the scalability for retrieving top scoring neighbors on large graphs, since to the best of our knowledge, there does not exist an algorithm to retrieve the top scoring neighbors for HadamardMLP decoders in sublinear complexity.

Original: To make HadamardMLP scalable, we propose the Flashlight algorithm to accelerate the top scoring neighbor retrievals for HadamardMLP: a sublinear algorithm that progressively applies approximate maximum inner product search (MIPS) techniques with adaptively adjusted query embeddings.
Revised: To make HadamardMLP scalable, we propose the Flashlight algorithm to accelerate the top scoring neighbor retrievals for HadamardMLP: a sublinear algorithm that progressively applies approximate maximum inner product search (MIPS) techniques with adaptively adjusted query embeddings.

Original: Empirical results show that Flashlight improves the inference speed of LP by more than 100 times on the large OGBL-CITATION2 dataset without sacrificing effectiveness.
Revised: Empirical results show that Flashlight improves the inference speed of LP by more than 100 times on the large OGBL-CITATION2 dataset without sacrificing effectiveness.

Original: Our work paves the way for large-scale LP applications with the effective HadamardMLP decoders by greatly accelerating their inference.
Revised: Our work paves the way for large-scale LP applications with the effective HadamardMLP decoders by greatly accelerating their inference.

Original:  Introduction
Revised:  Introduction

Original: The goal of link prediction (LP) is to predict the missing links in a graph [1].
Revised: The goal of link prediction (LP) is to predict the missing links in a graph [1].

Original: LP is drawing increasing attention in the past decade due to its board practical applications [2].
Revised: LP is drawing increasing attention in the past decade due to its broad practical applications [2].

Original: For instance, LP can be used to recommend new friends on social media [3], and recommend attractive items to the costumers on
Revised: For instance, LP can be used to recommend new friends on social media [3], and recommend attractive items to the costumers on

Original: E-commerce sites [4], so as to improve the user experience.
Revised: E-commerce sites [4], so as to improve the user experience.

Original: During inference, these applications demand the LP methods to retrieve the top scoring neighbors for a source node at low latencies.
Revised: During inference, these applications demand the LP methods to retrieve the top scoring neighbors for a source node at low latencies.

Original: This is especially challenging on large graphs because the LP methods need to search many candidate nodes to find the top scoring neighbors.
Revised: This is especially challenging on large graphs because the LP methods need to search many candidate nodes to find the top scoring neighbors.

Original: There are two main kinds of architecture followed by the recent LP models.
Revised: There are two main kinds of architecture followed by the recent LP models.

Original: The first uses an encoder, e.g., GCN [5], to obtain the node-level embeddings and uses a decoder, e.g., Dot Product, to get the edge scores between the paired nodes [6].
Revised: The first uses an encoder, e.g., GCN [5], to obtain the node-level embeddings and uses a decoder, e.g., Dot Product, to get the edge scores between the paired nodes [6].

Original: The second crops a subgraph for every edge and computes the edge score from the subgraph directly [7].
Revised: The second crops a subgraph for every edge and computes the edge score from the subgraph directly [7].

Original: The inference speed of the second is much lower than the first, so we focus on the first kind of models to achieve fast inference on large graphs.
Revised: The inference speed of the second is much lower than the first, so we focus on the first kind of models to achieve fast inference on large graphs.

Original: In the last years, extensive research focuses on developing more expressive LP encoders [6, 8].
Revised: In the last years, extensive research focuses on developing more expressive LP encoders [6, 8].

Original: However, much less work pays attention to the essential impacts of the choice of decoders on LP’s performance.
Revised: However, much less work pays attention to the essential impacts of the choice of decoders on LP’s performance.

Original: In this work, we theoretically and empirically analyze two popular LP decoders: Dot Product and
Revised: In this work, we theoretically and empirically analyze two popular LP decoders: Dot Product and

Original: HadamardMLP (a MLP following the Hadamard Product), and find that the latter is generally more effective than the former.
Revised: HadamardMLP (a MLP following the Hadamard Product), and find that the latter is generally more effective than the former.

Original: In practical applications, we should not only consider the effectiveness of LP, but also inference efficiency.
Revised: In practical applications, we should not only consider the effectiveness of LP, but also inference efficiency.

Original: Many LP applications generally require fast retrieval of the top scoring neighbors for lowlatency services [3, 9, 10].
Revised: Many LP applications generally require fast retrieval of the top scoring neighbors for lowlatency services [3, 9, 10].

Original: For a Dot Product decoder, this retrieval can be approximated efficiently at the sublinear time complexity [11].
Revised: For a Dot Product decoder, this retrieval can be approximated efficiently at the sublinear time complexity [11].

Original: However, to the best of our knowledge, no such sublinear algorithms exist for the top scoring neighbor retrievals of the HadamardMLP decoders.
Revised: However, to the best of our knowledge, no such sublinear algorithms exist for the top scoring neighbor retrievals of the HadamardMLP decoders.

Original: This means that for every source node, we have to iterate over all the nodes in the graph to compute the scores so as to find the top scoring neighbors for HadamardMLP, which is of linear complexity and cannot scale to large graphs.
Revised: This means that for every source node, we have to iterate over all the nodes in the graph to compute the scores so as to find the top scoring neighbors for HadamardMLP, which is of linear complexity and cannot scale to large graphs.

Original: To allow LP applications to enjoy the high effectiveness of HadamardMLP decoders while avoiding the poor inference scalability, we propose the scalable top scoring neighbor search algorithm named
Revised: To allow LP applications to enjoy the high effectiveness of HadamardMLP decoders while avoiding the poor inference scalability, we propose the scalable top scoring neighbor search algorithm named

Original: Flashlight .
Revised: Flashlight .

Original: Our Flashlight progressively calls the well-developed approximate maximum inner product search (MIPS) techniques for a few iterations.
Revised: Our Flashlight progressively calls the well-developed approximate maximum inner product search (MIPS) techniques for a few iterations.

Original: At every iteration, we analyze the retrieved neighbors and adaptively adjust the query embedding for Flashlight to find the missed high scoring neighbors.
Revised: At every iteration, we analyze the retrieved neighbors and adaptively adjust the query embedding for Flashlight to find the missed high scoring neighbors.

Original: Our Flashlight algorithm holds sublinear time complexity on finding top scoring neighbors for HadamardMLP decoders, allowing for fast and scalable inference.
Revised: Our Flashlight algorithm holds sublinear time complexity on finding top scoring neighbors for HadamardMLP decoders, allowing for fast and scalable inference.

Original: Empirical results show that
Revised: Empirical results show that

Original: Flashlight accelerates the inference of LP models by more than 100 times on the large OGBLCITATION2 dataset without sacrificing the effectiveness.
Revised: Flashlight accelerates the inference of LP models by more than 100 times on the large OGBLCITATION2 dataset without sacrificing the effectiveness.

Original: Overall, our work paves the way for the use of effective LP decoders in practical settings by greatly accelerating their inference.
Revised: Overall, our work paves the way for the use of effective LP decoders in practical settings by greatly accelerating their inference.

Original:  Revisiting Link Prediction Decoders
Revised:  Revisiting Link Prediction Decoders

Original: In this section, we formalize the link prediction (LP) problem and the LP decoders.
Revised: In this section, we formalize the link prediction (LP) problem and the LP decoders.

Original: Typically, many
Revised: Typically, many

Original: LP models include an encoder that learns the node-level embeddings x
Revised: LP models include an encoder that learns the node-level embeddings x

Original: i , i ∈ V , where V is the set of nodes, and an decoder ϕ : R d × R d → R that combines the node-level embeddings of a pair of nodes: x i , x j into a single score: s ij .
Revised: i , i ∈ V , where V is the set of nodes, and an decoder ϕ : R d × R d → R that combines the node-level embeddings of a pair of nodes: x i , x j into a single score: s ij .

Original: If s ij is higher, the link between nodes i and j is more likely to exist.
Revised: If s ij is higher, the link between nodes i and j is more likely to exist.

Original: The state-of-the-art models generally use graph neural networks as the encoders [5, 6, 8, 12, 13].
Revised: The state-of-the-art models generally use graph neural networks as the encoders [5, 6, 8, 12, 13].

Original: From here on, we mainly focus on the decoder ϕ .
Revised: From here on, we mainly focus on the decoder ϕ .

Original: 2.1 Dot Product Decoder
Revised: 2.1 Dot Product Decoder

Original: The most common decoder of link prediction is the Dot Product [6, 8, 10]:
Revised: The most common decoder of link prediction is the Dot Product [6, 8, 10]:

Original: [Equation]
Revised: [Equation]

Original: where • denotes the dot product.
Revised: where • denotes the dot product.

Original: Training a link prediction model with the Dot Product decoder encourages the embeddings of the connected nodes to be close to each other.
Revised: Training a link prediction model with the Dot Product decoder encourages the embeddings of the connected nodes to be close to each other.

Original: Intuitively, the score s ij can be thought as a measure of the squared Eulidean distance between the node embeddings x i , x j , as ∥ x i − x j ∥ 2 = ∥ x i ∥ 2 − 2 x i • x j + ∥ x j ∥ 2 , if the ∥ x j ∥ is constant over the neighbors j ∈ N , e.g., after normalization [14].
Revised: Intuitively, the score s ij can be thought as a measure of the squared Eulidean distance between the node embeddings x i , x j , as ∥ x i − x j ∥ 2 = ∥ x i ∥ 2 − 2 x i • x j + ∥ x j ∥ 2 , if the ∥ x j ∥ is constant over the neighbors j ∈ N , e.g., after normalization [14].

Original: Because the node embeddings represent the semantic information of nodes, Dot Product assumes the homophily of graph topology, i.e., the semantically similar nodes are more likely to be connected.
Revised: Because the node embeddings represent the semantic information of nodes, Dot Product assumes the homophily of graph topology, i.e., the semantically similar nodes are more likely to be connected.

Original: 2.2 HadamardMLP (MLP following Hadamard Product) Decoder
Revised: 2.2 HadamardMLP (MLP following Hadamard Product) Decoder

Original: Multi layer perceptrons (MLPs) are known to be universal approximators that can approximate any continuous function on a compact set [15].
Revised: Multi layer perceptrons (MLPs) are known to be universal approximators that can approximate any continuous function on a compact set [15].

Original: A MLP layer can be defined as a function f : R d in →
Revised: A MLP layer can be defined as a function f : R d in →

Original: R
Revised: R

Original: [Equation]
Revised: [Equation]

Original: which is parameterized by the learnable weight W ∈ R d out × d in (the bias, if exists, can be represented by an additional column in W and an additional channel in the input x with the value as 1 ).
Revised: which is parameterized by the learnable weight W ∈ R d out × d in (the bias, if exists, can be represented by an additional column in W and an additional channel in the input x with the value as 1 ).

Original: ReLU is the activation function [16].
Revised: ReLU is the activation function [16].

Original: In a MLP, several layers of f are stacked, e.g., a 3-layer MLP can be formalized as f W 3 ( f W 2 ( f W 1 ( x ))) .
Revised: In a MLP, several layers of f are stacked, e.g., a 3-layer MLP can be formalized as f W 3 ( f W 2 ( f W 1 ( x ))) .

Original: The state-of-the-art models widely use a MLP following the Hadamard Product between the paired nodes as the decoder (short as the HadamardMLP decoders) [6, 8, 10, 17]:
Revised: The state-of-the-art models widely use a MLP following the Hadamard Product between the paired nodes as the decoder (short as the HadamardMLP decoders) [6, 8, 10, 17]:

Original: [Equation]
Revised: [Equation]

Original: where ⊙ denotes the Hadamard Product.
Revised: where ⊙ denotes the Hadamard Product.

Original: Fig.
Revised: Fig.

Original: 1 illustrates these two models the Dot Product and
Revised: 1 illustrates these two models the Dot Product and

Original: HadamardMLP decoders.
Revised: HadamardMLP decoders.

Original: 2.3 Other Link Prediction Decoders
Revised: 2.3 Other Link Prediction Decoders

Original: In principle, every function that takes two vectors as the input and outputs a scalar can act as the decoder.
Revised: In principle, every function that takes two vectors as the input and outputs a scalar can act as the decoder.

Original: For example, there are bilinear dot product decoder (short as Bilinear decoder ) [6]:
Revised: For example, there are bilinear dot product decoder (short as Bilinear decoder ) [6]:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where W is the learnable weight, and the MLPs following the concatenate decoder [6, 10] (short as
Revised: where W is the learnable weight, and the MLPs following the concatenate decoder [6, 10] (short as

Original: ConcatMLP decoder ):
Revised: ConcatMLP decoder ):

Original: [Equation]
Revised: [Equation]

Original: (5) , etc.
Revised: (5) , etc.

Original: These two decoders are used much less than Dot Product and HadamardMLP in the state-ofthe-art LP models possibly due to their lower effectiveness [6, 8, 10, 17].
Revised: These two decoders are used much less than Dot Product and HadamardMLP in the state-ofthe-art LP models possibly due to their lower effectiveness [6, 8, 10, 17].

Original: 2.4 HadamardMLP is Generally More Effective than Other Decoders
Revised: 2.4 HadamardMLP is Generally More Effective than Other Decoders

Original: Dot Product demands the homophily of graph data to effectively infer the link between nodes.
Revised: Dot Product demands the homophily of graph data to effectively infer the link between nodes.

Original: In contrast, thanks to the universal approximation capability, MLP can approximate any continuous function, and thus does not demand the homophily of graph data for effective LP.
Revised: In contrast, thanks to the universal approximation capability, MLP can approximate any continuous function, and thus does not demand the homophily of graph data for effective LP.

Original: This gap in the expressiveness accounts for the performance difference of these two decoders on many datasets (see
Revised: This gap in the expressiveness accounts for the performance difference of these two decoders on many datasets (see

Original: Sec. 5.3).
Revised: Sec. 5.3).

Original: We additionally show in Appendix.
Revised: We additionally show in Appendix.

Original: A that using a HadamardMLP is easy to learn Dot
Revised: A that using a HadamardMLP is easy to learn Dot

Original: Product, which also partially accounts for the better effectiveness of the HadamardMLP decoders over the Dot Product.
Revised: Product, which also partially accounts for the better effectiveness of the HadamardMLP decoders over the Dot Product.

Original: Existing work also finds that the effectiveness of Bilinear and ConcatMLP is generally worse than the HardmardMLP or Dot Product decoder [6, 8, 10, 17].
Revised: Existing work also finds that the effectiveness of Bilinear and ConcatMLP is generally worse than the HardmardMLP or Dot Product decoder [6, 8, 10, 17].

Original: We confirm these findings more rigorously in the empirical results in Sec.
Revised: We confirm these findings more rigorously in the empirical results in Sec.

Original:  Scalability of Link Prediction Decoders
Revised:  Scalability of Link Prediction Decoders

Original: Most academic studies focus on training runtime when discussing scalability.
Revised: Most academic studies focus on training runtime when discussing scalability.

Original: However, in industrial applications, the inference speed is often more important.
Revised: However, in industrial applications, the inference speed is often more important.

Original: The inference of many LP applications needs to retrieve the top scoring neighbors given a source node, e.g., recommending friends to a user for friend recommendation.
Revised: The inference of many LP applications needs to retrieve the top scoring neighbors given a source node, e.g., recommending friends to a user for friend recommendation.

Original: Given a source node, if there are n nodes in the graph, then the inference time complexity is O ( n ) if the decoder needs to iterate over all the n nodes to compute the edge scores.
Revised: Given a source node, if there are n nodes in the graph, then the inference time complexity is O ( n ) if the decoder needs to iterate over all the n nodes to compute the edge scores.

Original: For large scale applications, n is typically in the range of millions, or even larger.
Revised: For large scale applications, n is typically in the range of millions, or even larger.

Original: The empirical results show that the inference time of finding the top scoring neighbors for a source node is longer than one second for HadamardMLP on the OGBL-CITATION2 dataset of nearly three million nodes (see Sec. 5.5).
Revised: The empirical results show that the inference time of finding the top scoring neighbors for a source node is longer than one second for HadamardMLP on the OGBL-CITATION2 dataset of nearly three million nodes (see Sec. 5.5).

Original: For a Dot Product decoder, the problem of finding the top scoring neighbors can be approximated efficiently.
Revised: For a Dot Product decoder, the problem of finding the top scoring neighbors can be approximated efficiently.

Original: This is a well-studied problem, known as approximate maximum inner product search (MIPS) [18, 19] (see Sec.
Revised: This is a well-studied problem, known as approximate maximum inner product search (MIPS) [18, 19] (see Sec.

Original: 6.2 for a comprehensive literature review).
Revised: 6.2 for a comprehensive literature review).

Original: MIPS techniques allow Dot
Revised: MIPS techniques allow Dot

Original: Product’ inference to be completed in a few milliseconds, even with millions of neighbors.
Revised: Product’ inference to be completed in a few milliseconds, even with millions of neighbors.

Original: There exists some work that tries to extend MIPS to the ConcatMLP [20, 21].
Revised: There exists some work that tries to extend MIPS to the ConcatMLP [20, 21].

Original: These methods hold strict assumptions on the models’ training and are not directly applicable to the HadamardMLP.
Revised: These methods hold strict assumptions on the models’ training and are not directly applicable to the HadamardMLP.

Original: To the best of our knowledge, no such sublinear techniques exist for the top scoring neighbor retrieval with the
Revised: To the best of our knowledge, no such sublinear techniques exist for the top scoring neighbor retrieval with the

Original: HadamardMLP [10], which is a complex nonlinear function.
Revised: HadamardMLP [10], which is a complex nonlinear function.

Original: To summarize, the HadamardMLP decoder is not scalable for the real time LP services on large graphs, while the Dot Product decoder allows fast retrieval using the well established MIPS techniques.
Revised: To summarize, the HadamardMLP decoder is not scalable for the real time LP services on large graphs, while the Dot Product decoder allows fast retrieval using the well established MIPS techniques.

Original:  Flashlight : Scalable Link Prediction with Effective Decoders
Revised:  Flashlight : Scalable Link Prediction with Effective Decoders

Original: Sec. 2 has shown that the HadamardMLP decoder enjoys higher effectiveness than the Dot Product decoder, which supports the superior performance of HadamardMLP on many LP benchmarks.
Revised: Sec. 2 has shown that the HadamardMLP decoder enjoys higher effectiveness than the Dot Product decoder, which supports the superior performance of HadamardMLP on many LP benchmarks.

Original: On the other hand, Sec. 3 has shown that the HadamardMLP is not scalable for real time LP applications on large graphs, while Dot Product supports the fast inference using the well-established MIPS techniques.
Revised: On the other hand, Sec. 3 has shown that the HadamardMLP is not scalable for real time LP applications on large graphs, while Dot Product supports the fast inference using the well-established MIPS techniques.

Original: In this section, we aim to devise fast inference algorithms for HadamardMLP to enable scalable LP with effective decoders.
Revised: In this section, we aim to devise fast inference algorithms for HadamardMLP to enable scalable LP with effective decoders.

Original: We try to exploit the advances in the well-developed MIPS techniques to accelerate the inference of
Revised: We try to exploit the advances in the well-developed MIPS techniques to accelerate the inference of

Original: HadamardMLP.
Revised: HadamardMLP.

Original: Specifically, we divide the top scoring retrievals for HadamardMLP predictors into a sequence of MIPS.
Revised: Specifically, we divide the top scoring retrievals for HadamardMLP predictors into a sequence of MIPS.

Original: Our algorithm works in a progressive manner.
Revised: Our algorithm works in a progressive manner.

Original: The query embedding in every search is adaptively adjusted to find the high scoring neighbors missed in the last search.
Revised: The query embedding in every search is adaptively adjusted to find the high scoring neighbors missed in the last search.

Original: The challenge of retrieving the neighbors of highest scores for HadamardMLP is rooted in the unawareness of which neurons are activated, since if we know which neurons are activated, the nonlinear HadamardMLP degrades to a linear model.
Revised: The challenge of retrieving the neighbors of highest scores for HadamardMLP is rooted in the unawareness of which neurons are activated, since if we know which neurons are activated, the nonlinear HadamardMLP degrades to a linear model.

Original: On the l th MLP layer, we define the mask matrix M A ,l ∈ R d l × d l to represent the set of activated neurons A as
Revised: On the l th MLP layer, we define the mask matrix M A ,l ∈ R d l × d l to represent the set of activated neurons A as

Original: [Equation]
Revised: [Equation]

Original: With M A ,l , we reformulate the HadamardMLP decoder as:
Revised: With M A ,l , we reformulate the HadamardMLP decoder as:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Because the vector W T 1 M A , 1 . . .
Revised: Because the vector W T 1 M A , 1 . . .

Original: W TL − 1 M T A ,L − 1 w L is determined by the weights of MLP and
Revised: W TL − 1 M T A ,L − 1 w L is determined by the weights of MLP and

Original: the activated neurons
Revised: the activated neurons

Original: A , we term it as MLP A ( · ) :
Revised: A , we term it as MLP A ( · ) :

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Given the source node i , because the score s ij is obtained by the dot product between ( W T 1 M A , 1 . . .
Revised: Given the source node i , because the score s ij is obtained by the dot product between ( W T 1 M A , 1 . . .

Original: W TL − 1 M T A ,L − 1 w L ⊙ x i ) and the neighbor embedding x j , we term the former vector as the query embedding q :
Revised: W TL − 1 M T A ,L − 1 w L ⊙ x i ) and the neighbor embedding x j , we term the former vector as the query embedding q :

Original: [Equation]
Revised: [Equation]

Original: In this way, we can reformulate the output of decoder ϕ
Revised: In this way, we can reformulate the output of decoder ϕ

Original: MLP ( x i , x j ) as
Revised: MLP ( x i , x j ) as

Original: [Equation]
Revised: [Equation]

Original: In practice, we can use the q as the query embedding in MIPS to retrieve the neighbors of highest inner products, which correspond to the highest scores.
Revised: In practice, we can use the q as the query embedding in MIPS to retrieve the neighbors of highest inner products, which correspond to the highest scores.

Original: Here, how to get the activated neurons
Revised: Here, how to get the activated neurons

Original: A so as to obtain the query embedding q is an issue.
Revised: A so as to obtain the query embedding q is an issue.

Original: Different node pairs activate different neurons A .
Revised: Different node pairs activate different neurons A .

Original: Initially, without knowing which neurons are activated, we first assume all the neurons are activated, i.e., we have the initial query embedding as:
Revised: Initially, without knowing which neurons are activated, we first assume all the neurons are activated, i.e., we have the initial query embedding as:

Original: [Equation]
Revised: [Equation]

Original: This initial design can reflect the general trends of increasing the edge scores on LP, without restricting which neurons are activated.
Revised: This initial design can reflect the general trends of increasing the edge scores on LP, without restricting which neurons are activated.

Original: We use q [1] as the query embedding to retrieve the highest inner product neighbors as N [1] in the first iteration.
Revised: We use q [1] as the query embedding to retrieve the highest inner product neighbors as N [1] in the first iteration.

Original: Then, given the retrieved neighbors in the t th iteration as
Revised: Then, given the retrieved neighbors in the t th iteration as

Original: N [ t ] , we analyze the N [ t ] and adaptively adjust the query embedding q [ t + 1] that we use in the next iteration to find more high scoring neighbors.
Revised: N [ t ] , we analyze the N [ t ] and adaptively adjust the query embedding q [ t + 1] that we use in the next iteration to find more high scoring neighbors.

Original: Specifically, we operate the feed-forward to MLP for N ( t ) .
Revised: Specifically, we operate the feed-forward to MLP for N ( t ) .

Original: We define the function A ( · , · ) that returns the set of activated neurons for a MLP (the first input) with the input x i ⊙ x j (the second input).
Revised: We define the function A ( · , · ) that returns the set of activated neurons for a MLP (the first input) with the input x i ⊙ x j (the second input).

Original: Then we can use it to extract A as:
Revised: Then we can use it to extract A as:

Original: [Equation]
Revised: [Equation]

Original: Then, we obtain the set of activated neurons of the highest scored neighbor at the t th iteration as:
Revised: Then, we obtain the set of activated neurons of the highest scored neighbor at the t th iteration as:

Original: [Equation]
Revised: [Equation]

Original: where j ⋆
Revised: where j ⋆

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: ← A
Revised: ← A

Original: N
Revised: N

Original: 10: return
Revised: 10: return

Original: N
Revised: N

Original: This implies that the neighbors activating A [ t ] can obtain the high edge scores.
Revised: This implies that the neighbors activating A [ t ] can obtain the high edge scores.

Original: Then, if we take A [1] as the set of neurons that we activate at the next query, we could find more high scoring neighbors.
Revised: Then, if we take A [1] as the set of neurons that we activate at the next query, we could find more high scoring neighbors.

Original: In this way, we set the neurons that we assume to activate in the next iteration as A [ t ] .
Revised: In this way, we set the neurons that we assume to activate in the next iteration as A [ t ] .

Original: We repeat the above iterations until enough neighbors are retrieved.
Revised: We repeat the above iterations until enough neighbors are retrieved.

Original: The algorithm is summarized in Alg. 1.
Revised: The algorithm is summarized in Alg. 1.

Original: We name our algorithm as Flashlight because it works like a flashlight to progressively “illuminates” the semantic space to find the high scoring neighbors.
Revised: We name our algorithm as Flashlight because it works like a flashlight to progressively “illuminates” the semantic space to find the high scoring neighbors.

Original: The query embeddings are like the lights sent from the flashlight.
Revised: The query embeddings are like the lights sent from the flashlight.

Original: And our process of adjusting the query embeddings is just like progressively adjusting the “lights” from the “flashlight” by checking the “objects” found in the last “illumination”.
Revised: And our process of adjusting the query embeddings is just like progressively adjusting the “lights” from the “flashlight” by checking the “objects” found in the last “illumination”.

Original: In the experiments, we find that our Flashlight algorithm is effective to find the top scoring neighbors from the massive candidate neighbors.
Revised: In the experiments, we find that our Flashlight algorithm is effective to find the top scoring neighbors from the massive candidate neighbors.

Original: For example, in Fig. 2, our Flashlight is able to find the top 100 scoring neighbors from nearly three million candidates by retrieving only 200 neighbors in the large OGBL-CITATION2 graph dataset for the HadamardMLP decoders.
Revised: For example, in Fig. 2, our Flashlight is able to find the top 100 scoring neighbors from nearly three million candidates by retrieving only 200 neighbors in the large OGBL-CITATION2 graph dataset for the HadamardMLP decoders.

Original: Discussion.
Revised: Discussion.

Original: The convergence of our Flashlight is guaranteed since it iterates over a limited number of times to find the maximum inner product neighbors with different queries, as shown in lineof Algorithm 1.
Revised: The convergence of our Flashlight is guaranteed since it iterates over a limited number of times to find the maximum inner product neighbors with different queries, as shown in lineof Algorithm 1.

Original: Both the Inner Product Maximization search in every Flashlight iteration and the top scoring neighbors from HadamardMLP outputs are to maximize the sum of different paths in the MLP across neurons from the input layer to the output layer.
Revised: Both the Inner Product Maximization search in every Flashlight iteration and the top scoring neighbors from HadamardMLP outputs are to maximize the sum of different paths in the MLP across neurons from the input layer to the output layer.

Original: The only difference is that for the former all the paths contribute to the output while for the later only those paths with all the neurons activated contribute to the final result.
Revised: The only difference is that for the former all the paths contribute to the output while for the later only those paths with all the neurons activated contribute to the final result.

Original: No matter whether activated or not, every path and every neuron’s monotonicity is not changed.
Revised: No matter whether activated or not, every path and every neuron’s monotonicity is not changed.

Original: In the initial iteration of Flashlight, maximizing the inner product is to encourage the values from all paths to be larger, which reflect the general increasing trend of the HadamardMLP.
Revised: In the initial iteration of Flashlight, maximizing the inner product is to encourage the values from all paths to be larger, which reflect the general increasing trend of the HadamardMLP.

Original: In the later iterations, based on the activation patterns found on the top scoring neighbors, our Flashlight adaptively adjusts the query embedding to find more top scoring neighbors accurately.
Revised: In the later iterations, based on the activation patterns found on the top scoring neighbors, our Flashlight adaptively adjusts the query embedding to find more top scoring neighbors accurately.

Original: Complexity Analysis.
Revised: Complexity Analysis.

Original: Using MLP decoders to compute the LP probabilities of all the neighbors holds the complexity as O ( N ) , where N is the number of nodes in the whole graph.
Revised: Using MLP decoders to compute the LP probabilities of all the neighbors holds the complexity as O ( N ) , where N is the number of nodes in the whole graph.

Original: Finding the top scoring neighbors from the exact probabilities of all the neighbors also holds the linear complexity
Revised: Finding the top scoring neighbors from the exact probabilities of all the neighbors also holds the linear complexity

Original: O ( N ) .
Revised: O ( N ) .

Original: Overall, using MLP decoders to find the top scoring neighbors is of the time complexity
Revised: Overall, using MLP decoders to find the top scoring neighbors is of the time complexity

Original: O ( N ) .
Revised: O ( N ) .

Original: In contrast, our Flashlight progressively calls the MIPS techniques for a constant number of times invariant to the graph data, which leads to the sublinear complexity as same as MIPS.
Revised: In contrast, our Flashlight progressively calls the MIPS techniques for a constant number of times invariant to the graph data, which leads to the sublinear complexity as same as MIPS.

Original: In conclusion, our Flashlight improves the scalability and applicability of HadamardMLP decoders by reducing their inference time complexity from linear to sublinear time.
Revised: In conclusion, our Flashlight improves the scalability and applicability of HadamardMLP decoders by reducing their inference time complexity from linear to sublinear time.

Original:  Experiments
Revised:  Experiments

Original: In this section, we first compare the effectiveness of different LP decoders.
Revised: In this section, we first compare the effectiveness of different LP decoders.

Original: We find that
Revised: We find that

Original: the
Revised: the

Original: HadamardMLP decoders generally perform better than other decoders.
Revised: HadamardMLP decoders generally perform better than other decoders.

Original: Then, we implement our
Revised: Then, we implement our

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Flashlight algorithm with LP models to show that Flashlight effectively retrieves the top scoring neighbors for the HadamardMLP decoders.
Revised: Flashlight algorithm with LP models to show that Flashlight effectively retrieves the top scoring neighbors for the HadamardMLP decoders.

Original: As a result, the inference efficiency and scalability of
Revised: As a result, the inference efficiency and scalability of

Original: HadamardMLP decoders are improved significantly by our work.
Revised: HadamardMLP decoders are improved significantly by our work.

Original: In Table
Revised: In Table

Original: [Table]
Revised: [Table]

Original: we report the performance of LP methods with different decoders: Dot Product, Bilinear,
Revised: we report the performance of LP methods with different decoders: Dot Product, Bilinear,

Original: ConcatMLP, HadamardMLP, HadamardMLP with Flashlight, as denoted in different column names.
Revised: ConcatMLP, HadamardMLP, HadamardMLP with Flashlight, as denoted in different column names.

Original: In Fig.
Revised: In Fig.

Original: [Table]
Revised: [Table]

Original: we report the experimental results with the decoder HadamardMLP and HadamardMLP with Flashlight, which hold much better effectiveness on link prediction than other decoders, as discussed in Sec. 6.3.
Revised: we report the experimental results with the decoder HadamardMLP and HadamardMLP with Flashlight, which hold much better effectiveness on link prediction than other decoders, as discussed in Sec. 6.3.

Original: 5.1 Datasets
Revised: 5.1 Datasets

Original: We evaluate the link prediction on Open Graph Benchmark (OGB) data [23].
Revised: We evaluate the link prediction on Open Graph Benchmark (OGB) data [23].

Original: We use four OGB datasets with different graph types, including OGBL-DDI, OGBL-COLLAB, OGBL-CITATION2, and OGBL-PPA.
Revised: We use four OGB datasets with different graph types, including OGBL-DDI, OGBL-COLLAB, OGBL-CITATION2, and OGBL-PPA.

Original: OGBL-DDI is a homogeneous, unweighted, undirected graph, representing the drugdrug interaction network.
Revised: OGBL-DDI is a homogeneous, unweighted, undirected graph, representing the drugdrug interaction network.

Original: Each node represents a drug.
Revised: Each node represents a drug.

Original: Edges represent interactions between drugs.
Revised: Edges represent interactions between drugs.

Original: OGBL-COLLAB is an undirected graph, representing a subset of the collaboration network between authors indexed by MAG.
Revised: OGBL-COLLAB is an undirected graph, representing a subset of the collaboration network between authors indexed by MAG.

Original: Each node represents an author and edges indicate the collaboration between authors.
Revised: Each node represents an author and edges indicate the collaboration between authors.

Original: All nodes come with 128-dimensional features.
Revised: All nodes come with 128-dimensional features.

Original: OGBL-CITATION2 is a directed graph, representing the citation network between a subset of papers extracted from MAG.
Revised: OGBL-CITATION2 is a directed graph, representing the citation network between a subset of papers extracted from MAG.

Original: Each node is a paper with 128-dimensional word2vec features.
Revised: Each node is a paper with 128-dimensional word2vec features.

Original: OGBL-PPA is an undirected, unweighted graph.
Revised: OGBL-PPA is an undirected, unweighted graph.

Original: Nodes represent proteins from 58 different species, and edges indicate biologically meaningful associations between proteins.
Revised: Nodes represent proteins from 58 different species, and edges indicate biologically meaningful associations between proteins.

Original: The statistics of these datasets is presented in Table.
Revised: The statistics of these datasets is presented in Table.

Original: 5.2 Hyper-parameter Settings
Revised: 5.2 Hyper-parameter Settings

Original: For all experiments in this section, we report the average and standard deviation over ten runs with different random seeds.
Revised: For all experiments in this section, we report the average and standard deviation over ten runs with different random seeds.

Original: The results are reported on the the best model selected using validation data.
Revised: The results are reported on the the best model selected using validation data.

Original: We set hyper-parameters of the used techniques and considered baseline methods, e.g., the batch size, the number of hidden units, the optimizer, and the learning rate as suggested by their authors.
Revised: We set hyper-parameters of the used techniques and considered baseline methods, e.g., the batch size, the number of hidden units, the optimizer, and the learning rate as suggested by their authors.

Original: We use the recent MIPS method ScaNN [19] in the implementation of our Flashlight.
Revised: We use the recent MIPS method ScaNN [19] in the implementation of our Flashlight.

Original: For the hyper-parameters of our Flashlight, we have found in the experiments that the performance of
Revised: For the hyper-parameters of our Flashlight, we have found in the experiments that the performance of

Original: Flashlight is robust to the change of hyper-parameters in a board range.
Revised: Flashlight is robust to the change of hyper-parameters in a broad range.

Original: Therefore, we simply set the number of iterations of our Flashlight as T = 3 and the number of retrieved neighbors constant as 200 per iteration by default.
Revised: Therefore, we simply set the number of iterations of our Flashlight as T = 3 and the number of retrieved neighbors constant as 200 per iteration by default.

Original: We run all experiments on a machine with 80 Intel(R)
Revised: We run all experiments on a machine with 80 Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: E5v4 @ 2.20GHz CPUs, and a single NVIDIA V100 GPU with 16GB RAM.
Revised: E5v4 @ 2.20GHz CPUs, and a single NVIDIA V100 GPU with 16GB RAM.

Original: 5.3 Effectiveness of Link Prediction Decoders
Revised: 5.3 Effectiveness of Link Prediction Decoders

Original: We follow the standard benchmark settings of OGB datasets to evaluate the effectiveness of LP with different decoders.
Revised: We follow the standard benchmark settings of OGB datasets to evaluate the effectiveness of LP with different decoders.

Original: The benchmark setting of OGBL-DDI is to predict drug-drug interactions given information on already known drug-drug interactions.
Revised: The benchmark setting of OGBL-DDI is to predict drug-drug interactions given information on already known drug-drug interactions.

Original: The performance is evaluated by Hits@20: each true drug interaction is ranked among a set of approximately 100,000 randomly-sampled negative drug interactions, and count the ratio of positive edges that are ranked at 20-place or above.
Revised: The performance is evaluated by Hits@20: each true drug interaction is ranked among a set of approximately 100,000 randomly-sampled negative drug interactions, and count the ratio of positive edges that are ranked at 20-place or above.

Original: The task of OGBL-COLLAB is to predict the future author collaboration relationships given the past collaborations.
Revised: The task of OGBL-COLLAB is to predict the future author collaboration relationships given the past collaborations.

Original: Evaluation metric is Hits50, where each true collaboration is ranked among a set of 100,000 randomly-sampled negative collaborations.
Revised: Evaluation metric is Hits50, where each true collaboration is ranked among a set of 100,000 randomly-sampled negative collaborations.

Original: The task of OGBL-PPA is to predict new association edges given the training edges.
Revised: The task of OGBL-PPA is to predict new association edges given the training edges.

Original: Evaluation metric is Hits@100, where each positive edge is ranked among 3,000,000 randomly-sampled negative edges.
Revised: Evaluation metric is Hits@100, where each positive edge is ranked among 3,000,000 randomly-sampled negative edges.

Original: The task of OGBL-CITATION2 is predict missing citation given existing citations.
Revised: The task of OGBL-CITATION2 is predict missing citation given existing citations.

Original: The evaluation metric is Mean Reciprocal Rank (MRR), where the reciprocal rank of the true reference among 1,000 sampled negative candidates is calculated for each source nodes, and then the average is taken over all source nodes.
Revised: The evaluation metric is Mean Reciprocal Rank (MRR), where the reciprocal rank of the true reference among 1,000 sampled negative candidates is calculated for each source nodes, and then the average is taken over all source nodes.

Original: We implement different decoders as introduced in Sec. 2, including the Dot Product, Bilinear,
Revised: We implement different decoders as introduced in Sec. 2, including the Dot Product, Bilinear,

Original: ConcatMLP, and the HadamardMLP decoders, over the LP encoders, including GCN [5], GraphSAGE [12], and Node2Vec [22], to compare the effects of different decoders on the LP effectiveness.
Revised: ConcatMLP, and the HadamardMLP decoders, over the LP encoders, including GCN [5], GraphSAGE [12], and Node2Vec [22], to compare the effects of different decoders on the LP effectiveness.

Original: We present the results on the OGBL-DDI, OGBL-COLLAB, OGBL-PPA, and OGBL-CITATION datasets in Table.
Revised: We present the results on the OGBL-DDI, OGBL-COLLAB, OGBL-PPA, and OGBL-CITATION datasets in Table.

Original: We observe that the HadamardMLP decoder outperforms other decoders on all encoders and datasets.
Revised: We observe that the HadamardMLP decoder outperforms other decoders on all encoders and datasets.

Original: Our Flashlight algorithm can effectively retrieve the top scoring neighbors for the HadamardMLP decoder and keep the exact LP probabilities of HadamardMLPs’ output, which leads to the same results of the HadamardMLP decoder with and without Flashlight.
Revised: Our Flashlight algorithm can effectively retrieve the top scoring neighbors for the HadamardMLP decoder and keep the exact LP probabilities of HadamardMLPs’ output, which leads to the same results of the HadamardMLP decoder with and without Flashlight.

Original: In Table 2, different columns refer to the performances of different decoders.
Revised: In Table 2, different columns refer to the performances of different decoders.

Original: Our Flashlight is to to reduce the search space of HadamardMLP to improve the inference efficiency.
Revised: Our Flashlight is to to reduce the search space of HadamardMLP to improve the inference efficiency.

Original: For the top scoring neighbors, the final exact link prediction scores and orders are still determined by HadamardMLP without being influenced by our Flashlight.
Revised: For the top scoring neighbors, the final exact link prediction scores and orders are still determined by HadamardMLP without being influenced by our Flashlight.

Original: Our Flashlight is able to accurately retrieve the candidate neighbors that include the top scorning ones for the HadamardMLP decoder.
Revised: Our Flashlight is able to accurately retrieve the candidate neighbors that include the top scorning ones for the HadamardMLP decoder.

Original: This is why in Table 2, the HadamardMLP with and without our Flashlight exhibit the same performance.
Revised: This is why in Table 2, the HadamardMLP with and without our Flashlight exhibit the same performance.

Original: Note that the benchmark settings of these datasets sample a small portion of negative edges for the test evaluation, which is not challenging enough to evaluate the scalability of LP decoders on retrieving the top scoring neighbors from massive candidates in practice.
Revised: Note that the benchmark settings of these datasets sample a small portion of negative edges for the test evaluation, which is not challenging enough to evaluate the scalability of LP decoders on retrieving the top scoring neighbors from massive candidates in practice.

Original: 5.4 The Flashlight Algorithm Effectively Finds the Top Scoring Neighbors
Revised: 5.4 The Flashlight Algorithm Effectively Finds the Top Scoring Neighbors

Original: To evaluate the effectiveness of our Flashlight on retrieving the top scoring neighbors for the
Revised: To evaluate the effectiveness of our Flashlight on retrieving the top scoring neighbors for the

Original: HadamardMLP decoder, we propose a more challenging test setting for the OGB LP datasets.
Revised: HadamardMLP decoder, we propose a more challenging test setting for the OGB LP datasets.

Original: Given a source node, we takes its top 100 scoring neighbors of the HadamardMLP decoder as the ground-truth for retrievals.
Revised: Given a source node, we takes its top 100 scoring neighbors of the HadamardMLP decoder as the ground-truth for retrievals.

Original: We set the task as retrieving k neighbors for a source node that can match the ground-truth neighbors as much as possible.
Revised: We set the task as retrieving k neighbors for a source node that can match the ground-truth neighbors as much as possible.

Original: We formally define the metric as Recall @ k , which is the portion of the ground-truth neighbors being in the top k neighbors retrieved by different methods.
Revised: We formally define the metric as Recall @ k , which is the portion of the ground-truth neighbors being in the top k neighbors retrieved by different methods.

Original: We sample 1000 nodes as the source nodes from the OGBL-DDI and OGBL-CITATION2 datasets respectively for evaluation.
Revised: We sample 1000 nodes as the source nodes from the OGBL-DDI and OGBL-CITATION2 datasets respectively for evaluation.

Original: We evaluate the effectivness of our Flashlight algorithm by checking whether it can find the top scoring neighbors for every source node.
Revised: We evaluate the effectivness of our Flashlight algorithm by checking whether it can find the top scoring neighbors for every source node.

Original: We set the number of Flashlight iterations as 10 and the number of retrieved neighbors per iteration as 50.
Revised: We set the number of Flashlight iterations as 10 and the number of retrieved neighbors per iteration as 50.

Original: We present the Recall @ k for k from 1 to 500 averaged over all the source nodes in Fig.
Revised: We present the Recall @ k for k from 1 to 500 averaged over all the source nodes in Fig.

Original: The “oracle” curve represents the performance of a optimum searcher, of which the retrieved top k neighbors are exactly the top k scoring neighbors of HadamardMLP.
Revised: The “oracle” curve represents the performance of a optimum searcher, of which the retrieved top k neighbors are exactly the top k scoring neighbors of HadamardMLP.

Original: When k = 100 , the 100 neighbors retrieved by our Flashlight can cover more than 80% ground-truth neighbors.
Revised: When k = 100 , the 100 neighbors retrieved by our Flashlight can cover more than 80% ground-truth neighbors.

Original: When k ≥ 200 , the recall reaches 100%.
Revised: When k ≥ 200 , the recall reaches 100%.

Original: As a comparison, if we randomly sample the
Revised: As a comparison, if we randomly sample the

Original: [Figure]
Revised: [Figure]

Original: candidate neighbors for retrievals, the Recall @
Revised: candidate neighbors for retrievals, the Recall @

Original: k grows linearly with k and is less than 1 × 10 − 4 for k = 100 on the OGBL-CITATION2 dataset.
Revised: k grows linearly with k and is less than 1 × 10 − 4 for k = 100 on the OGBL-CITATION2 dataset.

Original: The curves of Flashlight is close the optimum curve of the “oracle”.
Revised: The curves of Flashlight is close the optimum curve of the “oracle”.

Original: These results demonstrate the highly effectiveness of our Flashlight on finding the top scoring neighbors.
Revised: These results demonstrate the highly effectiveness of our Flashlight on finding the top scoring neighbors.

Original: Given the large OGBL-Citation2 dataset and smaller DDI dataset, our Flashlight exhibits similar
Revised: Given the large OGBL-Citation2 dataset and smaller DDI dataset, our Flashlight exhibits similar

Original: Recall @
Revised: Recall @

Original: k performance given different numbers k of retrieved neighbors.
Revised: k performance given different numbers k of retrieved neighbors.

Original: This implies that our
Revised: This implies that our

Original: Flashlight can accurately find the top scoring neighbors for both small and large graphs.
Revised: Flashlight can accurately find the top scoring neighbors for both small and large graphs.

Original: 5.5 Inference Efficiency of Link Prediction with Our Flashlight Algorithm
Revised: 5.5 Inference Efficiency of Link Prediction with Our Flashlight Algorithm

Original: We use the throughputs to evaluate the inference speed of neighbor retrieval of different methods.
Revised: We use the throughputs to evaluate the inference speed of neighbor retrieval of different methods.

Original: The throughput is defined as how many source nodes that a method can serve to retrieve the top 100 scoring neighbors per second.
Revised: The throughput is defined as how many source nodes that a method can serve to retrieve the top 100 scoring neighbors per second.

Original: Except for the LP models that follow the encoder and decoder architectures, e.g., GraphSAGE [12], GCN [5], and PLNLP [6], there are some subgraph based LP models, e.g., SUREL [7] and SEAL [24].
Revised: Except for the LP models that follow the encoder and decoder architectures, e.g., GraphSAGE [12], GCN [5], and PLNLP [6], there are some subgraph based LP models, e.g., SUREL [7] and SEAL [24].

Original: The common issue of the subgraph based models is the poor efficiency: they have to crop a seperate subgraph for every node pair to calculate the LP probability on the node pair.
Revised: The common issue of the subgraph based models is the poor efficiency: they have to crop a seperate subgraph for every node pair to calculate the LP probability on the node pair.

Original: In this sense, the node embeddings cannot be shared on the LP calculation for different node pairs.
Revised: In this sense, the node embeddings cannot be shared on the LP calculation for different node pairs.

Original: This leads to the much lower inference speed of the subgraph based LP models than the encoder-decoder LP models.
Revised: This leads to the much lower inference speed of the subgraph based LP models than the encoder-decoder LP models.

Original: We compare the inference effeciency of different methods on the OGBL-CITATION2 dataset in Fig.
Revised: We compare the inference effeciency of different methods on the OGBL-CITATION2 dataset in Fig.

Original: 3, where we present the inference speed of different methods when achieving the 100% Recall for the top 100 scoring neighbors.
Revised: 3, where we present the inference speed of different methods when achieving the 100% Recall for the top 100 scoring neighbors.

Original: We observe that our Flashlight significantly accelerate the inference speed of LP models GraphSAGE [12], GCN [5], and PLNLP [6] with the HadamardMLP decoders by more than 100 times.
Revised: We observe that our Flashlight significantly accelerate the inference speed of LP models GraphSAGE [12], GCN [5], and PLNLP [6] with the HadamardMLP decoders by more than 100 times.

Original: This gap will be even larger for the datasets of larger scales, because the inference with our Flashlight holds the sublinear time complexity while the HadamardMLP decoders holds the linear complexity.
Revised: This gap will be even larger for the datasets of larger scales, because the inference with our Flashlight holds the sublinear time complexity while the HadamardMLP decoders holds the linear complexity.

Original: Note that the y-axis is in logoratimic scale.
Revised: Note that the y-axis is in logoratimic scale.

Original: The subgraph based methods SUREL [7] and SEAL [24] hold the inference speed of throuputs lower than 1 × 10 − 2 and 1 × 10 − 3 respectively, which is not applicable to the practical services that require the low latency of milliseconds.
Revised: The subgraph based methods SUREL [7] and SEAL [24] hold the inference speed of throuputs lower than 1 × 10 − 2 and 1 × 10 − 3 respectively, which is not applicable to the practical services that require the low latency of milliseconds.

Original: Taking a further step, we comprehensively evaluate the tradeoff between the inference speed and the effectiveness of finding the top scoring neighbors.
Revised: Taking a further step, we comprehensively evaluate the tradeoff between the inference speed and the effectiveness of finding the top scoring neighbors.

Original: Taking GraphSAGE as the encoder, we present the tradeoff curves between the throughputs and the Recall on retrieving the top 100 neighbors for the
Revised: Taking GraphSAGE as the encoder, we present the tradeoff curves between the throughputs and the Recall on retrieving the top 100 neighbors for the

Original: OGBL-CITATION2 and OGBL-PPI datasets in Fig.
Revised: OGBL-CITATION2 and OGBL-PPI datasets in Fig.

Original: In comparison with our Flashlight, we take the HadamardMLP decoder with the Random Sampling as the baseline for comparison.
Revised: In comparison with our Flashlight, we take the HadamardMLP decoder with the Random Sampling as the baseline for comparison.

Original: We take the random sampling as a baseline method to reduce the search space for HadmardMLP because it is easy to understand and can act as a default choice when reducing the search space.
Revised: We take the random sampling as a baseline method to reduce the search space for HadmardMLP because it is easy to understand and can act as a default choice when reducing the search space.

Original: We observe that on the
Revised: We observe that on the

Original: OGBL-CITATION2 dataset, when achieving the Recall as more than 80%, the HadamardMLP with our Flashlight can serve more than 200 source nodes per second, while the HadamardMLP with the
Revised: OGBL-CITATION2 dataset, when achieving the Recall as more than 80%, the HadamardMLP with our Flashlight can serve more than 200 source nodes per second, while the HadamardMLP with the

Original: [Figure]
Revised: [Figure]

Original: u
Revised: u

Original: [Figure]
Revised: [Figure]

Original: hpu t ( n o
Revised: hpu t ( n o

Original: [Figure]
Revised: [Figure]

Original: s
Revised: s

Original: [Figure]
Revised: [Figure]

Original: s ec o nd )
Revised: s ec o nd )

Original: [Figure]
Revised: [Figure]

Original: random sampling can only serve less than 1 node per second.
Revised: random sampling can only serve less than 1 node per second.

Original: Overall, our Flashlight achieves much better inference speed and effectiveness tradeoff than the HadamardMLP with random sampling.
Revised: Overall, our Flashlight achieves much better inference speed and effectiveness tradeoff than the HadamardMLP with random sampling.

Original: 5.6 Ablation Study
Revised: 5.6 Ablation Study

Original: We analyze the sensitivity of Flashlight to the hyper-parameter: the number of iterations, and the number of retrieved neighbors per iteration.
Revised: We analyze the sensitivity of Flashlight to the hyper-parameter: the number of iterations, and the number of retrieved neighbors per iteration.

Original: The recall result on retrieving the top 100 scoring neighbors for the OGBL-CITATION2 dataset is presented in the Table 3.
Revised: The recall result on retrieving the top 100 scoring neighbors for the OGBL-CITATION2 dataset is presented in the Table 3.

Original: We alter number the number of iterations among {1, 2, 3, 4, 5} and number of retrieved neighbors per iteration among {100, 200, 300, 400}.
Revised: We alter number the number of iterations among {1, 2, 3, 4, 5} and number of retrieved neighbors per iteration among {100, 200, 300, 400}.

Original: For the result corresponding to Flashlight using the all-one mask neuron activation, please refer to the first column.
Revised: For the result corresponding to Flashlight using the all-one mask neuron activation, please refer to the first column.

Original: The performance of Flashlight is relatively smooth when parameters are within certain ranges.
Revised: The performance of Flashlight is relatively smooth when parameters are within certain ranges.

Original: However, extremely small values of the number of iterations and the number of retrieved neighbors per iteration result in poor performances.
Revised: However, extremely small values of the number of iterations and the number of retrieved neighbors per iteration result in poor performances.

Original: A too small number of iterations make the Flashlight unable to adaptively adjust its querying embedding
Revised: A too small number of iterations make the Flashlight unable to adaptively adjust its querying embedding

Original: [Table]
Revised: [Table]

Original: on finding neighbors, while a too small number of retrieved neighbors per iteration make Flashlight unable to retrieve enough neighbors to cover the top 100 scoring neighbors, i.e., the ground-truth ones.
Revised: on finding neighbors, while a too small number of retrieved neighbors per iteration make Flashlight unable to retrieve enough neighbors to cover the top 100 scoring neighbors, i.e., the ground-truth ones.

Original: Empirically, increasing the number of iterations for Flashlight can boost the performance more fast than increasing the number of retrieved neighbors per iteration.
Revised: Empirically, increasing the number of iterations for Flashlight can boost the performance more fast than increasing the number of retrieved neighbors per iteration.

Original: The reason is that with more iterations, Flashlight can find the neuron activation patterns of the high scoring neighbors more effectively and thus be able to adaptively adjust the query embeddings.
Revised: The reason is that with more iterations, Flashlight can find the neuron activation patterns of the high scoring neighbors more effectively and thus be able to adaptively adjust the query embeddings.

Original: Overall, only a poorly set hyper-parameter does not lead to significant performance degradation, which demonstrates that our Flashlight framework is able to find the high scoring neighbors among massive candidates on large graphs.
Revised: Overall, only a poorly set hyper-parameter does not lead to significant performance degradation, which demonstrates that our Flashlight framework is able to find the high scoring neighbors among massive candidates on large graphs.

Original:  Related Work 6.1 Link Prediction Models
Revised:  Related Work 6.1 Link Prediction Models

Original: Existing LP models can be categorized into three families: heuristic feature based [3, 9, 25–27], latent embedding based [12, 22, 28–31], and neural network based ones.
Revised: Existing LP models can be categorized into three families: heuristic feature based [3, 9, 25–27], latent embedding based [12, 22, 28–31], and neural network based ones.

Original: The neural network-based link prediction models are mainly developed in recent years, which explore non-linear deep structural features with neural layers.
Revised: The neural network-based link prediction models are mainly developed in recent years, which explore non-linear deep structural features with neural layers.

Original: Variational graph auto-encoders [13] predict links by encoding graph with graph convolutional layer [5].
Revised: Variational graph auto-encoders [13] predict links by encoding graph with graph convolutional layer [5].

Original: Another two state-of-the-art neural models WLNM [32] and SEAL [33] use graph labeling algorithm to transfer union neighborhood of two nodes (enclosing subgraph) as meaningful matrix and employ convolutional neural layer or a novel graph neural layer DGCNN [34] for encoding.
Revised: Another two state-of-the-art neural models WLNM [32] and SEAL [33] use graph labeling algorithm to transfer union neighborhood of two nodes (enclosing subgraph) as meaningful matrix and employ convolutional neural layer or a novel graph neural layer DGCNN [34] for encoding.

Original: More recently, [6, 8] summarized the architectures LP models, and formally define the encoders and decoders.
Revised: More recently, [6, 8] summarized the architectures LP models, and formally define the encoders and decoders.

Original: Different from the previous work, we focus on analyzing the effectiveness of different LP decoders and improving the scalability of the effective LP decoders.
Revised: Different from the previous work, we focus on analyzing the effectiveness of different LP decoders and improving the scalability of the effective LP decoders.

Original: In practice, we find that the Hadamard decoders exhibit superior effectiveness but poor scalability for inference.
Revised: In practice, we find that the Hadamard decoders exhibit superior effectiveness but poor scalability for inference.

Original: Our work significantly accelerates the inference of HadamardMLP decoders to make the effective LP scalable.
Revised: Our work significantly accelerates the inference of HadamardMLP decoders to make the effective LP scalable.

Original: 6.2 Maximum Inner Product Search
Revised: 6.2 Maximum Inner Product Search

Original: Finding the top scoring neighbors for the Dot Product decoder at the sublinear time complexity is a well studied research problem, known as the approximate maximum inner product search (MIPS).
Revised: Finding the top scoring neighbors for the Dot Product decoder at the sublinear time complexity is a well studied research problem, known as the approximate maximum inner product search (MIPS).

Original: There are several approaches to MIPS: sampling based [11, 35, 36], LSH-based [37–40], graph based [41–43], and quantization approaches [18, 19].
Revised: There are several approaches to MIPS: sampling based [11, 35, 36], LSH-based [37–40], graph based [41–43], and quantization approaches [18, 19].

Original: MIPS is a fundamental building block in various application domains [44–49], such as information retrieval [50, 51], pattern recognition [52, 53], data mining [54, 55], machine learning [56, 57], and recommendation systems [58, 59].
Revised: MIPS is a fundamental building block in various application domains [44–49], such as information retrieval [50, 51], pattern recognition [52, 53], data mining [54, 55], machine learning [56, 57], and recommendation systems [58, 59].

Original: With the explosive growth of datasets’ scale and the inevitable curse of dimensionality, MIPS is essential to offer the scalable services.
Revised: With the explosive growth of datasets’ scale and the inevitable curse of dimensionality, MIPS is essential to offer the scalable services.

Original: However, the HadamardMLP decoders are nonlinear and there do not exist the well studied sublinear complexity algorithms to find the top scoring neighbors for
Revised: However, the HadamardMLP decoders are nonlinear and there do not exist the well studied sublinear complexity algorithms to find the top scoring neighbors for

Original: HadamardMLP [10].
Revised: HadamardMLP [10].

Original: In this work, we utilize the well studied approximate MIPS techniques with the adaptively adjusted query embeddings to find the top scoring neighbors for the MLP decoders in a progressive manner.
Revised: In this work, we utilize the well studied approximate MIPS techniques with the adaptively adjusted query embeddings to find the top scoring neighbors for the MLP decoders in a progressive manner.

Original: Our method supports the plug-and-play use during inference and significantly acclerates the LP inference with the effective MLP decoders.
Revised: Our method supports the plug-and-play use during inference and significantly acclerates the LP inference with the effective MLP decoders.

Original:  Conclusion
Revised:  Conclusion

Original: Our theoretical and empirical analysis suggests that the HadamardMLP decoders are a better default choice than the Dot Product in terms of LP effectiveness.
Revised: Our theoretical and empirical analysis suggests that the HadamardMLP decoders are a better default choice than the Dot Product in terms of LP effectiveness.

Original: Because there does not exist a welldeveloped sublinear complexity top scoring neighbor searching algorithm for HadamardMLP, the
Revised: Because there does not exist a welldeveloped sublinear complexity top scoring neighbor searching algorithm for HadamardMLP, the

Original: HadamardMLP decoders are not scalable and cannot support the fast inference on large graphs.
Revised: HadamardMLP decoders are not scalable and cannot support the fast inference on large graphs.

Original: To resolve this issue, we propose the Flashlight algorithm to accelerate the inference of LP models with
Revised: To resolve this issue, we propose the Flashlight algorithm to accelerate the inference of LP models with

Original: HadamardMLP decoders.
Revised: HadamardMLP decoders.

Original: Flashlight progressively operates the well-studied MIPS techniques for a few iterations.
Revised: Flashlight progressively operates the well-studied MIPS techniques for a few iterations.

Original: We adaptively adjust the query embeddings at every iteration to find more high scoring neighbors.
Revised: We adaptively adjust the query embeddings at every iteration to find more high scoring neighbors.

Original: Empirical results show that our Flashlight accelrates the inference of LP models by more than 100 times on the large OGBL-CITATION2 graph.
Revised: Empirical results show that our Flashlight accelrates the inference of LP models by more than 100 times on the large OGBL-CITATION2 graph.

Original: Overall, our work paves the way for the use of strong LP decoders in practical settings by greatly accelerating their inference.
Revised: Overall, our work paves the way for the use of strong LP decoders in practical settings by greatly accelerating their inference.

Original: Acknowledgements
Revised: Acknowledgements

Original: This paper is supported by NUS ODPRT
Revised: This paper is supported by NUS ODPRT

Original: Grant R252-000-A81-133 and Singapore Ministry of
Revised: Grant R252-000-A81-133 and Singapore Ministry of

Original: Education Academic Research Fund Tier 3 under MOEs official grant number MOE2017-T3-1-007.
Revised: Education Academic Research Fund Tier 3 under MOEs official grant number MOE2017-T3-1-007.

Original: A Learning a Dot Product decoder with a HadamardMLP decoder is Easy
Revised: A Learning a Dot Product decoder with a HadamardMLP decoder is Easy

Original: Before we have discussed the limitations of the Dot Product decoder.
Revised: Before we have discussed the limitations of the Dot Product decoder.

Original: An interesting questions is whether the HadamardMLP decoder can replace the Dot Product decoder by approximating it.
Revised: An interesting questions is whether the HadamardMLP decoder can replace the Dot Product decoder by approximating it.

Original: If the
Revised: If the

Original: MLP decoder can learn a dot product easily
Revised: MLP decoder can learn a dot product easily

Original: [Figure]
Revised: [Figure]

Original: it is safe to use MLP decoder instead of the dot product ones in most cases.
Revised: it is safe to use MLP decoder instead of the dot product ones in most cases.

Original: There are similar problems actively studied in machine learning.
Revised: There are similar problems actively studied in machine learning.

Original: Existing work imply that the difficulty scales polynomial with dimensionality d and 1 /ϵ in theory [10, 60, 61].
Revised: Existing work imply that the difficulty scales polynomial with dimensionality d and 1 /ϵ in theory [10, 60, 61].

Original: This motivates us to investigate the question empirically.
Revised: This motivates us to investigate the question empirically.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: We set up a synthetic learning task where given two embeddings x i , x j ∈ R d and a label x i • x j , we want to obtain a MLP function that approximates the x i • x j with the inputs x i , x j ∈ R d .
Revised: We set up a synthetic learning task where given two embeddings x i , x j ∈ R d and a label x i • x j , we want to obtain a MLP function that approximates the x i • x j with the inputs x i , x j ∈ R d .

Original: For this experiment, we create the datasets including the embedding matrix as E ∈ R 10 6 × d .
Revised: For this experiment, we create the datasets including the embedding matrix as E ∈ R 10 6 × d .

Original: We draw every row in E from N (0 , I ) independently.
Revised: We draw every row in E from N (0 , I ) independently.

Original: Then, we uniformly sample (without replacement) 10 4 and S embedding pair combinations from E to form the test and training sets (no overlap) respectively.
Revised: Then, we uniformly sample (without replacement) 10 4 and S embedding pair combinations from E to form the test and training sets (no overlap) respectively.

Original: We train the MLP on the training and evalute it on the test set.
Revised: We train the MLP on the training and evalute it on the test set.

Original: For the architecture of the MLP, we keep it simple: we follow the existing work [6, 8] to set the number of layers as 2 and the number of hidden units as same as the input embeddings: d .
Revised: For the architecture of the MLP, we keep it simple: we follow the existing work [6, 8] to set the number of layers as 2 and the number of hidden units as same as the input embeddings: d .

Original: For the optimizer, we also folow the existing work [6, 8] to choose the Adam optimizer.
Revised: For the optimizer, we also folow the existing work [6, 8] to choose the Adam optimizer.

Original: As for evaluation metrics, we compute the MSE (Mean Squared Error) differences between the predicted score of the MLP and the dot product decoders.
Revised: As for evaluation metrics, we compute the MSE (Mean Squared Error) differences between the predicted score of the MLP and the dot product decoders.

Original: We measure the MSE of a naive model that predicts always 

Revised: We measure the MSE of a naive model that predicts always

Original: (the average rating).
Revised: (the average rating).

Original: Every experiment is repeated 5 times and we report the mean.
Revised: Every experiment is repeated 5 times and we report the mean.

Original: Fig.
Revised: Fig.

Original: 5 shows the approximation errors on the MLP per epoch given different number of training pairs and dimensions.
Revised: 5 shows the approximation errors on the MLP per epoch given different number of training pairs and dimensions.

Original: The figure suggests that an MLP can easily approximate the dot product with enough training data.
Revised: The figure suggests that an MLP can easily approximate the dot product with enough training data.

Original: Consistent with the theory, the number of samples needed scales polynominally with the increasing dimensions and reduced errors.
Revised: Consistent with the theory, the number of samples needed scales polynominally with the increasing dimensions and reduced errors.

Original: Ancedotally, we observe the number of needed training samples is about O ( d α /ϵ β ) for α ≈ 2 , β ≪ 1 (see Fig. 6).
Revised: Ancedotally, we observe the number of needed training samples is about O ( d α /ϵ β ) for α ≈ 2 , β ≪ 1 (see Fig. 6).

Original: In all cases, the MSE errors of the MLP decoder are negligible compared with the naive output.
Revised: In all cases, the MSE errors of the MLP decoder are negligible compared with the naive output.

Original: This experiment shows that an MLP can easily approximate the dot product with enough training data.
Revised: This experiment shows that an MLP can easily approximate the dot product with enough training data.

Original: We hope this can explain, at least partially, why the MLP decoder generally performs better than the dot product.
Revised: We hope this can explain, at least partially, why the MLP decoder generally performs better than the dot product.

Original: Our conclusion seems to be distinct to to the existing work [10], which claims that the ConcatMLP is hard to learn a Dot Product.
Revised: Our conclusion seems to be distinct to to the existing work [10], which claims that the ConcatMLP is hard to learn a Dot Product.

Original: Actually, our conclusion is not conflicted with that in [10].
Revised: Actually, our conclusion is not conflicted with that in [10].

Original: This
Revised: This

Original: ConcatMLP decoder processes the concatenation of the paired embeddings instead of the Hadamard product of the paired embeddings as the HadamardMLP.
Revised: ConcatMLP decoder processes the concatenation of the paired embeddings instead of the Hadamard product of the paired embeddings as the HadamardMLP.

Original: The HadamardMLP holds the inductive bias similar to the Dot Product, which makes the former easily learns the latter.
Revised: The HadamardMLP holds the inductive bias similar to the Dot Product, which makes the former easily learns the latter.

Original: We show that a simple two-layer MLP with only two hidden units is equivalent to the Dot Product with specific weights.
Revised: We show that a simple two-layer MLP with only two hidden units is equivalent to the Dot Product with specific weights.

Original: We assign the first layer weights for two hidden units as 1 and − 1 and the second layer weights as 1 and − 1 for the two hidden units respectively.
Revised: We assign the first layer weights for two hidden units as 1 and − 1 and the second layer weights as 1 and − 1 for the two hidden units respectively.

Original: Then, we have its output as:
Revised: Then, we have its output as:

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to the Dot Product decoder.
Revised: which is equivalent to the Dot Product decoder.

Original: From this result, we find that any MLP decoder with the careful initialization is equivalent to the Dot Product decoder and thus can learn the Dot Product easily.
Revised: From this result, we find that any MLP decoder with the careful initialization is equivalent to the Dot Product decoder and thus can learn the Dot Product easily.

Original: B More Discussion on ConcatMLP and HadamardMLP
Revised: B More Discussion on ConcatMLP and HadamardMLP

Original: Although ConcatMLP is as expressive as HadamardMLP in theory because MLP is a universal function approximator [15], such an argument neglects the difficulty of learning a target function using ConcatMLP.
Revised: Although ConcatMLP is as expressive as HadamardMLP in theory because MLP is a universal function approximator [15], such an argument neglects the difficulty of learning a target function using ConcatMLP.

Original: The ConcatMLP decoder processes the concatenation of the paired embeddings instead of the Hadamard product of the paired embeddings.
Revised: The ConcatMLP decoder processes the concatenation of the paired embeddings instead of the Hadamard product of the paired embeddings.

Original: In contrast, HadamardMLP takes the
Revised: In contrast, HadamardMLP takes the

Original: Hadamard product of the paired embeddings as the input.
Revised: Hadamard product of the paired embeddings as the input.

Original: The inductive bias of HadamardMLP enables HadamardMLP to decode the semantic connectivity of different embeddings more easily in practice (see the empirical analysis in Appendix A).
Revised: The inductive bias of HadamardMLP enables HadamardMLP to decode the semantic connectivity of different embeddings more easily in practice (see the empirical analysis in Appendix A).

Original: Actually, our work is not the first to discuss the practical limitations of ConcatMLP decoders.
Revised: Actually, our work is not the first to discuss the practical limitations of ConcatMLP decoders.

Original: For example, the existing work [10, 62, 63] has pointed out that In deep neural network designs, it is very common to replace a ConcatMLP with a more specialized structure that has an inductive bias that represents the problem better, which is crucial for advancing the state of the art of deep learning, although in theory they can all be approximated by ConcatMLPs.
Revised: For example, the existing work [10, 62, 63] has pointed out that In deep neural network designs, it is very common to replace a ConcatMLP with a more specialized structure that has an inductive bias that represents the problem better, which is crucial for advancing the state of the art of deep learning, although in theory they can all be approximated by ConcatMLPs.

Original: The inefficiency of ConcatMLPs to capture dot and tensor products has been studied by [64] in the context of recommender systems.
Revised: The inefficiency of ConcatMLPs to capture dot and tensor products has been studied by [64] in the context of recommender systems.

Original: Similar to our analysis in Sec. 2 and Appendix A, [64] points out that it is hard for ConcatMLPs to approximate dot products and tensor products with ConcatMLPs empirically.
Revised: Similar to our analysis in Sec. 2 and Appendix A, [64] points out that it is hard for ConcatMLPs to approximate dot products and tensor products with ConcatMLPs empirically.

Original: C Comparison with More Baselines
Revised: C Comparison with More Baselines

Original: Finding the neighbors that maximize the dot product between the target node and neighbors’ embeddings to reduce the search space for the HadmardMLP is a good baseline to compare with.
Revised: Finding the neighbors that maximize the dot product between the target node and neighbors’ embeddings to reduce the search space for the HadmardMLP is a good baseline to compare with.

Original: For the simplicity of expression, we term this baseline as DotMax.
Revised: For the simplicity of expression, we term this baseline as DotMax.

Original: We conducted the experiments on finding the top scoring neighbors for HadmardMLP with DotMax.
Revised: We conducted the experiments on finding the top scoring neighbors for HadmardMLP with DotMax.

Original: We found that DotMax needs to retrieve much more neighbors than our Flashlight to achieve the same Recall as Flashlight on finding the top 100 scoring neighbors.
Revised: We found that DotMax needs to retrieve much more neighbors than our Flashlight to achieve the same Recall as Flashlight on finding the top 100 scoring neighbors.

Original: For example, on the OGBL-CITATION2 dataset, which is the largest dataset among the used data holding nearly 3 million nodes as shown in Table 1, we follow the experimental settings as introduced in Sec.
Revised: For example, on the OGBL-CITATION2 dataset, which is the largest dataset among the used data holding nearly 3 million nodes as shown in Table 1, we follow the experimental settings as introduced in Sec.

Original: 5.4 to test DotMax.
Revised: 5.4 to test DotMax.

Original: DotMax achieves the following Recall on finding the top 100 scoring neighbors for HadmardMLP with different numbers of retrieved neighbors:
Revised: DotMax achieves the following Recall on finding the top 100 scoring neighbors for HadmardMLP with different numbers of retrieved neighbors:

Original: [Table]
Revised: [Table]

Original: In comparison, under the same experimental setting, when retrieving only 100 neighbors, our
Revised: In comparison, under the same experimental setting, when retrieving only 100 neighbors, our

Original: Flashlight can cover more than 80% of the top 100 scoring neighbors for HadmardMLP (as shown in
Revised: Flashlight can cover more than 80% of the top 100 scoring neighbors for HadmardMLP (as shown in

Original: Fig. 2).
Revised: Fig. 2).

Original: When retrieving more than 200 neighbors, Flashlight achieves the recall of 100%.
Revised: When retrieving more than 200 neighbors, Flashlight achieves the recall of 100%.

Original: Overall, our proposed Flashlight can reduce the search space for the HadmardMLP decoder much more effectively than the DotMax.
Revised: Overall, our proposed Flashlight can reduce the search space for the HadmardMLP decoder much more effectively than the DotMax.

Original: The reason is that Dot Product demands the homophily of graph data to effectively infer the link between nodes.
Revised: The reason is that Dot Product demands the homophily of graph data to effectively infer the link between nodes.

Original: In contrast, thanks to the universal approximation capability, MLP can approximate any continuous function, and thus does not demand the homophily of graph data for effective LP.
Revised: In contrast, thanks to the universal approximation capability, MLP can approximate any continuous function, and thus does not demand the homophily of graph data for effective LP.

Original: This constraint makes the Dot Product and HadamardMLP prefer different patterns of nodes’ semantic embeddings on computing the link prediction scores.
Revised: This constraint makes the Dot Product and HadamardMLP prefer different patterns of nodes’ semantic embeddings on computing the link prediction scores.

Original: As a result, using DotMax to reduce the search space cannot effectively cover the top scoring neighbors for HadamardMLP with a small number of retrieved neighbors.
Revised: As a result, using DotMax to reduce the search space cannot effectively cover the top scoring neighbors for HadamardMLP with a small number of retrieved neighbors.

Original: We observe that, under the same experimental setting, when retrieving only 150 neighbors, our
Revised: We observe that, under the same experimental setting, when retrieving only 150 neighbors, our

Original: Flashlight can cover more than 90% of the top 100 scoring neighbors for HadmardMLP, while
Revised: Flashlight can cover more than 90% of the top 100 scoring neighbors for HadmardMLP, while

Original: DotMax needs to retrieve more than 100000 neighbors to achieve the recall of only around 60%.
Revised: DotMax needs to retrieve more than 100000 neighbors to achieve the recall of only around 60%.

Original: Dot
Revised: Dot

Original: Product demands the homophily of graph data to effectively infer the link between nodes, which makes it hard to effectively work on the heterogenous graphs.
Revised: Product demands the homophily of graph data to effectively infer the link between nodes, which makes it hard to effectively work on the heterogenous graphs.

Original: In contrast, our Flashlight effectively retrieve the high scoring neighbors for HadamardMLP, which does not rely on the homophily of graph data for effective link prediction.
Revised: In contrast, our Flashlight effectively retrieve the high scoring neighbors for HadamardMLP, which does not rely on the homophily of graph data for effective link prediction.