Original: L OOK IN T HE M IRROR : M OLECULAR G RAPH C ONTRASTIVE L EARNING WITH L INE G RAPH
Revised: L OOK IN T HE M IRROR : M OLECULAR G RAPH C ONTRASTIVE L EARNING WITH L INE G RAPH

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Trapped by the label scarcity in molecular property prediction and drug design, graph contrastive learning came forward.
Revised: Trapped by the label scarcity in molecular property prediction and drug design, graph contrastive learning came forward.

Original: A general contrastive model consists of a view generator, view encoder, and contrastive loss, in which the view mainly controls the encoded information underlying input graphs.
Revised: A general contrastive model consists of a view generator, view encoder, and contrastive loss, in which the view mainly controls the encoded information underlying input graphs.

Original: Leading contrastive learning works show two kinds of view generators, that is, random or learnable data corruption and domain knowledge incorporation.
Revised: Leading contrastive learning works show two kinds of view generators, that is, random or learnable data corruption and domain knowledge incorporation.

Original: While effective, the two ways also lead to molecular semantics altering and limited generalization capability, respectively.
Revised: While effective, the two ways also lead to molecular semantics altering and limited generalization capability, respectively.

Original: Thus, a decent view that can fully retain molecular semantics and is free from profound domain knowledge is supposed to come forward.
Revised: Thus, a decent view that can fully retain molecular semantics and is free from profound domain knowledge is supposed to come forward.

Original: To this end, we relate molecular graph contrastive learning with the line graph and propose a novel method termed LGCL.
Revised: To this end, we relate molecular graph contrastive learning with the line graph and propose a novel method termed LGCL.

Original: Specifically, by contrasting the given graph with the corresponding line graph, the graph encoder can freely encode the molecular semantics without omission.
Revised: Specifically, by contrasting the given graph with the corresponding line graph, the graph encoder can freely encode the molecular semantics without omission.

Original: While considering the information inconsistency and over-smoothing derived from the learning process because of the mismatched pace of message passing in two kinds of graphs, we present a new patch with edge attribute fusion and two local contrastive losses for performance fixing.
Revised: While considering the information inconsistency and over-smoothing derived from the learning process because of the mismatched pace of message passing in two kinds of graphs, we present a new patch with edge attribute fusion and two local contrastive losses for performance fixing.

Original: Compared with state-of-the-art (SOTA) methods for view generation, superior performance on molecular property prediction suggests the effectiveness of line graphs severing as the contrasting views.
Revised: Compared with state-of-the-art (SOTA) methods for view generation, superior performance on molecular property prediction suggests the effectiveness of line graphs severing as the contrasting views.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: A deep understanding of molecular properties plays a vital role in the chemical and pharmaceutical domains.
Revised: A deep understanding of molecular properties plays a vital role in the chemical and pharmaceutical domains.

Original: In order to computationally discover novel materials and drugs, the molecules will be abstractly regarded as graphs, in which atoms are vertices and bonds are edges Gilmer et al.
Revised: In order to computationally discover novel materials and drugs, the molecules will be abstractly regarded as graphs, in which atoms are vertices and bonds are edges Gilmer et al.

Original: Goh et al.
Revised: Goh et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: (2018a).
Revised: (2018a).

Original: Thus, the marriage between molecular property prediction and graph learning captured a bunch of researchers and showed their happiness in several fields Yang et al.
Revised: Thus, the marriage between molecular property prediction and graph learning captured a bunch of researchers and showed their happiness in several fields Yang et al.

Original: ; Song et al.
Revised: ; Song et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: ; Wu et al.
Revised: ; Wu et al.

Original: (2022a).
Revised: (2022a).

Original: However, this relationship faces the challenges of label scarcity, as deep learning methods are known to consume massive amounts of labeled data, and annotated data are often of limited size and hard to acquire when considering many specific domains.
Revised: However, this relationship faces the challenges of label scarcity, as deep learning methods are known to consume massive amounts of labeled data, and annotated data are often of limited size and hard to acquire when considering many specific domains.

Original: In addition, given the immense differentiation among chemical molecules, existing supervised models could be barely reused in unseen cases Hu et al.
Revised: In addition, given the immense differentiation among chemical molecules, existing supervised models could be barely reused in unseen cases Hu et al.

Original: Rong et al.
Revised: Rong et al.

Original: Therefore, there are increasing demands for molecular representation learning in an unsupervised or self-supervised manner.
Revised: Therefore, there are increasing demands for molecular representation learning in an unsupervised or self-supervised manner.

Original: Plenty of works have attempted to learn molecule representations discarding the supervision of labels, like graph context prediction Liu et al.
Revised: Plenty of works have attempted to learn molecule representations discarding the supervision of labels, like graph context prediction Liu et al.

Original: graph-level motif prediction Rong et al.
Revised: graph-level motif prediction Rong et al.

Original: and masked attribute prediction Hu et al.
Revised: and masked attribute prediction Hu et al.

Original: In light of the contrastive learning from computer vision, researchers go one step further to model molecules in a contrastive manner with data augmentations You et al.
Revised: In light of the contrastive learning from computer vision, researchers go one step further to model molecules in a contrastive manner with data augmentations You et al.

Original: Suresh et al.
Revised: Suresh et al.

Original: Considering the inherent characteristics of chemical molecules, graph contrastive learning incorporating well-designed domain knowledge has also shown excellent capacity in molecular properties prediction Sun et al.
Revised: Considering the inherent characteristics of chemical molecules, graph contrastive learning incorporating well-designed domain knowledge has also shown excellent capacity in molecular properties prediction Sun et al.

Original: (2021); Fang et al.
Revised: (2021); Fang et al.

Original: Analogously, everything comes with a price.
Revised: Analogously, everything comes with a price.

Original: Inspecting the generated views in previous molecular graph contrastive learning unveils two intrinsic limitations.
Revised: Inspecting the generated views in previous molecular graph contrastive learning unveils two intrinsic limitations.

Original: First, data augmentation-based methods adopting random or learnable corruption (e.g., node/edge dropping and graph generation) would lead to inevitable variance in the crucial semantics and further misguide the contrastive learning
Revised: First, data augmentation-based methods adopting random or learnable corruption (e.g., node/edge dropping and graph generation) would lead to inevitable variance in the crucial semantics and further misguide the contrastive learning

Original: [Figure]
Revised: [Figure]

Original: You et al.
Revised: You et al.

Original: (2020); Sun et al.
Revised: (2020); Sun et al.

Original: Second, based on predefined sub-structure substitution rules
Revised: Second, based on predefined sub-structure substitution rules

Original: Sun et al.
Revised: Sun et al.

Original: (2021) or contrasted with 3-dimension geometric views Liu et al.
Revised: (2021) or contrasted with 3-dimension geometric views Liu et al.

Original: ; St ¨ ark et al.
Revised: ; St ¨ ark et al.

Original: (2022), domain knowledge-based methods intend to alleviate the problem of semantic alteration.
Revised: (2022), domain knowledge-based methods intend to alleviate the problem of semantic alteration.

Original: While effective, they are stinted to the profound domain knowledge that is unfriendly to researchers without such knowledge, thus limiting their generalization capability in other domains.
Revised: While effective, they are stinted to the profound domain knowledge that is unfriendly to researchers without such knowledge, thus limiting their generalization capability in other domains.

Original: In this context, we are seeking for a decent view that will not be bothered by prefabricated domain knowledge and can maintain the molecular semantic information integrally.
Revised: In this context, we are seeking for a decent view that will not be bothered by prefabricated domain knowledge and can maintain the molecular semantic information integrally.

Original: Fortunately, we met the line graph, also known as congruent graph in graph theory Whitney (1932);
Revised: Fortunately, we met the line graph, also known as congruent graph in graph theory Whitney (1932);

Original: Harary & Norman (1960); Jung (1966).
Revised: Harary & Norman (1960); Jung (1966).

Original: In a line graph, the nodes correspond to the edges of the original graph, and the edges refer to the common nodes of the pair edges in the original graph.
Revised: In a line graph, the nodes correspond to the edges of the original graph, and the edges refer to the common nodes of the pair edges in the original graph.

Original: In particular, the isomorphism of two line graphs is judged to be consistent with the corresponding two original graphs
Revised: In particular, the isomorphism of two line graphs is judged to be consistent with the corresponding two original graphs

Original: Whitney (1932); Jung (1966), which ensures the congruent semantic structure after line graph transformation.
Revised: Whitney (1932); Jung (1966), which ensures the congruent semantic structure after line graph transformation.

Original: In light of the line graph, we propose a method termed LGCL to tackle our expectations.
Revised: In light of the line graph, we propose a method termed LGCL to tackle our expectations.

Original: The framework of LGCL is shown in Figure 1.
Revised: The framework of LGCL is shown in Figure 1.

Original: Specifically, to fill the framework demanding two views, all input molecular graphs are transformed into the corresponding line graph.
Revised: Specifically, to fill the framework demanding two views, all input molecular graphs are transformed into the corresponding line graph.

Original: On such a basis,
Revised: On such a basis,

Original: LGCL equips with a dual-helix graph encoder to learn the hidden representation of two views.
Revised: LGCL equips with a dual-helix graph encoder to learn the hidden representation of two views.

Original: Note that, due to the different pace of message passing in the original graph and the corresponding line graph, two issues derive from the learning process, that is, information inconsistency and oversmoothing.
Revised: Note that, due to the different pace of message passing in the original graph and the corresponding line graph, two issues derive from the learning process, that is, information inconsistency and oversmoothing.

Original: For information consistency, we further update the graph encoder with edge attribute fusion to bridge the edge attributes between the two kinds of graphs.
Revised: For information consistency, we further update the graph encoder with edge attribute fusion to bridge the edge attributes between the two kinds of graphs.

Original: Over-smoothing is addressed by a novel intra-local contrastive loss based on the idea of NT-Xent loss; put differently, the intralocal contrastive loss aims to maximize the consense between the edge pairs in the two corresponding views and minimize the consense between different edge pairs within the same views.
Revised: Over-smoothing is addressed by a novel intra-local contrastive loss based on the idea of NT-Xent loss; put differently, the intralocal contrastive loss aims to maximize the consense between the edge pairs in the two corresponding views and minimize the consense between different edge pairs within the same views.

Original: Moreover, we further give an inter-local contrastive loss to enhance the representation learning.
Revised: Moreover, we further give an inter-local contrastive loss to enhance the representation learning.

Original: The effectiveness of LGCL is verified under the ubiquitous setting of transfer learning for molecular property prediction Hu et al.
Revised: The effectiveness of LGCL is verified under the ubiquitous setting of transfer learning for molecular property prediction Hu et al.

Original: Through pre-training on two million molecular graphs from
Revised: Through pre-training on two million molecular graphs from

Original: ZINC15, LGCL shows superior performance on six out of eight benchmarks for molecular property prediction and acquires the highest position on both average ROC-AUC and average ranking.
Revised: ZINC15, LGCL shows superior performance on six out of eight benchmarks for molecular property prediction and acquires the highest position on both average ROC-AUC and average ranking.

Original: Additionally, we delve deeper into the proposed components via analytical experiments to further assess their benefits.
Revised: Additionally, we delve deeper into the proposed components via analytical experiments to further assess their benefits.

Original: The contributions are elaborated below:
Revised: The contributions are elaborated below:

Original: • To the best of our knowledge, we are the first to figure out a way to freely and fully excavate molecular semantics within graph contrastive learning.
Revised: • To the best of our knowledge, we are the first to figure out a way to freely and fully excavate molecular semantics within graph contrastive learning.

Original: • Inspired by the line graph, we present an approach, termed LGCL, to tackle our expectations, in which edge attribute fusion and two local contrastive losses are united to address the concomitant issues and enhance molecular representation learning.
Revised: • Inspired by the line graph, we present an approach, termed LGCL, to tackle our expectations, in which edge attribute fusion and two local contrastive losses are united to address the concomitant issues and enhance molecular representation learning.

Original: • Leveraging eight benchmarks for molecular property prediction under the setting of transfer learning, LGCL exhibits its superiority against the SOTA methods for view generation.
Revised: • Leveraging eight benchmarks for molecular property prediction under the setting of transfer learning, LGCL exhibits its superiority against the SOTA methods for view generation.

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: This research focuses on molecular graph contrastive view generation, especially considering the case that is free from the intricate domain knowledge.
Revised: This research focuses on molecular graph contrastive view generation, especially considering the case that is free from the intricate domain knowledge.

Original: We will elaborate on these topics below.
Revised: We will elaborate on these topics below.

Original: Molecular graph contrastive learning.
Revised: Molecular graph contrastive learning.

Original: Along with the development of graph contrastive learning, plenty of research efforts have been devoted to designing contrastive learning models for molecular graphs Sun et al.
Revised: Along with the development of graph contrastive learning, plenty of research efforts have been devoted to designing contrastive learning models for molecular graphs Sun et al.

Original: ; Xu et al.
Revised: ; Xu et al.

Original: (2021); Fang et al.
Revised: (2021); Fang et al.

Original: ; St¨ark et al.
Revised: ; St¨ark et al.

Original: Liu et al.
Revised: Liu et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: (2022a).
Revised: (2022a).

Original: Besides random or learnable corruption, several works presented various contrastive learning models to embed the molecular geometry information by means of contrasting the generic 2D graph with its 3D conformers Liu et al.
Revised: Besides random or learnable corruption, several works presented various contrastive learning models to embed the molecular geometry information by means of contrasting the generic 2D graph with its 3D conformers Liu et al.

Original: ; St¨ark et al.
Revised: ; St¨ark et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: (2022a).
Revised: (2022a).

Original: They indeed get rid of the semantic altering issue caused by random corruption on molecular graphs, while introducing another semantic altering issue caused by 3D conformers, because a single 2D molecular graph generally has multiple conformers with different chemical properties St ¨ ark et al.
Revised: They indeed get rid of the semantic altering issue caused by random corruption on molecular graphs, while introducing another semantic altering issue caused by 3D conformers, because a single 2D molecular graph generally has multiple conformers with different chemical properties St ¨ ark et al.

Original: To enhance the performance in molecular property prediction, the domain knowledge-driven contrastive learning frameworks were proposed to preserve the semantics of graphs in the augmentation process Sun et al.
Revised: To enhance the performance in molecular property prediction, the domain knowledge-driven contrastive learning frameworks were proposed to preserve the semantics of graphs in the augmentation process Sun et al.

Original: (2021); Fang et al.
Revised: (2021); Fang et al.

Original: However, their learning capability heavily relies on the dissolved domain knowledge, that is the well-designed substitution rules in MoCL Sun et al.
Revised: However, their learning capability heavily relies on the dissolved domain knowledge, that is the well-designed substitution rules in MoCL Sun et al.

Original: (2021) and the prefabricated associations among chemical elements in KCL Fang et al.
Revised: (2021) and the prefabricated associations among chemical elements in KCL Fang et al.

Original: Furthermore, the domain knowledge varies across domains, which limits the application of these methods.
Revised: Furthermore, the domain knowledge varies across domains, which limits the application of these methods.

Original: Recently, besides the contrasting view exploration, GraphLoG Xu et al.
Revised: Recently, besides the contrasting view exploration, GraphLoG Xu et al.

Original: and OEPG Yang &
Revised: and OEPG Yang &

Original: Hong (2022) are built upon the generic graph contrastive learning methods to discover the global semantic structure underlying the whole dataset and present excellent performance.
Revised: Hong (2022) are built upon the generic graph contrastive learning methods to discover the global semantic structure underlying the whole dataset and present excellent performance.

Original: In this work, we are devoted to the domain of contrasting view generation, which is orthogonal to the works for dataset semantic structure exploration; put differently, extensive works for contrasting view generation can work with the framework of GraphLoG and OEPG to produce more superior performance.
Revised: In this work, we are devoted to the domain of contrasting view generation, which is orthogonal to the works for dataset semantic structure exploration; put differently, extensive works for contrasting view generation can work with the framework of GraphLoG and OEPG to produce more superior performance.

Original: Line graph.
Revised: Line graph.

Original: The line graph is a classic concept and has a long history in graph theory Whitney (1932); Harary & Norman (1960); Jung (1966).
Revised: The line graph is a classic concept and has a long history in graph theory Whitney (1932); Harary & Norman (1960); Jung (1966).

Original: In a line graph, the nodes correspond to the edges of the original graph, and the edges refer to the common nodes of the pair edges in the original graph.
Revised: In a line graph, the nodes correspond to the edges of the original graph, and the edges refer to the common nodes of the pair edges in the original graph.

Original: Thus, the graph neural networks (GNNs) built on line graphs are capable of encoding edge features and enhancing feature learning on graphs.
Revised: Thus, the graph neural networks (GNNs) built on line graphs are capable of encoding edge features and enhancing feature learning on graphs.

Original: Recently, based on the line graph structures, several line graph neural networks have shown promising performance on various graph-related tasks Chen et al.
Revised: Recently, based on the line graph structures, several line graph neural networks have shown promising performance on various graph-related tasks Chen et al.

Original: (2018b)
Revised: (2018b)

Original: ; Jiang et al.
Revised: ; Jiang et al.

Original: ; Bandyopadhyay et al.
Revised: ; Bandyopadhyay et al.

Original: In the chemistry domain, the structure of a compound can be treated as a graph, where the edges derived from chemical bonds link the corresponding atom nodes.
Revised: In the chemistry domain, the structure of a compound can be treated as a graph, where the edges derived from chemical bonds link the corresponding atom nodes.

Original: Thus, the edges in such graphs have different properties and various functions.
Revised: Thus, the edges in such graphs have different properties and various functions.

Original: In generic GNNs, however, message-passing operations among nodes do not pay enough attention to the edge properties.
Revised: In generic GNNs, however, message-passing operations among nodes do not pay enough attention to the edge properties.

Original: Fortunately, the line graph structure enables generic GNNs to take advantage of the edges as equals as nodes Jiang et al.
Revised: Fortunately, the line graph structure enables generic GNNs to take advantage of the edges as equals as nodes Jiang et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: (2018b).
Revised: (2018b).

Original: To this far, there is still no graph contrastive learning model to encode the molecular semantics integrally without well-designed domain knowledge.
Revised: To this far, there is still no graph contrastive learning model to encode the molecular semantics integrally without well-designed domain knowledge.

Original: In this paper, we revisit the line graph from the angle of graph contrastive learning.
Revised: In this paper, we revisit the line graph from the angle of graph contrastive learning.

Original: Based on it, we design a novel contrastive model, termed
Revised: Based on it, we design a novel contrastive model, termed

Original: LGCL, to freely and fully excavate molecular semantics.
Revised: LGCL, to freely and fully excavate molecular semantics.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: Here, we first present some preliminary concepts and notations.
Revised: Here, we first present some preliminary concepts and notations.

Original: In this work, let G =
Revised: In this work, let G =

Original: { G 1 , G 2 , · · · , G N } be a graph dataset with size N , and a molecular graph can be formulated as
Revised: { G 1 , G 2 , · · · , G N } be a graph dataset with size N , and a molecular graph can be formulated as

Original: G =
Revised: G =

Original: ( V, E, X V , X E ) , where V is the node set, E is the edge set, X V ∈ R | V |× V denotes the node features, and X E ∈ R | E |× E stands for the edge attributes.
Revised: ( V, E, X V , X E ) , where V is the node set, E is the edge set, X V ∈ R | V |× V denotes the node features, and X E ∈ R | E |× E stands for the edge attributes.

Original: 3. G RAPH R EPRESENTATION L EARNING
Revised: 3. G RAPH R EPRESENTATION L EARNING

Original: In generic GNNs, the message-passing scheme is adopted for information transmission among nodes
Revised: In generic GNNs, the message-passing scheme is adopted for information transmission among nodes

Original: Xu et al.
Revised: Xu et al.

Original: ; Wu et al.
Revised: ; Wu et al.

Original: (2022b).
Revised: (2022b).

Original: Through stacking L layers, a GNN will produce a hidden representation h v ∈ R with L -hop neighbor information for each node and a feature vector h G ∈ R via a global readout function for the entire graph G .
Revised: Through stacking L layers, a GNN will produce a hidden representation h v ∈ R with L -hop neighbor information for each node and a feature vector h G ∈ R via a global readout function for the entire graph G .

Original: Each node v is initialized with node feature X v and sent to the GNN input.
Revised: Each node v is initialized with node feature X v and sent to the GNN input.

Original: Formally, the l -th layer of a GNN Xu et al.
Revised: Formally, the l -th layer of a GNN Xu et al.

Original: (2019) can be written as
Revised: (2019) can be written as

Original: [Equation]
Revised: [Equation]

Original: where h ( l ) v represents the feature vector of node v at the l -th iteration, N ( v ) covers the 1-hop neighbors of v , AGGREGATE denotes the crucial message-passing scheme in GNNs, and COMBINE is used to update the hidden feature of v via merging information from its neighbors and itself.
Revised: where h ( l ) v represents the feature vector of node v at the l -th iteration, N ( v ) covers the 1-hop neighbors of v , AGGREGATE denotes the crucial message-passing scheme in GNNs, and COMBINE is used to update the hidden feature of v via merging information from its neighbors and itself.

Original: Finally, a GNN can produce the feature vector h G of the entire graph with a prefabricated readout function:
Revised: Finally, a GNN can produce the feature vector h G of the entire graph with a prefabricated readout function:

Original: [Equation]
Revised: [Equation]

Original: where READOUT aggregates the final set of node representations.
Revised: where READOUT aggregates the final set of node representations.

Original: 3. G RAPH C ONTRASTIVE L EARNING
Revised: 3. G RAPH C ONTRASTIVE L EARNING

Original: In a generic graph contrastive learning model, two correlated views from the same graph G i are required for contrasting and generally produced by two augmentation operations.
Revised: In a generic graph contrastive learning model, two correlated views from the same graph G i are required for contrasting and generally produced by two augmentation operations.

Original: Here, we denote the augmented views as ˜ G 1 i and ˜ G 2 i .
Revised: Here, we denote the augmented views as ˜ G 1 i and ˜ G 2 i .

Original: Then, a graph encoder and a projection head are stacked behind the two augmentation operators to map the correlated views into an embedding space and yield corresponding feature vectors
Revised: Then, a graph encoder and a projection head are stacked behind the two augmentation operators to map the correlated views into an embedding space and yield corresponding feature vectors

Original: h 1 i and h 2 i .
Revised: h 1 i and h 2 i .

Original: The released hidden representations are supposed to contain the essential features of the original graph G so that they can recognize themselves from the others.
Revised: The released hidden representations are supposed to contain the essential features of the original graph G so that they can recognize themselves from the others.

Original: Thus, the objective of graph contrastive learning is to maximize the consensus between the two positive views via the widespread NT-Xent loss Chen et al.
Revised: Thus, the objective of graph contrastive learning is to maximize the consensus between the two positive views via the widespread NT-Xent loss Chen et al.

Original: [Equation]
Revised: [Equation]

Original: where N is the batch size, τ refers to the temperature parameter, and sim ( h 1 , h 2 ) denotes a cosine similarity function
Revised: where N is the batch size, τ refers to the temperature parameter, and sim ( h 1 , h 2 ) denotes a cosine similarity function

Original: h
Revised: h

Original: [Equation]
Revised: [Equation]

Original: ||·||
Revised: ||·||

Original: h 2 || .
Revised: h 2 || .

Original: The numerator part is the similarity of the correlated views as positive pair.
Revised: The numerator part is the similarity of the correlated views as positive pair.

Original: The rest pairs that consist of views from different graphs are regarded as negative pairs and act as the denominator part.
Revised: The rest pairs that consist of views from different graphs are regarded as negative pairs and act as the denominator part.

Original: Note that the negative pairs can come from two directions, put differently, h 1 i can pair with all h 2 j , and h 2 i can pair with all h 1 j .
Revised: Note that the negative pairs can come from two directions, put differently, h 1 i can pair with all h 2 j , and h 2 i can pair with all h 1 j .

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: In this section, we bring about the proposed graph contrastive learning framework, termed LGCL , by revisiting the line graph of corresponding molecules.
Revised: In this section, we bring about the proposed graph contrastive learning framework, termed LGCL , by revisiting the line graph of corresponding molecules.

Original: Given the issue of label scarcity in realworld graph data, LGCL is designed to encode the molecular semantics integrally and free from well-designed domain knowledge.
Revised: Given the issue of label scarcity in realworld graph data, LGCL is designed to encode the molecular semantics integrally and free from well-designed domain knowledge.

Original: Specifically, to produce two contrastive views without any loss of molecular semantics, we first need to transform the given molecule to the corresponding line graph.
Revised: Specifically, to produce two contrastive views without any loss of molecular semantics, we first need to transform the given molecule to the corresponding line graph.

Original: On such a basis, LGCL equips with a dual-helix graph encoder to learn the hidden representation of two views with edge attribute fusion.
Revised: On such a basis, LGCL equips with a dual-helix graph encoder to learn the hidden representation of two views with edge attribute fusion.

Original: In particular, besides the ubiquitous contrastive loss for the readout graph representations, we further propose two local contrastive losses to enhance representation learning and alleviate the over-smoothing issue in deep GNNs.
Revised: In particular, besides the ubiquitous contrastive loss for the readout graph representations, we further propose two local contrastive losses to enhance representation learning and alleviate the over-smoothing issue in deep GNNs.

Original: Next, we elaborate on the LGCL framework below.
Revised: Next, we elaborate on the LGCL framework below.

Original: 4. L INE G RAPH T RANSFORMATION
Revised: 4. L INE G RAPH T RANSFORMATION

Original: Here, we first present an illustration of line graph transformation from a simple graph.
Revised: Here, we first present an illustration of line graph transformation from a simple graph.

Original: As shown in Figure 2, let G =
Revised: As shown in Figure 2, let G =

Original: ( V, E ) be a simple undirected graph, the output line graph L ( G ) after transformation is such a graph that reveals the adjacencies of edges in G .
Revised: ( V, E ) be a simple undirected graph, the output line graph L ( G ) after transformation is such a graph that reveals the adjacencies of edges in G .

Original: Specifically, each edge in G
Revised: Specifically, each edge in G

Original: [Figure]
Revised: [Figure]

Original: is mapped into a node of L ( G ) and each edge in L ( G ) indicates that the corresponding two vertices have a common node in G .
Revised: is mapped into a node of L ( G ) and each edge in L ( G ) indicates that the corresponding two vertices have a common node in G .

Original: Formally, the line graph can be written as L ( G ) =
Revised: Formally, the line graph can be written as L ( G ) =

Original: ( V L , E L ) , where V L = { ( v i , v j ) | ( v i , v j ) ∈ E ) } and E L = { (( v i , v j ) , ( v j , v k ))
Revised: ( V L , E L ) , where V L = { ( v i , v j ) | ( v i , v j ) ∈ E ) } and E L = { (( v i , v j ) , ( v j , v k ))

Original: |{ ( v i , v j ) , ( v j , v k ) } ⊂ E } .
Revised: |{ ( v i , v j ) , ( v j , v k ) } ⊂ E } .

Original: At this point, we have settled the topology transformation of line graphs.
Revised: At this point, we have settled the topology transformation of line graphs.

Original: Besides the relationships among nodes and edges, the node and edge attributes underlying the molecular graph should also be delivered to the corresponding line graph.
Revised: Besides the relationships among nodes and edges, the node and edge attributes underlying the molecular graph should also be delivered to the corresponding line graph.

Original: In this paper, based on the one-to-one correspondence between the edges of G and the nodes of L ( G ) , the node attributes of the line graph can be directly obtained from the edge attributes of original graphs, that is, X V L = X E .
Revised: In this paper, based on the one-to-one correspondence between the edges of G and the nodes of L ( G ) , the node attributes of the line graph can be directly obtained from the edge attributes of original graphs, that is, X V L = X E .

Original: As for the edge attributes of L ( G ) , because several edges in the line graph would correspond to the same node in the original graph, a mapping function with such relationships is required to endow the line graph edge with the original node attribute.
Revised: As for the edge attributes of L ( G ) , because several edges in the line graph would correspond to the same node in the original graph, a mapping function with such relationships is required to endow the line graph edge with the original node attribute.

Original: In the light of E L after the line graph transformation, the edge attribute mapping function can be formulated as M ( e L ) =
Revised: In the light of E L after the line graph transformation, the edge attribute mapping function can be formulated as M ( e L ) =

Original: ( v i , v j ) ∩ ( v j , v k ) , thus the edge attributes of the line graph can be obtained via X E L = MX V .
Revised: ( v i , v j ) ∩ ( v j , v k ) , thus the edge attributes of the line graph can be obtained via X E L = MX V .

Original: Finally, the line graph of a molecular graph is given by L ( G ) =
Revised: Finally, the line graph of a molecular graph is given by L ( G ) =

Original: ( V L , E L , X V L , X E L ) .
Revised: ( V L , E L , X V L , X E L ) .

Original: According to Roussopoulos’s algorithm Roussopoulos (1973), the time complexity of line graph transformation is O (max( | V | , | E | )) .
Revised: According to Roussopoulos’s algorithm Roussopoulos (1973), the time complexity of line graph transformation is O (max( | V | , | E | )) .

Original: As stated in the Whitney graph isomorphism theorem Whitney (1932), the isomorphism of two line graphs is judged to be consistent with the corresponding two original graphs, which convinces us that the semantic structure information of G is encoded in the line graph L ( G ) .
Revised: As stated in the Whitney graph isomorphism theorem Whitney (1932), the isomorphism of two line graphs is judged to be consistent with the corresponding two original graphs, which convinces us that the semantic structure information of G is encoded in the line graph L ( G ) .

Original: In particular, as described in the line graph transformation, there is a one-to-one correspondence between the edges in the graph G and the vertices in the line graph L ( G ) .
Revised: In particular, as described in the line graph transformation, there is a one-to-one correspondence between the edges in the graph G and the vertices in the line graph L ( G ) .

Original: Therefore, a vertex with e edges in G will produce e × ( e − 1) / 2 edges in L ( G ) .
Revised: Therefore, a vertex with e edges in G will produce e × ( e − 1) / 2 edges in L ( G ) .

Original: Meanwhile, the message-passing frequency around this node will drift from O ( e ) in G to O ( e 2 ) in L ( G ) , put differently, this node feature in G is only passed to e neighbors, while the corresponding line graph will pass such information to e × ( e − 1) / 2 nodes.
Revised: Meanwhile, the message-passing frequency around this node will drift from O ( e ) in G to O ( e 2 ) in L ( G ) , put differently, this node feature in G is only passed to e neighbors, while the corresponding line graph will pass such information to e × ( e − 1) / 2 nodes.

Original: Actually, in light of the line graph capacity in Table A.1, the line graph encoder only requires about 50% more computation than the original graph encoder.
Revised: Actually, in light of the line graph capacity in Table A.1, the line graph encoder only requires about 50% more computation than the original graph encoder.

Original: While this nature of the line graph could cause two inevitable issues in the contrastive learning framework with stacked graph convolutional layers, that is information inconsistency and over-smoothing.
Revised: While this nature of the line graph could cause two inevitable issues in the contrastive learning framework with stacked graph convolutional layers, that is information inconsistency and over-smoothing.

Original: In this paper, we propose two approaches, edge attribute fusion and two local contrastive losses, to alleviate the two issues and strengthen molecular representation learning.
Revised: In this paper, we propose two approaches, edge attribute fusion and two local contrastive losses, to alleviate the two issues and strengthen molecular representation learning.

Original: Next, we give a detailed description.
Revised: Next, we give a detailed description.

Original: 4. E DGE A TTRIBUTE F USION
Revised: 4. E DGE A TTRIBUTE F USION

Original: In the chemistry domain, the structure of a compound can be treated as a graph, where the edges derived from the chemical bonds link the corresponding atom nodes.
Revised: In the chemistry domain, the structure of a compound can be treated as a graph, where the edges derived from the chemical bonds link the corresponding atom nodes.

Original: Thus, the edges in such graphs have different properties and various functions.
Revised: Thus, the edges in such graphs have different properties and various functions.

Original: Besides the topology information weaved by atoms, a well-designed graph convolution with edge attributes plays a crucial role in molecular property and protein function prediction.
Revised: Besides the topology information weaved by atoms, a well-designed graph convolution with edge attributes plays a crucial role in molecular property and protein function prediction.

Original: Given a molecular graph, its input node features and edge features are both represented as a 2dimensional categorical vector (see Appendix A for details), denoted as X V ∈ R | V |× 2 and X E ∈
Revised: Given a molecular graph, its input node features and edge features are both represented as a 2dimensional categorical vector (see Appendix A for details), denoted as X V ∈ R | V |× 2 and X E ∈

Original: R | E |× 2 , respectively.
Revised: R | E |× 2 , respectively.

Original: In previous works regarding molecular property prediction Hu et al.
Revised: In previous works regarding molecular property prediction Hu et al.

Original: (2020), the raw node categorical vectors are embedded in the input layer by
Revised: (2020), the raw node categorical vectors are embedded in the input layer by

Original: [Equation]
Revised: [Equation]

Original: where x 0 v and x 1 v are the atomic number and chirality tag of node v , respectively.
Revised: where x 0 v and x 1 v are the atomic number and chirality tag of node v , respectively.

Original: EMBEDDING ( · ) denotes an embedding function that transfers a single integer into a d -dimensional vector space.
Revised: EMBEDDING ( · ) denotes an embedding function that transfers a single integer into a d -dimensional vector space.

Original: Meanwhile, the raw edge categorical vectors are embedded in each layer by
Revised: Meanwhile, the raw edge categorical vectors are embedded in each layer by

Original: [Equation]
Revised: [Equation]

Original: where x 0 e and x 1 e represent the bond type and bond direction, respectively, and l denotes the index of GNN layers.
Revised: where x 0 e and x 1 e represent the bond type and bond direction, respectively, and l denotes the index of GNN layers.

Original: At the l -th layer, the node representation can be updated by
Revised: At the l -th layer, the node representation can be updated by

Original: [Equation]
Revised: [Equation]

Original: where σ ( · ) is an activation function, and ( v, v ) represents the self-loop edge.
Revised: where σ ( · ) is an activation function, and ( v, v ) represents the self-loop edge.

Original: Under this GNN architecture, the output molecular representations will be decorated with edge attributes.
Revised: Under this GNN architecture, the output molecular representations will be decorated with edge attributes.

Original: However, as discussed above, there is a significant difference in message-passing frequency between the original graph and the corresponding line graph, which could lead to information inconsistency between the outputs.
Revised: However, as discussed above, there is a significant difference in message-passing frequency between the original graph and the corresponding line graph, which could lead to information inconsistency between the outputs.

Original: Here, we present a novel edge attribute fusion approach to tackle this issue.
Revised: Here, we present a novel edge attribute fusion approach to tackle this issue.

Original: Specifically, we bridge the edge information between the molecular graph and line graph to help the original graph encoder keep pace with the line graph encoder.
Revised: Specifically, we bridge the edge information between the molecular graph and line graph to help the original graph encoder keep pace with the line graph encoder.

Original: The edge and node embeddings are still employed as the initial edge attributes in the first layer (i.e., l = 0 ).
Revised: The edge and node embeddings are still employed as the initial edge attributes in the first layer (i.e., l = 0 ).

Original: As for l ≥ 1 , the edge attributes of the original graph are obtained from the node hidden features in the line graph, which is formally given by
Revised: As for l ≥ 1 , the edge attributes of the original graph are obtained from the node hidden features in the line graph, which is formally given by

Original: [Equation]
Revised: [Equation]

Original: where ( v i , v j ) ∈ E
Revised: where ( v i , v j ) ∈ E

Original: and ( v i , v j ) ∈ V L .
Revised: and ( v i , v j ) ∈ V L .

Original: Correspondingly, the edge attributes of the line graph can be updated by the node hidden features in the original graph, which is formally formulated as:
Revised: Correspondingly, the edge attributes of the line graph can be updated by the node hidden features in the original graph, which is formally formulated as:

Original: [Equation]
Revised: [Equation]

Original: where ( v i , v j ) ∈ E , ( v j , v k ) ∈ E and (( v i , v j ) , ( v j , v k ))
Revised: where ( v i , v j ) ∈ E , ( v j , v k ) ∈ E and (( v i , v j ) , ( v j , v k ))

Original: ∈ E L .
Revised: ∈ E L .

Original: Based on the dual-helix graph encoder with edge attribute fusion, the hidden features from the line graph will be dissolved into the original graph representations, allowing information consistency between the two contrastive views and enhancing molecular representation learning.
Revised: Based on the dual-helix graph encoder with edge attribute fusion, the hidden features from the line graph will be dissolved into the original graph representations, allowing information consistency between the two contrastive views and enhancing molecular representation learning.

Original: 4. I NTRA -L OCAL C ONTRASTIVE L OSS
Revised: 4. I NTRA -L OCAL C ONTRASTIVE L OSS

Original: In this part, we look forward to tackling the over-smoothing issue introduced by the line graph.
Revised: In this part, we look forward to tackling the over-smoothing issue introduced by the line graph.

Original: Motivated by NT-Xent loss for contrastive learning, an intra-local contrastive loss is proposed.
Revised: Motivated by NT-Xent loss for contrastive learning, an intra-local contrastive loss is proposed.

Original: The design concept of NT-Xent loss aims to maximize the representation similarities of positive pairs consisting of hidden features of the same molecules and enforce dissimilarity of negative pairs comprising hidden features of different molecules simultaneously.
Revised: The design concept of NT-Xent loss aims to maximize the representation similarities of positive pairs consisting of hidden features of the same molecules and enforce dissimilarity of negative pairs comprising hidden features of different molecules simultaneously.

Original: Similarly, the proposed intra-local contrastive loss seeks to optimize the consensus between the same nodes as opposed to different nodes in a single graph.
Revised: Similarly, the proposed intra-local contrastive loss seeks to optimize the consensus between the same nodes as opposed to different nodes in a single graph.

Original: Considering the one-to-one correspondence between the edges in G and the vertices in L ( G ) , the contrastive samples of this loss are composed of the edge hidden features in G and the node hidden features in L ( G ) .
Revised: Considering the one-to-one correspondence between the edges in G and the vertices in L ( G ) , the contrastive samples of this loss are composed of the edge hidden features in G and the node hidden features in L ( G ) .

Original: Thus, given a graph G , the intra-local contrastive loss of one edge pair is formally defined as:
Revised: Thus, given a graph G , the intra-local contrastive loss of one edge pair is formally defined as:

Original: [Equation]
Revised: [Equation]

Original: where e i = ( v m , v n ) , e i ∈ E and ( v m , v n ) ∈ V L .
Revised: where e i = ( v m , v n ) , e i ∈ E and ( v m , v n ) ∈ V L .

Original: In particular, the edge representations from G are formed by such hidden features of the two endpoints of each edge, that is, ˜ h G · e i =
Revised: In particular, the edge representations from G are formed by such hidden features of the two endpoints of each edge, that is, ˜ h G · e i =

Original: MLP ([ h v m , h v n ]) .
Revised: MLP ([ h v m , h v n ]) .

Original: In light of this contrastive loss designed inside the graph, we look forward to reducing the similarity between different nodes and further alleviating the over-smoothing.
Revised: In light of this contrastive loss designed inside the graph, we look forward to reducing the similarity between different nodes and further alleviating the over-smoothing.

Original: 4. I NTER -L OCAL C ONTRASTIVE L OSS
Revised: 4. I NTER -L OCAL C ONTRASTIVE L OSS

Original: Here, we give another contrastive loss to enhance molecular graph contrastive learning.
Revised: Here, we give another contrastive loss to enhance molecular graph contrastive learning.

Original: As we are capable of enforcing dissimilarity between different edge representations, we move forward to generalizing the edge dissimilarity to all contrasted samples.
Revised: As we are capable of enforcing dissimilarity between different edge representations, we move forward to generalizing the edge dissimilarity to all contrasted samples.

Original: Our critical insight is that the widespread NT-Xent loss only provides graph-level contrast, while a contrastive angle based on node representations would also be meaningful in crucial structure identification.
Revised: Our critical insight is that the widespread NT-Xent loss only provides graph-level contrast, while a contrastive angle based on node representations would also be meaningful in crucial structure identification.

Original: In light of the intralocal contrastive loss, the inter-local contrastive loss can be formally formulated as:
Revised: In light of the intralocal contrastive loss, the inter-local contrastive loss can be formally formulated as:

Original: [Equation]
Revised: [Equation]

Original: where e i ∈ E G , e j ∈ V L ( ˆ G ) and G represents a training batch.
Revised: where e i ∈ E G , e j ∈ V L ( ˆ G ) and G represents a training batch.

Original: Note that the negative pairs of the inter-local contrastive loss also come from two directions.
Revised: Note that the negative pairs of the inter-local contrastive loss also come from two directions.

Original: Currently, we have presented the main components of the proposed LGCL that aims to help molecular graph contrastive learning free from well-designed domain knowledge and maintain the semantics.
Revised: Currently, we have presented the main components of the proposed LGCL that aims to help molecular graph contrastive learning free from well-designed domain knowledge and maintain the semantics.

Original: For unsupervised molecular graph representation learning, the final objective function of LGCL for pre-training is given by
Revised: For unsupervised molecular graph representation learning, the final objective function of LGCL for pre-training is given by

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) where L G denotes the NT-Xent loss, α and β are two hyper-parameters for loss weight controlling.
Revised: ) where L G denotes the NT-Xent loss, α and β are two hyper-parameters for loss weight controlling.

Original:  E XPERIMENT
Revised:  E XPERIMENT

Original: In this section, we are devoted to evaluating LGCL with extensive experiments 1 .
Revised: In this section, we are devoted to evaluating LGCL with extensive experiments 1 .

Original: Following the procedure of pre-training and fine-tuning, we validate the effectiveness of our approach against
Revised: Following the procedure of pre-training and fine-tuning, we validate the effectiveness of our approach against

Original: SOTA competitors for view generation.
Revised: SOTA competitors for view generation.

Original: Furthermore, we carry out analytical studies to assess each proposed component.
Revised: Furthermore, we carry out analytical studies to assess each proposed component.

Original: Unsupervised and semi-supervised learning are shown in appendix.
Revised: Unsupervised and semi-supervised learning are shown in appendix.

Original: 5. E XPERIMENTAL S ETUP
Revised: 5. E XPERIMENTAL S ETUP

Original: To be in line with the previous graph contrastive learning methods without prefabricated domain knowledge and make the comparisons fair, we follow the experimental setup under the guidance of
Revised: To be in line with the previous graph contrastive learning methods without prefabricated domain knowledge and make the comparisons fair, we follow the experimental setup under the guidance of

Original: Hu et al.
Revised: Hu et al.

Original: Pre-training dataset.
Revised: Pre-training dataset.

Original: ZINC

Revised: ZINC

Original: Sterling & Irwin (2015) dataset is adopted for LGCL pre-training.
Revised: Sterling & Irwin (2015) dataset is adopted for LGCL pre-training.

Original: In particular, a subset with two million unlabeled molecular graphs are sampled from the ZINC15.
Revised: In particular, a subset with two million unlabeled molecular graphs are sampled from the ZINC15.

Original: Pre-training details.
Revised: Pre-training details.

Original: In the graph encoder setting in Hu et al.
Revised: In the graph encoder setting in Hu et al.

Original: (2020), a Graph Isomorphism Network (GIN)
Revised: (2020), a Graph Isomorphism Network (GIN)

Original: Xu et al.
Revised: Xu et al.

Original: (2019) with five convolutional layers is adopted for message passing.
Revised: (2019) with five convolutional layers is adopted for message passing.

Original: In particular, the hidden dimension is fixed to 300 across all layers and a pooling readout function that averages graph nodes is hired for NT-Xent loss calculation with the scale parameter τ = 0 .
Revised: In particular, the hidden dimension is fixed to 300 across all layers and a pooling readout function that averages graph nodes is hired for NT-Xent loss calculation with the scale parameter τ = 0 .

Original: The hidden representations at the last layer are injected into the average pooling function.
Revised: The hidden representations at the last layer are injected into the average pooling function.

Original: An Adam optimizer
Revised: An Adam optimizer

Original: Kingma & Ba (2015) is employed to minimize the integrated losses produced by the 5-layer GIN encoder.
Revised: Kingma & Ba (2015) is employed to minimize the integrated losses produced by the 5-layer GIN encoder.

Original: The batch size is set as 256, and all training processes will run 100 epochs.
Revised: The batch size is set as 256, and all training processes will run 100 epochs.

Original: The two hyper-parameters (i.e., α and β ) for loss weight controlling are both set as 1.
Revised: The two hyper-parameters (i.e., α and β ) for loss weight controlling are both set as 1.

Original: Fine-tuning dataset.
Revised: Fine-tuning dataset.

Original: We employ the eight ubiquitous benchmarks from the MoleculeNet dataset
Revised: We employ the eight ubiquitous benchmarks from the MoleculeNet dataset

Original: Wu et al.
Revised: Wu et al.

Original: (2018) to validate LGCL as downstream experiments.
Revised: (2018) to validate LGCL as downstream experiments.

Original: These benchmarks include a variety of molecular tasks like physical chemistry, quantum mechanics, physiology, and biophysics.
Revised: These benchmarks include a variety of molecular tasks like physical chemistry, quantum mechanics, physiology, and biophysics.

Original: For dataset split, the scaffold split scheme Chen et al.
Revised: For dataset split, the scaffold split scheme Chen et al.

Original: (2012) is adopted for train/validation/test set generation.
Revised: (2012) is adopted for train/validation/test set generation.

Original: Table A.1 summarizes the basic characteristics of the datasets, such as the size, tasks and molecule statistics.
Revised: Table A.1 summarizes the basic characteristics of the datasets, such as the size, tasks and molecule statistics.

Original: Detailed descriptions can be found in Appendix A.
Revised: Detailed descriptions can be found in Appendix A.

Original: Fine-tuning details.
Revised: Fine-tuning details.

Original: For downstream tasks, a linear layer is stacked after the pre-trained graph encoders for final property prediction.
Revised: For downstream tasks, a linear layer is stacked after the pre-trained graph encoders for final property prediction.

Original: The downstream model still employs the Adam optimizer for 100 epochs fine-tuning.
Revised: The downstream model still employs the Adam optimizer for 100 epochs fine-tuning.

Original: All experiments on each dataset are performed for ten runs with different
Revised: All experiments on each dataset are performed for ten runs with different

Original: Baselines.
Revised: Baselines.

Original: In this paper, we choose the SOTA competitors that follows the experimental setup in Hu et al.
Revised: In this paper, we choose the SOTA competitors that follows the experimental setup in Hu et al.

Original: The first category is self-supervised graph learning algorithms, including EdgePred,
Revised: The first category is self-supervised graph learning algorithms, including EdgePred,

Original: AttrMsking, ContexPred Hu et al.
Revised: AttrMsking, ContexPred Hu et al.

Original: (2020), Infomax Velickovic et al.
Revised: (2020), Infomax Velickovic et al.

Original: (2019), and GraphMAE Hou et al.
Revised: (2019), and GraphMAE Hou et al.

Original: The second category are graph contrastive learning methods for view generation, such as GraphCL You et al.
Revised: The second category are graph contrastive learning methods for view generation, such as GraphCL You et al.

Original: (2020), JOAO(v2)
Revised: (2020), JOAO(v2)

Original: You et al.
Revised: You et al.

Original: LP-Info You et al.
Revised: LP-Info You et al.

Original: AutoGCL
Revised: AutoGCL

Original: Yin et al.
Revised: Yin et al.

Original: (2022), GraphMVP Liu et al.
Revised: (2022), GraphMVP Liu et al.

Original: , RGCL Li et al.
Revised: , RGCL Li et al.

Original: (2022b)
Revised: (2022b)

Original: and
Revised: and

Original: D-SLA Kim et al.
Revised: D-SLA Kim et al.

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: The results of LGCL along with SOTA competitors for molecular property prediction on eight benchmarks are shown in Table 1.
Revised: The results of LGCL along with SOTA competitors for molecular property prediction on eight benchmarks are shown in Table 1.

Original: To summarize, the proposed graph contrastive learning framework with the line graph, LGCL, obtains superior performance compared with the previous works.
Revised: To summarize, the proposed graph contrastive learning framework with the line graph, LGCL, obtains superior performance compared with the previous works.

Original: Specifically, in light of the last column for average rank, our method seizes the highest ranking position from SOTA contrastive learning methods as well as self-supervised learning methods, and a significant ranking improvement can be witnessed as opposed to the second place (D-SLA gives the
Revised: Specifically, in light of the last column for average rank, our method seizes the highest ranking position from SOTA contrastive learning methods as well as self-supervised learning methods, and a significant ranking improvement can be witnessed as opposed to the second place (D-SLA gives the

Original: A.R. 5.0).
Revised: A.R. 5.0).

Original: In particular, LGCL achieves the best performance on six out of eight benchmarks, and the best comprehensive performance is also with us (see the penultimate column).
Revised: In particular, LGCL achieves the best performance on six out of eight benchmarks, and the best comprehensive performance is also with us (see the penultimate column).

Original: Thus, we can conclude that LGCL captures the molecular semantic information well in the absence of well-designed domain knowledge, and the line graph provides an excellent contrastive view without altering the molecular semantics
Revised: Thus, we can conclude that LGCL captures the molecular semantic information well in the absence of well-designed domain knowledge, and the line graph provides an excellent contrastive view without altering the molecular semantics

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 5. A BLATION S TUDY
Revised: 5. A BLATION S TUDY

Original: Here, we delve deeper into the performance influence of each proposed component.
Revised: Here, we delve deeper into the performance influence of each proposed component.

Original: First, we analyze the performance boosting from the introduction of the line graph and edge attribute fusion without ZINC15 pre-training.
Revised: First, we analyze the performance boosting from the introduction of the line graph and edge attribute fusion without ZINC15 pre-training.

Original: As for the two local contrastive losses, we present the test results of various combinations from these parts following the transfer learning settings.
Revised: As for the two local contrastive losses, we present the test results of various combinations from these parts following the transfer learning settings.

Original: The detailed discussions are as follows.
Revised: The detailed discussions are as follows.

Original: The effect of line graph.
Revised: The effect of line graph.

Original: In Figure 3, we analyze the effect of the line graph.
Revised: In Figure 3, we analyze the effect of the line graph.

Original: In comparison with the red bar (i.e., ‘No Pre-Train’) that denotes the results from random initialization, introducing the line graph (i.e., ‘No Pre-Train w/LG’) shows an overall superior performance, which empirically suggests that the semantics underlying the edges can be better captured by the line graph.
Revised: In comparison with the red bar (i.e., ‘No Pre-Train’) that denotes the results from random initialization, introducing the line graph (i.e., ‘No Pre-Train w/LG’) shows an overall superior performance, which empirically suggests that the semantics underlying the edges can be better captured by the line graph.

Original: The effect of edge attribute fusion.
Revised: The effect of edge attribute fusion.

Original: Based on the performance boosting of the line graph, we further present the results with edge attribute fusion in Figure 3 (i.e., ‘No Pre-Train w/LG w/AF’).
Revised: Based on the performance boosting of the line graph, we further present the results with edge attribute fusion in Figure 3 (i.e., ‘No Pre-Train w/LG w/AF’).

Original: Besides the general promotion compared with the results of random initialization, the edge attribute
Revised: Besides the general promotion compared with the results of random initialization, the edge attribute

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: fusion also brings five out of eight better results in contrast to the solo line graph.
Revised: fusion also brings five out of eight better results in contrast to the solo line graph.

Original: Furthermore, following the setting of transfer learning, the performance differences between the first and fifth rows as well as the fourth and sixth rows in Table 2 also validate the effectiveness of edge attribute fusion.
Revised: Furthermore, following the setting of transfer learning, the performance differences between the first and fifth rows as well as the fourth and sixth rows in Table 2 also validate the effectiveness of edge attribute fusion.

Original: Thus, we may conclude that edge attribute fusion can alleviate information inconsistency and enhance molecular graph representation learning.
Revised: Thus, we may conclude that edge attribute fusion can alleviate information inconsistency and enhance molecular graph representation learning.

Original: The effect of intra-local contrastive loss.
Revised: The effect of intra-local contrastive loss.

Original: The test results under the supervision of the proposed losses are shown in Table 2.
Revised: The test results under the supervision of the proposed losses are shown in Table 2.

Original: To achieve a comprehensive comparison, we first give a baseline only pre-trained with the NT-Xent loss (see the first row).
Revised: To achieve a comprehensive comparison, we first give a baseline only pre-trained with the NT-Xent loss (see the first row).

Original: The effectiveness of the proposed intra-local contrastive loss is confirmed by the performance differences between the second and first rows as well as the fourth and third rows, in which the only experimental setup difference is the L IntraC .
Revised: The effectiveness of the proposed intra-local contrastive loss is confirmed by the performance differences between the second and first rows as well as the fourth and third rows, in which the only experimental setup difference is the L IntraC .

Original: Specifically, at least six out of eight better results are obtained via deploying this contrastive loss, which informs us of its effectiveness in over-smoothing addressing.
Revised: Specifically, at least six out of eight better results are obtained via deploying this contrastive loss, which informs us of its effectiveness in over-smoothing addressing.

Original: The effect of inter-local contrastive loss.
Revised: The effect of inter-local contrastive loss.

Original: Analogously, when comparing the results of the first and third rows as well as the second and fourth rows in Table 2, we can observe that six and five datasets achieve performance exaltation, respectively.
Revised: Analogously, when comparing the results of the first and third rows as well as the second and fourth rows in Table 2, we can observe that six and five datasets achieve performance exaltation, respectively.

Original: This metric promotion indicates the effectiveness of the inter-local contrastive loss in crucial structure identification.
Revised: This metric promotion indicates the effectiveness of the inter-local contrastive loss in crucial structure identification.

Original: Finally, despite several failures within these ablation studies, the last row that simultaneously adopts all proposed components performs best; thus, the proposed parts of LGCL are complementary to each other in molecular semantic exploration regardless of the intricate domain knowledge.
Revised: Finally, despite several failures within these ablation studies, the last row that simultaneously adopts all proposed components performs best; thus, the proposed parts of LGCL are complementary to each other in molecular semantic exploration regardless of the intricate domain knowledge.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this work, we try to figure out a decent view for molecular graph contrastive learning that can maintain the integrity of molecular semantic information and is friendly to researchers without profound domain knowledge.
Revised: In this work, we try to figure out a decent view for molecular graph contrastive learning that can maintain the integrity of molecular semantic information and is friendly to researchers without profound domain knowledge.

Original: Inspired by the line graph, we propose a method, called LGCL, to tackle our expectations.
Revised: Inspired by the line graph, we propose a method, called LGCL, to tackle our expectations.

Original: Due to the different pace of message passing in the original graph and the corresponding line graph, we further present three crucial components to address the concomitant issues and enhance molecular graph representation learning.
Revised: Due to the different pace of message passing in the original graph and the corresponding line graph, we further present three crucial components to address the concomitant issues and enhance molecular graph representation learning.

Original: Under the setting of transfer learning, we empirically present the superior performance of LGCL over SOTA works.
Revised: Under the setting of transfer learning, we empirically present the superior performance of LGCL over SOTA works.

Original: Look in The Mirror: Molecular Graph Contrastive Learning with Line Graph A D ETAILS OF M OLECULAR D ATASETS
Revised: Look in The Mirror: Molecular Graph Contrastive Learning with Line Graph A D ETAILS OF M OLECULAR D ATASETS

Original: As discussed in Section 4.
Revised: As discussed in Section 4.

Original: [Table]
Revised: [Table]

Original: a vertex with e edges in G will produce e × ( e − 1) / 2 edges in L ( G ) , which could lead to severe runtime complexity when the original graphs are dense.
Revised: a vertex with e edges in G will produce e × ( e − 1) / 2 edges in L ( G ) , which could lead to severe runtime complexity when the original graphs are dense.

Original: Therefore, our method only suits sparse graphs.
Revised: Therefore, our method only suits sparse graphs.

Original: As for molecules adopted in this work, we can see from
Revised: As for molecules adopted in this work, we can see from

Original: Table A.1 that the computation of the line graph encoder is 1.5 times that of the original graph encoder according to the average degrees in the transformed line graphs.
Revised: Table A.1 that the computation of the line graph encoder is 1.5 times that of the original graph encoder according to the average degrees in the transformed line graphs.

Original: However, as for the realistic time required for model pre-training, detailed comparisons are shown in Appendix C.2.
Revised: However, as for the realistic time required for model pre-training, detailed comparisons are shown in Appendix C.2.

Original: Input graph representation.
Revised: Input graph representation.

Original: For simplicity, we use a minimal set of node and bond features that unambiguously describe the two-dimensional structure of molecules.
Revised: For simplicity, we use a minimal set of node and bond features that unambiguously describe the two-dimensional structure of molecules.

Original: We use RDKit Landrum (2013) to obtain these features.
Revised: We use RDKit Landrum (2013) to obtain these features.

Original: • Node features: – Atom number: [1, 118] – Chirality tag: { unspecified, tetrahedral cw, tetrahedral ccw, other } • Edge features: – Bond type: { single, double, triple, aromatic } – Bond direction: { –, endupright, enddownright }
Revised: • Node features: – Atom number: [1, 118] – Chirality tag: { unspecified, tetrahedral cw, tetrahedral ccw, other } • Edge features: – Bond type: { single, double, triple, aromatic } – Bond direction: { –, endupright, enddownright }

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: • BBBP Martins et al.
Revised: • BBBP Martins et al.

Original: Blood-brain barrier penetration (membrane permeability), involves records of whether a compound carries the permeability property of penetrating the blood-brain barrier.
Revised: Blood-brain barrier penetration (membrane permeability), involves records of whether a compound carries the permeability property of penetrating the blood-brain barrier.

Original: • Tox21 Tox (2014).
Revised: • Tox21 Tox (2014).

Original: Toxicity data on 12 biological targets, which has been used in theTox21 Data Challenge and includes nuclear receptors and stress response pathways.
Revised: Toxicity data on 12 biological targets, which has been used in theTox21 Data Challenge and includes nuclear receptors and stress response pathways.

Original: • ToxCast Richard et al.
Revised: • ToxCast Richard et al.

Original: Toxicology measurements based on over 600 in vitro highthroughput screenings.
Revised: Toxicology measurements based on over 600 in vitro highthroughput screenings.

Original: • SIDER Kuhn et al.
Revised: • SIDER Kuhn et al.

Original: Database of marketed drugs and adverse drug reactions (ADR), grouped into 27 system organ classes and also known as the Side Effect Resource.
Revised: Database of marketed drugs and adverse drug reactions (ADR), grouped into 27 system organ classes and also known as the Side Effect Resource.

Original: • ClinTox Novick et al.
Revised: • ClinTox Novick et al.

Original: ; Gayvert et al.
Revised: ; Gayvert et al.

Original: Qualitative data classifying drugs approved by the FDA and those that have failed clinical trials for toxicity reasons.
Revised: Qualitative data classifying drugs approved by the FDA and those that have failed clinical trials for toxicity reasons.

Original: • MUV Gardiner et al.
Revised: • MUV Gardiner et al.

Original: Subset of PubChem BioAssay by applying a refined nearest neighbor analysis, designed for validation of virtual screening techniques.
Revised: Subset of PubChem BioAssay by applying a refined nearest neighbor analysis, designed for validation of virtual screening techniques.

Original: • HIV HIV.
Revised: • HIV HIV.

Original: Experimentally measured abilities to inhibit HIV replication.
Revised: Experimentally measured abilities to inhibit HIV replication.

Original: • BACE Subramanian et al.
Revised: • BACE Subramanian et al.

Original: Qualitative binding results for a set of inhibitors of human β -secretase 1.
Revised: Qualitative binding results for a set of inhibitors of human β -secretase 1.

Original: Dataset splitting.
Revised: Dataset splitting.

Original: For molecular prediction tasks, following Ramsundar et al.
Revised: For molecular prediction tasks, following Ramsundar et al.

Original: (2019), we cluster molecules by scaffold (molecular graph substructure) Bemis & Murcko (1996), and recombine the clusters by placing the most common scaffolds in the training set, producing validation and test sets that contain structurally different molecules.
Revised: (2019), we cluster molecules by scaffold (molecular graph substructure) Bemis & Murcko (1996), and recombine the clusters by placing the most common scaffolds in the training set, producing validation and test sets that contain structurally different molecules.

Original: Prior work has shown that this scaffold split provides a more realistic estimate of model performance in prospective evaluation compared to random split
Revised: Prior work has shown that this scaffold split provides a more realistic estimate of model performance in prospective evaluation compared to random split

Original: Chen et al.
Revised: Chen et al.

Original: (2012); Sheridan (2013).
Revised: (2012); Sheridan (2013).

Original: The split for train/validation/test sets is 80%:10%:10%.
Revised: The split for train/validation/test sets is 80%:10%:10%.

Original: B M OLECULAR P ROPERTY P REDICTION VIA U NSUPERVISED L EARNING AND S EMI -S UPERVISED L EARNING
Revised: B M OLECULAR P ROPERTY P REDICTION VIA U NSUPERVISED L EARNING AND S EMI -S UPERVISED L EARNING

Original: As discussed above, LGCL only suits sparse graphs to avoid too bad runtime complexity.
Revised: As discussed above, LGCL only suits sparse graphs to avoid too bad runtime complexity.

Original: As shown in Table A.2, the capacities of all social network datasets and dense bioinformatics dataset (i.e.,
Revised: As shown in Table A.2, the capacities of all social network datasets and dense bioinformatics dataset (i.e.,

Original: DD) heavily increase after the line graph transformation, which leads to unaffordable computation consumption in graph representation learning.
Revised: DD) heavily increase after the line graph transformation, which leads to unaffordable computation consumption in graph representation learning.

Original: Therefore, we only employ sparse bioinformatics datasets for unsupervised and semi-supervised learning.
Revised: Therefore, we only employ sparse bioinformatics datasets for unsupervised and semi-supervised learning.

Original: Experiment details are elaborated below.
Revised: Experiment details are elaborated below.

Original: [Table]
Revised: [Table]

Original: B. U NSUPERVISED L EARNING
Revised: B. U NSUPERVISED L EARNING

Original: Datasets.
Revised: Datasets.

Original: Three sparse bioinformatics datasets are adopted from TUDataset Morris et al.
Revised: Three sparse bioinformatics datasets are adopted from TUDataset Morris et al.

Original: (2020) for unsupervised learning, including NCI1 and MUTAG, and PROTEINS.
Revised: (2020) for unsupervised learning, including NCI1 and MUTAG, and PROTEINS.

Original: Table A.2 summarizes the characteristics of the three employed datasets.
Revised: Table A.2 summarizes the characteristics of the three employed datasets.

Original: • NCI1 is a dataset made publicly available by the National Cancer Institute (NCI) and is a subset of balanced datasets containing chemical compounds screened for their ability to suppress or inhibit the growth of a panel of human tumor cell lines; this dataset possesses 37 discrete labels.
Revised: • NCI1 is a dataset made publicly available by the National Cancer Institute (NCI) and is a subset of balanced datasets containing chemical compounds screened for their ability to suppress or inhibit the growth of a panel of human tumor cell lines; this dataset possesses 37 discrete labels.

Original: • MUTAG has seven kinds of graphs that are derived from 188 mutagenic aromatic and heteroaromatic nitro compounds.
Revised: • MUTAG has seven kinds of graphs that are derived from 188 mutagenic aromatic and heteroaromatic nitro compounds.

Original: • PROTEINS is a dataset where the nodes are secondary structure elements (SSEs), and there is an edge between two nodes if they are neighbors in the given amino acid sequence or in 3D space.
Revised: • PROTEINS is a dataset where the nodes are secondary structure elements (SSEs), and there is an edge between two nodes if they are neighbors in the given amino acid sequence or in 3D space.

Original: The dataset has 3 discrete labels, representing helixes, sheets or turns.
Revised: The dataset has 3 discrete labels, representing helixes, sheets or turns.

Original: Configuration.
Revised: Configuration.

Original: To keep in line with GraphCL You et al.
Revised: To keep in line with GraphCL You et al.

Original: (2020), the same GNN architectures are employed with their original hyper-parameters under individual experiment settings.
Revised: (2020), the same GNN architectures are employed with their original hyper-parameters under individual experiment settings.

Original: Specifically,
Revised: Specifically,

Original: GIN Xu et al.
Revised: GIN Xu et al.

Original: (2019) with 3 layers is set up in unsupervised representation learning.
Revised: (2019) with 3 layers is set up in unsupervised representation learning.

Original: The encoder hidden dimensions are fixed for all layers to keep in line with GraphCL under individual experiment setting.
Revised: The encoder hidden dimensions are fixed for all layers to keep in line with GraphCL under individual experiment setting.

Original: Models are trained 20 epochs and tested every 10 epochs.
Revised: Models are trained 20 epochs and tested every 10 epochs.

Original: Hidden dimension is 32, and batch size is ∈ { 32 , 128 } .
Revised: Hidden dimension is 32, and batch size is ∈ { 32 , 128 } .

Original: An Adam optimizer Kingma & Ba (2015) is employed to minimize the contrastive lose and learning rate is ∈ { 0 . 01 , 0 .
Revised: An Adam optimizer Kingma & Ba (2015) is employed to minimize the contrastive lose and learning rate is ∈ { 0 . 01 , 0 .

Original: Learning protocol.
Revised: Learning protocol.

Original: Following the learning setting in SOTA works, the corresponding learning protocols are adopted for a fair comparison.
Revised: Following the learning setting in SOTA works, the corresponding learning protocols are adopted for a fair comparison.

Original: In unsupervised representation learning Sun et al.
Revised: In unsupervised representation learning Sun et al.

Original: (2020), all data is used for model pre-training and the learned graph embeddings are then fed into a non-linear SVM classifier to perform classification.
Revised: (2020), all data is used for model pre-training and the learned graph embeddings are then fed into a non-linear SVM classifier to perform classification.

Original: Experiments are performed for 5 times each of which corresponds to a 10-fold evaluation as Sun et al.
Revised: Experiments are performed for 5 times each of which corresponds to a 10-fold evaluation as Sun et al.

Original: (2020), with mean and standard deviation of accuracies (%) reported.
Revised: (2020), with mean and standard deviation of accuracies (%) reported.

Original: Compared methods.
Revised: Compared methods.

Original: We adopt nine baselines that are composed of three categories.
Revised: We adopt nine baselines that are composed of three categories.

Original: The published hyper-parameters of these methods are adopted.
Revised: The published hyper-parameters of these methods are adopted.

Original: The first set is three SOTA kernel-based methods that include GL Shervashidze et al.
Revised: The first set is three SOTA kernel-based methods that include GL Shervashidze et al.

Original: (2009), WL Shervashidze et al.
Revised: (2009), WL Shervashidze et al.

Original: (2011) and DGK Yanardag & Vishwanathan (2015).
Revised: (2011) and DGK Yanardag & Vishwanathan (2015).

Original: The second set is four heuristic self-supervised methods, including node2vec Grover & Leskovec (2016), sub2vec Adhikari et al.
Revised: The second set is four heuristic self-supervised methods, including node2vec Grover & Leskovec (2016), sub2vec Adhikari et al.

Original: (2018), graph2vec Annamalai Narayanan & Jaiswal (2017), and InfoGraph Sun et al.
Revised: (2018), graph2vec Annamalai Narayanan & Jaiswal (2017), and InfoGraph Sun et al.

Original: The final compared methods are GraphCL
Revised: The final compared methods are GraphCL

Original: You et al.
Revised: You et al.

Original: (2020), JOAO(v2)
Revised: (2020), JOAO(v2)

Original: You et al.
Revised: You et al.

Original: , AD-GCL Suresh et al.
Revised: , AD-GCL Suresh et al.

Original: AutoGCL
Revised: AutoGCL

Original: Yin et al.
Revised: Yin et al.

Original: and RGCL Li et al.
Revised: and RGCL Li et al.

Original: (2022b).
Revised: (2022b).

Original: Results.
Revised: Results.

Original: The results of LGCL along with SOTA competitors on three benchmarks are shown in
Revised: The results of LGCL along with SOTA competitors on three benchmarks are shown in

Original: Table A.3.
Revised: Table A.3.

Original: To summarize, the proposed graph contrastive learning framework with the line graph,
Revised: To summarize, the proposed graph contrastive learning framework with the line graph,

Original: LGCL, obtains superior performance compared with the previous works.
Revised: LGCL, obtains superior performance compared with the previous works.

Original: In particular, except for
Revised: In particular, except for

Original: NCI1, LGCL achieves the best performance on two out of three benchmarks.
Revised: NCI1, LGCL achieves the best performance on two out of three benchmarks.

Original: Thus, we can conclude that LGCL captures the molecular semantic information well in the setting of unsupervised learning.
Revised: Thus, we can conclude that LGCL captures the molecular semantic information well in the setting of unsupervised learning.

Original: [Table]
Revised: [Table]

Original: S UPERVISED L EARNING
Revised: S UPERVISED L EARNING

Original: Datasets.
Revised: Datasets.

Original: Two sparse bioinformatics datasets are adopted from TUDataset Morris et al.
Revised: Two sparse bioinformatics datasets are adopted from TUDataset Morris et al.

Original: (2020) for semi-supervised learning, including NCI1 and PROTEINS.
Revised: (2020) for semi-supervised learning, including NCI1 and PROTEINS.

Original: Configuration.
Revised: Configuration.

Original: ResGCN with 128 hidden units and 5 layers is set up in semi-supervised learning.
Revised: ResGCN with 128 hidden units and 5 layers is set up in semi-supervised learning.

Original: For all datasets we perform experiments with 10% label rate for 5 times, each of which corresponds to a 10-fold evaluation as You et al.
Revised: For all datasets we perform experiments with 10% label rate for 5 times, each of which corresponds to a 10-fold evaluation as You et al.

Original: (2020), with mean and standard deviation of accuracies (%) reported.
Revised: (2020), with mean and standard deviation of accuracies (%) reported.

Original: For pre-training, learning rate is tuned in { 0 . 01 , 0 .
Revised: For pre-training, learning rate is tuned in { 0 . 01 , 0 .

Original: 001 , 0 . 0001 } and epoch number in { 20 , 40 , 60 , 80 , 100 } where grid search is performed.
Revised: 001 , 0 . 0001 } and epoch number in { 20 , 40 , 60 , 80 , 100 } where grid search is performed.

Original: For fine-tuning, we following the default setting in
Revised: For fine-tuning, we following the default setting in

Original: , that learning
Revised: , that learning

Original: , hidden dimension and the pre-trained models are
Revised: , hidden dimension and the pre-trained models are

Original: Learning protocols.
Revised: Learning protocols.

Original: Following the learning setting in SOTA works, the corresponding learning protocols are adopted for a fair comparison.
Revised: Following the learning setting in SOTA works, the corresponding learning protocols are adopted for a fair comparison.

Original: In semi-supervised learning You et al.
Revised: In semi-supervised learning You et al.

Original: (2020), there exist two learning settings.
Revised: (2020), there exist two learning settings.

Original: For datasets with a public training/validation/test split, pre-training is performed only on training dataset, finetuning is conducted with 10% of the training data, and final evaluation results are from the validation/test sets.
Revised: For datasets with a public training/validation/test split, pre-training is performed only on training dataset, finetuning is conducted with 10% of the training data, and final evaluation results are from the validation/test sets.

Original: For datasets without such splits, all samples are employed for pre-training while finetuning and evaluation are performed over 10 folds.
Revised: For datasets without such splits, all samples are employed for pre-training while finetuning and evaluation are performed over 10 folds.

Original: Compared methods.
Revised: Compared methods.

Original: Under the setting of semi-supervised learning, eight baselines are adopted, including (1) the naive GCN without pre-training You et al.
Revised: Under the setting of semi-supervised learning, eight baselines are adopted, including (1) the naive GCN without pre-training You et al.

Original: (2020), which is directly trained with 10% labeled data from random initialization; (2) GAE Kipf & Welling (2016), a predictive method by edge-based reconstruction in the pre-training phase; (3) Infomax Velickovic et al.
Revised: (2020), which is directly trained with 10% labeled data from random initialization; (2) GAE Kipf & Welling (2016), a predictive method by edge-based reconstruction in the pre-training phase; (3) Infomax Velickovic et al.

Original: (2019), a node embedding method with global-local representation consistency; (4) ContextPred Hu et al.
Revised: (2019), a node embedding method with global-local representation consistency; (4) ContextPred Hu et al.

Original: (2020), a method via sub-structure information preserving; (5)
Revised: (2020), a method via sub-structure information preserving; (5)

Original: GraphCL You et al.
Revised: GraphCL You et al.

Original: (2020), the first graph contrastive learning method with data augmentations.
Revised: (2020), the first graph contrastive learning method with data augmentations.

Original: (6) JOAO(v2)
Revised: (6) JOAO(v2)

Original: You et al.
Revised: You et al.

Original: (2021), an optimization framework to automatically select data augmentations; (7) AD-GCL Suresh et al.
Revised: (2021), an optimization framework to automatically select data augmentations; (7) AD-GCL Suresh et al.

Original: (2021), a framework aim to exclude redundant information during the training by optimizing adversarial graph augmentation strategies; (8) AutoGCL Yin et al.
Revised: (2021), a framework aim to exclude redundant information during the training by optimizing adversarial graph augmentation strategies; (8) AutoGCL Yin et al.

Original: (2022), a model with learnable graph view generators orchestrated by an auto augmentation strategy.
Revised: (2022), a model with learnable graph view generators orchestrated by an auto augmentation strategy.

Original: Results.
Revised: Results.

Original: The results of LGCL along with SOTA competitors on the two benchmarks are shown in Table A.4, in which LGCL suppresses the SOTA view generation works on the two employed datasets.
Revised: The results of LGCL along with SOTA competitors on the two benchmarks are shown in Table A.4, in which LGCL suppresses the SOTA view generation works on the two employed datasets.

Original: Thus, we can conclude that LGCL captures the molecular semantic information well in the setting of semi-supervised learning.
Revised: Thus, we can conclude that LGCL captures the molecular semantic information well in the setting of semi-supervised learning.

Original: [Table]
Revised: [Table]

Original: C F URTHER A BLATION S TUDY C. H YPER - PARAMETER S ENSITIVITY
Revised: C F URTHER A BLATION S TUDY C. H YPER - PARAMETER S ENSITIVITY

Original: In the design of LGCL, besides the general hyper-parameters (i.e., learning rate, batch size, dropout ratio, etc.), we introduce two hyper-parameters, α and β , for loss balance in pre-training stage.
Revised: In the design of LGCL, besides the general hyper-parameters (i.e., learning rate, batch size, dropout ratio, etc.), we introduce two hyper-parameters, α and β , for loss balance in pre-training stage.

Original: To clearly show the essential effectiveness of the two losses rather than the two hyper-parameters, we fix α and β to 1 in the main text.
Revised: To clearly show the essential effectiveness of the two losses rather than the two hyper-parameters, we fix α and β to 1 in the main text.

Original: The other hyper-parameters in pre-training phase are also fixed and consistent with Hu et al.
Revised: The other hyper-parameters in pre-training phase are also fixed and consistent with Hu et al.

Original: To further inspect the hyper-parameter sensitivity of LGCL, we tuned the candidates of α and β in the range of [0.01, 0.1, 1, 10, 100], respectively.
Revised: To further inspect the hyper-parameter sensitivity of LGCL, we tuned the candidates of α and β in the range of [0.01, 0.1, 1, 10, 100], respectively.

Original: In the tunning of α , we fix the β to 1 and vice versa.
Revised: In the tunning of α , we fix the β to 1 and vice versa.

Original: In the fine-tuning stage, the learning is fixed to 0.001; the batch size is fixed to 32; the dropout ratio is fixed to 0.
Revised: In the fine-tuning stage, the learning is fixed to 0.001; the batch size is fixed to 32; the dropout ratio is fixed to 0.

Original: [Figure]
Revised: [Figure]

Original: experiments on each dataset are performed for ten runs with different seeds, and the results are the averaged ROC-AUC scores (%) ± standard deviations.
Revised: experiments on each dataset are performed for ten runs with different seeds, and the results are the averaged ROC-AUC scores (%) ± standard deviations.

Original: The average ROC-AUC scores of SIDER, ClinTox, and BACE are shown in Figure A.1.
Revised: The average ROC-AUC scores of downstream tasks are shown in Figure A.1.

Original: As can be seen, different downstream tasks prefer different loss controls.
Revised: As can be seen, different downstream tasks prefer different loss controls.

Original: Specifically, ClinTox and BACE prefer small α but large β , while SIDER would like large α and is insensitive to β , which suggests that ClinTox and BACE suffer more from the over-smoothing.
Revised: Specifically, ClinTox and BACE prefer small α but large β , while SIDER would like large α and is insensitive to β , which suggests that ClinTox and BACE suffer more from the over-smoothing.

Original: It is worth noting that more superior results can be obtained from tuning these two hyper-parameters, such as the 85.88% of ClinTox with α = 1 and β = 10 compared to 77.59% in the main text with α = 1 and β = 1 , which implies the huge potential of LGCL with the tuning of the two hyper-parameters.
Revised: It is worth noting that more superior results can be obtained from tuning these two hyper-parameters, such as the 85.88% of ClinTox with α = 1 and β = 10 compared to 77.59% in the main text with α = 1 and β = 1 , which implies the huge potential of LGCL with the tuning of the two hyper-parameters.

Original: C. E FFICIENCY
Revised: C. E FFICIENCY

Original: In this paper, to address the issues of molecular semantics alteration and generalization capability in molecular contrastive learning, we introduce the line graph and further propose edge attribute fusion, intra-local contrastive loss, and inter-local contrastive loss to enhance contrastive learning.
Revised: In this paper, to address the issues of molecular semantics alteration and generalization capability in molecular contrastive learning, we introduce the line graph and further propose edge attribute fusion, intra-local contrastive loss, and inter-local contrastive loss to enhance contrastive learning.

Original: In
Revised: In

Original: Section 5, we have shown the superior performance of LGCL.
Revised: Section 5, we have shown the superior performance of LGCL.

Original: Here, we further present the efficiency of LGCL by comparing the pre-training time on 2 million molecular graphs of ZINC15 with the baselines.
Revised: Here, we further present the efficiency of LGCL by comparing the pre-training time on 2 million molecular graphs of ZINC15 with the baselines.

Original: In addition, the actual runtime of LGCL through introducing each item is also given, including LGCL with only line graph (i.e., LGCL w/o AF L IntraC L InterC ); LGCL with line graph and edge attribute fusion (i.e., LGCL w/o L IntraC L InterC ); LGCL with line graph, edge attribute fusion and intra-local contrastive loss (i.e., LGCL w/o L InterC ).
Revised: In addition, the actual runtime of LGCL through introducing each item is also given, including LGCL with only line graph (i.e., LGCL w/o AF L IntraC L InterC ); LGCL with line graph and edge attribute fusion (i.e., LGCL w/o L IntraC L InterC ); LGCL with line graph, edge attribute fusion and intra-local contrastive loss (i.e., LGCL w/o L InterC ).

Original: The setting of pre-training time comparison is consistent with the main text Hu et al.
Revised: The setting of pre-training time comparison is consistent with the main text Hu et al.

Original: All baselines and LGCL are rerun in our platform (Tesla V100 GPU and Intel(R)
Revised: All baselines and LGCL are rerun in our platform (Tesla V100 GPU and Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Silver 4214 CPU).
Revised: Silver 4214 CPU).

Original: Note that, we set the num workers in dataloader to default for fair comparison.
Revised: Note that, we set the num workers in dataloader to default for fair comparison.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: Figure A.2 shows the pre-training time required for 2 million molecular graphs from ZINC15 pretraining with 100 epochs.
Revised: Figure A.2 shows the pre-training time required for 2 million molecular graphs from ZINC15 pretraining with 100 epochs.

Original: As shown in Figure A.2a, because the contrastive views of LGCL are static, the time required by LGCL is much less than the time needed by baselines except AutoGCL, which reveals that LGCL has not only superior performance but also excellent efficiency.
Revised: As shown in Figure A.2a, because the contrastive views of LGCL are static, the time required by LGCL is much less than the time needed by baselines except AutoGCL, which reveals that LGCL has not only superior performance but also excellent efficiency.

Original: In partic- ular, LP-Info requires almost 16 hours for one epoch pre-training, that is about 1,600 hours forepochs, thus we do not report its pre-training time in Figure A.2a.
Revised: In partic- ular, LP-Info requires almost 16 hours for one epoch pre-training, that is about 1,600 hours forepochs, thus we do not report its pre-training time in Figure A.2a.

Original: Furthermore, as shown in Figure A.2b, comparing to the model only with line graph (i.e., LGCL w/o
Revised: Furthermore, as shown in Figure A.2b, comparing to the model only with line graph (i.e., LGCL w/o

Original: AF L IntraC L InterC ), the additional time from the introduction of edge attribute fusion is nearly negligible.
Revised: AF L IntraC L InterC ), the additional time from the introduction of edge attribute fusion is nearly negligible.

Original: The biggest time consumption gap comes from the proposal of two local contrastive losses, but they still only occupy 16.26% of the total time, which is a small price in contrast to the accompanying performance boosting.
Revised: The biggest time consumption gap comes from the proposal of two local contrastive losses, but they still only occupy 16.26% of the total time, which is a small price in contrast to the accompanying performance boosting.

Original: C. V ISUALIZATION OF O VER - SMOOTHING A LLEVIATING
Revised: C. V ISUALIZATION OF O VER - SMOOTHING A LLEVIATING

Original: [Figure]
Revised: [Figure]

Original: intra we calculate the similarities among nodes within the same graph from the pre-training dataset via the model after 100-epoch pre-training.
Revised: intra we calculate the similarities among nodes within the same graph from the pre-training dataset via the model after 100-epoch pre-training.

Original: As shown in
Revised: As shown in

Original: Figure A.3, we give the node similarity distribution of GraphCL, LGCL, and LGCL without the two local contrastive losses.
Revised: Figure A.3, we give the node similarity distribution of GraphCL, LGCL, and LGCL without the two local contrastive losses.

Original: First, based on the distribution shown in Figure A.3a, we can see that the introduction of line graph leads to a right shift of node similarity distribution, that is, over-smoothing.
Revised: First, based on the distribution shown in Figure A.3a, we can see that the introduction of line graph leads to a right shift of node similarity distribution, that is, over-smoothing.

Original: Then, the left shift of node similarity distribution in Figure A.3b suggests that the proposed loss could alleviate the over-smoothing.
Revised: Then, the left shift of node similarity distribution in Figure A.3b suggests that the proposed loss could alleviate the over-smoothing.

Original: C. V ISUALIZATION OF I NFORMATION I NCONSISTENCY A LLEVIATING
Revised: C. V ISUALIZATION OF I NFORMATION I NCONSISTENCY A LLEVIATING

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original: In the ablation study, we empirically validate the effectiveness of edge attribute fusion in information inconsistency alleviating.
Revised: In the ablation study, we empirically validate the effectiveness of edge attribute fusion in information inconsistency alleviating.

Original: Here, visualization case studies are  presented tofurther directly show howthe edge attribute fusion works.
Revised: Here, visualization case studies are further presented to show how edge attribute fusion works.

Original: First, we calculate the view similarity of positive pairs from the pretraining dataset via the model after 100-epoch pre-training.
Revised: First, we calculate the view similarity of positive pairs from the pre-training dataset via the model after 100-epoch pre-training.

Original: As shown in Figure A.4a, we present the view similarity distribution of LGCL with and without the edge attribute fusion 2 .
Revised: As shown in Figure A.4a, we present the view similarity distribution of LGCL with and without the edge attribute fusion 2 .

Original: We can see that the introduction of edge attribute fusion significantly increases the view similarity between positive pairs.
Revised: We can see that the introduction of edge attribute fusion significantly increases the view similarity between positive pairs.

Original: Second, we inspect the convergence of NT-Xent loss of LGCL with and without edge attribute fusion.
Revised: Second, we inspect the convergence of NT-Xent loss of LGCL with and without edge attribute fusion.

Original: As shown in Figure A.4b, LGCL equipped with edge attribute fusion converges faster and has a lower loss, which suggests the higher alignment of the view embeddings.
Revised: As shown in Figure A.4b, LGCL equipped with edge attribute fusion converges faster and has a lower loss, which suggests the higher alignment of the view embeddings.

Original: In summary, edge attribute fusion can address the information inconsistency accompanying the line graph.
Revised: In summary, edge attribute fusion can address the information inconsistency accompanying the line graph.

Original: D O THER B ASELINES
Revised: D O THER B ASELINES

Original: Here, we further compare LGCL with other self-supervised molecular representation learning:
Revised: Here, we further compare LGCL with other self-supervised molecular representation learning:

Original: • GROVER Rong et al.
Revised: • GROVER Rong et al.

Original: (2020) predicts the contextual properties based on atom embeddings to encode contextual information into node embeddings.
Revised: (2020) predicts the contextual properties based on atom embeddings to encode contextual information into node embeddings.

Original: • MGSSL Zhang et al.
Revised: • MGSSL Zhang et al.

Original: (2021) aims to capture the rich information insubgraphs or graph motifs.
Revised: (2021) aims to capture the rich information in graph motifs.

Original: • 3D-Infomax St ¨ ark et al.
Revised: • 3D-Infomax St ¨ ark et al.

Original: (2022) aims to reason about the geometry of molecules given only their 2D molecular graphs by pre-training model with existing 3D molecular datasets.
Revised: (2022) aims to reason about the geometry of molecules given only their 2D molecular graphs by pre-training model with existing 3D molecular datasets.

Original: The results are shown in Table A.5.
Revised: The results are shown in Table A.5.

Original: As can be seen, LGCL suppresses GROVER on all datasets,
Revised: As can be seen, LGCL suppresses GROVER on all datasets,

Original: MGSSL on six out of eight datasets, and 3D-Infomax on six out of seven datasets.
Revised: MGSSL on six out of eight datasets, and 3D-Infomax on six out of seven datasets.

Original: Moreover, LGCL also achieves the highest average results among the these baselines.
Revised: Moreover, LGCL also achieves the highest average results among the these baselines.

Original: datasets.

Original: A.R. denotes the average rank.
Revised: R. denotes the average rank.

Original: - indicates the data missing in the such works. 
Revised: - indicates the data missing in the such works. [Table]

Revised: E T HEORETICAL U NDERSTANDING OF LGCL

Revised: Besides the superior performance of LGCL shown in the main text for molecular property prediction, here, we further present a theoretical understanding of how LGCL obtains better performance

Original: [Table]
Revised: [Table]

Revised: Definition E.1.

Revised: (Graph Quotient Space).

Revised: Define the equivalence ∼ = between two graphs G 1 ∼ =

Revised: Gif G 1 , G 2 cannot be distinguished by the 1-WL test.

Revised: Define the quotient space G = G / ∼ = .

Revised: So every element in the quotient space, i.e., G ∈ G , is a representative graph from a family of graphs that cannot be distinguished by the 1-WL test.

Revised: Note that our definition also allows attributed graphs.

Revised: Theorem E.2.

Revised: Suppose G is a countable space and thus G ′ is a countable space.

Revised: Because G and G ′ are countable, P G and P G ′ are defined over countable sets and thus discrete distribution.

Revised: Later we may call a function z ( · ) can distinguish two graphs G 1 , G 2 if z ( G 1 ) ̸

Revised: = z ( G 2 ) .

Revised: Moreover, for notational simplicity, we consider the following definition.

Revised: Suppose the encoder f is implemented by a GNN.

Revised: The optimal encoder f ∗ is the best model which GNN can find.

Revised: Because f ∗ is as powerful as the 1-WL test.

Revised: Then, for any two graphs G 1 , G 2 ∈ G , G 1 ∼ = G 2 , f ∗ ( G 1 ) =

Revised: f ∗ ( G 2 ) .

Revised: We may define a mapping over G ′ , also denoted by f ∗ which simply satisfies f ∗ ( G ′ ) : ≜ f ∗ ( G ) , where G ∼ = G ′ , and G ∈ G and G ′ ∈ G ′ .

Revised: Suppose t ( · ) is the data augmentation function and L ( · ) is the line graph transformation function.

Revised: We have

Revised: I ( L ( G ); G ) ≥ I ( t ( G ′ ); G ′ ); 2.

Revised: I ( L ( G ); Y ) ≥ I ( t ( G ); Y ) .

Revised: The statement 1 in Theorem E.2 indicates that the retained information after line graph transformation from given graphs is more than contrastive views with data augmentation.

Revised: The statement 2 in Theorem E.2 suggests that the essential information underlying the line graph for target prediction is more than contrastive views with data augmentation.

Revised: Proof.

Revised: Given G , G ⇒ L ( G ) is an injective deterministic mapping.

Revised: Therefore, for any random variable Q ,

Revised: [Equation]

Revised: Of course, we may set Q = G .

Revised: So,

Revised: [Equation]

Revised: Then, we have

Revised: [Equation]

Revised: where ( a ) is because the data processing inequality Cover (1999).

Revised: Moreover, because f ∗ could be as powerful as the 1-WL test and is injective in G ′ .

Revised: Meanwhile, as stated in the Whitney graph isomorphism theorem Whitney (1932), the isomorphism of two line graphs is judged to be consistent with the corresponding two original graphs, thus we have

Revised: [Equation]

Revised: Here, the second equality is because the transformation of line graph will not change the isomorphic relationship between two graphs G ′ and G , meanwhile f ∗ ( G ′ )

Revised: =

Revised: f ∗ ( G ) .

Revised: Therefore, we achieve the statement 1:

Revised: [Equation]

Revised: Again, because by definition f ∗ = argmax f I

Revised: [Equation]

Revised: f ( G ); G ) , f ∗ must be injective.

Revised: Given G ∗ , G ∗ ⇒ f ∗ ( G ∗ ) is an injective deterministic mapping.

Revised: Of course, we may set Q = Y .

Revised: So,

Revised: [Equation]

Revised: Because G ⇒ L ( G ) is an injective deterministic mapping.

Revised: [Equation]

Revised: Further because of the data processing inequality Cover (1999),

Revised: [Equation]

Revised: Combining above equations, we have the statement 2:

Revised: [Equation]

Revised: which concludes the proof of the essential information of line graph.