Paper: UMMAN: UNSUPERVISED MULTI-GRAPH MERGE ADVERSARIAL NETWORK FOR DISEASE PREDICTION BASED ON INTESTINAL FLORA (ICLR-2024)

Reviewer comment (Novelty):
The technique contribution of this work is not very high as it mainly uses the GNN for disease network.

Author response:
“The technical contribution of this work is not very high as it mainly uses the GNN for disease network.” It may not sound very correct, as Amazon chief scientist Li Mu said: “A good paper does not have to appear all the time before. This is a module that no one has ever seen, but can you think of such a method?" First of all, as far as we know, previous GNN methods have never used multiplex relationships to construct graphs. At the same time, considering the particularity of this field, that is, the correlation of intestinal microorganisms is complicated and difficult to obtain, even multi-graphs cannot guarantee the accuracy of extraction. property, so we built Shuffled-Graph for adversarial training. At the same time, we supplemented ablation studies and tried it on Cirrhosis and T2D data sets. We found that if this method is not used, accuracy and auc metrics will drop by 2 ~6 percentage points, specific experimental data will be supplemented in the paper.

Reviewer comment (Novelty):
The technique contribution of this work is not very high as it mainly uses the GNN for disease network.

Author response:
“The technical contribution of this work is not very high as it mainly uses the GNN for disease network.” It may not sound very correct, as Amazon chief scientist Li Mu said: “A good paper does not have to appear all the time before. This is a module that no one has ever seen, but can you think of such a method?" First of all, as far as we know, previous GNN methods have never used multiplex relationships to construct graphs. At the same time, considering the particularity of this field, that is, the correlation of intestinal microorganisms is complicated and difficult to obtain, even multi-graphs cannot guarantee the accuracy of extraction. property, so we built Shuffled-Graph for adversarial training. At the same time, we supplemented ablation studies and tried it on Cirrhosis and T2D data sets. We found that if this method is not used, accuracy and auc metrics will drop by 2 ~6 percentage points, specific experimental data will be supplemented in the paper.

Reviewer comment (Experiments):
The baselines are old, and the code/dataset are not available.

Author response:
Finally, "The baselines are old, and the code/dataset are not available.", you may not have read the full text carefully. First of all, there is very little related work in this field, and it is also one of the bottlenecks. Our method is the first time to apply GNN to Methods in this field, so previous work has never used GNN to solve this problem, and our supporting material contains the complete code, which is inconsistent with what you said.

Reviewer comment (Experiments):
The experiment section is not very sound. Recent SOTA (state-of-the-art) expressive GNN baselines are not included. Since the gut microbiota disease prediction is studied as a graph prediction task, these powerful GNN baselines are necessary.

Author response:
We sincerely appreciate your suggestions for improving the details of the paper. Our method is the first to apply GNN to this field, so previous work has never used GNN to solve this problem. In view of this, we used GraphSAGE as a control experiment in the comparison experiment in the original paper, as shown in table1 . In the past two days, we have also added GAT experiments, and the results are similar to the GraphSAGE in the table. The specific data will also be displayed in the final paper. In addition, previous methods to extract the global features of a graph usually use the average of all nodes or global pooling, but the method is simple and the effect is not ideal. In order to verify the stability of the NFGI module, we conducted experiments on the value of K and found that the model with K ranging from 10 to 20 is generally more stable. Thank you again for your suggestions for improving the details of the paper. We will continue to revise and polish it.

Reviewer comment (Evaluation):
The paper concludes that the proposed UMMAN is stable. The experimental section does not support it.

Author response:
We sincerely appreciate your suggestions for improving the details of the paper. Our method is the first to apply GNN to this field, so previous work has never used GNN to solve this problem. In view of this, we used GraphSAGE as a control experiment in the comparison experiment in the original paper, as shown in table1 . In the past two days, we have also added GAT experiments, and the results are similar to the GraphSAGE in the table. The specific data will also be displayed in the final paper. In addition, previous methods to extract the global features of a graph usually use the average of all nodes or global pooling, but the method is simple and the effect is not ideal. In order to verify the stability of the NFGI module, we conducted experiments on the value of K and found that the model with K ranging from 10 to 20 is generally more stable. Thank you again for your suggestions for improving the details of the paper. We will continue to revise and polish it.

Reviewer comment (Experiments):
It is not clear why the unsupervised training is used. What's the advantage over supervised learning on the gut microbiota disease prediction?

Author response:
"It is not clear why the unsupervised training is used." Because the hosts in the field of bioinformatics are usually patients or related personnel, and because the label involves user privacy, unsupervised learning is a very important research direction in bioinformatics. We have never used the host's label in the process of obtaining the embedding of each node, so this is our explanation for this problem.

Reviewer comment (Novelty):
Technical contribution is a bit marginal as both the applied model and the idea of contrast learning are somehow take-off-shelf.

Author response:
I have carefully read all your suggestions and agree with most of the opinions. As you said, we should introduce the form of tabular data and the shortcomings of basic CNNs in more detail in the paper, and will continue to supplement the paper in these aspects. However, "technical contribution is a bit marginal" may not be very reasonable, because as Amazon chief scientist Li Mu said: "A good paper is not about appearing modules that no one has seen before, but about such methods. Can you think of it?" First of all, as far as we know, previous GNN methods have never used multiplex relationships to construct graphs. At the same time, considering the particularity of this field, that is, the correlation of intestinal microorganisms is complicated and difficult to obtain, even multi-graphs cannot guarantee the accuracy of extraction. property, so we built Shuffled-Graph for adversarial training. At the same time, we supplemented ablation studies and tried it on Cirrhosis and T2D data sets. We found that if this method is not used, accuracy and auc metrics will drop by 2 ~6 percentage points, specific experimental data will be supplemented in the paper.

Reviewer comment (Presentation):
How the existing works conducted on the tabular data can be introduced in the Related Works with more details. For example, what a typical tabular data looks like? A figure with a few columns and rows from the tabular data should be enough.

Author response:
I have carefully read all your suggestions and agree with most of the opinions. As you said, we should introduce the form of tabular data and the shortcomings of basic CNNs in more detail in the paper, and will continue to supplement the paper in these aspects. However, "technical contribution is a bit marginal" may not be very reasonable, because as Amazon chief scientist Li Mu said: "A good paper is not about appearing modules that no one has seen before, but about such methods. Can you think of it?" First of all, as far as we know, previous GNN methods have never used multiplex relationships to construct graphs. At the same time, considering the particularity of this field, that is, the correlation of intestinal microorganisms is complicated and difficult to obtain, even multi-graphs cannot guarantee the accuracy of extraction. property, so we built Shuffled-Graph for adversarial training. At the same time, we supplemented ablation studies and tried it on Cirrhosis and T2D data sets. We found that if this method is not used, accuracy and auc metrics will drop by 2 ~6 percentage points, specific experimental data will be supplemented in the paper.

Reviewer comment (Theory):
The proposed adversarial training scheme seems a bit ad-hoc. More analysis could be provided on why this particular approach is effective.

Author response:
Thank you for your suggestion. First, for "adversarial training", we supplemented ablation studies and tried it on Cirrhosis and T2D data sets. We found that if this method is not used, accuracy and auc metrics will drop by 2 to 6 percentage points. Specific experimental data It will be supplemented in the paper. As for the motivation of Shuffled-Graph, the paper actually involves it but does not elaborate on it. Specifically, the connection between intestinal microorganisms and the host is intricate. Even if we use multiple relationships to build graphs, it is difficult to ensure the accuracy of the correlation acquisition. This is also one of the biggest pain points in this field, so we thought of another way. , that is, constructing Shuffled-Graph. During training, the model expects the difference between Shuffled-Graph and Original-Graph to be maximized, that is, to maximize the similarity between Original-Graph and true embedding. In addition, our method is the first time to introduce GNN in this field. Although there have been many GNN methods before, such as STGCN, GAT, and GraphSAGE, among which GraphSAGE is also used as part of the experiment, it is not applicable in this field.

Reviewer comment (Experiments):
Ablation studies only evaluate the contribution of individual components; would be good to see the ablation of a full adversarial training scheme.

Author response:
Thank you for your suggestion. First, for "adversarial training", we supplemented ablation studies and tried it on Cirrhosis and T2D data sets. We found that if this method is not used, accuracy and auc metrics will drop by 2 to 6 percentage points. Specific experimental data It will be supplemented in the paper. As for the motivation of Shuffled-Graph, the paper actually involves it but does not elaborate on it. Specifically, the connection between intestinal microorganisms and the host is intricate. Even if we use multiple relationships to build graphs, it is difficult to ensure the accuracy of the correlation acquisition. This is also one of the biggest pain points in this field, so we thought of another way. , that is, constructing Shuffled-Graph. During training, the model expects the difference between Shuffled-Graph and Original-Graph to be maximized, that is, to maximize the similarity between Original-Graph and true embedding. In addition, our method is the first time to introduce GNN in this field. Although there have been many GNN methods before, such as STGCN, GAT, and GraphSAGE, among which GraphSAGE is also used as part of the experiment, it is not applicable in this field.

Reviewer comment (Experiments):
More discussion could be provided on the choice of graph construction methods.

Author response:
Thank you for your suggestion. First, for "adversarial training", we supplemented ablation studies and tried it on Cirrhosis and T2D data sets. We found that if this method is not used, accuracy and auc metrics will drop by 2 to 6 percentage points. Specific experimental data It will be supplemented in the paper. As for the motivation of Shuffled-Graph, the paper actually involves it but does not elaborate on it. Specifically, the connection between intestinal microorganisms and the host is intricate. Even if we use multiple relationships to build graphs, it is difficult to ensure the accuracy of the correlation acquisition. This is also one of the biggest pain points in this field, so we thought of another way. , that is, constructing Shuffled-Graph. During training, the model expects the difference between Shuffled-Graph and Original-Graph to be maximized, that is, to maximize the similarity between Original-Graph and true embedding. In addition, our method is the first time to introduce GNN in this field. Although there have been many GNN methods before, such as STGCN, GAT, and GraphSAGE, among which GraphSAGE is also used as part of the experiment, it is not applicable in this field.