Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper introduces an assessment framework for an in-depth analysis of the effect of graph convolutional filters and proposes a novel graph neural network with adaptable filters based on the analysis. The assessment framework builts on the Fisher discriminant score of features and can also be used as an additional (regularization) term for choosing optimal filters in training. The assessment result shows that there is no single graph filter for all types of graph structures. Experiments on both synthetic and real-world benchmark datasets demonstrate that the proposed adaptive GNN can learn appropriate filters for different graph tasks. 

The proposed analysis using the Fisher score is reasonable and interesting, giving us an insight into the role of graph filters. Even though the analysis is limited (using simple graph models and filter family) and the result is not surprising (given no free lunch theorem, there is very likely to be no single silver bullet fo graph filters), I appreciate the analysis and the result. But, I have some concerns as follows. 

1) The proposed GNN and the optimization process
The proposed method is to extend CNN to a simple linear combination of different filter bases with learnable weights, which I don't think is very novel. Adding the GFD score as an additional constraint term is interesting, but the way of optimizing the whole objective function is unclear. (In addition, I think calling it the "regularization term" is inadequate since the term actually involves data observation, rather than a prior on parameters only.) 
In the case of AFGNN_inf, I don't think it is equivalent to applying infinite lamda. If lamda is infinite, L_CE needs to be completely ignored. This needs to be clarified. 
In the case of AFGNN1, I don't clearly understand how the whole objective function is properly optimized with fixed data representation. Is it also iteratively optimized? I hope this is also clarified in more detail. 

2) Unconvincing experiments
The results on three real datasets do not show significant gains, and two of them are even worse than those of GAT. Furthermore, inductive learning (e.g., protein-protein interaction (PPI) dataset used in GAT) is not tested, which I think needs to be also evaluated. While two synthetic datasets (SmallGap and SmallRatio) created by the authors show significant improvement, these datasets appear to be extreme and unrealistic and look carefully selected in favor of the proposed method. I recommend the authors use for evaluation more realistic datasets that can be found in related research.  

review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: In this paper, the authors raise and address three questions about graph neural network: (1) Whether there is a best filter that works for all graphs. (2) Which properties of the graph will influence the performance of the graph filter. (3) How to design a method to adaptively find the optimal filter for a given graph.
The paper proposes an assessment method called the Graph Filter Discriminant Score based on the Fisher Score. It measures how well the graph convolutional filter discriminate node representations of different classes in the graph by comparing the Fisher score before and after the filter.
Based on the GFD scores of different normalization strategy and different order of the graph filter in the experiments on synthetic data, the authors answer the first two questions: (1) There is no optimal normalization for all graphs. (2) row normalization performs better with lower power-law coefficient, but works worse with imbalanced label classes and large density gap.
For the third question, the authors propose a learnable linear combination of a limited family of graph convolutional filters as the layer of model AFGNN, which can learn the optimal arguments of the combination based on the FGD score.
The paper focuses on a significant topic and proposes an assessment tool for the graph filters. Based on that, it also introduces a model to choose filters from a family of filters for any specific graph.
The description of preliminaries is clear.
The observations of the impact of the graph properties on the filter choice are interesting and explanations are provided.
The results of the test accuracy on both bench mark and synthetic datasets demonstrate the good performance of the proposed model.
It is good that the paper provides proof for the claim that the graph convolutional can help the non-linear separable data to be linearly separable, so it is reasonable to use Fisher score. However, does this claim support the second term in equation (3), where the Fisher score is used to evaluate before the filters applied?
The presentation of the last paragraph of “graph filter discriminant score” in page 4 can be improved. The Figure references seem incorrect and confusing.
The analysis of the influence of label ratio seems not accurate enough.
For the GFD score comparison in Figure 4, why choose order 1,3,7 for density and different order 2,3,6 for density gap?
What is the meaning of the symbol psi(l)?
It would be better if the explanation of the raining loss section is more detailed and clear.
What is “AFGNN_P” in the experiment analysis?
It could be interesting to see the comparison of time between the proposed method and the GAT.
For the graph filter discriminate analysis, is it fair to compare the learned layer with the other base filter using the GFD score? Since the learned layer is picked with highest GFD score. Maybe one or two sentences on this will be helpful.
The writing of the paper must be improved. Too many typos and grammar problems will impair the presentation and the reader can be distracted.
Minor comments:
The layout of the sub caption of Figure 1 can be improved.
The usage of capital letter in the phrase “density gap” is inconsistent.
“As shown in figure” instead of “As is shown in figure”.
Many sentences miss article.
There are many typos in the writing.
For example, “Note that for given (feature)”, “…make the representation of nodes in different (class) more separable.”, “Noted that there are some other (variant) of GNN filters that (does) not fall into…” in page 4.
“Here we give (a) empirical explanation to this phenomenon”, “this normalization strategy (take) into account…”, “Thus even in the case that the other two (doesn’t) perform well…” in page 5.
“…a very important factor that (influence) the choice of normalization strategy”, “when power-law coefficient (decrease)”, “when the (sizes) of each class (become) more imbalanced”,  “This is because column normalization better (leverage) …” , “in a similar manner (with) label ratio”, “when the density or density gap (increase)”, “high-order filters can help gather… and thus (makes) the representations…”, “when the density gap (increase)” in page 6.
These can be continued but it is obvious that this paper needs proofreading.

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: This is a very interesting study about GNN. Authors proposed to extend LDA as a discrimination evaluator for graph filleters. Also authors proposed Adaptive Filter Graph Neural Network to find the optimal filter within a limited family of graph convolutional filters. The whole study is novel, and beneficial for the community of graph neural network study. It provides a new way to understand and evaluate GNN.

There are some questions authors should clarify. And some writing errors to correct.
Eq (3) defines GFD for a pair of classes i and j. For a graph with more than two classes, the GFD will be the average of all pairs? Will class imbalance will have any impact on this GFD measure?  
Errors:
•	we studies the roles of this two components 
•	there exist a best choice 
•	we only consider to to find

review_title: Official Blind Review #2

Meta-review: The paper investigates graph convolutional filters, and proposes an adaptation of the Fisher score to assess the quality of a convolutional filter. Formally, the defined Graph Filter Discriminant Score assesses how the filter improves the Fisher score attached to a pair of classes (considering the nodes in each class, and their embedding through the filter and the graph structure, as propositional samples), taking into account the class imbalance.

An analysis is conducted on synthetic graphs to assess how the hyper-parameters (order, normalization strategy) of the filter rule the GFD score depending on the graph and class features. As could have been expected there no single killer filter.

A finite set of filters, called base filters, being defined by varying the above hyper-parameters, the search space is that of a linear combination of the base filters in each layer. Three losses are considered: with and without graph filter discriminant score, and alternatively optimizing the cross-entropy loss and the GFD; this last option is the best one in the experiments.

As noted by the reviewers and other public comments, the idea of incorporating LDA ideas into GNN is nice and elegant. The reservations of the reviewers are mostly related to the experimental validation: of course getting the best score on each dataset is not expected; but the set of considered problems is too limited and their diversity is limited too (as demonstrated by the very nice Fig. 5).

The area chair thus encourages the authors to pursue this very promising line of research and hopes to see a revised version backed up with more experimental evidence. 
Decision: Reject