Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper provides a framework for domain adaptation in graph neural networks. Algorithmically, the framework relies on addition of two regularizer terms to the loss function. The construction of the regularizer terms is motivated from theoretical results derived using the theory of optimal transport-based domain adaptation and stability of graph filters from the existing literature. Experiments on real world datasets illustrate the utility of the proposed framework. 
strength_and_weaknesses: Strengths:
Domain adaptation is a very practically relevant problem. The proposed framework is theoretically motivated and provides a solid contribution in this regard. Experiments are well-designed and provide a convincing evidence for the approach. 

Weaknesses:
I don't identify any major weaknesses. However, I couldn't verify the 'padding with isolated nodes' procedure. For $A_1$ in Lemma 1 (and other results where this procedure is mentioned), is this similar to extending the matrix diagonally with dummy nodes to make it of size $N_G \times N_G$? If so, this does not appear to be an optimal procedure for domain adaptation. 


clarity,_quality,_novelty_and_reproducibility: The paper is well written and easy to follow. Various technical steps taken by the authors are described with the help of examples and figures. The results in the paper are motivated by optimal-transport theory and the authors leverage the existing results on stability of graph filters to get around the challenges offered in the graph domain. 

I would recommend that the authors provide additional details on the empirical evaluation of the Wasserstein distance $\hat W_1$ and the construction of graphs in the protein-protein interaction networks. 
summary_of_the_review: I think this is a solid contribution in domain adaptation on graphs. The algorithms are well-motivated from theory and the experiments are illuminating and convincing.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The main idea of the article is to adapt the results about bounds in domain adaptation (from [Redko et al., 2017] and following wowrk) to situations involving graphs instead of Euclidean space. After a theoretical analysis, very close to the existing results in non-graph settings, the authors focus only on one term which drives the Lipschitz property of GNN (acting as a feature extractor) on graphs. Their methodological work then amounts to add a regularization over the graph spectrum based on spectral smoothness and/or maximum frequency response of the graph filter inside the GNN. Some numerical experiments on two types of datasets (Protein-Protein Interactions and Topic classification of articles) to study the possible performance of the method.
strength_and_weaknesses: Strength:

1- the idea of adapting  [Redko et al., 2017]  to graphs is relevant ;

2- the obtained theoretical result (Corollary 1 and the two Lemmas of 4.1) appear to be sound.

Weaknesses:

1- the theoretical  results are obtained in a straightfoward re-purposing of the aforementioned works ; they don't look wrong, but not that original in lights of existing works.

2- the reach of this results is limited because, as soon as it is obtained, the authors limit themselves to the $C_f$ Lipschitz constant and the proposed method appear then to be only a slight improvement on existing architectures ;

3- the proposed regularizations, SSReg or MFRReg have an impact which is almost minor in Tables 1 and 2 ; and again in Tables 3 and 4 when comparing to GNN with DA-W ;

4- most of the really novel content is of experimental nature (in Section 5) and the observations reported in 5.1 and 5.2 appear to be quite fragile: i) I am not convinced by the paragraph "Hypotheses" in 5.1 on the two tasks on PPI and I think that more experimental validations should be sought for before concluding that ; and ii) the discussion under "Results" in 5.1 is not fully convincing about the impact of DA. Ifw we follow the metrics in term of ranks proposed by the authors, methods with DA-W appear to perform better in Tables 1 and 2 than without. Looking at specific instances is toio much cherry-picking of results that seem to saturate (the improvement by the proposed method is often on the third digit in percentage)

5- the article leverages many "big" models (Transformes, GNN with many layers) yet there is no major study about the computation power / time / expense incurred when doing this study (the only reported times are in Tables 13 and 14 in SI); The obtained marginal improvements should be discussed and put in balance with the increased computational costs. For instance, is the less than 5% increase in performance w.r.t. GraphCL  useful ? What's the difference in price (or resource consumption) between what is proposed and this latter method, for instance.
     
clarity,_quality,_novelty_and_reproducibility: Based on the previous comments on Strength and Weakness, the article has only marginal novelties, with a strongt limitation of only working on the regularization w.r.t the Lipschitz constant.

The reproducibility is fair, as a code is provided (I didn't have the time to try it), yet not really good because the numerical cost is heavy. The authors mentioned for instance in SI, section E, the hardware used which appear to be a lot  with "computer clusters" in unprecised numbers.

The clarity and quality and faire as well, as I think that the experimental observations lack more robust insights and conclusions. 
summary_of_the_review: As a summary, given the aforementioned weaknesses, I currently vote against the acceptation of the article, which appears to me to be incremental both on the theoretical aspect and on the methodological+experimental side.

Update: scores raised after the answers.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes theory-driven graph domain adaptation regularization terms, including spectral smoothing (SS) and maximum frequency response (MFR). The authors try to extend the theorem in generic domain adaptation into the graph field and focus on the definition of Lipschitz on GNN. Via analyzing the graph Lipschitz property, the authors develop two reguarlizations that directly minimize the upper-bound of the target risk (the right hand of the inequality). The experiments also verify the effectiveness of the idea. 
strength_and_weaknesses: ### Strength 
- The paper is well-written and the idea is well-motivated. 
- The proposed theory-driven regularization terms are reliable and interesting. To be honest, I enjoy reading this (kind of) paper. 
- The paper is technically sound. 

### Weakness
- The theoretical analysis is limited to the single-layer GNN and it seems non-trivial to extend it into the multi-layer cases. 
- The core theorem is mainly from the existing work of generic domain adaptation. The real theoretical contributions of this paper seem to be the two lemmas about the discussions of Lipschitz on graphs. 

clarity,_quality,_novelty_and_reproducibility: - The paper is well-written and easy to follow.
- Although the basis theorem is from the existing work of generic domain adaptation, the theory-driven regularization terms are impressive. 
- The details of experiments are provided and the codes are submitted as supplementary.
summary_of_the_review: I'd like to increase my score during the discussion period.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies learning with graphs under the covariate shift assumption. The predictor is decomposed into two functions; one is the feature extractor of the graph, f, and the other is the discriminator, g. The generalization performance of the predictor h(G)=g(f(G)) for a graph G is governed by the Lipschitz constant of f and g. Through analysis of the theoretical properties of the feature extractor defined on the set of graphs, the authors proposed two regularization methods; spectral smoothness(SS) for the node transfer and maximum frequency response(MFR) for the link transfer. The theoretical findings well explain numerical results for link prediction and node classification. 
strength_and_weaknesses: Strength
- In the paper, the transfer learning on the graph is roughly categorized into node transfer and link transfer. For each setting of the transfer learning, an appropriate regularization for the feature extractor is presented. An important insight provided by this paper is to recognize two types of information transformations on the graph. For each type, a reasonable regularization is proposed. 

Weaknesses
- Only a single-layer architecture is used as the feature extractor f(G). Extension to multi-layer networks may be rather straightforward, though computational cost becomes high. A brief comment on such an extension can be beneficial for readers.
- The categorization of node transfer and link transfer is interesting and important. Usually, we may not have sufficient prior knowledge of the scenario. Is it possible to determine which scenario occurs in practice? Which kind of information is beneficial to identify which scenario occurs?

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written, and the readability is high. Clear recognition of the node transfer and link transfer is significant, though I'm not sure the concept is new. The regularization tailored for each scenario is an interesting insight in this work.
summary_of_the_review: The paper is clearly written. The regularization methods for node transfer and link transfer are an interesting contribution of this paper.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper tackles the problem of covariate shifts when the input domain is graphs. Through analysis of the theoretical properties of the feature extractor defined on the set of graphs, the paper proposes two regularization methods; spectral smoothness(SS) for the node transfer and maximum frequency response(MFR) for the link transfer.

All reviewers agree that the paper is well written and the proposal is supported by sound theoretical analysis. The problem is well dissected and studied in the sense that transfer learning on graphs is categorized into node transfer and link transfer. For each setting of the transfer learning, an appropriate regularization for the feature extractor is presented (xYtM). 

Regarding weaknesses, one concern raised was that Corollary 1 (risk bound) is a direct application of existing results (no2m, bQEk). Separately another concern was that theoretical analysis of graph neural networks is only presented for the single-layer case. Both of these concerns have been satisfactorily addressed by a revision and an appropriate rebuttal. A new analysis was added to the appendix for the two-layer case. Although there is a concern (no2m) on minor performance improvement in experiments, the overall theoretical contributions appear to be novel and outweigh this concern. 

I recommend that the paper be accepted.

*Suggestions to the authors*:
Please address all comments from the reviewers. Please add a discussion on memory consumption and compute time to the revised version. This may be an expanded version of responses given to reviewer no2m on these points.

justification_for_why_not_higher_score: There is a concern (no2m) on minor performance improvement in experiments.

justification_for_why_not_lower_score: The paper is well written and the proposal is supported by sound theoretical analysis. Risk bound on graph domain is also relative less common compared to ones on Euclidean domain. I think the paper is worth publishing.
Decision: Accept: poster