Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a model and estimator for the "contagion" effect of treatments applied to a node's immediate neighbors in a graph structure. They control for homophily by conditioning on the node's graph embedding. Identifiability is formalized and the method is validated on semi-synthetic datasets.
strengths_and_weaknesses: Strengths:
* The idea is clever and simple, yet highly relevant to the field of causal inference (on graphs in particular.)
* Most of the paper is clear and easy to read.
* The theorems are significant and their accompanying explanations are intuitive. This is especially the case for Theorem 2.

Weaknesses:
* I am a bit hesitant about the novelty of this work, as the method is quite similar to that introduced in [VWB19]. *Operationally*, the only major difference appears to be the aggregation operator that defines $V_i$ (Definition 1), which, coincidentally, is kept on rather vague terms until much later in the paper. The latter issue can be a source of confusion.
* The results are limited to synthetic outcome models. In the first experiment (6.1), the outcome's parametrization is linear with i.i.d Gaussian error, which matches the proposed loss function perfectly. The formulation of the second (6.2) is quite simple, which is compelling, but not a comprehensive evaluation. It would be helpful to explore the performance under minor model mismatch.


Smaller concerns:
* The `do` calculus should be introduced before using it in Equation 3.1. The notation $(T=t^*)$ should more clearly reflect the fact that it is a vector assignment. Also, the context preceding that equation is confusing. How can it be viewed as an intuitive choice for capturing the "average influence of a node's neighbors' treatments on its outcome" if there is no explicit incorporation of neighborhoods? That specification comes a whole section later, leaving the reader in the dark. 
* The justification given in footnote #2 for considering the embeddings nonparametric is a bit of a stretch. Some distributional assumptions appear to be made implicitly, through the outer product and sigmoid in the loss function. Please clarify this claim.
* It would be helpful for the reader to clarify that $m_G$ is used to model a node's *potential outcomes*.
* In Section 7, the first paragraph on RCTs is outside the scope of this topic. Perhaps some of this space can be allocated for a conclusion section.
questions: Please refer to the concerns above. I would be happy to amend my score if any of the major weaknesses are addressed in the rebuttal.

Maybe further expanding on the role of the aggregation function can illuminate the novelty of this approach.
limitations: I would suggest for the authors to briefly discuss the fundamental limitations in using a social-network structure to draw conclusions about social contagion in humans, which is really driven by many other factors.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors tackle the well-known challenging problem in social networks that influence and homophily effects are not trivially distinguishable. In particular, this problem is more challenging because confounding factors are typically unobserved. The proposed method uses the embeddings to represent latent confounding factors and adjusts such confounding factors by adding a network regularization term in the estimation of causal influence effects. Authors show the validity of the proposed method by providing enough theoretical analysis. Also, empirical analysis under semi-synthetic experimental setting verifies the benefits of the proposed method as opposed to baseline estimations.
strengths_and_weaknesses: Strengths
- Authors address a very well-known but critical problem, which fits well into the conference. The final estimation method is straightforward and easily understandable.
- Authors provide enough theoretical support when developing each step of the proposed method. Such support helps understanding without any big logical gap.
- Experimental results show that the proposed method outperforms baseline methods, particularly when the confounding factors become problematic in the estimation of peer influence effect.

Weaknesses
- While notations are clearly defined in theory, it is not easy to follow the meaning or condition (e.g. what kind of value that each variable can have) of  some variables. Some examples would be helpful for straightforward understanding.
- Experimental setup needs to be enforced because the current results show limited aspect of the proposed method. Please take a look at the suggestions below.
- Unlike the typical ML problem, the parameter q is not trivially tunable through AutoML or any hyperparameter tuning because q balances two non-comparable objectives. Authors need to provide some guidance.
questions: In general, experimental part is the weakest part throughout the manuscript. Here are some suggestions regarding the experiments.

- While experimental results can be affected by network structure and the relationships between networks and node attributes, authors need to present richer experimental results on various datasets and simulation models. In particular, since contagion is simulated, the proposed experiment should be applicable for many public datasets.
- Also, variations on the parameter values of \beta's would be great to check. Currently, only \beta_0 = 0 or \beta_0 >= \beta_1 cases are present. However, from the indecisive results for "Zero" case, we may think that the proposed method might be better when \beta_0 is large enough compared to \beta_1 but not necessarily good enough for small \beta_0 case. Further experiments can check this out.
- Moreover, the fixed \beta_1 value does not show the performance over SNR (signal-to-noise ratio; here it should be \beta_0 over the standard deviation of the noise). It would be interesting to see whether SNR impacts the performance more or \beta_{1} / \beta_{0} impacts more.
- Isn't the formula in page 8 eventually \beta_{0}? If so, it would be great to explicitly describe so in order to prevent any confusion. Before that formula, everything is described with respect to \beta, but suddenly it seems as if a new estimator were introduced. Actually, I would suggested introducing this in the earlier part of the paper, instead of in the experiment section.

Here are some other questions.
- Formula (5.2) and experimental setup is based on the continuous variable regression. However, the introductory example is based on the binary outcome. Should Formula (5.2) remain the same regardless of the type of Y values? How about the simulation process in Page 7? Without some management, Y_{i} cannot be guaranteed to be binary.
- In Page 4, line 128, authors mention that the identification of causal effects would be straightforward if we know the latent C_i's. However, the identification does not seem trivial when we do not know the impact of each C_i's on the final response variable. Authors need to provide explanation on this argument.
- As described above, the choice of q value is not straightforward. Authors need to provide some guidance.
- Authors describe that 
limitations: Authors somehow present the limitation of the proposed method through the discussion about experimental results. However, it would be great if authors can summarize the limitation of the proposed methods.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper present a method that use node embedding to causally estimate the peer influence in social networks. In this task, the key challenge is to distinguish the similarity of linked users and peer influence. This paper claims that for a graph that is sparse enough, the hidden attributes can be ignored if we can provide sufficient information on graph structure. And then under this theoretic results, they propose that jointly adding a term relugarizing the node embedding to help reconstruct the graph structure can help us distinguish the peer similarity and peer influence. Experimental results on a semi-synthetic show that the proposed method can bring a significant boost.
strengths_and_weaknesses: Strengths:
1. The topic of this paper is interesting. 
2. The presentation of this paper is good.
3. The idea of this paper is easy to understand, yet with theoretical support.

Weakness:
1. In this framework, how should we remove the bias brought by the correlation between treatment and covariates? In traditional frameworks, such a challenge is usually addressed by inverse probability of treatment weights (IPTW). But I do not think that computing IPTW in the proposed framework is trivial.
2. The experiments are conducted on a synthetic dataset with real-world graph structure. I am wondering if it is possible to run the model on a complete real-world dataset. I understand that on real-world dataset, it is hard to get ground-truth labels. However, we can try to use the model to identify some social effect that are found by social scientists.
questions: How is the q in Eq. 5.2 decided if we do not have ground truth causal effect?
limitations: See the summary.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a formulation for casual estimation of treatment effects in the presence of network influence. The proposed formulation allows for the covariates affecting both network formation and treatment adoption to be unseen and estimated from an observed network. More specifically, the proposed method uses network embeddings as a means of accounting for network influence in a causality model. 
strengths_and_weaknesses: ___Strengths___: The paper presents a clear, relatively novel method that is well supported theoretically. The use of network embeddings is a straightforward extension of incorporating network structure into a causality model based upon previous works that used techniques like stochastic blockmodels (SBMs) or latent space models to incorporate network information. The proposed method, in contrast to previous ones, does not require as many assumptions about the nature of the network being incorporated into the model as other techniques, like SBMs do. The paper is also attacking an important problem, that of determining causality in treatments in network settings, which has important implications for very serious problems like understanding vaccine choices. 
	
The paper also has strong theoretical support for the proposed method. The convergence of the various forms of the estimand only requires mild, often true assumptions like the relatively large size of the network and sparsity of the network. Also, the incorporation of a loss function into the ERM embedding that supports the causal inference model is also noteworthy and makes the method more complete. 
	
Finally, the article's presentation is clear and the writing is generally free of errors. 

___Weaknesses___: The paper does not present a thorough empirical validation and has some practical limitations that are not addressed. The paper only considers one empirical data set and artificially crafts the treatment based on observed covariates. A more convincing empirical validation would feature more than one data set and, ideally, one that would have observed treatment effects from a real-world scenario, like vaccine uptake or voicing of political support of a candidate, or something else similar. Additionally, the paper does not investigate the nature of the embedding procedure on its results; the paper only considers one type of deep learning-based embedding technique. Thus it's not clear how much the embedding technique can affect the results. 
	
The paper also does not address some important real-world limitations of the proposed model. For one, not all node attributes contribute to link formation in any given network. For example, the attribute of gender is usually inversely related to link formation in something like a sexual contact network. Additionally, this problem can be further compounded by non-assortativity in networks, which is another occurrence in real-world social networks. Furthermore, within the empirical results the paper identifies that when there is zero confounding, the performance of the proposed method suffers. So, it would seem there is a significant limitation in this method’s usage that can have deleterious effects on results in actual usage. 

questions: In addition to the points raised in the weaknesses, one other question may be helpful for future work. Can the proposed method be extended to the case where there is more than one network observed, maybe with some assumptions on independence between the networks? For scenarios like social media, there are often several networks present (following, friendship, replying, etc.) all of which may have confounding effects on treatment.
limitations: See the strength and weaknesses section about limitations for use in practical settings. The authors have addressed any limitations with regard to potential negative societal impacts.
review_title: null

Meta-review: metareview: This paper introduces a method for causal estimation of peer influence in networks in the presence of a confounding effect of homophily. The reviewers  agreed that the idea proposed in the paper is novel and well-motivated, and the accompanying theoretical analysis is useful. While there were concerns about the lack of experimental variation in more realistic scenarios, the consensus is that the contribution is sufficiently  novel and significant to warrant its acceptance. 
confidence: Certain
Decision: Accept