Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: Given multiple data sources, each related to a subset of variables, authors study the problem of deciding which data source to sample from at each time step. Then, by using the samples, the goal is to estimate the model parameters as good as possible. The proposed framework is based on the generalized method of moments. Their contributions are as follows:

1. In Section 4, it is assumed that each instance has the same cost across all data sources. Then, authors first show that any fixed policy that differs from the optimal one will attain a constant asymptotic regret. To overcome this issue, two adaptive methods are proposed OMS-ETC and OMS-ETG. Here, OMS-ETC resembles the classical explore-then-commit algorithm in the context of bandits, where there is an exploration step to estimate the model parameters and then the model commits to a data source that minimizes the asymptotic variance of the target parameter. In contrast, OMS-ETG keeps updating the model parameters as more data is gathered. Under certain conditions, authors show that both adaptive methods will attain a zero asymptotic regret.

2. In Section 5, both adaptive methods are modified as to attain zero regret in the case where instances have different costs across data sources.

3. Experiments are provided in Section 6, where authors depict the behavior of the regret for different causal DAGs. Semi-synthetic dataset are also provided to showcase the usability of their method.
main_review: Here I present my review along the axes from the NeurIPS reviewing guidelines:

**Originality**

- Are the tasks or methods new? 
  - Yes. I believe the proposed setting is new as it differs from the data fusion problem.

- Is the work a novel combination of well-known techniques?
  - Yes. The methods are motivated by similar ideas applied in online settings. 

- Is it clear how this work differs from previous contributions? 
  - Yes. The main contrast starts with the setting presented in the paper, as it can be regarded as a "online data fusion" problem.

- Is related work adequately cited?
  - Yes, authors compare their methods against relevant work.


**Quality** 

- Is the submission technically sound? 
  - Yes, the theoretical analysis relies heavily on martingale theory and leverages existing concentration and asymptotic results from other settings.

- Are claims well supported (e.g., by theoretical analysis or experimental results)? 
  - Yes. Proofs are given in the appendix, and experiments effectively support the theoretical claims.

- Are the methods used appropriate? 
  - Yes, given the non-iid nature of moment samples, martingale is an appropriate tool for the analysis.

- Is this a complete piece of work or work in progress? 
  - This work is complete.

- Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?
  - Yes, authors state the conditions/assumptions required for their methods.


**Clarity** 

- Is the submission clearly written? 
  - Yes. I particularly appreciate Example 1, which helped grasp the setting faster.

- Is it well organized? 
  - Yes, I was able to follow the main results without issues.

- Does it adequately inform the reader? 
  - Yes, several details are provided to reproduce the results.


**Significance** 

- Are the results important? 
  - Yes, the setting along with the algorithms provided are of interest to the machine learning community. In the introduction, authors provide a compelling story to motivate the setting.

- Are others (researchers or practitioners) likely to use the ideas or build on them? 
  - Very likely in my opinion. The setting itself might get the attention of other researchers to propose alternative methods or improve upon the proposed ones.

- Does the submission address a difficult task in a better way than previous work? 
  - As the setting differs to existing ones, there is no direct comparison to other methods.

- Does it advance the state of the art in a demonstrable way? 
  - The methods presented can be regarded as the first set of results for the setting under study.

# Questions / Comments

I believe the paper discusses an interesting setting, where two non-trivial algorithms (motivated by existing methods from the online learning community) are proposed. My questions relate to the assumptions required to carry out the regret analyses of the methods.

1. The identifiability assumption given in Assumption 2 was initially hard to follow to me. Can authors provide more intuition/interpretation on this assumption?

2. Do authors have any comments about the necessary conditions for their methods?

3. I understand that the reason for the lack of comparison against other methods is due to the setting. Still, I wonder if there is a possibility to (even naively) apply existing methods for ATE estimation under this setting. 

---
I thank the authors for their response. I remain positive about the paper and still recommend acceptance.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes online moment selection (OMS), a framework in which
structural assumptions are encoded as moment conditions. This
algorithm allows users to actively acquire data (variables) given the
very moments that identify the functional of interest. The paper shows
that there is substantial empirical gains in doing so.
main_review: The paper proposes online moment selection (OMS), a framework in which
structural assumptions are encoded as moment conditions. This
algorithm allows users to actively acquire data (variables) given the
very moments that identify the functional of interest. The paper shows
that there is substantial empirical gains in doing so.

The theoretical and empirical results of the paper make sense. I have
a few major comments.

One comment is on the potential bias due to actively collecting data.
In particular, several works have identified that actively acquired
data / variables have bias due to the active nature of the procedure.
[*] Does the proposed algorithm suffer from similar bias too? Why or
why not? Is there any correction procedure that would avoid this
issue?

[*] Nie, X., Tian, X., Taylor, J., & Zou, J. (2018, March). Why
adaptively collected data have negative bias and how to correct for
it. In International Conference on Artificial Intelligence and
Statistics (pp. 1261-1269). PMLR.

Farquhar, S., Gal, Y., & Rainforth, T. (2020, September). On Statistical Bias In Active Learning: How and When to Fix It. In International Conference on Learning Representations.

The other comment is that the theoretical results of the paper focus
on zero asymptotic regret, which is a fairly weak guarantee. Loosely,
it says eventually we will converge to the truth. Do we know how fast
the algorithm converge to the truth? Can we bound the finite step
regret? Does the algorithm admit optimal finite horizon regret? How
does regret grow with the number of time steps?

limitations_and_societal_impact: The paper discuss the limitations in terms of the assumptions it require.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes online moment selection, an approach for choosing which data sources to draw from in an online setting. The goal is estimation of causal effects, the structural causal model is encoded as moment conditions, and the data sources drawn from are chosen to minimize the variance of the estimate. Explore then commit and explore then greedy approaches are suggested and analyzed. A limitation is that the structural equations need to be parametric and have a clear noise term. Improvement is shown over simple fixed policy baselines.
main_review: I found the problem interesting and the results good. I would have liked to see more discussion of the many assumptions and some exploration of whether they hold in practice and a theoretical discussion of under what types of causal models we can expect them to hold, e.g. the uniqueness assumptions etc. 

Line 163: I assume it is the case that this oracle fixed policy is also optimal across adaptive policies, if so this is important to clarify as the regret is being defined for an active policy. 

Minor: may be of interest to include related work on active learning of causal models, e.g. (Squires et al). 
limitations_and_societal_impact: no social impact
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The asymptotic property of GMM estimator is proposed by some previous papers. In this paper, from the theoretical aspect, the authors generalize that to the setting with multiple domains and apply the results in the online estimation of causal effects. They evaluate the regret under three online policies to select which dataset to observe. 
main_review: After rebuttal: the authors have addressed my quesions. The paper is very interesting and I am happy to see it accepted. I increase my score to 8.

------------------------------------
At first, I want to say that I am not familiar with asymptotic theory. Although I read the proof and some related papers [15,27], I do not feel quite qualified to judge at the theoretical level. Hence it is hard for me to give some suggestions from theoretical viewpoint. Hence I mainly focus on the algorithm.

The main reason that I give a borderline score is that the problem tackled seems not reasonable. This makes me think the method right now is solid, but does not contribute a lot to causality literature, especially in a causal effect estimation task.

It seems that in reality, the optimal policy is always as [0,0,\cdots,0,1,0,\cdots,0] (It implies one dataset is better then the others). The estimator under different datasets have different asymptotic distributions. In this case, the optimal policy is to select the dataset with which the estimator has the least asymptotic variance. Hence it seems the main problem is to estimate the asymptotic variance for each dataset. Although the authors propose a more general method, which could be applied no matter what the optimal policy is like, it seems the contribution does not have a strong connection with causal effect estimation task. Could the authors further provide an example in reality when the optimal policy is not as [0,0,\cdots,0,1,0,\cdots,0]?


In addition, there are a few suggestions, questions, or reminders of typos:
-The authors had better propose the explanation of symbol \tilde{g_t}(\theta) in Line 115.
-Line 181: Appendix D -> Appendix C
-I do not understand the equation on Line 170. Why is the order of difference constant when the policy is fixed?
-In reality, it seldom holds that the parameter could be identified by each dataset. As said by the example of doctor on Line 19,  the different datasets are possible from different interventions (tests). In this case, it is hard to guarantee that the parameter is identifiable in each dataset. Hence I think it will be better if the authors could mention that they discard the dataset by which the parameter is unidentifiable.



I look forward to the authors' rebuttal. And I will be happy to increase my score if the authors could provide some persuasive opinions, no matter by correcting me or responding to my questions above.
limitations_and_societal_impact: Yes
review_title: null

Meta-review: The authors propose a methodology for estimating statistical and causal parameters in an online setting in which multiple available sources of data contain information only on a subset of relevant variables. Repeatedly, an agent much choose which data source to interact with in order to as efficiently as possible estimate the desired parameter. Conditions are given under which the proposed estimators (OMS-ETC, OMS-ETG) achieve vanishing regret, both in uniform and non-uniform cost settings. The approach is evaluated on  synthetic and real-world data. 

The reviewers all recognised the novelty of the setting and the value of the proposed algorithm and analysis. There were no major questions or issues remaining after the discussion period, and all four recommended accepting the paper. I believe the unconventional take on the causal estimation problem could be of interest to the wider NeurIPS community. 
Decision: Accept (Spotlight)