Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper studies how to construct predictive intervals from a contextual bandit policy such that the true labels are within the predictive intervals with a user specific probability, using some bandit feedback data, which they call the calibration data. 

The paper identifies that the data distributions are different under the logging policy and the target policy in contextual bandits, which impedes using conventional conformal prediction methods to provide predictive intervals with coverage guarantees. So they adopt the conformal prediction method with covariate shifts [13] to this problem to provide coverage guarantees despite the data distribution shift. 

They provide marginal coverage (adapted from prior works) and conditional coverage analysis (with strong assumptions) of the method, and demonstrate its effectiveness in empirical evaluation. 
strengths_and_weaknesses: The paper is well-written and easy to follow. 

They provide coverage analysis of the proposed method. 

The empirical evaluation demonstrates the effectiveness of the proposed method. 

Weaknesses:

1. The method does not provide any optimality guarantee. The method does provide coverage guarantee (i.e. the prediction set contains the true label with high probability). But it does not have any guarantee on the size of the prediction sets. I can easily construct prediction sets with perfect coverage guarantee, i.e. the set of all labels. 

2. The paper does not seem to provide new technical contributions. The proofs seem to be directly adapted from prior works. 

3. I am also a bit confused about whether the guarantees are desirable. In proposition 4.1, the randomness is in both the calibration data and the one test data point, right? Does this mean that for each single test data, we need n calibration data to construct the prediction sets to provide the coverage guarantee? Would it be possible to get this kind of guarantee instead: with high probability in the calibration data, the probability that the true label is in the prediction set is greater than a user-specified level 1-\alpha? If this is true, it then makes sense to construct a single set prediction function using calibration data, and then apply it to a set of test data. 


questions: See weaknesses. 
limitations: I did not find limitations and potential negative societal impact.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper studies conformal prediction for the (mixed) outcome in contextual bandits under a target policy, using data from a different behavior policy. To tackle the shift from observations to targets, the authors provide the explicit formula for the distributional shift, then propose to use density estimation as well as a Monte Carlo sampling method to compute the conformal prediction intervals. The authors also build on existing methods to show the marginal coverage and asymptotic conditional coverage of the proposed methods. Finally, extensive experiments show the performance of the proposed method. 
strengths_and_weaknesses: Originality: 

This paper studies a new problem. The techniques contain some novel points but somehow heavily rely on established results. The approach and the results are not too surprising when reading the paper. 


----- 
Significance: 

This paper studies a (moderately new) problem of quantifying the uncertainty in outcomes from contextual bandits. Instead of predicting counterfactuals, the authors consider the mixed overall outcome. This adds to the application of conformal inference, but I am not sure whether this is indeed interesting. 


-----
Quality: 

The overall quality of this paper is good. The presentation is clear; the theoretical results are easy to understand with nice interpretation and discussions; the comparison to the literature is detailed and comprehensive; the conclusions from experiments are clear. 

Besides the strengths, I do have a few concerns. 

1. Theories involving weight function estimation. 

It is unclear to me how it depends on the Monte Carlo approximation with $h$ samples. Although Proposition 4.2 provides an error bound that is quite similar to Lei and Candes (2021), I am a bit concerned that it is not the most appropriate for this place. Instead, it might be more informative if the error bound could be in terms of $\hat{P}(y | x, a)$ because this quantity is estimated, while $\hat{w}$ is built on top of it. 

2. Estimation of $\hat{P}(y | x,a)$. 

The density estimation of $\hat{P}(y | x,a)$ is known to be hard because of data scarcity (more difficult than propensity score estimation, at least). It would be helpful to justify this approach by referring to some classical literature on estimation performance under appropriate conditions. 


3. Comparison to SBA.

I am not convinced why the authors compare their method to SBA. It is not a conformal-based algorithm and not studied in the literature. 

-----
Clarity: 

The presentation of this paper is clear and easy to follow. The results are interpreted clearly from time to time. One concern is for the weight function estimation, as stated above. 






questions: 1. How does the miscoverage depend on the estimation error of $\hat{P}(y|x,a)$ instead of the hard-to-interpret $\hat{w}$? 

2. The density estimation of $\hat{P}(y | x,a)$ is known to be hard because of data scarcity (more difficult than propensity score estimation, at least). It would be helpful to justify this approach by referring to some classical literature on estimation performance under appropriate conditions. 

3. Please provide more explanation for comparing to SBA in your experiments. 
limitations: The authors have addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies off-policy value predictions at particular X with either marginal or conditional coverage guarantees.  Unlike prior art, this paper preserves the context distribution and models the shift in the outcome distribution via the shift in the action distribution.   The methodology thus applies to stochastic policies and in infinite action spaces (with some caveats).   For marginal guarantees, the methodology works with either a known historical policy or estimated propensity scores.  As is typical in conformal prediction, conditional coverage guarantees require restrictive additional assumptions.
strengths_and_weaknesses: This paper is in the happy position of (essentially) being the first proposed solution to a problem of clear practical importance.

This suffers from the page limit.  Personally, I would elide more of the requisite but uninteresting experimental details to the appendix in order to expand the discussion around proposition 4.3 in the case of known weights.  Alternatively, section 2.2 is ultimately unnecessary: yes prior art used a different covariate shift model, but you could reduce this to a sentence or two in the related work and perhaps a reference to the appendix, and get back some valuable space.
questions: This paper repeatedly states that known weights are not typical, which contradicts my extensive personal experience with large scale recommendation systems with known randomization.  In causal modeling with observational data propensities are often estimated, but the paper emphasizes the contextual bandit application area.   The estimated weight results are appreciated; but for instance the paper misses a chance to present a simplified corollary of proposition 4.3 for the case when w(x,y) is known.

The moment condition in proposition 4.2 is presented without comment.  Some discussion about how this affects the estimation of \hat{w} would be appreciated (e.g., suggest a regularizer; or comment upon the moment properties of equation (7)).
limitations: This reviewer is satisfied.
review_title: null

Meta-review: metareview: The paper develops conformal prediction tools for OPE under the contextual bandit model. The reviewers find the problem of clear importance and the quality of the paper sufficiently high.  All reviewers are in favor of acceptance. The authors are encouraged to incorporate the reviewers' feedback and have a clear discussion of the technical novelty in the final version.
confidence: Less certain
Decision: Accept