Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper considers the stochastic setting where utility comes from some unknown distribution and the goal is to learn the distribution from the utility-maximizing actions of agents. They consider two distributions: Gaussian and delta-corruption. In both settings, they derive Bayesian approaches to learn distribution and provide some theoretical guarantees. They also provide approximation techniques and present some numerical experiments. 
strengths_and_weaknesses: Strenghts:
They motivate the guarantees based on posterior distribution using sound explanation and examples.

Weaknesses:
The experiments show that their approximation algorithms are quite weak and accuracy suffers in high dimensions.If we follow the authors argument that the primary reason is curse of dimensionality then this work only helps small dimensional settings.
questions: I think paper can benefit from broader discussion on tractability and  inaccuracy in high dimension settings.
limitations: The authors address the limitations.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper studies the problem of revealed preference under a stochastic setting. The utility of the agents follows an unknown distribution and we estimate the distribution from only the observation of the utility-maximizing actions of the agents. In this paper, the authors consider two underlying distributions: a Gaussian setting and a $\delta$-corruption setting, and provide Bayesian approaches for the problems with theoretical guarantees.
strengths_and_weaknesses: *Strengths:*

- The paper studies the revealed preference problem under a stochastic setting, where existing analyses assume that the agents share the same utility function.
- The authors propose Bayesian approaches for two utility function distributions and the method may potentially be extended to other settings beyond Gaussian and $\delta$-corruption settings.
- The authors also provide theoretical guarantees to the Bayesian models.

*Weaknesses:*

- The method requires sampling from a high-dimensional space, which incurs the curse of dimensionality that negatively impact the performance of the algorithms.
questions: - A typo in L261, where an extra "0" appears at the end of the sentence.
- Figure 1 plus Figure 2 appears to be too wide.
limitations: The authors seem to have addressed the limitations of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the problem of learning utilities from purchasing behavior, or, learning from revealed preference. While multiple papers have studied the sample complexity of backing out a single set of feasible item utilities that explain purchasing data drawn from a distribution, the authors study stochastic revealed preferences: the item utilities themselves are drawn from a distribution, and the dataset consists of purchases with respect to iid draws of the item utilities. The authors consider two main settings: in the first the utilities are drawn from a von Mises-Fisher distribution, in the second there is a deterministic utility vector that explains the purchase w.h.p, but the purchase (with complementary probability) might be explained by a utility vector drawn from an arbitrary distribution (representing a probability-delta corruption). They provide sampling and optimization techniques for both settings, with guarantees, and verify convergence of the learned utility parameters experimentally via actual implementations of the techniques.
strengths_and_weaknesses: I think this is an interesting contribution to the area of learning from choice data. The paper is nicely written and I was able to follow it with no issues. The techniques are interesting: the stochastic nature of the problem makes it more difficult than simply "backing out" a feasible utility vector. To this end I found the techniques used to overcome this challenge to be novel and interesting. I also appreciated the fact that the authors directly gave algorithms for arriving at the distribution P_u that could be experimentally verified, rather than stopping at sample complexity bounds.

One potential weakness is that the paper considers only two somewhat disparate settings, and there are not any connections (as far as I could tell) developed between the techniques used for both settings. It does seem like this setting is challenging and more general remarks would be tougher (e.g. there’s probably no general distribution-independent statement that can be made) but I wonder if there is a more general abstract learnability result in terms of parameters of P_u. 
questions: Is there any relation to the literature on learning stochastic choice? See, e.g., the manuscript “Prediction and Stochastic Choice” on SSRN and references therein. That literature deals mostly with binary preference relations so maybe there’s not much of a relation, but could be worth looking into (I’m not suggesting that these works be cited). 
limitations: N/A
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies the problem of learning an known utility function via the actions (of an agent) in the stochastic setting, specifically a Gaussian setting and a $\delta$-corruption setting. Under both settings, methods with theoretical guarantees are proposed to learn the known utility function paramerized by a vector. Two respective algorithms are described and implemented for numerical evaluation to verify the theoretical guarantees (i.e., convergence).
strengths_and_weaknesses: This paper rigorously studies the problem of learning some unknown utility function under the stochastic setting (two specific settings). The problem setting is clearly laid-out and the proposed approaches seem sound (to the best of my ability). In particular, Theorems 1 and 2, provide the necessary theoretical guarantees for the approaches, respectively, which are later empirically verified. Regarding weaknesses, the setting of "a set of agents" can be made clearer and the motivation for considering the Gaussian setting and $\delta$-corruption setting togeher in one paper can be made clearer, too.

Strengths
- The overall theoretical rigor is good: in problem formulation, discussing the approaches and providing the necessary theoretical guarantees.
- The presentation is clear (i.e., writing and organization of content).

Weaknesses
- The setting of "a set of agents" is not so clear. A clear contrast (to existing works) this paper claims is "all the existing algorithms and analyses uder this topic relay on the assumption that all the agents share one common utility function (or a common utility parmeter vector) and thus can fail in the stochastic setting."
    - Are there multiple agents (with different utility functions) at the same time for the learner to learn? Why does stochasticity give rise to multiple utility functions (as opposed to one common utility function), since there is only one true $\mathcal{P}_{\mu}$ (for the Gaussian case) or $\boldsymbol{u}^*$ (for the $\delta$-corruption case)?
    - Following this, the motivation of this setting can be made clearer with references in addition to the described use-case in lines 28-30.
- The motivation for specifically considering both the Gaussian case and the $\delta$-corruption can be made clearer. Is it a standard approach to consider these two options (if so, are there references for it), or is there a significant motivation for considering these two case together in one paper?

 
questions: - Please see the first Weakness about "a set of agents".
- In Theorem 1, and in lines 130-132, why is it that $b$ no longer has a subcript $t$? What does $b$ (without the subscript $t$) refer to?
- In lines 137, "the Wasserstein distance in the theorem is not critical and it can be replaced with other distances such as the total variation distance and the Hellinger distance." 
    - Does the bound in the theorem depend on the specific choice of distance in terms of _expression_ and _properties_ (e.g., does it need to be a proper metric)?
    - If multiple distances are possible, which should be used? In other words, what are some considerations one should make when picking the suitable distance?
limitations: The authors have identified in Sec. 5 that the approaches seem to scale poorly w.r.t. the dimension of the problem and argued that it can be mitigated by better and more efficient implementation. It would be good to briefly discuss this (perhaps in Appendix if there is no space) i) to pinpoint the component in the Algorithms that lead to this issue; ii) some intuitions for better implementation (e.g., why would parallel computing improve estimation accuracy, as I thought parallel computing is meant to distribute the computational load).
review_title: null

Meta-review: metareview: An interesting approach to stochastically revealed preferences  
confidence: Certain
Decision: Accept