Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper formalizes the recommendation process as a bandit task. The agent recommends items based on a preference model of the user learning to maximize rewards based on a diversity constraint.  They also introduce locally learnable functions that can be learned from a small neighborhood i.e., behavior over the entire domain can be learned from a small region 


strengths_and_weaknesses: Strengths:
The paper gives insights on the importance of diversity in recommendations
The paper also has proofs in formalizing a benchmark for regret
Locally learnable functions are defined fir learning from a small neighborhood


Weakness
Though there are proofs showcasing the results, there is no evaluation on real data to support the claims of the paper.

questions: Some notations are used before being defined. What is s_(v,i)?, d_TV

How do you correlate entropy and diversity?

limitations: I was unable to find discussions on limitations
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper 'Diversified Recommendations for Agents with Adaptive Preferences' looks at the problem of recommending a set of items in an online setting where customer preferences adapt over time to satisfy both the diversified recommendations criteria as well as to avoid an effect of the "rabbit hole". The authors formulate the problem as an adversarial bandit task and demonstrate a negative result in terms of finding an algorithm that satisfies all the proposed constraints. The authors then suggest an algorithm for bandit linear optimisation. 
strengths_and_weaknesses: My review of strengths and weaknesses will be relatively brief. As I'm quite inexperienced with the topic, I admit that it was very difficult to read and understand. The paper poses a very interesting problem (high originality) and conducts several theoretical analyses to provide insights into the problem. As the intended audience must be much more familiar with the concepts, I can't comment on this part, but I might recommend to incorporate empirical results or clarifying simulations/visualisations based on the suggested Algorithm 2 or RC-FKM. This might help to grasp the results and the outlined concepts for the more inexperienced audience. Therefore, I will omit the quality and significance from my review and would rate clarity on a medium-to-low level. 
questions: * in the abstract (line 23) you mention that you give a set of negative results justifying your assumptions. Do you refer to Theorem 1?
* It would be nice to add references to the appendix, like in the case of the Theorem 1 proof.
* while I do not mind reading papers with unconventional structure, I found in this case that hiding the literature overview in the appendix is making it even more difficult to read. 
* there is an interesting claim that the further users go down the rabbit hole, the more likely they are to continue. While I agree, I think it would be good to substantiate this claim. (line 31)
* (lines 60-61) you write that you propose a 'natural benchmark'. I'm not sure I understand what it means in this context. 
* (lines 108, 126) typos
* (line 152) I couldn't find where $d_{TV}$ is introduced
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper discusses the problem of online recommendation for a single agent with adaptive preferences as an adversarial bandit problem. The authors show that it is necessary to consider a different benchmark, as there are no linear regret algorithms that can compete against the typical best menu/item distribution. Since the available bandit optimization algorithms do not apply to contracting decision sets, the authors provide a new algorithm for this setting that also handles imprecision from estimation. Under a class of locally learnable functions and a mild assumption on each preference model, the authors supply a local learning algorithm to learn an estimated model that achieves $\tilde{O}(T^{3/4})$ regret. 
strengths_and_weaknesses: Strengths:
- The paper investigates an interesting problem of recommendation with adaptive preferences and provides novel theoretical results in their lower bounds.

- The theoretical contributions are significant in advancing the study of diversification in recommendation problems.

- The authors provide a comprehensive set of assumptions and sufficient explanations for why they are necessary. 

- The first algorithm (RC-FKM) is intuitive, and the second algorithm has sufficient discussion.

- The authors adequately address the current literature in their related work section and the appendix.

- The paper is well-organized and easy to follow.

Weaknesses:

- The current model only considers a single agent, which is restrictive.

- In section 2, when the authors first introduce the diversity constraint, the variable $d_{TV}$ was not defined before (line 152). 

- In section 3, algorithm 1 and theorem 4 lack sufficient discussion of the results. 

- The theoretical results could benefit from having a small proof sketch next to them.

- In line 261, there is an extra "and" at the end.
questions: - How can this model be applied to the setting with a single recommender and $N$ different agents? Could the current model handle having more agents whose preferences and behaviors are correlated?

- Could there be experimental sections that show the performance of the second algorithm? While the discussions of the subroutines in section 4.2 are helpful, it could be useful seeing how the algorithm performs on some synthetic data.  
limitations: - The authors sufficiently address the limitations in Section 5.

- While the authors address the negative societal impact of having a small set of items recommended, more discussion on the societal effect of having diversified recommendations could be useful in section 1.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies a game in which menus of items are shown to a user.  The user has unknown preferences which influence the choice of the item from the menu (exfactly one item is chosen).

The chosen item has two consequences:
1) it modifies the agents future behavior and makes it more likely to select the same action
2) it gives the recommender a reward which is non-personalized and depends only on the chosen item.


strengths_and_weaknesses: 
The problem of feedback loops in recommender systems is interesting and there is a lack of good papers studying this question so it was a pleasure to see this paper address the topic.

That said this problem is not a clear analogue of a recommendation problem I know.  I expect people who work on industrial Recsys might have difficulty seeing the connection to the problem being studied and real Recsys problems, although this may not be the intent.

The complete formulation of a recommender system in a bandit framework contains user behavior interspersed with (usually) slates of recommendations and response to the recommendations.  Reward as measured at A/B test time is computed over the time line and may include clicks but also sales or engagement.  The intervention that an A/B test measures is therefore a combinatorial object and 'diversity' is measured within the slates and within time.  Reward itself is not attached to the recommendations but to the timeline.  It is entirely possible that a diverse collection of recommendations result in receiving reward, but this can only be understood if we attach the reward to the timeline not to the individual recommendations.  This paper like most of the recsys literature attaches the reward to the individual recommendations.

Diversity across different user timelines is also necessary for exploration purposes.

This paper makes interesting observations in a simplified model where there are causal implications in the timeline of a recommendation.  Although it isn't to my mind diversity in any of the above senses.  Indeed if I understand correctly only the selected action gives a reward and this is a non-personalized reward although the selected reward has long term impacts on the user (which is very novel).

The writing is quite dense for a recommender systems audience right now.  I would suggest augmenting the mathematical presentation with examples to increase accessibility.
questions: 
I think the presentation would be improved by examples / case studies and possibly simulation studies as well.
limitations: The authors contribution to me is a type of bandit problem rather than taylored to RecSys.  Producing a more convincing RecSys motivation would be interesting.  Although perhaps the paper will stand on its own as a "bandit" paper - I will leave that for others to judge.

The term "adversarial" is used.  I take this to mean something "zero sum" or game theoretic is going on.  I don't see why this would occur, usually the recommender system typically has incentives to improve the user experience.
review_title: null

Meta-review: metareview: This paper studies a sequential recommendation problem where user preferences change over time based on the items selected by the user. The authors show that no sublinear regret algorithm exists for this setting. Under the constraint on preference diversity, they derive an algorithm with $O(T^{3 / 4})$ regret. The original ratings of the paper were 7, 7, 5, and 5; and they did not change after the rebuttal. The reviewers generally like the paper because it studies both adaptive users and diverse recommendations, while most other works try to avoid these topics. This is the strongest point of the paper and my recommendation for acceptance is based on that. On the other hand, the paper is overly theoretical and lacks experiments. Therefore, it is unlikely to appeal to a general recommender systems audience.
confidence: Less certain
Decision: Accept