Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors first point out that the requirement of efficiency of IW- and DA-based methods is the overlapping between the source domain and target domain. Meanwhile, theoretical analysis also shows that insufficient overlapping problems can cause the hardness or impossibility of IW- and DA-based learning methods. In light of this, this paper aims to propose a novel method to optimize recommendation results by solving such insufficient overlapping problems. Specifically, a principled transportation-constraint risk minimization objective function is devised to optimize the recommendation results, which is able to transport the learned patterns from the source domain to the intervention domain better. Then a two-layer adversarial model (GDA) is deployed to optimize the transportation-constraint risk minimization objective function, and sufficient analysis of the GDA is provided. Extensive experiments were conducted on both real datasets and semi-synthetic datasets to show the superiority of the proposed method.    
main_review: Pros: 
	1. This paper identifies and tries to tackle an important problem in recommender systems - insufficient overlapping. The theoretical proof is given to show the uncertainty of the results.  
	2. Generally speaking, the paper is well-written and well-presented. Though many mathematical theorems are contained in this paper, it is easy to follow for readers.
	3. The proposed method is novel and can inspire future research in recommender systems. 
	4. The experiments are sufficient to validate the claims.

Cons:
	1. There are some small typos that should be fixed before publication. 
		1. In claim 1, E_Q R(f) should be E_Pw R(f).
		2. In Table 1, the Hit@10 of DT-MCF on LastFM is 81.81(20). I am confused as 20 is pretty high, and maybe here is (0.20) or (.02).
		3. In Section 5, we randomly sample m < |I| items and see if i' is among the top-K of these m items. items should be unrelated items. Also, the same questions occurred in other parts. I suggest to double-check that.

	2. \mu is also an important hyperparameter, and authors also mentioned that a suitable \mu will give us the opportunity to better explore the whole feedback data. I recommend that the parameter sensitivity of \mu should be conducted. 
	3. The main contribution is to address the insufficient overlapping problem that causes the uncertainty of IW and DA-based methods. However, in the experiment part, IW and DA-based methods are not compared with. I recommend that DT-IW and DT-WA should be included.
	4. The motivation is somehow unclear and the presentation of introduction could be further improved. A more detailed explanation about the relationships between intervention and domain adaption, intervention and the proposed method, why the proposed method can address insufficient overlapping problems, etc should be introduced. 

summary_of_the_review: The novelty of this paper is high.  However, the motivation is not quite clear, and  more IW- and DA-based methods are expected to compare in the experiment.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a new perspective for recommender systems: rephrase optimizing recommendation as finding an intervention that best transports the patterns it learns from the observed domain to its intervention domain. To optimize the recommendation algorithms in this setting, the authors proposed a transportation-constraint risk minimization objective and convert it to a two-player minimax game. Theoretical and empirical studies demonstrated the effectiveness of the proposed method. 
main_review: The paper views recommender systems from a new perspective, which is similar but different to existing works in causal collaborative filtering. The theoretical analyses are comprehensive. The experimental studies are conducted in synthetic data, offline data and online data, which can well demonstrate the effectiveness of the proposed method.

I have the following questions/concerns about this paper:
1.	It seems to me that the proposed method can outperform IW methods when the overlap between the source and target domains are insufficient. However, there is no measurement about the insufficiency. It will be better to have some theoretical or empirical analysis about how the overlap between different domains can affect the performance.
2.	It is hard to directly compare the bounds derived by the paper, e.g., between Theorem 3 and Proposition 1. This will be important to intuitively understand the advances of the proposed method.
3.	The online evaluation used the Deep and Wide method. But the same method is not evaluated in the offline setting, which seems unreasonable.
4.	The proposed method tries to optimize recommendation algorithms via a minimax game, which could be much more complicated than other baselines. Thus, it would be necessary to analyze the efficiency of the proposed method in the experiments.

summary_of_the_review: In summary, this paper targets at an interesting problem and shows promising theoretical and empirical results, which could be helpful for the community of recommender systems.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a novel way to learn new recommendation deterministic policies from feedback collected under different recommendation policies.
The authors start by pointing out that the majority of current approaches, either Counterfactual (Inverse Propensity Weighting) either Domain Adaptation-based, 
do not work well when the support of the current and future recommendation policies do not fully overlap (one of the main causes being that the current recommendation is deterministic and therefore not exploring all actions).
To this end, the authors propose a novel way of re-weighting and regularizing empirical risk that encourages the search for policies that have better reward (lower risk) and are still overlapping in evidence with the current policy.
To escape the support issue that is present in the existing approaches, the authors switch the regularization parameter to an IPM metric (WASSERSTEIN) that can handle non-overlapping supports.
 The authors provide both theoretical and empirical evidence that the resulting algorithm outperforms existing SOTA methods.
main_review: Strengths:
* The subject of the paper is quite relevant and the identified issue of non-overlapping support for RW recommendations serious
* The application of existing methods from causal inference to deterministic recommendation policies is valuable and the empirical section is well-developed and the results significant 

Weaknesses:
* The motivation for the particular approach is not extremely clear. The authors build the argument around “transportability of patterns”, but I feel the point is never fully explained. If an action is never taken under a certain context in the past, (meaning is in the non-overlapping set), the algorithm will have to extrapolate the outcome. In order for the extrapolation to hold some assumptions need to be made about the reward model, and I find the authors do not really spend time on this.
* The resulting objective is closely resembling the counterfactual loss proposed in [1], eq. 3. I think the authors should expand on this approach in the related work, and highlight that unlike in the cited work, they can choose the counterfactual world in which they want to act. Furthermore, I think borrowing either the counterfactual notation or the policy learning notation and vocabulary would help quite a lot in explaining the approach.




summary_of_the_review: Overall, I liked the paper and I found the approach interesting. I think the authors should link their approach a lot more to the work presented in [1] and also covered in two of their references [2,3].
I think by doing so, it clarifies that the main difference in the problem domain is that we are in control of which of the counterfactual worlds we want to compute the risk in. 
Otherwise, all of the vocabulary and the core explanations of the soundness of the approach remain the same.

[1] @inproceedings{shalit2017estimating,
  title={Estimating individual treatment effect: generalization bounds and algorithms},
  author={Shalit, Uri and Johansson, Fredrik D and Sontag, David},
  booktitle={International Conference on Machine Learning},
  pages={3076--3085},
  year={2017},
  organization={PMLR}
}

[2] F. D. Johansson, D. Sontag, and R. Ranganath. Support and invertibility in domain-invariant representations. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 527–536. PMLR, 2019. 
[3] F. D. Johansson, U. Shalit, N. Kallus, and D. Sontag. Generalization bounds and representation learning for estimation of potential outcomes and causal effects. arXiv preprint arXiv:2001.07426, 2020.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors study the domain transportation issue in information retrieval (IR) systems. The main challenge is the insufficient overlapping problem. The authors propose a new adversarial learning method based on transportation-constraint risk minimization to address this challenge. The main insight behind this proposed method is that the interventional impact of recommendation is not exclusive to observed data, but also interferes with the target domain of interest. The authors first analyze the generalization bound of IW and DA methods, and then further give the consistency, generalization, and excessive risk bounds of the transportation-constraint risk minimization. The simulation, real-data analysis, and online experimentation show the effectiveness of the proposed method.
main_review: The paper is well written and easy to follow. Generally, the overall framework is novel. The authors give some theoretical analyses to the proposed transportation-constraint risk minimization. Extensive experiments are conducted to show the effectiveness of the proposed methods. 

My major concerns and questions are as follows:

(1) The motivation is not clear to me. One of the claims of the paper is that the proposed algorithm can address the insufficient overlapping problem. However, the authors do not give a clear motivation of why this problem is important in real-world RL systems and why the proposed methods can address this problem.

(2) The theoretical analyses are not clear and lack interpretations.  Proposition 1 is good to show that IW and DA can not address the insufficient overlapping problem. However, Theorem 1 & 2, Claim 1 and Theorem 3 are not clear.  Specifically,  Theorem 1 and Theorem 2 are only the finite sample bounds under $P$ but not the transfer generalization bound w.r.t the target distribution $Q$ like Proposition 1. It is unclear to me why  Theorem 1 and  Theorem 2 can resolve the insufficient overlapping problem mentioned for using DA in the IR system. For Claim 1, I think the left part in the equation is $\min_{d_{W}(P_{w}, P_{f}) \leq\rho}\mathbb{E}_{P_w} R(f)$ 

not the risk  $\min_{d_{W}(P_{w}, ~P_{f}) \leq \rho}\mathbb{E}_{Q} R(f)$ under the target $Q$. Thus the Claim 1 is also irrelevant to the target risk.  Theorem 3 indeed gives the generalization bound under the target domain, but it does not clearly show the proposed method can address the insufficient overlapping problem of IW and DA in Proposition 1.  Given above, I think the theoretical part is not well written and it would be great if the authors can provide some interpretations of the theoretical results. The presentation in these parts of the paper could be improved. 


(3) The literature review is not very good, and the context is missing about how the research work stands in the agenda of each subfield such as Wasserstein Robust Optimization, Unbiased Recommendation, Off-policy Learning and Causal Inference with Missing data. For instance, the authors miss some important related works [1-7] with deficient support in logging policy and unbiased learning in recommendation. The authors also need to discuss the differences between this paper and other Distributionally robust optimization papers with Wasserstein distance in terms of the generalization bounds. This paper does not make a clear connection with the presented theoretical results and contextualize this work in the existing literature. It is difficult to figure out what's new and what's already a part of the existing literature. 

(4) The experimental comparison is also not sufficient. This paper lacks some baselines and results on benchmark datasets.  The comparison should also be done with some DA [8] and advanced IW methods [2]. The authors also need to evaluate their methods on Yahoo and Coat datasets which are the benchmark datasets for counterfactual learning in the recommendation.  A more detailed analysis of the time complexity would be desirable.

[1] Off-policy bandits with deficient support. KDD 2020.

[2] Doubly Robust Joint Learning for Recommendation on Data Missing Not at Random. ICML 2019.

[3] A General Offline Reinforcement Learning Framework for Interactive Recommendation. AAAI 2021.

[4] Distributionally Robust Counterfactual Risk Minimization. AAAI 2020.

[5] Asymmetric Tri-training for Debiasing Missing-Not-At-Random Explicit Feedback. SIGIR 2020.

[6] Distributionally Robust Policy Evaluation and Learning in Offline Contextual Bandits. ICML 2020.

[7] A General Knowledge Distillation Framework for Counterfactual Recommendation via Uniform Data. SIGIR 2020.

[8] Causal Embeddings for Recommendation. RecSys 2018.

summary_of_the_review:  I think this paper is potential. The proposed domain transportation strategy for the recommendation is quite interesting. I suggest the authors focus on this part and give more discussions about the motivation, interpretations, and related works.
review_title: null

Meta-review: This paper presented a domain transportation perspective on optimizing recommender systems. The basic motivation is to view recommendation as applying some form of intervention, implying a distributional shift after the recommendation/intervention. Distribution shift brings tremendous difficulty to traditional causal inference or missing data theory perspective of recommender systems as it violates the distributional overlapping assumption: in simple terms, if the model recommends radically different set of items, there isn't much you can say about its generalization ability; on the other hand, if the model only recommends items that it already observed during training (no distribution shift at all), it would inherent all the biases which already exist in the data. To that end, this paper proposed a domain transportation perspective by introducing a Wasserstein distance constrained risk minimization to find interventions that can best transport the patterns it learns from the observed domain to the post-intervention domain.

The paper received overall borderline scores. All the reviewers acknowledged that the proposed perspective is novel and has the potential to spark a new direction for future work. The reviewers raised concerns, ranging from the bounds in the paper, sensitivity of the optimization w.r.t. the hyperparameter, to some relevant but missing baselines. The authors provided very detailed response and revised the paper quite substantially to address most of the feedback. I also read the paper myself given the borderline scores, and I think the authors did a reasonably good job improving the paper and I agree this paper provides an interesting and novel perspective on viewing recommendation, though I also agree with one reviewer that the idea of "partially extrapolation" can be further explored. 

My major complaint is around experimental evaluation. It seems to me that only the semi-synthetic experiment actually makes sense in this context (where the measure is based on the unobserved relevance as opposed to observed click), as the traditional random-split-on-clicks evaluation would inevitably favor models with little distributional shift (the training and test data essentially come from the same distribution, maybe not so with a sequential setting but still close). Furthermore, the inclusion of Yahoo R3 and Coat dataset is even more confusing, as the associated test set implies random exposure which is certainly not what this paper aims to address, unless I am missing something in which case more clarification would be nice. 

My overall assessment of the paper is still leaning towards positive but I also wouldn't be too upset if this paper doesn't end up making it. However, if accepted, I do want the authors to carefully revise the presentation of the experimental results for the final version.
Decision: Accept (Poster)