StableDR: Stabilized Doubly Robust Learning for Recommendation on Data Missing Not at Random
Authors: Haoxuan Li
Chunyuan Zheng
Peng Wu
Venue: ICLR 2023 poster (2023)
Keywords: Recommender System
Bias
Debias
Doubly Robust

Abstract: In recommender systems, users always choose the favorite items to rate, which leads to data missing not at random and poses a great challenge for unbiased evaluation and learning of prediction models. Currently, the doubly robust (DR) methods have been widely studied and demonstrate superior performance. However, in this paper, we show that DR methods are unstable and have unbounded bias, variance, and generalization bounds to extremely small propensities. Moreover, the fact that DR relies more on extrapolation will lead to suboptimal performance. To address the above limitations while retaining double robustness, we propose a stabilized doubly robust (StableDR) learning approach with a weaker reliance on extrapolation. Theoretical analysis shows that StableDR has bounded bias, variance, and generalization error bound simultaneously under inaccurate imputed errors and arbitrarily small propensities. In addition, we propose a novel learning approach for StableDR that updates the imputation, propensity, and prediction models cyclically, achieving more stable and accurate predictions. Extensive experiments show that our approaches significantly outperform the existing methods.

Published as a conference paper at ICLR 2023
STABLEDR: STABILIZED DOUBLY ROBUST LEARNING FOR RECOMMENDATION ON DATA MISSING NOT AT RANDOM
Haoxuan Li
Chunyuan Zheng*
Peng Wu
Pekinn University University of California, San Diego BBeijing Technology and Business University hxliestur pku. edu.cn, czheng@ucsd.
pengwu@btbu.edu.co
ABSTRACT
In recommender systems, users always choose the favorite items to rate, which leads to data missing not at random and poses a great challenge for unbiased eval- uation and learning of prediction models. Currently, the doubly robust (DR) meth- ods have been widely studied and demonstrate superior performance. However, in this paper, we show that DR methods are unstable and have unbounded bias, variance, and generalization bounds to extremely small propensities. Moreover, the fact that DR relies more on extrapolation will lead to suboptimal performance. propose a
To address the above limitations while retaining double robustness, stabilized doubly robust (StableDR) learning approach with a weaker reliance on extrapolation. Theoretical analysis shows that StableDR has bounded bias, vari- ance, and generalization error bound simultaneously under inaccurate imputed errors and arbitrarily small propensities. In addition, propose novel learn- ing approach for StableDR that updates the imputation, propensity, and prediction models cyclically, achieving more stable and accurate predictions. Extensive ex- periments show that our approaches significantly outperform the existing methods.
INTRODUCTION
Modern recommender systems (RSs) are rapidly evolving with the adoption of sophisticated deep learning models (Zhang al. 2019). However, it is well documented that directly using advanced deep models usually achieves sub-optimal performance due to the existence of various biases in RS (Chen al. 2020 Wu al. 20226), and the biases would be amplified over time (Mansoury ct al. 2020; Wen et al. 2022). A large number of debiasing methods have emerged and gradually become trend. For many practical tasks in RS, such as rating prediction (Schnabel et al. 2016 Wang et al. 2020a 2019), view click-through rate prediction (Guo et al. 2021). post-click con-
version rate prediction (Zhang al. 2020 Dai al. 2022). and uplift modeling (Saito al.) 2019
Sato et al. 2019 2020), critical challenge combat the selection bias and confounding bias that
leading to significantly difference between the trained sample and the targeted population (Hernán
& Robins 2020). Various methods were designed to address this problem and among them, doubly robust (DR) methods (Wang et al. 2019 Zhang et al.) 2020: Chen et al.) 202 Dai et al. 2022: Ding
ct al. 2022 play the dominant role due to their better performance and theoretical properties.
The success of DR is attributed to its double robustness and joint-learning technique. However, the DR methods still have many limitations. Theoretical analysis shows that inverse probability scoring (IPS) and DR methods may have infinite bias, variance, and generalization error bounds, in the presence of extremely small propensity scores (Schnabel et al. 2016) |Wan al. 2019 Guo et al. 2202]] et al.] 2023b). In addition, due to the fact that users are more inclined to eva uate the preferred items, the problem of data missing not at random (MNAR) often occurs in RS. This would cause selection bias and results in inaccuracy for methods that more rely on extrapolation, such as Steck 2013) and DR methods.
error imputation based (EIB) MMrlinn 2007
"Corresponding author.

---

Published as a conference paper at ICLR 2023
Imputation
LEIR
Prediction Model
Pn
Prediction Model
Model
Imputation Model
Prediction Model
SSR
4
Propensity
Les
Prediction Model
Propensity Model
Stabilizzation
Imputation Model
Model
L Doubly robust joint double learning.
(a) Two-phase learning using single model.
(6) Proposed eyele learning with stabilization.
Figure 1: During the training of updating a prediction model, two-phase learning (Marlin et al, 2007| Steck 2013 Schnabel et al. (016) uses fixed imputation/propensity model (Left), whereas DR-JL (Wagg et al. 2019), MRDR-DL. (Guo et al. 202T), and AutoDebias (Chen et al., 2021) uses alternative learning between the imputation/ppropenityy and the prediction model (Middle). The
proposed learning approach updates the three models cyclically with stabilization (Right).
To overcome the above limitations while maintaining double robustness, we propose a stabilized doubly robust (SDR) estimator with weaker reliance extrapolation, which reduces the negative impact of extrapolation and MNAR effect on the imputation model. Through theoretical analysis, demonstrate that the SDR has bounded bias and generalization error bound for arbitrarily small propensities, which further indicates that the SDR can achieve more stable predictions.
Furthermore, we propose novel cycle learning approach for SDR. Figure I shows the differences between the proposed cycle learning of SDR and the existing unbiased learning approaches. Two- phase learning (Marlin et al. 2007 Steck 2013 SSchnabll et al.] 2016) first obtains an imputa- tion/propensity model to estimate the ideal loss and then updates the prediction model by minimizing the estimated loss. DR-JL (Wang et al. 2019), MRDR-DL. (Guo et al. (02T), and AutoDebias (Chen) al. (021) alternatively update the model used to estimate the ideal loss and the prediction model. The proposed learning method cyclically uses different losses to update the three models with the aim of achieving more stable and accurate prediction results. We have conducted extensive exper- iments on two real-world datasets, and the results show that the proposed approach significantly improves debiasing and convergence performance compared to the existing methods.
2 PRELIMINARIES
2.1 PROBLEM SETTING
In RS, due to the fact that users are more inclined to evaluate the preferred items, the collected ratings are always missing not random (MNAR). We formulate the data MNAR problem using the widely adopted potential outcome framework (Neyman) [9900] Imbens & Rubin 2015). Let u = 21, U}, I (1,2,..., 7) and D = 4 I be the index sets of users, items, all user-item pairs. For each (u, i) E D. we have a treatment oa, E {0, 1}. a feature vector Fu.i, and an observed rating Tu,i, where Ou,i
if user rated the item i in the logging data, Ou.i 0 if the rating is missing, Let (1) is defined as the be the rating that would be observed if item i had been rated by user u, which is observable only for O = {(u, i) (u, i) D,o,,, 1}. Many tasks in RS can be formulated by predicting the potential outcome Ta, (1) using feature Tu, for each (u,
Let (1) f( σ) be a prediction model with parameters Φ. If all the potential outcomes (Ta.(L): (u, i) E D} were observed, the ideal loss function for solving parameters @ is given as
(o)
(DI-1
Σ Cu,i,
(a,))DD
where e is the prediction error, such as the squared loss Gu,i = (f.))))))))) Lideat()) can be regarded as benchmark of unbiased loss function, even though it is infeasible due to the miss- ingness of (1).: ou,; = 0}. As such, a variety of methods are developed through approximating Lideat(() to address the selection bias, in which the propensity- -hased estimators show the relatively superior performance (Schnabel et al. 2016 Wang et al. (201)), and the IPS and DR estimators are
Eips -D
Cu and EDR [D)-1
Σ (u,))D
(euss + Ou.i - Ev,i)
Pu.i
Pu,i
(D,))ED
where Pu,i is estimate of propensity score Pu,i P(ooui 1/xxx)) lu, is an estimate of ex.i.
2

---

Published as a conference paper at ICLR 2023
2.2 RELATED WORK
Debiased learning in recommendation. The data collected in RS suffers from various biases (Chen al. 2020 Wu al. 2022b). which are entangled with the true preferences of users and pose great challenge to unbiased learning. There is increasing interest in coping with different biases in recent years (Zhang al. 2021 Ai et al., 2018 Liu al. 2016 Liu et al. R0.T). |Schnabel et al. (2016) proposed using inverse propensity score (IPS) and self-normalizee
IPS (SNIPS) methods to address the selection bias on data missing not at random, Saito (2019) and Saito et al. (2020) extended them to implicit feedback data. Marlin et al. (2007) and Steck (2013) derived error imputation-basee (EIB) unbiased learning method. These three approaches adopt two-phase learning (Wang et al. 2021. which first learns propensity/imputaion model and then applies it to construct an unbi- ased estimator of the ideal loss to train the recommendation model. A doubly robust joint learning (DR-JL.) method (Wang et al. 2019) was proposed by combining the IPS and EIB approaches. Sub- sequently, strands of enhanced joint learning methods were developed, including MRDR (Guo et al.) 2021 Multi-task DR (Zhang et al., 2020), DR-MSE (Dai et al. 2002)) BRD-DR (Ding et al. 2022). TDR Li ct al. (023b), uniform data-aware methods (Bonner & Vasile 2018 Liu et al. 2020 Chen
ct al. 2021 Wang et. al. 2021 Li et al. 2023c) that aimed to seek better recommendation strategies by leveraging a small uniform dataset, and multiple robust method (Li et al.
2023a) that specifies multiple propensity and imputation models and achieves unbiased learning if any of the propensity models, imputation models, or even linear combination of these models can accurately estimate the true propensities or prediction errors. Chen et al. (2020) reviewed various biases in RS and discussed the recent progress on debiasing tasks. |Wu et al. (2022b) established the connections between the biases in causal inference and the biases, thereby presenting the formal causal definitions for RS.
Stabilized causal effect estimation. The proposed method builds on the stabilized average causal effect estimation approaches in causal inference. Molenberghs et al. (2015) summarized the limita- tions of doubly robust methods, including unstable to small propensities (Kang & Schafer 2007 Wu et al. 2022a). unboundedness (van der Laan & Rose, [001]) and large variance [Tan] 2007)) These issues inspired a series of stabilized causal effect estimation methods in statisties (Kang & Schafer) 2007 [Bng & Robins, 2005; van der Laan & Rose| 2011] Molenberghs et al.) 2013). Unlike previ-
ous works that focused only on achieving learning with unbiasedness in RS, this paper provides a new perspective to develop doubly robust estimators with much more stable statistical properties.
STABILIZED DOUBLY ROBUST ESTIMATOR
In this section, elaborate the limitations of DR methods and propose a stabilized DR (SDR) estimator with a weaker reliance on extrapolation. Theoretical analysis shows that SDR has bounded bias and generalization error bound for arbitrarily small propensities, while IPS and DR don't.
3.1 MOTIVATION
Even though DR estimator has double robustness property, its performance could be significantly improved if the following three stabilization aspects can be enhanced.
More stable to small propensities. As shown in Schnabel al. (2016). Wang al. (2019) and Guo et al. (202T). if there exist some extremely small estimated propensity scores, the IPS/DR estimator and its bias, variance, and tail bound are unbounded, deteriorating the prediction accuracy. What's more, such problems are widespread in practice, given the fact that there are many long-tailed users and items in RS, resulting in the presence of extreme propensities.
More stable through weakening extrapolation. DR relies more on extrapolation because the imputation model in DR learned from the exposed events O and extrapolated to the unexposed events. If the distributional disparity of e ou, 0 and o.. 1 is large, the imputed errors likely to be inaccurate on the unexposed events and incur bias of DR. Therefore, is beneficial to reduce bias if we can develop an enhanced DR method with weaker reliance on extrapolation.
More stable training process of updating a prediction model. In general, alternating training be- tween models results in better performance. From Figure [ Wang et al.) (011)) proposes joint learn ing for DR, alternatively updating the error imputation and prediction models with given estimated propensities. Double learning (Guo et al. 2021) further incorporates parameter sharing between the
3

---

Published as a conference paper at ICLR 2023
imputation and prediction models. Bi-level optimization (Wang et al. 2021 Chen et al. 2021 can be viewed as alternately updating the prediction model and the other parameters used to estimate the loss. To the best of our knowledge, this is the first paper that proposes a algorithm the three models (i.e., error imputation model, propensity model, and prediction model) separately using different optimizers, which may resulting more stable and accurate rating predictions.
update
3.2 STABILIZED DOUBLY ROBUST ESTIMATOR
We propose stabilized doubly robust (SDR) estimator that has weaker dependence on extrapola- tion and is robust to small propensities. The SDR estimator consists of the following three steps.
Step 1
(Initialize imputed errors). Pre-train imputation model E,,5, let & - ID E(WnED Expi
Step 2 (Learn constrained propensities). Learn a propensity model Pu,e satisfying
1
Ou,
(uui 8) 0.
D
Pu,i
(u,i)ED
Step 3 (SDR estimator). The SDR estimator is given as
ESDR
Σ
Ou,iCu,s
Σ
Oui
Σ
Wu,iCa,i
Puçi
Pu.i
(u,i)ED
(a.))ED
(u,J)ED
Ou.i It can be seen that SDR estimator has the same form as SNIPS estimator, but the propensities learned differently. In SDR, the estimation of propensity model relies on the imputed errors, whereas not in SNIPS.
Ou.i where Wa,i Pa.: E(.sjeD Pu.:"
Each step in the construction of SDR estimator plays different role. Specifically, the Step 2 is designed to enable double robustness property as shown in Theorem [seee Appendix(A.T]6 proofs).
Theorem (Double Robustness). ESDR is an asymptotically unbiased estimator of Lideat, when either the learned propensities Pu,s the imputed errors e., accurate for all user-item pairs.
We provide an intuitive way to illustrate the rationale of SDR. On the one hand, if the propensities can be accurately estimated (i.e., Pu,i Pu,s) by using common model (c.g.. logistic regression) Then the expectation of the left hand side of constraint (T) becomes
without imposing constraint ①
Eo
Σ
(Bu.t- Ou.e
Σ (u,iED
(c) =0
D
Pu,s
D
(u.))ED
which indicates the constraint I always holds the sample size goes to infinity by the strong law large 2 irrespective of the accuracy of the imputed errors & This implies that the constraint (T) imposes almost restriction the estimation of propensities. In this case, the SDR estimator is almost equivalent to the original SNIPS estimator. On the other hand, the propensities cannot be accurately estimated by using a common model, but the imputed enrors accurate (i.e., ε, eu,i). In this case, E is an unbiased estimator. Specifically, ESDR satisfies
Ow.ileu,s ESDR)
0.
D
Pu.i
Combining the constraint T and equation 2 gives
Ou.ileu. - Ema)
on.lEE ESDR)
0,
(3)
D
Du.i
Du,i
(M.))ED
where the first term equals 0 if é.,; Cu,i. it implies that Espr E, then the unbiasedness of ESdr follows immediately from the unbiasedness of É.
Assymttotccllyy unbiased means unbiasedness as the sample size goes to infinity. This the reason why adopt the notation of "asymptotically unbiased".
4

---

Published as a conference paper at ICLR 2023
In addition, Step is designed for two main reasons achieve stability. First, ESDR is more robust to extrapolation compared with DR. This is because the propensities are learned from the entire data and thus have less requirement on extrapolation. Second, Espn is more stable to small propensities, since the seff-normalization imposes the weight Wa,i to fall on the interval [0.1].
In summary, forcing the propensities to satisfy the constraint 1 makes the SDR estimator not only doubly robust, but also captures the advantages of both SNIPS and DR estimators. The design of SDR enables the constrained propensities adaptively find the direction of debiasing if either the learned propensities without imposing constraint or the imputed errors are accurate.
3.3 THEORETICAL ANALYSIS STABLENESS
Through theoretical analysis, we note that previous debiasing estimators such as IPS (Schnabel et al., 2016) and DR-based methods (Wang et al. 2019; Guo et al.) 2021) tend to have infinite biases, variances, tail bound, and corresponding generalization error bounds, in the presence of extremely small estimated propensities. Remarkably, the proposed SDR estimator doesn't it suffer from such problems and is stable arbitrarily small propensities, shown in the following Theorems (see Appendixes A.2 A.3 and A.4 for proofs).
Theorem (Bias SDR). Given imputed errors ēus and learned propensities Pu, satisfying the stabilization constraint with Pu, 0 for all user-item pairs, the bias of EsDa is
E(U.DED SunPus/Purt 8u,
Bias(E =
Σ
EjuseD Puji/Dui PusiepPuss/bus
+O(1D)))
D
(M,))D
where 6,, = Cu,s - Cus is the error deviation.
Cheorem@showw the bias of the SDR estimator consisting of dominant term given by the difference between du,i and its weighted average, and negligible term of order 0(12)-1). The fact that the ou,i and its convex combinations are bounded, shows that the bias is bounded for arbitrarily small Pa,i. Compared to the Bias (Errs) |DI Σn,ed (Pu,t Pus)euu s/.aal and Bias (Enn) =
I (Pu,i - Pu,))u,,/Pu.. it indicates that IPS and DR will have extremely large bias when there exists an extremely small pu.i.
Theorem (Variance of SDR). Under the conditions of Theorem 2 the variance of EsDR is
Var (ESDR) = E(G.))DD Pu. Ph, /P.
SuuijEDPu.1/PP 2 + +O(DD)3)
(Pu,s/uwkk is a bounded where hu, = (Cu,i - eu,)) E (EDD) (Pusleu, - eu.)) P/, // (I.)) difference between Cu,b ēu,s and its weighted average.
Theorem 3 shows the variance of the SDR estimator consisting of dominant term and neg- ligible term of order - The boundedness of the variance for arbitrarily small Pa. given directly from the fact that SDR has bounded range given by the self-normalized form. Compared to the Var (EIPs) |D] Eu,ep P1, Pe, Ph, and Var (EdR)
"DI'Z EntiE Pu.; (1 (eu.) - eu. / it indicates that IPS and DR will have extremely large variance (tend to infinity) when there exist extremely small Pu,, (tends to 0).
Theorem 4 (Tail Bound of SDR). Under the conditions of Theorem [ for any prediction model, with probability TI, the deviation of Espa from its expectation has the following tail bound
|ESDR Eo(ESDR)
S
1
log
4 17
Σ
(6max 5,,)) + (8u,5 8min) (1 (ED Pu,s/Putt - e¹)32
2
(u,))D
where 8min min (a.()D Ju 8max (ax du e'= 1og(4/nn)/ EEDr(a))) 1/F2, and D D (u,i) is the set of 'D excluding the element (u, i).
5

---

Published as a conference paper at ICLR 2023
Note that Epi(uN)) Pu.s/u.: O(DD) and é' (22) in Theorem it follows that the tail bound of the SDR estimator converges to 0 for large samples. In addition, the tail bound is bounded for arbitrarity small Pa,s. Compared to the tail bound of IPS and DR, with probability 1 - 7, we have
|EiPs- Bo [Erpø]]
log (2/n)
Cu, Pu,i
|Enn Eo [EnR]l s
log (2/m) 2|1
Σ
8us Pu,s
2/2/2
(K,))D
(M,))D
which are both unbounded when Pu.i - O. For SDR in the prediction model training phase, the boundedness of the generalization error bound (see Theorem [5] in Appendix [A.5) follows imme- diately from the boundedness of the bias and tail bound. The above analysis demonstrates that SDR can comprehensivelyy mitigate the negative effects caused by extreme propensities and results in more stable predictions. Theorems 2 are stated under the constraint (T). If we estimate the propensities with constraint (I). but finally constraint (1) somehow doesn" 't hold exactly, the associ- ated bias, variance, and generalization error bound of SDR are presented in Appendix B
4 CYCLE LEARNING WITH STABILIZATION
In this section, we propose a novel SDR-based cycle learning approach, that not only exploits the stable statistical properties of the SDR estimator itself, but also carefully designs the updating pro- cess among various models to achieve higher stability. In general, inspired by the idea of value iteration in reinforcement learning (Sutton & Barto 2018), alternatively updating the model tends to achieve better predictive performance, as existing debiasing training approaches suggested (Wang) iet al., 2019 Guo et al. 202I, Chen et al.) 2021]. As shown in Figure |I the proposed approach dynamically interacts with three models, utilizing the propensity model and imputation model si- multancously in a differentiated way, which can be regarded as an extension of these methods. In eycle learning, given pre-trained propensities, the inverse propensity weighted imputation error loss is used to first obtain an imputation model, and then take the constraint (I) as the regularization term train stabilized propensity model and ensure the double robustness of SDR. Finally, the prediction model is updated by minimizing the SDR loss and used to readjust the imputed errors. By repeating the above update processes cyclically, the cycle learning approach can fully utilize and combine the advantages of the three models to achieve more accurate rating predictions.
Specifically, the data MNAR leads to the presence of missing i so that all ea,i cannot be used directly. Therefore, we obtain imputed errors by learning pseudo-labeling model (1) parameterized by 3, and the imputed errors é = CC((f,, (1)) are updated by minimizing
Cun - en,i) ((F,,ii))
Le (0, a, = |DI-1
Σ (u,1)ED
+ 1e113111
where eu,i = (L). Xe 2. Pu.; = T(Zu, is the propensity model, || ||2 is the Frobenius norm. For each observed ratings, the inverse of the estimated propensities are used for weighting to account for MNAR effects. Next, we consider two methods for estimating propensity scores, which Naive Bayes with Laplace smoothing and logistic regression. The former provides a wide range of opportunities for achieving stability constraint (I) through the selection of smoothing coefficients. The latter requires user and item embeddings. which are obtained by employing MF before performing cycle learning. The learned propensities need to both satisfy the accuracy, which evaluated with cross entropy, and meet the constraint I for stabilization and double robustness. The propensity model π (T a) is updated by using the loss Lee (Φ, ar (3) + n Estable (ø, at 99, where Loe (Ø, ar, () cross entropy loss of propensity model and
Letable (0, a, B) [DI-
Σ (u,))D
On.
(Bu.s E))3 Aetabtellaalf..
π (Tu, a)
where Astable 2 0. and is a ayper--arameter for trade-off. Finally, the prediction model fz Φ) updated by minimizing the SDR loss
Lodr (0, a, B)
Σ (a.J)D
Ou,iCu,s 7 ( a))
ou. T(Eutia)) + dedr-lob1?,
(u,))D

---

Published as a conference paper at ICLR 2023
Algorithm 1: The Proposed Stable DR (MRDR) Cycle Learning, Stable-DR (MRDR)
Input:
observed ratings Y and v7, Xe: Astable; Asar while stopping criteria is not satisfied do for number of steps for training the imputation model do
Sample batch of user-item pairs {[ug, 82) & from O; Update 3 by descending along the gradient V VaCe (Ø, a, B); end
for number of steps for training the propensity model do
Sample a batch of user-item pairs {(ux, ix) )== from D:
Calculate the gradient of propensity cross entropy error V. Coe a0,
Calculate the gradient of propensity stable constraint (1) V Estabble (0, (,,a, a,,
10
Update by descending along the gradient VaLce (0, a, B) + 11 VaLstable (Ø, a, B)
end
12
for number of steps for training the prediction model do
Sample batch of user-item pairs {w, i from O: end
13
Update by descending along the gradient V V&Ladr (6. a. B);
14
15
end
where the first term is equivalent to the left hand side of equation 33. and Asdr 2 0. In cycle learning, the updated prediction model will be used for re-update the imputation model using the next sample batch. Notably, the designed algorithm strictly follows the proposed SDR estimator in Section 3.2 From Figure ! and Alg. [ our algorithm first updates imputed errors by Step 1, and then learns a propensity p based on learned é to satisfy the constraint (1) in Step 2. The main purpose of the first two steps is to ensure that the SDR estimator in Step 3 has double robustness and has a lower extrapolation dependence compared to the previous DR methods. Finally, from Step 3 we update the predicted rating f by minimizing the estimation of the ideal loss using the proposed SDR estimator. For the next round, instead of re-initializing, Step updates the imputed errors according to the new prediction model, then Step re-updates the constrained propensities p, and then uses Step 3 to update the prediction model f again, and so on. We summarized the cycle learning approach in Alg.
REAL-WORLD EXPERIMENTS
In this section, several experiments are conducted to evaluate the proposed methods on two real- world benchmark datasets. We conduct experiments to answer the following questions:
RQL. Do the proposed Stable-DR and Stable-MRDR approaches improve in debiasing perfor- mance compared to the existing studies?
RQ2. Do our methods stably perform well under the various propensity models?
RQ3. How does the performance of our method change under different strengths of the stabiliza- tion constraint?
5.1 EXPERIMENTAL SETUP
Dataset and preprocessing. To answer the above RQs, we need to use the datasets that contain both MNAR ratings and missing-at- at-radoom (MAR) ratings. Following the previous studies (Schnabel et al., 2016; Wang et al.) 2019: Guo et al. 2021] Chen et al. 2021). we conduct experiments on the two commonly used datasets: Coat contains ratings from 290 users to 300 items. Each user evaluates 24 items, containing 6,960 MNAR ratings in total. Meanwhile, each user evaluates 16 items randomly, which generates 4,640 MAR ratings. Yahoo! contains totally 311,704 MNAR and 54,000 MAR ratings from 15,400 users to 1,000 items. Baselines. In our experiments, take
Matrix Factorization (MF) Koren et al. 2009). Sparse LInear Method (SLIM) (Ning & Karypis,
hhtps://www.cs.come/ www.:eoooooccccaeaee.....e hww.mm//....u/sccahss///aaaaaa hhtpp///wccsccppesaaddii wwwoo..coaaoooo///oooo

---

Published as a conference paper at ICLR 2023
Table Performance on Coat and Yahoo!R3, using MF, SLIM, and NCF as the base models.
Dataset
Coat
Yahoo!R3
Method
MSE
AUC
N@5
@@ 10
MSE
AUC
N@5
N@ 10
MF
0.2428
0.7063
0.6025
0.6774
0.2500
0.6722
0.6374
0.7634
IPS
0.2316
0.7166
0.6184
0,6897
0.2194
0.6742
0.6304
0.7556
SNIPS
0.2333
0.7070
0.6222
0.6851
0.1931
0.6831
0.6348
0.7608
AS-IPS
0.2121
0.7180
0.6160
0.6824
0.2391
0.6770
0,6364
0.7601
CVIB
0,2195
0.7239
0.6285
0.6947
0,2625
0.6853
0.6513
0.7729
DR
0.2298
0.7132
0.6243
0.6918
0.2093
0.6873
0.6574
0.7741
DR-JL
0,2254
0.7209
0.6252
0.6961
0.2194
0.6863
0.6525
0.7701
Stable-DR (Ours)
0.2159
0.7508
0.6511
0.7073
0.2090
0.6946
0.6620
0.7786
MRDR--L
0.2252
0.7318
0.6375
0.6989
0.2173
0.6830
0.6437
0.7652
Stable-MRDR (Ours)
0.2076
0.7548
0.6532
0.7105
0.2081
0.6915
0.6585
0.7757
SLIM
0,2419
0.7074
0.7064
0.7650
0.2126
0.6636
0.7190
0.8134
IPS
0.2411
0.7058
0.7235
0.7644
0.2046
0.6583
0.7285
0.8244
SNIPS
0.2420
0.7071
0.7369
0.7672
0.2155
0.6720
0.7303
0.8227
AS-IPS
0,2133
0.7105
0.6238
0.6975
0.1946
0.6769
0.6508
0.7702
CVIB
0,2413
0.7108
0.7214
0.7638
0.2024
0.6790
0.7335
0.8221
DR
0,2334
0.7064
0.7267
0.7649
0.2054
0.6771
0.7344
0.8248
DR-JL
0,2407
0.7090
0.7279
0.7655
0.2044
0.6792
0.7360
0.8260
Stable-DR (Ours)
0,2356
0.7201
0.7389
0.7724
0,2080
0.6874
0,7473
0.8349
MRDR-JL
0,2409
0.7074
0.7329
0.7679
0.2016
0.6791
0.7338
0.8239
StaDle-MRDR (Ours)
0.2369
0.7148
0.7378
0.7711
0,2086
0.6842
0.7435
0.8308
NCF
0,2050
0.7670
0.6228
0.6954
0.3215
0.6782
0.6501
0.7672
+ IPS
0,2042
0.7646
0.6327
0.7054
0.1777
0.6719
0.6548
0.7703
SNIPS
0.1904
0.7707
0.6271
0.7062
0.1694
0.6903
0.6693
0.7807
AS-IPS IPS
0.2061
0.7630
0.6156
0.6983
0.1715
0.6879
0.6620
0.7769
CVIB
0.2042
0.7655
0.6176
0.6946
0.3088
0.6715
0.6669
0.7793
DR
0,2081
0.7578
0.6119
0.6900
0.1705
0.6886
0.6628
0.7768
DR-JL
0,2115
0.7600
0.6272
0.6967
0.2452
0.6818
0.6516
0.7678
Stable-DR (Ours)
0.1896
0.7712
0.6337
0.7095
0.1664
0.6907
0.6756
0.7861
-RRD--L
0,2046
0.7609
0.6182
0.6992
0.2367
0.6778
0.6465
0.7664
Stable-MRDR (Ours)
0.1899
0.7710
0.6380
0.7082
0.1671
0.6910
0.6734
0.7846
2011 J. and Neural Collaborative Filtering (NCF) (He et al. 2017) as the base model respectively,
and compare against the proposed methods with the following baselines: Base Model, IPS (Saito jtt al. 2020 (chhabel et al. (016)) SNIPS (Swaminathan Joachims 2015), IPS-AT (Saito) 2020). CVIB (Wang et al. (0200)). DR (Saito) 2020), DR-JI. (Wang et al. 2019), and MRDR-JI (Guo cct al. 2021). In addition, Naive Bayes with Laplace smoothing and logistic regression are used to establish the propensity model respectively.
Experimental protocols and details. The following four metrics are used simultaneously in the evaluation of debiasing performance: MSE, AUC, NDCG@5, and NDCG@ 10. All the experi- ments are implemented on PyTorch with Adam as the optimizer We tune the learning rate in (0.005, 0.01, 0.05, 0.1). weight decay in {le 6, 5e 6. 5e 3, le 122)), 2), constrain parameter eta in (50, 100, 150, 200) for Coat and (500, 1000, 1500, 2000) for Yahoo! R3, and batch size in {128, 256, 512, 1024, 2048} for Coat and (1024, 2048, 4096, 8192, 16384) for Yahoo! R3. In ad- dition, for the Laplacian smooth parameter in Naive Bayes model, the initial value is set to and the learning rate tuned (5, 10, 15, 203 for Coat and in (50, 100, 150, 2003 for Yahoo! R3.
5.2 PERFORMANCE COMPARISON (RQ1)
Table summarizes the performance of the proposed Stable-DR and Stable-MRDR methods com- pared with previous methods. First, the mausaly-inspired methods perform better than the base model, verifying the necessity of handling the selection bias in rating prediction. For previous meth-
For all experiments, we NVIDIA GeForce RTX 3090 as the computing resource.
8

---

Published as a conference paper at ICLR 2023
DK-IL SLAL-UR
NAUR-LL StAle-MOUK
DX-IK 3000K-1A
DRAIL StAA--O
NRUK-
DR-IK SIUD-IK
SeefeAKKK
0.76
6.00%
-8.00%
0.23
4.00%
-6.00%
0.74
MSE
0.22
-4.00%
IR
AUC
0.72
2.00%
IR
0.00%
2.00%
0.20
0.70
-2.00%
-0.00%
0.19
0.68
4.00%
150
2.00
100
150
200
Stabilization hyper-parameter
Stabilization Hyyer-paramette
6.00%
4.00%
0.230
3.00%
-4.00%
MSE
0.225
-2.00%
IR
AUC
0.72
2.00%
0.220
1.00%
0.215
-0.00%
0.210
-1.00%
0
50
100
150
200
0
50
100
150
200
Stabilization hyper-paramcter 1
Stabilizattion hyper-paramcter y
Figure 2: MSE, AUC and Increasing Ratio (IR) of Stable-DR and Stable-MRDR comparing with two baseline algorithms DR-JL and MRDR-JL in two different propensity model setting: Naive Bayes with Laplace smoothing (Top) and logistic regression (Bottom) respectively.
ods, SNIPS, CVIB and DR demonstrate competitive performance. Second, the proposed Stable-DR and Stable-MRDR have the best performance in all four metrics. On one hand, our methods outper- form SNIPS, attributed to the inclusion of the propensity model in the training process, well as the boundedness and double robustness of SDR. On the other hand, our methods outperform DR-JL and MRDR-JL, attributed to the stabilization constraint introduced in the training of the propensity model. This further demonstrates the benefit of cycle learning, in which the propensity model is acted as the mediation between the imputation and prediction model during the training process, rather than updating the prediction model from the imputation model directly.
5.3 ABLATION AND PARAMETER SENSITIVITY STUDY (RQ2, RQ3)
The debiasing performance under different stabilization constraint strength and propensity models shown in Figure 2 First, the proposed Stable-DR DR and StableMRDDR outperform DR-JL and MRDR- JL. when either Naive Bayes with Laplace smoothing or logistic regression is used as propensity models. It indicates that methods have better debiasing ability in both the feature containing and collaborative filtering scenarios. Second, when the strength of the stabilization constraint is zero, our method performs similarly SNIPS and slightly worse than the DR-JL and MRDR-JL, which indicates that simply using cross-entropy loss to update the propensity model is not effective in im- proving the model performance. However, as the strength of the stabilization constraint increases, Stable-DR DR and Stable-MRDD using cycle learning have stable and significant improvement com- pared to DR-JL and MRDR-JL. Our methods achieve the optimal performance the appropriate constraint strength, which can be interpreted as simultaneous consideration of accuracy and stability to ensure boundedness and double robustness of SDR.
6 CONCLUSION
In this paper, we propose an SDR estimator for data MNAR that maintains double robustness and improves the stability of DR in the following three aspects: first, we show that SDR has weaker extrapolation dependence than DR and can result in more stable and accurate predictions in the presence of MNAR effects. Next, through theoretical analysis, we show that the proposed SDR has bounded bias, variance, and generalization error bounds under inaccurate imputed errors and arbi- trarily small estimated propensities, while DR does not. Finally, we propose a novel learning ap- proach for SDR that updates the imputation, propensity, and prediction models cyclically, achieving more stable and accurate predictions. Extensive experiments show that our approach significantly outperforms the existing methods in terms of both convergence and prediction accuracy.

---

Published as a conference paper at ICLR 2023
ETHICS STATEMENT
This work is mostly theoretical and experiments are based on synthetic and public datasets. We claim that this work does not present any foresceable negative social impact.
REPRODUCIBILITY STATEMENT
Code is provided in Supplementary Materials to reproduce the experimental results.
ACKNOWLEDGMENTS
The work was supported by the National Key R&D Program of China under Grant No. 2019YFB1705601.
REFERENCES
Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W Bruce Croft. Unbiased learning to rank with unbiased propensity estimation. In SIGIR, 2018.
Heejung Bang and James M. Robins. Doubly robust estimation in missing data and causal inference models. Biometricx, 61:962-972, 2005.
Stephen Bonner and Flavian Vasile. Causal embeddings for recommendation. In RecSys, 2018.
Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng. Meng Wang, and Xiangnan He. Bias and debias in recommender system: survey and future directions. arXiv:2010.03240, 2020.
Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. Autodebias: Learning debias for recommendation. In SIGIR, 2021.
Quanyu Dai, Haoxuan Li, Peng Wu, Zhenhua Dong, Xiao-Hua Zhou, Rui Zhang, Xiuqiang He, Rui Zhang, and Jie Sun. A generalized doubly robust learning framework for debiasing post-click conversion rate prediction. KDD, 2022.
Sihao Ding. Peng Wu, Fuli Feng, Xiangnan He, Yitong Wang, Yong Liao, and Yongdong Zhang- Addressing unmeasured confounder for recommendation with sensitivity analysis. In KDD, 2022.
Siyuan Guo, Lixin Zou, Yiding Liu, Wenwen Ye, Suqi Cheng, Shuaiqiang Wang, Hechang Chen, Dawei Yin, and Yi Chang. Enhanced doubly robust learning for debiasing post-click conversion rate estimation. In SIGIR, 2021.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nic, Xia Hu, and Tat-Seng Chua. Neural collab- orative filtering. In WWW, 2017.
Miguel A. Hernán and James M. Robins. Causal Inference: What If. Boca Raton: Chapman and Hall/CRC, 2020.
Guido W. Imbens and Donald B. Rubin. Causal Inference For Statistics Social and Biomedical Science. Cambridge University Press, 2015.
Joseph D.Y. Kang and Joseph L. Schafer. Demystifying double robustness: comparison of alter- native strategies for estimating a population mean from incomplete data. Statistical Science, 22: 523-539, 2007.
Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, 2009.
Haoxuan Li. Quanyu Dai, Yuru Li, Yan Lyu, Zhenhua Dong, Xiao-Hua Zhou, and Peng Wu. Multi- ple robust learning for recommendation. In AAAI, 2023a.
Haoxuan Li, Yan Lyu, Chunyuan Zheng, and Peng Wu. TDR-CL: Targeted doubly robust collabo- rative learning for debiased recommendations. In ICLR, 2023b.
10

---

Published as a conference paper at ICLR 2023
Haoxuan Li, Yanghao Xiao, Chunyuan Zheng. and Peng Wu. Balancing unobserved confounding with a few unbiased ratings in debiased recommendations. In WWW, 2023c.
Dugang Liu, Pengxiang Cheng, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. A general knowledge distillation framework for counterfactual recommendation via uniform data. In SIGIR, 2020.
Dugang Liu, Pengxiang Cheng, Hong Zhu, Zhenhua Dong, Xiugiang He, Weike Pan, and Zhong Ming. Mitigating confounding bias in recommendation via information bottleneck. In RecSys, 2021.
Yiming Liu, Xuezhi Cao, and Yong Yu. Are you influenced by others when rating?: Improve rating prediction by conformity modeling. In RecSys, 2016
Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, and Bamshad Mobasher. Feedback loop and bias amplification in recommender systems. In WWW, 2020.
Benjamin Marlin, Richard S Zemel, Sam Roweis, and Malcolm Slaney, Collaborative filtering and the missing random assumption. UAI, 2007.
Geert Molenberghs, Garrett Fitzmaurice, Michael G. Kenward, Anastasios Tsiatis, and Geert Ver- beke. Handbook of Missing Data Methodology. Chapman & Hall/CRC, 2015.
Jerzy Splawa Neyman. On the application of probability theory to agricultural experiments. essay on principles. section 9, Statistical Science, 5:465 -425, 1990.
Xia Ning and George Karypis. Slim: Sparse linear methods for top-n recommender systems. In 2011 IEEE 11th international conference on data mining, pp. 497-506. IEEE, 2011.
Yuta Saito. Unbiased pairwise learning from implicit feedback. In NewrIPS Workshop, 2019.
Yuta Saito. Asymmetric tri-trainn tri-training for debiasing missing-not aat-ggg-no-------deeo explicit feedback. In SI- GIR, 2020.
Yuta Saito. Doubly robust estimator for ranking metries with post-click conversions. In RecSys, 2020.
Yuta Saito, Hayato Sakata, and Kazuhide Nakata. Doubly robust prediction and evaluation methods improve uplift modeling for observational data. In SIAM, 2019.
Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. Unbiased rec- ommender learning from mossing-not-at-raate
implicit feedback. In WSDM, 2020.
Masahiro Sato, Janmajay Singh, Sho Takemori, Takashi Sonoda, Qian Zhang, and Tomoko Ohkuma. Uplif-based evaluation and optimization of recommenders. In RecSys, 2019.
Masahiro Sato, Sho Takemori, Janmajay Singh, and Tomoko Ohkuma. Unbiased learning for the causal effect of recommendation. In RecSys, 2020.
Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. Recommendations as treatments: Debiasing learning and evaluation. ICML, 2016.
Harald Steck. Evaluation of recommendations: rating predictirdiition and ranking. In RecSys, 2013.
Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
Adith Swaminathan and Thorsten Joachims. The self neff-normalize estimator for counterfactual learn- ing. In NewrIPS, 2015.
Zhiqiang Tan. Comment: understanding or, ps and dr. Statistical Science, 22:560-568. 2007.
Mark J. der Laan and Sherri Rose. Targeted Learning: Causal Inference for Observational and Experimental Data. Springer, 2011.
Xiaojie Wang. Rui Zhang, Yu Sun, and Jianzhong Qi. Doubly robust joint learning for recommen- dation data missing not random. In ICML. 2019.

---

Published as a conference paper at ICLR 2023
Xiaojie Wang. Rui Zhang, Yu Sun, and Jianzhong Qi. Combating selection biases in recommender systems with a few unbiased ratings. In WSDM, 2021.
Yixin Wang, Dawen Liang, Laurent Charlin, and David M. Blei. Causal inference for recommender systems. In RecSys, 2020a.
Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Ercan Kuruoglu, and Yefeng Zheng. Informa- tion theoretic counterfactual learning from missing-not-at-raddon feedback. NeurIPS, 2020b.
Hongyi Wen, Xinyang Yi, Tiansheng Yao, Jiaxi Tang, Lichan Hong, and Ed H. Chi. Distributionally- robust recommendations for improving worst-case user experience. In WwW, 2022.
Peng Wu, Shasha Han, Xingwei Tong, and Runze Li. Propensity score regression for causal infer- with treatment heterogeneity. Statistica Sinica, 2022a.
Peng Wu, Haoxuan Li, Yuhao Deng, Wenjie Hu, Quanyu Dai. Zhenhua Dong, Jie Sun, Rui Zhang, and Xiao-Hua Zhou. On the opportunity of causal learning in recommendation systems: Founda- tion, estimation, prediction and challenges. IJCAI, 2022b.
Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay, Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), 52(1):1-38, 2019.
Wenhao Zhang, Wentian Bao, Xiao Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani. Large-scale causal approaches to debiasing post-click conversion rate estimation with multi-task learning. In WWW, 2020.
Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. Causal intervention for leveraging popularity bias in recommendation. In SIGIR, 2021.
12

---

Published as a conference paper at ICLR 2023
APPENDIX
Throughout, following existing studies (Schnabel et al. 2016 Wang et al. 2019; Guo et al. 2021 Dai et al., 2022), we assume that the indicator matrix contains independent random variables and each Ou,i follows a Bernoulii distribution with probability Pu.i.
A PROOF OF THEOREMS
A.1 PROOF OF THEOREM 1
Proof of Theorem 1. demonstrate the double robustness of the SDR, first note that P (limn +00 EşDR Lideal) = if the learned propensities are accurate (Swaminathan & Joachims 2015), since |D| ((.))eD Ou,i/u,, converges to I almost surely as [D] goes to in- hnnity and IPS unbiased. Besides, the constraint 4 is constructed to ensure the unbiasedness of ESDR if the error imputation model is correctly specified. In fact, ESDR satisfies
Ou,i ESDR)
(4)
Pu,i
(u.i)DD
Combining the constraint
T and equation
gives
ou Ewxi)
ou.lE - ESDR)
0,
D
Puçi
Pu,i
(u,))DD
where the first term equals to when the imputation model is correctly specified, it implies that
ESDR = E, then the unbiasedness of ESDa follows immediately from the unbiasedness of
A.2 PROOF OF THBOREM 2
Proof of Theorem 2. Equation 3 implies that ESDR can be expressed as
Ou,ileu,i Cu,i E)
Ou,i
ESDR
(5)
D
Pur
D
Pa
(u.))DD
(u,))E
For notational simplicity, let
Wy,i
A ou.i Pu,i
and
Vu,i A on,leen Cngi + 8)/
then ESda be written ratio statistic
ESDR D
Σ
Vus /
Σ
Whi - f(ū, w)
D
(u.i)DD
(u,i)ED
where f(u, w) = v/w, 8= |D] "( (a.iee Un,is and |D]-
Σ
(u.jED Wuii
Applying the Taylor expansion around (Nor Mho) IE E.
( yields that
[(6, w) [Mv, + (e pww (ū (m)) + S' (Hhu: Mm) (w mww)
+ 1 { ( )ww (U 2 2 + 25 (Hr: pww (ō - po) (w - -ww + f (w - Hhe)
2
+ R(ūū T)
where R(ö, w) is the remainder term. Note that (He (He, pw) = 0, f (Mvr Pw) 11 and
f (Hos (ww) 2p p.g then taking an expectation on both sides of the Taylor expansion leads
B(7/ū)
Hv. P/v (uw))
Cov(U, w) Var(w) E[R(O,
(Ww)
13

---

Published as a conference paper at ICLR 2023
By some calculations, have Cov(ū, (U, w) o(( Thus, the bias of ESDR is given as
O((D)=1), Var(w) =, ū)]
1 [D]
[(u,))D Su.iPu,/Pus.
) ) +0(1D)-)
Bias(EsDR)
8uni
PlaNsEDPus/Du. Pus//ust
(u,i)ED
A.3 PROOF OF THEOREM 3
Proof of Theorem 3. According to the proof Theorem 2, we have
Blo/u] p O(1
(6)
Then the variance of &SDR can be decomposed into
Var(EsDR) = Var(ë/m) E { f [(ūm))}
= E {0/w (000 -0/0 Mo/ + 0(DD
Vi + 1/2 + 0112)-
where 1/ A [ū/ū / pw 1 V/2 -00((11-1) (B(T/)) Mw//ww] Equation 6 implies
that 1/2 = = -2).
Denote f(v, w) = v/w, and apply delta method around (Mv yields that
(E [0]. EE [
to calculate V1
Vi = [ ( ) + fo pw) (6 - um) + (Mes pw) (w um)) +pp( - (M)r
= E [F. Hw) (o um) + ft. (Hber (Hortte (10 - pw) + Op Op( (p(
= 1. (ύ p + 25 (14 (m)) (ō po) I' (H4r pao) (1o - koo)
+ f (pe, Hw) (w0 pw) ) + 0(1
5 (Mo, Var(T) + 2 (Hos pho )oo) Sd (Mo: Hew) Cov w) + I (vo; Mw) Var(w) O(ID)
Note that H' pm)) = 1/ and l (1v, Hww) = - /w/ M Then we have
V1
1
Var(ō) + -kv Cov(U, w)
(ma)²
Var(w) ( ((DI)2
(Hw))
(mw)
(Hw))
)
Var(T) Cov(ū, w)
Var(w)
+O(ID)-2)
(Mww)
(man)
(mw))
E
to
Hw
0(12)-2)
2
E(noyy Pu.i(1 Pu,i) s + +0(D1-
/
E(mnn Pus//
where hu,t (Ea,s - Su,s) - E(u,n) -u.,) Σ E(a.a) s/..) a bounded differ- ence between Cu,i - én,s and its weighted average. The conclusion that the SDR variance bounded for any propensities is given directly by the self-normalized form of SDR, i.e., the bounded range of SDR is [8min, 5max]-
14

---

Published as a conference paper at ICLR 2023
A.4 PROOF OF THEOREM 4
Proof of Theorem 4. The McDiarmid's inequality states that for independent bounded random vari-
ables X1, X2,... X,, where X, X; for all and mapping : Xi x X2 Nr R. Assume there exist constant C1, 02, On such that for all i,
sup
If (..,..., 21-1, 2411," , f (..,.... 2;-1, 24, setl, ... 3n) S Gs.
ML 2y 2 2 ( An
Then, for any O,
2ca
P X ... .x,) - (X,,, ,X,,II 2 2 2xpp
2i=1
In fact, equation (5) implies that the SDR estimator can be written as
εSDR
Σ
Qu.(eu,t Eu,i)
Σ
Out Pu,i
+ E,
Pu.i
(M.))D
(u.))D
denoted as f0111,.....
Ou,i, O,,I). Note that (01,1...... On. ou,1) - (o1.1,.... .r..,),,.))
sup
ou.i
Susi/uui + ED((u.s) Pu,i/Pu
max
if ou,i S (ōmin + 6maxx)
1/Du,i + EDr(u,)) Ou.i/uni Edr(u)) Ou,i/ Ou.i/Pu, Pu,i + Dus/Puni
Senns if ousi > (ōmin + Smaxx) /2,
Ed((u))) / + 1/Du,i
where D \ (u, i) is the set of D excluding the element (u, i).
Next, we focus on analyzing the Ed((w)) Ou.//uu.t The Hoeffding's s inequality states that for independent bounded random variables X1,..., X, that take values in intervals sizes Pi....p with probability and for any >
-26² Σ Pk.
P
Σ
X
E
Xx) 26)
< 22exp exp
For EDi(u.)) / have
26
P Σ (V,u))
Du.i/Pus
Σ Ditu.i)
Pu.i/ 2 2 2exp
Ed((AA) 1/F2,
Setting the last term equals n/2, and solving for gives that with probability at least n//,, the following inequality holds
Σ (V)(
Ou.i/Paft
Σ Pu.i
log
4
A c'
(8)
2
62
Di(u.))
((a,)))
D
Therefore, combining and 8 yields that with probability at least 1 - 17/ 1
If (oI.1,...,.u. Ou,i, .u, - (oi.1..... O'ns ..... ,
sup 01, On. o ou.
2max Sa./Duus + (Ed /i.. - c')mmax 1/Du,i + EDr(,,)) Pus//uui - 6)) EDrr(us) Pu,a/ 2')inn Su.i/uu.
if Suli (8min Smax)) )/2,
Smins
if du,i (Suin 5maxx) 1/2,
EDD(uu)) Pu,i/ El) + 1/Du,i
(Bmax - Sn,)/(1 pu, (E Pas
Pus// c)}}
duti S (8min + 8maxx) /2, if
<
(Bus 6min)/ + Pr. P (D)()))) Puii Prej c')}
if 6 > (8min + 8max) /2
15

---

Published as a conference paper at ICLR 2023
where ou,i Cu,, -u,, is the error deviation, 8min (1,i), ou,i and 6max max(u, Su,i- Invoking McDiarmid's inequality leads to that
P P(ESDRR EE(ESDR)] 2 c)
(6max 8,,1) ED-(uup (m.s) Pei Paia/D..s c²)3
<2exp -e?
Σ
Du,i E
(u,ilidu, Smin tamax
(o., 8min))
{1 + Pu.s 2x (w.6) Pus//uuii -2))²2
(u,s),bu, Amnn thans
-2c²
I exp
E(ans)(0maa )² + (du.; Smin) )2 //111111111111111111 + Pu.i DD-L,, Puss/ - e()}3
Setting the last term equals to n/,,, and solving for complete the proof.
A.5 GENERALIZATION BOUND UNDER INACCURATE MODELS
Theorem 5 (Generalization Bound under Inaccurate Models), For any finite hypothesis space of predictions H = -YY!.... ...., YIHI,, with probability 1 n. the true risk R(Y+) deviates from the SDR estimator with imputed errors Eug and learned propensities Pu,i satisfying the stabilization constraint 7 is bounded by
EL ot Pu. ElwnED Pu.s/uu.
R(Y1) ESDN(Yt) +
Σ
ot
[D]
(u,
Blas Term
(Ōmax 6 (6) 6min))
4//
log
(u {1 + Pu, ED Du.s/Pu,i e²)
Variance Term
where 6 is the error deviation corresponding to the prediction model
(6max 52,)2 + (65 - 8min)²
Y
argmaxy*en
Σ
[1 1D+Du,i(2 Du,i (u, Pu.i/ Pu.i/Pu: Pu,i e²) (u.))ED
Proof of Theurem 5. Proof. Theorem 4 shows that for all predictions Y H, we have
P P(ESDD((Y")) ElESDDA( 2c)
-26
s 2exp
E(un))((Bnax ,2 + (8hh - 8min)2)/( + Pr.))
16

---

Published as a conference paper at ICLR 2023
McDiarmid's inequality and union bound ensures the following uniform convergence results:
P (JESDR(Y)) (Y)) E[ESDR( |1} st) 2 21-1
max YIEH ESDA( E[ESDR [Y")] Se 2 21-1
P
V ESDR((")) E[ESDR((")))
- 26 <1
Y.EH
H2
P ( BLESDE Y 26) 26)<n <1
k=1
[
-2c
2 exp
E(u,)) [6mmax - 81, +6% 8min /21 Pu, (u Pu.//un ct)32
h=1
-2e
< |H|
exx
<1
((u.))((mma - 54 )2 + (5
mmn )2m1 + Pu. (E (u.i) Pa.i/ Pu.i/Pui - e¹}
Solving the last inequality for e, it is concluded that, with probability 1 17. the following inequality holds
4|H] r
(6max 60,)² + (5 mmin) [I Pu, PuriLEDruu Pu.i/ c¹)
B(ESDR(Y - EEsDR(Y)) s log
Σ
(u.))D
Theorem 2 shows that for the optimal prediction model Yt, the following inequality holds
E(ui)ED ot On,iuus/ Dus
R( - EEESDR( Yt) =
Σ
DD]
E(uneED Puji/Dus
(u.i)ED
The stated results can be obtained by adding the two inequalities above.
B FURTHER THEORETICAL ANALYSIS OF SDR
Without loss of generality, we assume
1
Ou, (Co,t E) E)= #
D
(u,))D Pust
In this case, the learned propensities must be inaccurate; otherwise, the constraint ① holds naturally the same size increases. Thus, if the imputed errors are accurate, then Lideat E. By exactly
same arguments of equation 3
we have
Ou.i ( ESDR) X,
D
Pu,s
(u.i)DD
which implies that
ESDR = Lideat / D
Ou,
Pu.i
(u,))DD
This means that the degree of violation of constraint 1
determines the size of the bias of SDR.
Furthermore, we can compute the bias, variance, tail bound, and generalization error bound of SDR. Specifically, if both the learned propensities and imputed errors are inaccurate, constraint (3) does not hold either. Then the bias SDR
Elunee Cu.iPu.i/Pus. Ca.é
)
Bias(Espn) =
O((1)-)),
D
Pu.s/ EtuoeepPus/P..
(u.)ED
17

---

Published as a conference paper at ICLR 2023
the variance of SDR becomes
Var (ESDR) E(un)) Puji(1 - Pu. + +((DD))))
(I Puss/Pus))
Pu, / E(unnED The tail bound of SDR is where ha,s Ea,i L given
[SDA - EE(ESDR) I
log
(+)
4 n7
Σ
(emax - Cu,i)² + (Cuii Cmin) )2
1 + Bu, Pus//uu, cc)]2²
(a,i)ED
where 8min = (in Omax = ea e' v llg(4/n))22 ED((w.)) 1/Fe, and
D' (u, a) is the set of excluding the element (u, t). In addition, we can derive the generation error bound of SDR. Given finite hypothesis space H. of the prediction model, then for any prediction model h E H. with probability - the true risk R(h) deviates from the SDR estimator is bounded by
4/H1
(emux )2,) + (cu, E )2 (1 + Pu, Pu. Pua/Puutt - c¹)}
R(b) S ESDR(h) Bias(Espn)
log(
Σ
(u,j)D
where et is the error deviation corresponding to the prediction model
(Emer e² (E Emin)
h = arg max NEW
Σ
( {1 + Pu (DD (u.)) Pu,s/ Dusi }
18


Reviews:
[{"review_id": "r2XCeUTuvYu", "reviewer": "Reviewer_f6Th", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Existing doubly robust (DR) estimators have multiple limitations, e.g, having an unbounded bias, variance, and generalization bound when propensities are extremely small. To address these limitations, this paper proposes a stabilized doubly robust (StableDR) estimator which has a weaker reliance on extrapolation than the existing DR estimators. The paper demonstrate that the StableDR estimator has a bounded bias and a bounded generalization bound even when propensities are arbitrarily small. Moreover, the paper proposes a novel cycle learning approach for training a prediction model, a propensity model, and an imputation model based on the StableDR estimator. The key idea of the proposed cycle learning approach is cyclically employing different losses to update the parameters of the three models. The paper conducts extensive experiments on two widely-used datasets containing both Missing Not At Random (MNAT) and Missing At Random (MAR) ratings. The experimental results show that the proposed cycle learning approach based on the StableDR estimator significantly outperforms existing approaches in terms of rating prediction.", "summary_of_the_review": "I think this paper is marginally below the acceptance threshold due to limited novelty and some not well-supported arguments.\n", "strength_and_weaknesses": "W1: The paper argues that existing DR estimators have an unbounded bias, variance, and generalization bound given extremely small propensities, while the proposed StableDR estimator does not. It would be better to formulate this argument formally into some lemma or proposition. To argue that the StableDR estimator is more stable than existing DR estimators, it is crucial to demonstrate that the StableDR estimator has a smaller variance than the DR estimators, both theoretically and empirically.\n\nW2: I am concerned about the novelty of the proposed approach. First, the StableDR estimator has the same form as an existing Self-Normalized Inverse-Propensity-Scoring (SNIPS) estimator. The property of the StableDR estimator having a bounded bias, variance, and generalization bound mostly comes from the SNIPS estimator. Second, it is not clear why the paper proposes to train models in a cyclic order of prediction model, imputation model, and propensity model instead of another cyclic order or joint learning of all three models. It is important to add more explanations, justifications, and theoretical or empirical results regarding this aspect.\n\nW3: Looks like the property of double robustness in Theorem 1 does not have the exact same definition as that in existing DR estimators. It is recommended to call out the difference and provide some comparisons between these different definition of being double robust.\n\nS1: The experimental results are quite extensive. The paper uses two representative datasets for recommendation debiasing. The paper compares the proposed approach against a wide variety of existing approaches under various metrics. The paper also present some ablation studies to understand the impacts of different components of the proposed approach.\n\nS2: The presentation of the paper is good and the references by the paper are sufficient.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IAZ5yPPBeYb", "reviewer": "Reviewer_HkXH", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a new Stable Doubly-Robust (SDR) method for learning a recommender system from Missing Not At Random (MNAR) feedback data. As compared to previous efforts on DR learning from MNAR data, current work provides theoretical guarantees of stability, specifically against very small propensities. Previous works like Inverse Propensity Scoring (IPS), and DR are prone to instability when certain items have very low propensities in the log data. Due to the inverse term, a very low propensity can cause very high variance. \n\nThe authors also propose a 'pipeline' for learning, where the imputation model, propensity model, and SDR model are trained serially/cyclically, to ensure stability in the learning process. Theoretical bounds of SDR learning guarantee robustness against small propensities, which is a downside of the existing methods, where small propensities can lead to larger generalization errors (as shown on Page 6).", "summary_of_the_review": "- The authors propose a new Stable Doubly-Robust (SDR) method for learning a recommender system from Missing Not At Random (MNAR) feedback data. \n- The method is proven to be robust under the presence of small propensities, a setting in which the existing methods might fail. \n- Empirical results on popular MNAR benchmarks (Coat and Yahoo! R3) demonstrate the efficacy of the proposed method against strong baselines.  \n\n-- Response to authors ---\n\nThe authors have addressed all the comments and some concerns I had initially. Overall I think this is a very interesting paper with extensive theoretical analysis of the generalization properties of the method. I think this paper would be very relevant to the ML & RecSys community. \n\n ", "strength_and_weaknesses": "- The paper is well-written and easy to understand. The related works are covered in detail and are up-to-date with the latest work in learning from MNAR data. \n- Theorm 4 (Tail Bound of SDR) is convincing to show that the proposed method is robust to small propensities in the data. \n- Given that the overall learning is done in three stages, I was expecting a relatively larger difference in training time, but Table 2 suggests the proposed method has a runtime similar to existing methods, which is impressive, given the stability guarantees of the methods. \n\nSome weaknesses of the paper:\n\n- SDR is asymptotically unbiased (Theorm 1), whereas baselines like IPS are unbiased for any given data size. This means, for smaller datasets, there is no guarantee of unbiasedness. Although the experiments on small datasets like Coat demonstrate superior performance to baselines, there's still there's no guarantee of unbiasedness when the dataset is small. Any comments on that? \n- Since the SDR estimator is similar in form to the SNIPS estimator, it would be interesting to compare the bias-variance of both estimators. Since SNIPS performs some kind of normalization, it should also be robust to small propensities. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "8YIf0swHLJf", "reviewer": "Reviewer_si5q", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "By applying a stabilizing constraint on propensity model learning given an error imputation model, the authors show that the result SNIPS estimator satisfies the double robustness property in Theorem 1. Hence this estimator is termed the SDR estimator. This naturally results in bounded bias (Theorem 2), a tail bound on the deviation (Theorem 4) and a generalization bound (Theorem 5), demonstrating mitigation of the effects caused by small propensities. \nMotivated by these theoretical results, for recommender systems, the authors propose iterative 3-step alternating optimization of the error imputation model, the propensity model and the prediction model. SDR and SMRDR outperform competitors on two public rating datasets using MF and NCF baselines for the prediction model.\n\n", "summary_of_the_review": "Despite the concern regarding general practical applicability mentioned under weaknesses, given the simple elegance of the proposed approach, the strong theoretical foundations and the convincing results, I recommend acceptance of this paper.\n", "strength_and_weaknesses": "Strengths:\n\n1. The proposed approach in Algorithm 1 is simple, well-motivated and backed by strong theoretical results.\n\n2. The experimental section is strong. For instance, ablation studies on the stabilization constraint in Sec 5.3 demonstrate its efficacy despite the use of two different propensity estimators.\n\nWeaknesses:\n\n1. It is not clear why NCF is used as a baseline in Table 1. In \"Are we really making much progress? A worrying analysis of recent neural recommendation approaches\", RecSys 2019, it was shown that the SLIM baseline outperforms NCF.\n\n2. The authors ignore temporal variability in recommender systems and assume that features $x_{ui} $ for a user-item pair are static rather than temporally varying.\nRecurrent Recommender Networks, WSDM 2017, shows that this assumption often does not hold for several real world recommender systems.\n\n3. Typos exist, e.g., in the sentence before Theorem 3, \"when there exist an...\" should be \"when there exists an ... \"\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster