TDR-CL: Targeted Doubly Robust Collaborative Learning for Debiased Recommendations
Authors: Haoxuan Li
Yan Lyu
Chunyuan Zheng
Peng Wu
Venue: ICLR 2023 poster (2023)
Keywords: Recommender System
Bias
Debias
Doubly Robust

Abstract: Bias is a common problem inherent in recommender systems, which is entangled with users' preferences and poses a great challenge to unbiased learning. For debiasing tasks, the doubly robust (DR) method and its variants show superior performance due to the double robustness property, that is, DR is unbiased when either imputed errors or learned propensities are accurate.
However, our theoretical analysis reveals that DR usually has a large variance. Meanwhile, DR would suffer unexpectedly large bias and poor generalization caused by inaccurate imputed errors and learned propensities, which usually occur in practice. In this paper, we propose a principled approach that can effectively reduce the bias and variance simultaneously for existing DR approaches when the error  imputation model is misspecified. In addition, we further propose a novel semi-parametric collaborative learning approach that decomposes imputed errors into parametric and nonparametric parts and updates them collaboratively, resulting in more accurate predictions. Both theoretical analysis and experiments demonstrate the superiority of the proposed methods compared with existing debiasing methods.

Published as a conference paper at ICLR 2023
TDR-CL: TARGETED DOUBLY ROBUST COLLABORA- TIVE LEARNING FOR DEBIASED RECOMMENDATIONS
Haoxuan Li
Yan Lyu'
Chunyuan Zheng
Peng Su
Peking University
University of California, San Diego
Beijing Technology and Business University
[hxli, SyuyanJ@stu, http://suuuuuuuuuue. edu.edu. en, czheng@ucsd edu,, pengwu@btbu. edu.
ABSTRACT
Bias common problem inherent in recommender systems, which is entangled with users' preferences and poses a great challenge to unbiased learning. For debiasing tasks, the doubly robust (DR) method and its variants show superior performance due to the double robustness property. that is, DR is unbiased when either imputed errors or learned propensities are accurate. However, our theoret- ical analysis reveals that DR usually has large variance. Meanwhile, DR would suffer uncxpectedly large bias and poor generalization caused by inaccurate im- puted errors and learned propensities, which usually occur in practice. In this paper, we propose principled approach that can effectively reduce the bias and variance simultaneously for existing DR approaches when the error imputation model is misspecified. In addition, we further propose a novel semi-parametric collaborative learning approach that decomposes imputed errors into parametric and nonparametric parts and updates them collaboratively. resulting in more accu- rate predictions. Both thcoretical analysis and experiments demonstrate the supe- riority of the proposed methods compared with existing debiasing methods.
INTRODUCTION
Addressing various tasks in recommender systems (RSs) with causality- -hased methods has become increasingly popular (Wu et al. 2002b)). Causality-based recommendation has shown its great po- tential in both numeric experiments and theoretical analyses across extensive literature (Chen al.) 2020 Wang et al., 2019). Generally, the basic question faced in RS is that "what would the feed- back be if recommending an item to user", requiring to estimate the causal effect of recom- mendation on user feedback. To answer the question, many methods have been proposed, such inverse propensity score (IPS) (Schnabel al. 2016)) self-normalized inverse propensity score (SNIPS) (Swaminathan & Joachims 2015). error imputation based (EIB) methods (Steck] (000)) and doubly robust (DR) methods (Chen al. 2021 Wang al. 2019 202 Dai et al. 2022 Ding et al., 2022). Among them, the DR method and its variants show superior performance. We compare and evaluate these methods in terms of three desired properties, including doubly robust (Hernán & Robins 2020 Wu et al.] 2022c), robust to small propensities (Rosenbaum) (2220), and low vari- ance Tan 2007). Failing to meet any them may lead to sub-optimal performance (Molenberghs) et al. 2015 van der Laan & Rose 2011). Our theoretical analysis shows that DR has much greater variance and is less robust to small propensities compared to EIB (Kang & Schafer 2007). even though the imputed errors and the learned propensities accurate. Mcanwhile, DR would suffer uncxpectedly large bias and poor generalization caused by inaccurate imputed errors and learned propensities, which usually occur in practice.
In this paper, we first propose a novel targeted doubly robust (TDR) method, that can capture the merits of both DR and EIB effectively, by leveraging the targeted learning technique (van der Laan & Rose 2011 2018). TDR can effectively reduce the bias and variance simultaneously for existing DR approaches when the imputed errors are less accurate. Remarkably, TDR provides model-
"Corresponding author.

---

Published as a conference paper at ICLR 2023
agnostic framework and be assembled into any DR method by updating its enror imputation model, resulting in more accurate predictions.
To further reduce the bias and variance during the training process, propose novel uniform- data-free TDR-based collaborative learning (TDR-CL) approach that decomposes imputed errors into a parametric imputation model part and nonparametric error part, where the latter adaptively rectifies the residual bias of the former. By updating the two parts collaboratively, TDR-CL achieves more accurate and robust prediction. Both theoretical analysis and experiments demonstrate the superiority of TDR and TDR-CL compared with existing methods.
2
PRELIMINARIES
Many debiasing tasks in RS can be formulated using the widely adopted potential outcome frame- work (Neyman 1990, Rubin 1974). Denote U {u). I {i} and D U I as the sets of users, items and user-item pairs, respectively. T.,i, Tu,é, and ou,i be the feature, feedback, and exposure status of user-item pair (u, i), where on 1 or 0 represents whether the item is exposed to or not. Define Tu, (1) as the potential outcome if ou,i had been set to 1, which is observed only when o 1. In RS, we are often interested in answering the causal question: "if recommend products to users, what would be the feedback?"" This question can be formulated as to learn the quantity E (rr,,(1) (1) i.e., requires to predict (1.:(1) using feature Fu,ss where denotes the expectation with respect to the target distribution P. Many classical tasks in RS can be defined as estimating this quantity, such as rating prediction (Schnabel et al. 2016) and post-click conversion rate prediction (Guo et al., 202T). More examples can be found in Wu et al. (022b))
Let fo(T...) )) be model used to predict ru,+(1) with parameter 0. Ideally, if all (1,((1) for (u, i) E D were observed, 0 can be trained directly by optimizing the following ideal loss
Lideal (DI-1
Σ Cu,i
(u,))D
where Ca.; is the prediction error, e.g., the squared loss Cu,i (ru,: (o. However, since (1) is observed only when Ou,i = 1, the ideal loss is non-computable. Restricting the analysis to non-missing data will obtain biased conclusions, the observed data may form unrepresenta- tive sample the target population. Different debiasing methods are designed to approximate and substitute the ideal loss. For example, the IPS and EIB estimators given
Lirs = |DI-
Σ (N.J)D
Ou.iRas/Pu.s LEIB = |DI-1
Σ (u,j)ED
(ou + (1- Ou,)
where Pu,s is an estimate of propensity score Pu,i = P(on,i 1/2n,)), Cu, is an estimate of prediction error Ju,i = Elen, [nn.), i.e., fits Bng using Tu,s. The DR estimator is formulated as
Σ (evt Ou, - [M.))D
CDR D
Pu,i
which enjoys doubly robust property, i.e., an unbiased estimator of ideal loss when either im- puted errors or learned propensities are accurate.
3 MOTIVATION
DR approaches have been extensively studied in RS for various debiasing tasks for its double ro- bustness, e.g.. rating prediction (Wang al., 2019 2020a; Li et al.) (023b|c). learning-to-ranl
(LTR) (Saito) 2020; Oosterhuis (022). and post-click conversion rate prediction (Guo et al.) [2021] Dai et al.] 2022), etc. However, the DR still have several limitations that need to resolved. first
show that DR has large variance and is sensitive to small propensities as shown in Proposition
(see Appendix[AAeo for proofs).
Proposition 1. iu, and ên,i are accurate estimates pu, and gm.i respectively, i.e Pu,i Pu. eu,i Su.i then IPS, EIB and DR estimators are unbiased, and their variances satisfy
Var(CEin) S Var(LDR) & Var(Lirr).
2

---

Published as a conference paper at ICLR 2023
where the equality holds if and only Pu.: for all (u,t) D. In addition, when Pu,i tends to O, (iP() and Var(CDR) tends to infinity, and Var(CEIB) tends to its minimum.
Proposition T shows that the EIB estimator is low-variance and robust to small propensities (Tan 2007] Imbens & Rubin 2015) Wu et al., 2021 2022a). In RS, some small propensities will ap- pear mevitably due to the sparsity of the exposed data, resulting in a significant difference between Var(LEIB) and Var(LDR) Nevertheless, EIB usually has a large bias and is not preferred in prac- tice. Proposition provides motivation to develop estimator that combines the low-variance and robustness to small propensities of EIB with the double robustness of DR.
In summary, DR outperforms IPS in terms of both bias and variance. When compared with EIB, if éu,i is inaccurate but Pu.i is accurate, DR tends to have a smaller bias, but if both én,; and Pu.i are accurate, then EIB has smaller variance. If é.,/ is accurate but Pa.: is inaccurate, then EIB may be superior to DR in terms of both bias and variance. In practice, both Pa.: and é are likely to be at least mildly inaccurate, so choosing from EIB and DR involves the bias-variance trade-off. Ideally, it is desirable to develop method that is robust to small propensities, with lower bias and variance compared to previous DR methods, while maintaining the double robustness.
COLLABORATIVE LEARNING DEBIASING FRAMEWORK
4.1 TARGETED DOUBLY ROBUST ESTIMATOR
We first bridge the explicit form of the DR estimator and the EIB estimator by noting that
Pu.i
LDR = [D]
lou,eCu,s + (1 ou,i) +
Ou,i(Cu, - Cu,s)
(1)
Puii
(u,i)ED
(a,i)ED
CRTA
correction
where LDR formally equivalent to adding correction term using learned propensities to LETB- The correction term has an important role in the bias-variance trade-off for the estimations of the ideal loss as shown in Proposition [ Specifically, compared with CEIB. LDR can reduce bias by adding the correction term. As compromise, the correction term will increase the variance of the DR estimator. Thus, i is computed a manner that ensures that
1- --uu. 0.
Ou,i Êu.i) (u,i)E
(2)
D
Du.
then the EIB estimator would have small bias and the DR estimator would have small variance.
For equation to hold, naive method is taking it as a constraint condition when training the ror imputation model. However, the constraint 2 may degrade the accuracy of the imputed errors because it will restrict the hypothesis space of the error imputation model. Instead of directly esti- mating ē satisfying the constraint (2). we propose to exploit the extra information on propensities when training the error imputation model. The basic idea of the proposed TDR estimator consists of the following two steps.
Step (Initialization). Let ēu,i be the imputed error obtained by using any of the existing DR methods.
Step 2 (Targeting). Update é by fitting an extended mne-parameter model as follows
(n) t..i (1(1//.. 1)
which includes single variable 1/pu.i - 1 and the offset h(zu,). The parameter 7j is solved by minimizing the squared loss between (1) and en,s in the exposed events. Then the proposed TDR estimator is given
LTDR D
Σ (u,i)ED
Eu. + on,i Eu.i)/Pu..)
The targeting step enlarges the hypothesis space of ē.,; compared to é and does not sacrifice the accuracy of the error imputation model. due to the introduce of an error correction term 1/pu,; - to estimate Gu.i- Theorem I shows the validity and preservation of TDR (see Appendix B for proofs).
3

---

Published as a conference paper at ICLR 2023
Theorem 1. The imputed error ē..; obtained with TDR satisfies the following properties:
(a) (validity) eus satisfies equation (2) which implies TDR would have smaller bias than EIB and smaller variance than DR based the initial imputed error En.i- (b) (preservation ij in the targeting step will converge to 0 and renders Cu,5 = Eug when tu, already satisfies equation
From Theorem ! TDR guarantees that equation (2) always holds, regardless of the choice of the initial imputed errors. In addition, TDR inherits the desirable properties of EIB, such as low-vaiiance and robust to small propensities, since equations ① and implies that the TDR estimator can be regarded as an EIB estimator.
TDR would reduce the variance of DR as shown in Theorem I further question is whether the variance-reduction will come at the expense of an increase in bias? Remarkably, TDR has no sacri- fice of bias. Specifically, it can be shown (see Appendix ( that the bias of both L and DR are composed of the product of the errors of the propensity model and imputation model weighted by 1/ Therefore, given the same Iearned propensities, the more accurate the imputed errors are, the smaller the bias is. Since TDR updates é., by adding an extra term 1/pu.i 1, so e.,á is expected to be more accurate than ε resulting in smaller bias for LTDR than CDR-
Importantly, the TDR provides a model-agnostic framework due to the free choice of the initial imputed errors in Step 1, which has great potentially strengths for recommendation. TDR can be assembled into any competing DR approach (Wang et al.. 2019 Guo et al. 2021] Dai et al. 2022). by updating its error imputation model with the targeting step. This extra targeting step tends to reduce both the bias and variance of the competing DR approach, resulting in more accurate predictions. Next, Theorem Srreemmiinde indicatess the double robustness of the TDR estimator (see Appendix ffr for proofs).
Theorem 2. The proposed TDR estimator have the following properties:
(a) (unbiasedness under accurate imputed errors) LTDR is unbiased if ēu, accurately estimates Gu...
(b) (unbiasedness under accurate learned propensities) Suppose that Pa,s accurate estimates Pu.. and the validity of Cu.s doesn't 't hold, then LEIB is biased, while LTDR is unbiased.
Besides, Theorem b)) reveals that TDR can remove the bias of LEIB even though the initial im- puted errors are inaccurate, provided the learned propensities are accurate.
4.2 SEMI-PARAMETRE COLLABORATIVE LEARNING
In this subsection, we propose novel TDR-based collaborative learning (TDR-CL) approach, in which the imputed errors εμ,; are decomposed into a parametric error imputation model part &n,; and a nonparametric targeting part Wu,i _ ry(1 (1/fu..; - 1) as in Section 4.1 i.e., eu, = ên, + Wh,i- The latter corrects the residuals of the error imputation model. By updating both the parametric and nonparametric parts collaboratively, the bias and variance of the TDR estimator can be further reduced, resulting more accurate predictions.
First, the embedding of each user u and item i is obtained by matrix factorization, and the stack layer gets the embedding Zu,i by concatenation. TDR-based learning methods require estimated propensities for all user-item pairs, thus the Naive Bayes approach is no longer applicable. To handle this problem, the pre-trained propensities are obtained by conducting logistic regression of
Oa,i on Pu,i, and the model parameters are used as the initialization of pe (T(,)) in the iterative learning process. Given both the parametric error imputation part é..; = go (Faas) a) and nonparametric targeting part Wa.i. the propensity model pe (z.,;) and the prediction model fo (z(,)) are updated simultaneously using the training loss
CTDR-CL(0,5
LTDR |DI-1
Σ (u,))ED
[-
-oo log Pu,; -(1 Ou,s) -log(1 pu.i))
where Pu,s = Pe(Tu..), Caji (fo(zu,) Tata(1)) εv,i = (fo (Iu,)) go (Zu,i) - Wait 1 (fo(zu,)))) with 1 the operator that sets the gradient of the operand to zero thus Vo 1 (fo (Fu,,) 0 and 1 (fo (z.,i) fo (z.,i).
Then, unlike traditional alternative learning algorithms that directly use the parametric part Io (I En the proposed collaborative learning additionally uses Wu,t non-parametric correction
4

---

Published as a conference paper at ICLR 2023
Algorithm 1: The Proposed Targeted Doubly Robust Collaborative Learning, TDR-CL
Input:
observed ratings R pre-trained learned propensities P, and wu,i 0.
while stopping criteria is not satisfied do for number of steps for training the prediction and propensity model do
Sample batch of user-item pairs {(u,, k))) from 'D;
Update 9 and ç by descending along the gradient VegErDD--L (0, 6);; end
for number of steps for training the imputation model with targeting step do Sample batch of user-item pairs { dx) [g=1 from O:
Update Φ by descending along the gradient V VaCa(O, C,
Sample batch of user-item pairs {(us, it) from D;
10
1f" <
arg min Σ (e, i (1/fu.. - 1))²
Update Wa,t + Was + (L/Pu,, 1) for all user-item pairs.
12
end
13 end
term summed with Io (Za,i) to correct the estimation of Cu,z. Specifically, given the prediction model and the propensity model, E., first updates its parametric part Io (Zu,a) by minimizing
(o,0,E -DI
Σ (u,))D
Ou, (ēu, eu,i) /Pu.i:
where 6n. (1,((1) So (In))) Eus = 9o (Inn,) + wa,i- Next, the targeting step described in Section I. applied to further update the imputed enrors Eu,i- Through calculating the optimal step size for line search 1) arg min Σ (()) - êa,i n11/fu,i ?, the non parametric
targeted error term Wa,s updated by adding " (1/p..: - 1).
In summary, the proposed learning approach collaboratively update the parametric term ê.,i Is (Ir,s) and the nonparametric term Wu,s achieve a better trade-off estimate cu,s, which can reduce the bias of the existing DR methods such as DR-JI. (Wang et al. 2019) and MRDR-DL. (Guo al. 202T), by further modeling for the fitted residuals of the parametric parts tu, On the other hand, as shown in Theorems I and when é is already an accurate estimate of Cu,i, the introduc- tion of the targeted error term T satisfics no-harm property and the unbiasedness is maintained. We summarized the proposed TDR-CL approach in Alg.
5 SEMI-SYNTHETE EXPERIMENTS
In this section, following the previous studies (Schnabel et al. [016 Wang et al.) 2019 Saito 2020) Guo et al. 2021), we aim to answer the following research question (RQ) on the semi-synthetic datasets:
RQL. Does the proposed TDR estimator in estimating the ideal loss have both the statistical properties of lower bias and variance in the presence of selection bias?
5.1 EXPERIMENTAL SETUP
Dataset and Preprocessing. MovieLens 100K (ML-100K) is a dataset of 100,000 missing-not-att random (MNAR) ratings from 943 users and 1,682 movies collected from movie recommendation ratings. MovieLenss 1M (ML-IM) is a larger dataset of 1,000,209 MNAR ratings from 6,040 users and 3,952 movies. Following the data preprocessing procedure of previous studies (Schnabel al.) 2016 Wang et al., 2019, Saito, 2020) Guo et al. 2021), we first use matrix factorization (Koren al. 2009) to complete the rating matrix the five-scale. Then for each predicted ratings Ry,
http:///goooyoonorrrgaaaasttvtvv 100k/
5

---

Published as a conference paper at ICLR 2023
Table 1: Mean and standard deviation of the relative error on the Naive, EIB, IPS, DR and TDR.
Dutaset
Method
ONE
THREE
FIVE
ROTATE
SKEW
CRS
Naive
0.0688 = 0.0025
0.0790
0.0028
0.1027 0.0028
0.1378 = 0.0011
0.0265
0.0021
0.1062
0.0022
E1B
0,5442 + 0.0016
0.5878 +: 0.0017
0.6167 = 0.0018
0.2533 + 0.0004
0,3584 = 0.0007
0,1443 = 0.0007
ML-100K
0,0338 0.0033
0.0390
0.0037
0.0511
0,0696
0.0129
0.0027
0.0526 0.0026
DR
0.0140 + 0.0034
0,0180
0.0037
0.0150 = 0.0034
0,0401
- 0.0016
0.0101 = 0.0027
0.0237 - 0.0025
TDR
0.0053 0.0026*
0.0035 + 0.0025*
0.0066 + 0.0032
0.0325
0.0015
0.0029
0.0020
0.0193 = 0.0025*
Naive
0.0682 +: 0.0007
0.0783 + 0.0007
0.1014 + 0.0008
0.1377
+ 0.0005
0.0256 = 0.0007
0.1054 0.0006
0.0005
0.6157
0,2531
+
0.0002
0.1442 0.0001
MI.-MM
IPS
0.0343 1 0.0009
0,0394
+: 0.0009
0.0508 1 0.0009
0.0687
+ 0.0006
0,0130
+ 0.0008
0.0528 1: 0.0007
0,0130 0.0009
0,0168 0.0009
0.0133 0.0009
0.0399 + 0.0005
0.0090
0.0008
0.0229 0.0007
TDR
0,0054 + 0.0009
0,0031 + 0.0009
0,0076 + 0.0009
0,0324 0.0005
0,0031 0.0008
0.0187 = 0,0007
Note:
means statistically significant results (p-value S 0.00 using the paired-t-test compared with the best baseline.
41,2,3,4,5), we assign the Pu.i E (0, with Pu.: po-max(1,5-RR..) Finally, we replace the predicted ratings Ru with Mne in {0.1, 0.3, 0.5, 0.9) and sample the binary click indicator and conversion label with the Bernoulli sampling ou, ~ Bern(pu,i), Tu, ~ (ern(rarr") V(u, i) E D, where Bern(-) denotes the Bernoulli distribution.
Predicted Metries. The following prediction metrics are used evaluate the debiasing performance under different scenarios.
ONE: fu,. is identical to the u,a except that iu, raue = 0.93| randomly selected با of 0.1
are flipped to 0.9.
THREE: Same as ONE, but flipping rae of 0.3 instead.
FIVE: Same as ONE, but flipping ru, of 0.5 instead.
ROTATE: fu,i = T 0.2 when Ta,i 2 0.3, and fa,: = 0.9 when ra,; = 0.1.
SKEW: fu,i follows the truncated Gaussian distribution N 1... (# = p = (1- - )/2).
CRS: fu,s = 0.2 if the tree < 0.6. Otherwise, fu,s = 0.6.
Experimental Details. For each prediction matrix R == (fu, (1) (u, i) D), the proposed TDR is compared with Naive (Koren et al. 2009). EIB (Hernández- IPS (Saito al. 2020 Schnabel et al. 2016), and DR (Wang al. 2019 Saito 2020 methods. We B)/Pu.t + 3//P.. where is randomly sampled from [0, 1] to introduce noises. Define e,,. = CE(E (n.)) Ta,iWn,i, Pup), where Wa,i = (1/pu,x) / ( 1/Pu.).. CE denotes the cross entropy loss. For EIB and DR, the imputed error is computed as ē.,s = &y,s, For TDR, ēu,i ēu,i + (1"Pp/ 1), where 'y" = arg min Σ (( éu,é - (1/Pp., 1))2. The performance of the estima- tors is based the absolute relative eπror (RE) of the estimated and ideal loss RE(Cest) [Cideal (R) -R)-Leed (R)) (R), where Lest denotes the estimator to be compared. RE evaluates the accuracy of the estimated loss, and smaller RE value indicates a higher estimation accuracy.
et al. 2014 Steck 2010)
obtain the propensities by T/Pu,; =
(M.jeD Ou and B
5.2 EXPERIMENT RESULTS (RQ1)
In Table we report the means and standard deviations of the RE of the five estimators for each predicted matrix over 20 times of sampling. On the one hand, the average RE of the IPS, DR and TDR methods significantly lower than that of the Naive method. verifying the validity of causal- based debiasing methods. The proposed TDR achieves the lowest RE in all settings, attributed to the introduced correction term Wa.; for estimating Cu,i, that further reduces the bias of DR. The direct application of the EIB method is even worse than the Naive method, attributed to the challenge to make an accurate estimate eu,.. On the other hand, same as the conclusion of Theorem [ the standard deviation of the EIB method is significantly lower compared to the IPS and DR methods. The proposed TDR method combines the advantages of the EIB in terms of lower standard devia- tions than IPS and DR in all settings, reflecting stronger robustness. It can be concluded that the estimation accuracy and robustness of the proposed method are significantly improved compared to the previous methods.

---

Published as a conference paper at ICLR 2023
6 REAL-WORLD EXPERIMENTS
In this section, conduct experiments evaluate the proposed methods two real-world bench- mark datasets containing bussing--t-randon (MAR) ratings. Throughout, our methods are imple- mented without uniform data estimate the propensities, which differs from the existing Naive Bayes approach. We aim to answer the following RQs:
RQ2. How do the proposed methods compare with the existing methods in terms of debiasing performance in practice?
RQ3. How does the collaborative learning phase design affect the performance of our methods?
RQ4. Do our methods stably perform well under different learned propensities?
6.1 EXPERIMENTAL. SETUP
Dataset and Preprocessing. MAR ratings are necessary to evaluate the performance of debiasing methods on real-world datasets. Following previous studies, we take the following two benchmark datasets: Coat Shopping has 4,640 MAR and 6,960 MNAR ratings of 290 users to 300 Coats. Music! R3 has 54,000 MAR and 311,704 MNAR ratings of 15,400 users to 1,000 songs.
Baselines. We take the widely used Matrix Factorization (MF) as the base model (Koren et al. 2009,, and compare the proposed methods with the following baselines: Base Model (Koren et al, 2009 IPS (Schnabel et al. 2016), SNIPS (Swaminathan & Joachims 2015). IPS with asymmetric training (IPS-AT) (Saito) 2020), CVIB (Wang et al. 2020bb,, DIB (Liu et al. (021). DR (Saito 2020),
DR-JL (Wang et 2011)) DR-CL, MRDR-JL JL (Guo et al. 2021). MRDR- CL, where DR-CL and but without the targeting step update (lines 9-11), also for comparison purpose. In addition, the proposed TDR-based methods include TDR, TDR-JL, and TMRDR-JL implemented by a single targeting step, and TDR-CL and TMRDR-CL implemented by collaborative learning approach shown in Alg. The real-world experimental protocols and details are provided in Appendix
MRDR-CL are performed using the proposed Alg.
6.2 PERFORMANCE COMPARISON (RQ2)
In Table 2. we report the performance of various debiasing methods using MSE, AUC, NDCG@S, and NDCG @ 10 as evaluation metrics. For previous de-biasing methods, propensity-basse IPS, SNIPS, IPS-AT, and information bottleneck-based CVIB and DIB all outperform the base model. The doubly robust methods, such DR-JL, DR-CL, ML, and MRDR-CL, using alternating learning and outperforming DR, which are considered as the most competitive baselines. The pro- posed TDR estimators are implemented by both single-step and collaborative learning, respectively, based on DR and MRDR as initialized error imputation models, outperforming the baseline meth- all AUC, NDCG@5, and NDCG@ 10 metries, attributed the effectiveness of the introduced nonparametric correction term. It is noted that the collaborative version of TDR achieves the optimal performance both within DR and MRDR, which implements the proposed tar- geting step repeatedly. The fact that TDR-J and TMRDR-I. implemented the targeting step only at the final training of the prediction models outperformed DR-JL. and MRDR-JL, respectively, further demonstrates the effectiveness of the proposed targeting step to correct imputed errors.
ods significantly
6.3 IN-DEPTH ANALYSIS (RQ3, RQ4)
Ablation Study (RQ3). To illustrate the specific reasons for the effectiveness of the TDR-CL al- gorithm, conduct ablation studies DR-based and MRDR-based methods, respectively. From Figure [ DR-CL and DR-JI. perform similarly on MSE, AUC and NDCG@5 metrics, and the MRDR approach has similar findings, which indicates that the directly use of collaborative learning approach without targeting steps cannot improve prediction performance. However, for the proposed TDR-CL and TMRDR-CL methods, there is significant performance improvement compared to
hhtpps//www.cs..c http:://wwwwwwwccollll..co/ schnahts/mmar/ hhtp:://ccsccopessanddd sandbox. Music.com/
7

---

Published as a conference paper at ICLR 2023
Table MSE, AUC, NDCG@5, and NDCG@ 10 on the MAR of Coat and Music. We bold the outperforming DR-based and MRDR-bas based models. The proposed TDR methods implemented by a single targeting step are marked with * and collaborative learning are marked with
Coat
Music
MSE
AUC
N@s
N@ 10
MSE
AUC
N@s
N@ 10
Base Model
0.2448
0.7047
0,5912
0.6667
0.2494
0.6795
0.6353
0.7644
+ IPS
0.2389
0.7041
0.6170
0.6852
0.2496
0.6824
0.6409
0.7674
+ SNIPS
0.2388
0.7061
0.6145
0.6945
0.2493
0.6815
0.6454
0.7701
IPS-AT
0.2344
0.7320
0.6102
0.6784
0.2480
0.6816
0.6409
0.7667
+ CVIB
0.2201
0.7234
0.6221
0.6991
0.2638
0.6823
0.6483
0.7719
+ DIB
0.2334
0.7104
0.6303
0.6986
0.2494
0.6832
0.6348
0.7633
+
0.2359
0.7031
0.6213
0.6967
0.2420
0.6867
0.6613
0.7791
+ DR-JL
0.2352
0.7155
0.6183
0.6925
0.2496
0.6853
0.6536
0.7738
+ DR-CL
0.2358
0.7183
0.6261
0,6927
0.2494
0.6808
0,6334
0.7622
TDR"
0,2268
0.7109
0.6300
0.7006
0.2115
0.7044
0.7008
0.8016
TDR-JL
0.2151
0.7236
0.6388
0.7047
0.2577
0.7036
0.6786
0.7884
+ TDR-CL
0.2119
0.7339
0.6526
0.7112
0.2472
0.7057
0.6758
0.7871
+ MRDR-JL
0.2162
0.7192
0.6360
0,7016
0.2496
0.6842
0.6487
0.7717
+ MRDR-CL
0.2155
0.7200
0.6427
0.7047
0.2494
0.6805
0,6345
0.7623
TMRDR-JL'
0.2114
0.7278
0.6498
0.7101
0.2557
0.7036
0.6785
0.7884
+ .TMRDD-CL
0.2114
0,7316
0.6428
0.7088
0.2473
0.7060
0.6803
0.7902
the DR-CL and MRDR-CL methods without targeting steps. This ablation study reveals that the im- provement in the proposed TDR-CL and TMRDR- originates from the nonparametric correction term of the imputed errors, not from introducing additional model parameters for updating.
Effect on Learned Propensities (RQ4). An important fact is that the nonparametric correction term in the TDR estimator is based given learned propensities. In order examine whether the proposed targeting steps stably help to improve the prediction accuracy under different learned propensities obtained by setting different clipping threshold, we conducted repeated experiments to quantify the sensitivity of the TDR-CL method to the propensity clipping threshold. From Figure 2, the proposed method outperforms the DR-JL and DR-CL methods in terms of AUC, NDCG@5, and NDCG@ 10 on all clipping thresholds. The optimal performance is reached when the clipping threshold equal to 0.15, which is interpreted as achieving the best trade-off between information utilization and robustness.
7 RELATED WORK
Debiasing in Recommendation. Bias is a common problem inherent in RS (Chen et al., 2020 Wu et al.) 2022b), such as popularity bias (Zhang et al. 2021). model selection bias (Yuan et al, 2019), user self-selection bias (Saito 2020), position bias (Ai et al. 2018), and conformity bias (Liu ct al. 2016). Various methods were proposed for unbiased learning. For example, [Schnabel et al. (2016) considered the recommendation treatment and introduced the IPS and self-normalized IPS (SNIPS) methods to debiasing in explicit feedback data. (Saito et al.] (2020) extended it to the im- plicit recommendation. WWang et al. (2019) proposed doubly robust joint learning approach that improved the IPS method. Subsequently, series of enhanced DR methods were developed, such as MRDR (Guo al.) (022). Multi-task DR (Zhang et al. 2020), DR-MSE (Dai al.. 2022), BRD- DR (Ding et al. (022), and SDR (Li et al., 2003c). Li et al. (2023a) proposed a muluple robust method that takes the advantages of multiple propensity and error imputation models. In addition, several new debiasing algorithm are designed via using an extra small uniform dataset (Bonner & Vasile 2018 Chen et al., 2021 I Liu et al. 2020 Wang et al. 2021 Li et al. 2023b). Chen et al. (2020) provided a thorough discussion the recent progress debiasing tasks in RS. Wu let al. (2022)) established unified causal analysis framework and gave formal causal definitions of various biases in RS from the perspective of causal inference. Unlike the existing enhanced DR approaches that purse a better bias-variance trade-off, the proposed TDR reduces both the bias and variance and is theoretically guaranteed.
8

---

Published as a conference paper at ICLR 2023
DRJJI
0.70
DR.CL
NDCG@5
0.60
0.65
MSE
0.275
0.60
0.55
0.250
0.55
0.225
DR-CL
0.50
DR-CI
TDR-CL
0.200
0.45
0.45
15
20
0
5
10
15
10
15
20
Training Epochs
Training Epochs
Training Epochs
(a) Comparison of DR-JL, DR- CL and TDR-CL in terms of MSE, AUC and NDCG@5.
0.325
0.75
0.65
MADR-JL
0.300
MADR CL IMHUR-CL
N
0.60
0.65
MSE
0.275
0.55
0.55
MRDR.JL MRDR-CL
MRDR.JL
0.50
TMRDR.CU
TMRDR CL
0.200
0
5
10
15
20
0.43
5
10
15
20
0.40
10
15
20
Training Epochs
Training Epochs
Training Epochs
(b) Comparison of MRDR-JL, MRDR-CL and TMRDR-CC CL in terms of MSE, AUC and NDCG@5.
Figure 1: Ablation studies on DR methods (top) and MRDR methods (bottom), where DR-Cl and
MRDR-CL skips the targeting steps in TDR-CL and TMRDR-CI.
DX--I
DR--TI
TIS-
TBR-II
TRR-T
TDL
DR-II
TDR-CT
0.74
0.72
0.73
0.54
0.71
NDCG..
NDCG210
0.71
0.69
0.00
0.69
0.68
0.58
0.65
0.05
0.05
Propensity Clipping Thresbold
Propensity Clipping Threshold
Propensity Clipping Threshold
Figure 2: Learning performance on MAR test set of AUC (left), NDCG@5 (middle), and NDCG@11 10 (right) with varying levels of propensity clipping threshold.
Targeted Learning. Targeted learning a general framework in causal inference (van der Laan & Rose 2011) that includes many field-specific approaches accommodate various scientific prob- lems in different fields, such as survival analysis (Stitelman et al. 2012). genomics (Gruber & der Laan 2010), epidemiology (Rose & der Laan 2014) and etc. More application scenar- ios about targeted learning can refer to the two excellent monographs (van der Laan & Rose [011] 2018). Shi al. (2019) proposed adapting neural networks for estimating the average treatment effects based on targeted learning. Different from the existing literature of targeted learning, this paper deals with the estimator and learning problem simultaneously. To the best of our knowledge, this is the first paper that extends targeted learning to the field of debiased recommendation.
CONCLUSION
In this paper, we propose TDR estimator for debiased recommendation that enjoys the properties of double robustness, boundedness, low variance, and robustness to small propensities simultancously. Theoretical analysis shows that TDR can effectively reduce the bias and variance simultancously for any DR estimator when the error imputation model is less accurate. In addition, we further propose novel uniform-data data-ffe TDR- based collaborative learning approach that adaptively implements the targeting step, thus making the prediction model more robust. We conducted experiments on both semi-synthetic and real-world data. The superiority of the proposed method is demonstrated when compared with the existing debiasing methods. Throughout, we adopt 1/βu, - 1 as a key choice of targeting step to satisfy equation (2). which can be regraded as first-order targeted learning Carone cet al.] (2014). In future work, we will explore higher-order targeted learning and more effective feature selection in the proposed targeting step.

---

Published as a conference paper at ICLR 2023
ETHICS STATEMENT
This work is mostly theoretical and experiments are based on synthetic and public datasets. We claim that this work does not present any foreseeable negative social impact.
REPRODUCIBILITY STATEMENT
Code is provided in Supplementary Materials to reproduce the experimental results.
ACKNOWLEDGMENTS
The work was supported by the National Key R&D Program of China under Grant No. 2019YFB170560..
REFERENCES
Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W Bruce Croft. Unbiased learning to rank with unbiased propensity estimation. SIGIR, 2018.
Stephen Bonner and Flavian Vasile. Causal embeddings for recommendation. In RecSys, 2018.
Marco Carone, Iván Díaz, and Mark van der Laan. Higher-order targeted minimum loss-based estimation. U.C. Berkeley Division of Biostatistics Working Paper Series, Paper 331, 2014.
Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. Bias and debias in recommender system: A survey and future directions. arXiv:2010.032200, 2020.
Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. Autodebias: Learning to debias for recommendation. In SIGIR, 2021.
Quanyu Dai, Haoxuan Li, Peng Wu, Zhenhua Dong, Xiao-Hua Zhou, Rui Zhang, Xiuqiang He, Rui Zhang, and Jie Sun. A generalized doubly robust learning framework for debiasing post-click conversion rate prediction. In KDD, 2022.
Sihao Ding, Peng Wu, Fuli Feng, Xiangnan He, Yitong Wang, Yong Liao, and Yongdong Zhang. Addressing unmeasured confounder for recommendation with sensitivity analysis. In KDD, 2022.
Susan Gruber and Mark J. van der Laan. An application of collaborative targeted maximum likeli- hood estimation causal inference and genomics. The International Journal of Biostatistics, 6: articel 18, 2010.
Siyuan Guo, Lixin Zou, Yiding Liu, Wenwen Ye, Suqi Cheng, Shuaiqiang Wang, Hechang Chen, Dawei Yin, and Yi Chang. Enhanced doubly robust learning for debiasing post-click conversion rate estimation. In SIGIR, 2021.
Miguel A. Hernán and James M. Robins. Causal Inference: What If. Boca Raton: Chapman and Hall/CRC, 2020.
José Miguel Hernández-Lobato, Neil Houlsby, and Zoubin Ghahramani. Probabilistic matrix fac- torization with non-random missing data. In ICML, 2014.
Guido W. Imbens and Donald B. Rubin. Causal Inference For Statistics Social and Biomedical Science. Cambridge University Press, 2015.
Joseph D.Y. Kang and Joseph L. Schafer. Demystifying double robustness: comparison of alter- native strategies for estimating a population mean from incomplete data. Statistical Science, 22: 523-539, 2007.
Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8))30-37, 2009.
10

---

Published as a conference paper at ICLR 2023
Haoxuan Li, Quanyu Dai, Yuru Li, Yan Lyu, Zhenhua Dong, Xiao-Hua Zhou, and Peng Wu. Multi- ple robust learning for recommendation. In AAAI, 2023a.
Haoxuan Li, Yanghao Xiao, Chunyuan Zheng. and Peng Wu. Balancing unobserved confounding with a few unbiased ratings in debiased recommendations. In wwW, 2023b.
Haoxuan Li, Chunyuan Zheng, and Peng Wu. StableDR: Stabilized doubly robust learning for recommendation data missing not at random. In ICLR, 2023c.
Dugang Liu, Pengxiang Cheng, Zhenhua Dong. Xiuqiang He, Weike Pan, and Zhong Ming. A general knowledge distillation framework for counterfactual recommendation via uniform data. In SIGIR, 2020.
Dugang Liu, Pengxiang Cheng, Hong Zhu, Zhenhua Dong, Xiugiang He, Weike Pan, and Zhong Ming. Mitigating confounding bias in recommendation via information bottleneck. In RecSys, 2021.
Yiming Liu, Xuezhi Cao, and Yong Yu. Are you influenced by others when rating?: Improve rating prediction by conformity modeling. In RecSys, 2016.
Geert Molenberghs, Garrett Fitzmaurice, Michael G. Kenward, Anastasios Tsiatis, and Geert Ver- beke. Handbook of Missing Data Methodology. Chapman & Hall/CRC, 2015.
Jerzy Splawa Neyman. On the application of probability theory to agricultural experiments. essay principles. section 9, Statistical Science, 5:465-477 -452, 1990.
Harrie Oosterhuis. Doubly-robul robust estimation for unbiased learning-to -tornnkk-t--kkkk from position-biased click feedback. arXiv preprint urXiv:2203.77114 2022.
Sherri Rose and Mark J. der Laan. A double robust approach causal effects in case-control studies. American journal of epidemiology, 179:663-66 669, 2014.
Paul R. Rosenbaum. Design of Observational Studies. Springer Nature Switzerland AG, second edition, 2020.
Donald B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational psychology, 66:688-701, 1974.
Yuta Saito. Asymmetric tri-training for debiasing missing- -ote--oot----- not-at-random explicit feedback. In GIR, 2020.
Yuta Saito. Doubly robust estimator for ranking metrics with post-click conversions. In SIGIR, 2020.
Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. Unbiased rec- ommender learning from missing-not-at-randoo implicit feedback. In WSDM, 2020.
Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. Recommendations as treatments: Debiasing learning and evaluation. ICML, 2016.
Claudia Shi, David M. Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. In NeurlPS, 2019.
Harald Steck. Training and testing of recommender systems on data missing not at random. In KDD, 2010.
Ori M. Stitelman, Victor De Gruttola, and Mark J. van der Laan. A general implementation of tmle for longitudinal data applied to causal inference in survival analysis. The International Journal of Biostatisties, 8tariicel 26, 2012.
Adith Swaminathan and Thorsten Joachims. The self-normalized estimator for counterfactual learn-
ing. In NewrIPS, 2015.
Zhiqiang Tan. Comment: understanding OR, PS and DR. Statistical Science, 22:560-568, 2007. 2007.

---

Published as a conference paper at ICLR 2023
Mark J. van der Laan and Sherri Rose. Targeted Learning: Causal Inference for Observational and Experimental Data. Springer, 2011.
Mark J. van der Laan and Sherri Rose. Targeted Learning in Data Science: Causal Inference for Complex Longitudinal Studies. Springer, 2018.
Xiaojie Wang. Rui Zhang. Yu Sun, and Jianzhong Qi. Doubly robust joint learning for recommen- dation on data missing not at random. In ICML, 2019.
Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. Combating selection biases in recommender systems with a few unbiased ratings. In WSDM, 2021.
Yixin Wang, Dawen Liang, Laurent Charlin, and David M. Blei. Causal inference for recommender systems. In RecSyx, 2020a.
Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Erean Kuruoglu, and Yefeng Zheng. In- formation theoretic counterfactual learning from missing-not coatinng-oota--aee att-at-random feedback. In NeurIPS, 2020b.
Peng Wu, Xinyi Xu, Xingwei Tong, Qing Jiang, and Bo Lu. Semiparametric estimation for average causal effects using propensity score-based spline. Journal of Statistical Planning and Inference, 212:153-168, 2021.
Peng Wu, Shasha Han, Xingwei Tong, and Runze Li. Propensity score regression for causal infer- ence with treatment heterogeneity. Statistica Sinica, 2022a.
Peng Wu, Haoxuan Li, Yuhao Deng, Wenjie Hu, Quanyu Dai, Zhenhua Dong, Jie Sun, Rui Zhang, and Xiao-Hua Zhou. On the opportunity of causal learning in recommendation systems: Founda- tion, estimation, prediction and challenges. IJCAI, 2022b.
Peng Wu, Zhiqiang Tan, Wenjie Hu, and Xiao-Hur Zhou. Model-assisted inference for covariate- specific treatment effects with high-dimensional data. Statistica Sinica, 2022c.
Bo-Wen Yuan, Yung Hsia, Meng-Yusd Yuan Yang, Hong Zhu, Chih-Yao Chang, Zhenhua Dong, and Chih-Jen Lin. Improving ad click prediction by considering non-displayed events. In CIKM, 2019.
Wenhao Zhang, Wentian Bao, Xiao Xiao-Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani. Large-scale causal approaches to debiasing post-click conversion rate estimation with multi-task learning. WwW, 2020.
Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. Causal intervention for leveraging popularity bias in recommendation. In SIGIR, 2021.
12

---

Published as a conference paper at ICLR 2023
A PROOF OF PROPOSITION
Recall that Pu.i = P(o.. = 1[ = Elou.| and Iu.i = [.,,l both of them functions of Tu,s- Throughout, we maintain the common unconfoundedness assumption (i.e., (1, IT Ou,i Fu,s) and the consistency assumption, (i.e., (1, = Tu, if ou, = 1). All the lower-case letters denote random variables for simplification.
Proof of Proposition I. The property of unbiasedness is obvious. Next, we focus analysing the variance. Define
0² (Tu,i) = Var(ea.:(eu.s) = [E( Ru,k)² Tu,aa
then Ef(2 [u.s = 0² (xr,t) + p2.s- The variance IPS estimator is given by
Var(Lrrs) = =1D|- |DI-1 -1, Var( Ou,eeni
Puçi
Ca.
{ E Cu. 2
=|D|-
p
Pa,s
I3.
El Fari) Fle22,
Elo [u.n) Ele (rnss) }²
|D]-1.
pa.
Pu,i
[DI-1.
(Blens)
}
Pu.i
IE 2 u,e (uni)) [Blemit)]
|DI-1.
Puj
(Inss) + Fuck
(on))((eee (B(eni))
[D]
E
Puçi
where the third equation follows by the law iterated expectations and the unconfoundedness assumption. The variance of DR estimator is derived by
Ou,i(Ea, - -9u,s)
[DI- Var(LDR) Var Ju,i +
Puç
Ou,i Pusd (Eaii - 9us))
Var Cu,i +
Pu,
Ouii - -aud (Bu,s mus))
=Var(eu,)) + Var
Duçi
Ouni - Pui (eu, )uiit)
=Ble2, [E(e,,))] + Var
Puçi
(Ons - Puni)?
=Eoo( + 2 [B(e 2
+
(eu, -uni)
p2.
EE E[(s (a,,)2(zanns)) (.nis
Elo?(wu,)) + + [B(e +
Pu, E((eu. - gui)) (lanid
Pu,a(1 po (Ia,))
= E(o2(nu:)) + 2a.l [B(eu,)) +
pd.
= =b| =B| (Inas) + 22.1 - [E(e,,)))
Pu.i
where the fifth equation holds by noting that
E Gu.i (Dn,s pu.s) (Ru,i - Duss) = E(ou, Pussleus)) Eleurileu.s - Fm = 0.
Pa.s
Pu,5
13

---

Published as a conference paper at ICLR 2023
Since iuu + (1 - 0u.,) = gu = il we have
[D] Var(LEIB) Var (u, + (u.i)
Lou (1 Ou. 121 [E((e,))]
E Ousseu + (1 Ouss) [(enx))
E Pu.i (Fus) + gu, + (1 - Pu,i [((e,.)]
(I + {u.)] [[(en,)]?
B PROOF OF THEOREM
Proof of Theorem I. The parameter rj is solved by minimizing
{euss
Σ Ou.i
(u, - Êu,i 77( Pu,i
1)}
(M.))D
Taking the first derivative of the above loss with respect to v and setting it to zero leads to that
Σ
Ouii
(eus êu, 17( 1
11} (1/Pu,t 1) 0,
(4)
Pus
(u.1)E
which implies that
Σ Ou!i fey.s- ēu,s} ) (1/Pu,i - 1) = 0,
(u,1)ED
namely, the equation (2) holds. This finishes the proof of Theorem 1(a). If êu, already satisfies equation [ then Os a solution of equation (4). Let î is another solution of equation 4 Since the solution of equation is unique, then will converges 0. This proves the conclusion of Theorem 1(b).
C PROOF OF THEOREM 2
Proof Theorem 2. The result of Theorem 2(a) is obvious. To show Theorem 2(b). We first claim that if ē is an accurate estimate of Ju.i i.e., Cu,i = Ju,is then it will satisfy equation 22. holds immediately from the following calculations
ou.iteu,i Lu,i) (-_ -1)
Paçi
(LJ)ED
1
ouiieuui Iu.it
1)
Pa.s
(u
E Juuiteus Iuitt [1-1
Puçi
E B(ou.slaua) Efeus - Iund
1)]
Puçi
0.
Thus, if é not satisfy equation
then ēu,i + gu.s. Given Eu,s and ēu,i. the bias of LETB is
Bias Bias(CEIB)
Elo,, + (1 sendl - E[eu,s]
E[( (E,s) ea,i)]
E[ ( Iu.))]:
14

---

Published as a conference paper at ICLR 2023
and the bias of LIDR is
(Ouss Purd (eui - Emi)) - Blewad
Bias(CTDR) = E
Cu,i +
Pu,
E(Ou,i Puralterrd) Bfens Eupeltu,st
E
Pu,
E
(Ju,i Rugi)
Pu,i
0.
This proves the result of Theorem 2(b).
Biases of DR and TDR. Given Pu,s and ē for all (2,,) E D, the bias of TDR is
Bias(LTDR) = E Eu,i (Ous - Duvi) (eni Emi)) Bleud Dusi
B(ou,i Pa.s) Blent Eue
E
Dus
(Pu,i Pajk) (9u, Eus)) Pa,s
E
Similarly, given Pa,s and é for all (u, E D. the bias of DR is
(Pu.i Pr.i)
Bias (
(gu,i Eass)
Pus
D REAL-WORLD EXPERIMENTAL PROTOCOLS AND DETAILS
Experimental protocols and details. For real-world experiments, the following four metrics were considered as the evaluation metrics: MSE, AUC, NDCG@ 5, and NDCG@ 10. For fast convergence in the learning phase, Adam is utilized the optimizer for all models. We the learning (0.001, 0.005, 0.01, 0.05, 0.1). weight decay in [1c- 6. 1c - 2] at 10x multiplicative ratio, and batch size in {128, 256. 512, 1024, 2048) for Coat and {1024, 2048, 4096, 8192, 16384) for Music! R3. Specifically for the propensity training, we tune the clipping threshold in (0.05, 0.10, 0.15, 0.20). After finding out the best configuration on the validation set, we evaluate the trained models on the MAR test set. Experiments are conducted using NVIDIA GeForce RTX 3090.
in
15


Reviews:
[{"review_id": "JBZpZ12e-v", "reviewer": "Reviewer_Z8P2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper argues that existing doubly robust (DR) estimators usually have a large variance, a large bias, and poor generalization ability given inaccurate error imputation. The paper proposes a novel estimator, called targeted doubly robust (TDR), to simultaneously reduce the bias and variance of existing DR estimators. The paper demonstrates that the imputed errors by the TDR estimator satisfy two important properties called validity and preservation in Theorem 1 and shows that the TDR estimator preserves the desired property of double robustness in Theorem 2. The paper further proposes a semi-parametric collaborative learning approach (TDR-CL) whose key idea is to decompose imputed errors into parametric term and a non-parametric correction term. The paper conducts semi-synthetic experiments to demonstrate that the proposed TDR estimator achieves the lowest absolute relative error among four baseline estimators (Naive, EIB, IPS, and DR) under six rating missing scenarios (ONE, THREE, FIVE, Rotate, SKEW, and CRS). The paper also conducts experiments on two real-world datasets (COAT and YAHOO) and the experimental results show that the proposed approach significantly outperforms baseline approaches under MSE, AUC, NDCG@5, and NDCG@10.\n\n", "summary_of_the_review": "This paper is marginally above the acceptance threshold based on its novelty and significance.", "strength_and_weaknesses": "S1: The idea of applying targeted learning techniques to effectively reduce the bias and variance of existing DR estimators is novel. The paper provides theoretical results showing that the imputed errors obtained with the proposed TDR estimator satisfy two desired properties (validity and preservation) and the proposed TDR estimator has the desired property of being doubly robust.\n\nS2: The paper conducts pretty extensive semi-synthetic experiments and real-world experiments to validate the effectiveness of the proposed TDR estimator and TDR-CL approach, respectively. In the semi-synthetic experiments, the paper simulates a total of six different rating missing scenarios and compares the proposed TDR estimator against four baseline estimators under these diverse scenarios. In the real-world experiments, the paper uses two real-world datasets, compares against a wide range of baseline approaches (IPS, SNIPS, CVIB, DR, DR-JL, etc.), applies the proposed TDR-CL approach to two representative DR approach (DR-JL and MRDR), and computes various evaluation metrics (MSE, AUC, NDCG@5, NDCG@10). The extensive experimental results make the argument that the proposed estimator and approach are more effective than existing approaches much convincing.\n\nW1: In Section PRELIMINARIES, it is not clear why a prediction error is defined as an expectation of the real prediction error given features. It is important to add more explanations and clarifications here.\n\nW2: It would be better if the paper can move the descriptions of the six rating missing scenarios, ONE, THREE, FIVE, Rotate, SKEW, and CRS, from the Appendix to the main text.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HkVBJ2NfnKQ", "reviewer": "Reviewer_VJ5g", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a new estimator for debiasing Recommender Systems called Targeted Doubly Robust (TDR) which addresses some limitations of the Doubly Robust (DR) and Error Imputation Based (EIB) estimators. \n\nThey show first that DR has a large variance and is sensitive to small propensities. The variance of DR, while being lower than variance of Inverse Propensity Score (IPS), is higher than variance of Error Imputation Based (EIB) method. The objective is to come up with an estimator with the best of the two worlds: DR and EIB.\n\nBy rewriting DR estimator as a sum of the EIB estimator and a correction term, they model the error imputation by any arbitrary function (without any assumptions) so that the correction loss is equal to zero to ensure the lowest variance of EIB but smaller bias than EIB (Theorem 1). \n\nTDR benefits from all other advantages of EIB: boundedness and robustness to small properties. The authors show TDR does not increase the bias compared to DR. Theorem 2 proves its double robustness. \n\nThe authors finally show how TDR is used in a collaborative-filtering-based recommender system. \n", "summary_of_the_review": "The paper proposes a new estimator TDR for debiasing Recommender Systems which is thoroughly investigated theoretically and well-supported experimentally. TDR obtains best performance on semi-synthetic and real datasets and gathered all the desirable theoretical properties. ", "strength_and_weaknesses": "Strengths:\n\nThe paper is quite clear and enjoyable to read. Motivation of the paper is clearly stated and this is easy to follow for the reader how the authors approached the problem and solved it. \n\nEach of the desirable properties for debiasing estimators (doubly robust, low variance, robust to small properties, without extrapolation and boundedness) is investigated theoretically. Definitions of the concepts are recalled in the appendix.\n\nThe theoretical contributions of the paper are significant and are supported by extensive experiments on synthetic and real datasets (best performances are obtained by TDR estimator and its variants). \n\nWeaknesses:\n\nThe section on “semi-parametric collaborative learning” might be a little bit dense.\n\nIt might be worth also to recall briefly the differences between the competing methods used for comparison in the Section “Real-World Experiments”.\n\nIt would be finally helpful for the reader to give a little bit more details on targeted learning and the link with the proposed estimator.\n\nWhich stopping criteria and number of steps do you suggest for Algorithm 1 in practice?\n\nMinor (typos):\n-p.4: “Specifically, It”", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "46b8cm07vma", "reviewer": "Reviewer_k4cv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThe authors propose a novel targeted doubly robust (TDR) algorithm for collaborative learning in recommender systems that alternately optimizes the predictor and propensity estimator encountered in a typical DR estimator along with the error imputation estimate and the step size estimate encountered in a targeted approach. Ablation studies on real data clearly demonstrate the efficacy of the proposed novel components. The authors also convincingly demonstrate improved metrics (AUC, NDCG, etc.) with respect to competing methods.\n", "summary_of_the_review": "Given the concerns regarding clarity discussed under weaknesses as well as the second weakness regarding general practical applicability, I weakly recommend acceptance of this paper.", "strength_and_weaknesses": "Strengths:\n\n1. The paper appears to be theoretically strong. Proposition 1, Theorem 1 and Theorem 2 appear to be novel and interesting.\n\n2. Algorithm 1 is itself well-motivated and the emprical results on semi-synthetic data in Table 2 and real data in Tables 3, 4 are quite convincing.\n\nWeaknesses:\n1. The authors correctly credit van der Laan and Rose for developing TDR estimators, but they do not relate the multi-step algorithm in Sec. 4.1 on TDR with previous work by van der Laan and Rose. Theorem 1 and Theorem 2 in Sec 4.1 appear to be new, but the reader is not explicitly made aware of this fact.\n\n2. The authors ignore temporal variability in recommender systems and assume that features $x_{ui} $ for a user-item pair are static rather than temporally varying.\nRecurrent Recommender Networks, WSDM 2017, shows that this assumption often does not hold for real world recommender systems.\n\n3. The presentation of the material could be improved. For instance, the columns \"Without extrapolation\" and \"Boundedness\" in Table 1 are not explained in the paper and one needs to refer to appendices, whereas the symbol \\eta in eq (4) changes to \\epsilon when we reach Algorithm 1 and its description. The meaning of the columns in Table 2 is not clear until one reads Appendix E.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster