TDR-CL: T ARGETED D OUBLY R OBUST C OLLABORATIVE L EARNING FOR D EBIASED R ECOMMENDATIONS
Anonymous authors
Paper under double-blind review
A BSTRACT
Bias is a common problem inherent in recommender systems, which is entangled with users’ preferences and poses a great challenge to unbiased learning.
For debiasing tasks, the doubly robust (DR) method and its variants show superior performance due to the double robustness property, that is, DR is unbiased when either imputed errors or learned propensities are accurate.
However, our theoretical analysis reveals that DR usually has a large variance.
Meanwhile, DR would suffer unexpectedly large bias and poor generalization caused by inaccurate imputed errors and learned propensities, which often occur in practice.
In this paper, we propose a principled approach that can effectively reduce the bias and variance simultaneously for existing DR estimators when the error imputation model is misspecified.
In addition, we further propose a novel semi-parametric collaborative learning approach that decomposes imputed errors into parametric and nonparametric parts and updates them collaboratively, resulting in more accurate predictions.
Both theoretical analysis and experiments demonstrate the superiority of the proposed methods compared with existing debiasing methods.
I NTRODUCTION
Addressing various tasks in recommender systems (RS) with causality-based methods has become increasingly popular (Wu et al., 2022).
Causality-based recommendation has shown its great potential in both numeric experiments and theoretical analyses across extensive literature (Chen et al., 2020;
Wang et al., 2019).
Generally, the basic question faced in RS is that ”what would the feedback be if recommending an item to a user”, requiring to estimate the causal effect of a recommendation on user feedback.
To answer the question, many methods have been proposed, such as inverse propensity score (IPS) (Saito et al., 2020; Schnabel et al., 2016), self-normalized inverse propensity score (SNIPS) (Swaminathan & Joachims, 2015), error imputation based (EIB) methods (Steck, 2010), and doubly robust (DR) methods (Chen et al., 2021;
Wang et al., 2019; 2021; Dai et al., 2022).
Among them, the DR method and its variants show superior performance.
To assess the performance of these methods, we introduce five perspectives to compare and evaluate different approaches in RS, including doubly robust, robust to small propensities, boundedness (Molenberghs et al., 2015), without extrapolation (Tan, 2007) and low variance (Tan, 2007).
Failing to meet any of them may lead to sub-optimal performance (Molenberghs et al., 2015; van der
Laan & Rose, 2011).
We compare various estimators in these five aspects and find that DR has much greater variance than EIB (Kang & Schafer, 2007), even though the imputed errors and the learned propensities are accurate.
Meanwhile, DR would suffer unexpectedly large bias and poor generalization caused by inaccurate imputed errors and learned propensities, which often occur in practice.
In this article, we first propose a novel targeted doubly robust (TDR) method, that can capture the merits of both DR and EIB effectively by leveraging the targeted learning technique (van der Laan & Rose, 2011; 2018).
TDR can effectively reduce the bias and variance simultaneously for existing
DR estimators when the imputed errors are less accurate.
Remarkably, TDR provides a modelagnostic framework and can be assembled into any DR method by updating its error imputation model, resulting in more accurate predictions.
To further reduce the bias and variance during the training process, we propose a novel uniformdata-free TDR-based collaborative learning (TDR-CL) approach that decomposes imputed errors into a parametric imputation model part and a nonparametric error part, where the latter adaptively rectifies the residual bias of the former.
By updating the two parts collaboratively, TDR-CL achieves a more accurate and robust prediction.
Both theoretical analysis and experiments demonstrate the superiority of TDR and TDR-CL compared with existing methods.
P RELIMINARIES
Many debiasing tasks in RS can be formulated using the widely adopted potential outcome framework (Neyman, 1990; Rubin, 1974).
Denote U = { u } , I =
{ i } and D = U
× I as the sets of users, items and user-item pairs, respectively.
Let x u,i , r u,i , and o u,i be the feature, feedback, and exposure status of user-item pair ( u, i ) , where o u,i = 1 or 0 represents whether the item i is exposed to user u or not.
Define r u,i (1) as the potential outcome if o u,i had been set to 1 , which is observed only when o u,i = 1 .
In RS, we are often interested in answering the causal question: ”if we recommend products to users, what would be the feedback?”.
This question can be formulated as to learn the quantity E ( r u,i (1) | x u,i ) , i.e., it requires to predict r u,i (1) using feature x u,i , where E denotes the expectation with respect to the target distribution P .
Many classical tasks in RS can be defined as estimating this quantity, such as rating prediction (Schnabel et al., 2016) and post-click conversion rate prediction (Guo et al., 2021).
More examples can be found in Wu et al.
Let f θ ( x u,i ) be a model used to predict r u,i (1) with parameter θ .
Ideally, if all r u,i (1) for ( u, i ) ∈ D were observed, θ can be trained directly by optimizing the following ideal loss
[Equation]
where e u,i is the prediction error, e.g., the squared loss e u,i = ( r u,i (1) − f θ ( x u,i )) 2 .
However, since r u,i (1) is observed only when o u,i = 1 , the ideal loss is non-computable.
Restricting the analysis to non-missing data will obtain biased conclusions, as the observed data may form an unrepresentative sample of the target population.
Different debiasing methods are designed to approximate and substitute the ideal loss.
For example, the IPS and EIB estimators are given as
[Equation]
where ˆ p u,i is an estimate of propensity score p u,i :
= P ( o u,i = 1 | x u,i ) , ˆ e u,i is an estimate of prediction error g u,i :=
E [ e u,i | x u,i ] , i.e., it fits e u,i using x u,i .
The DR estimator is formulated as
[Equation]
which enjoys doubly robust property, i.e., it is an unbiased estimator of ideal loss when either imputed errors or learned propensities are accurate.
M OTIVATION
DR approaches have been extensively studied in RS for various debiasing tasks for its double robustness, e.g., rating prediction (Wang et al., 2019; 2020a), learning-to-rank (LTR)
(Saito, 2020;
Oosterhuis, 2022), post-click conversion rate prediction (Guo et al., 2021), etc.
However, the DR still have several limitations that need to resolved.
We first show that DR has a large variance and is sensitive to small propensities as shown in Proposition 1 (see Appendix B for proofs).
Proposition 1.
If ˆ p u,i and ˆ e u,i are accurate estimates of p u,i and g u,i , respectively, i.e., ˆ p u,i = p u,i , ˆ e u,i
= g u,i , then IPS, EIB and DR estimators are unbiased, and their variances satisfy
[Equation]
where the equality holds if and only if p u,i
= 1 for all ( u, i ) ∈ D .
In addition, when p u,i tends to 0,
Var( L IPS ) and Var( L DR ) tends to infinity, and Var( L EIB ) tends to its minimum.
In RS, some small propensities will appear inevitably due to the sparsity of the exposed data, resulting in a significant difference between Var( L EIB ) and Var( L DR ) .
Nevertheless, EIB usually
[Table]
Note
[Table]
symbols ✓ , o and × denotes good, medium and bad, respectively.
has a large bias and is not preferred in practice.
Proposition 1 provides a motivation to develop an estimator to combine the low-variance property of EIB with the double robustness of DR.
Furthermore, we compare the typical debiasing estimators in terms of: doubly robust, robust to small propensities, boundedness (Molenberghs et al., 2015; van der Laan & Rose, 2011), without extrapolation (Tan, 2007), and low variance, as shown in Table 1 (See Appendix A for details).
In summary, DR outperforms IPS in terms of both bias and variance.
When compared with EIB, if ˆ e u,i is inaccurate but ˆ p u,i is accurate, DR tends to have a smaller bias, but if both ˆ e u,i and ˆ p u,i are accurate, then EIB has a smaller variance.
If ˆ e u,i is accurate but ˆ p u,i is inaccurate, then EIB may be superior to DR in terms of both bias and variance.
In practice, both ˆ p u,i and ˆ e u,i are likely to be at least mildly inaccurate, so choosing from EIB and DR involves the bias-variance trade-off.
Ideally, it is desirable to develop a method that is robust to small propensities, bounded, and has lower bias and variance compared to previous DR methods, while maintaining the double robustness.
C OLLABORATIVE L EARNING D EBIASING F RAMEWORK 4. T ARGETED D OUBLY R OBUST E STIMATOR
DR and EIB are related via the ”correction term”.
Specifically, note that
[Equation]
which indicates that the correction term uses propensity score to estimate how much L EIB overestimates or underestimates L DR and then subtracts it.
As a compromise, the correction term will increase the variance of the DR estimator.
Thus, if ˆ e u,i is computed in a manner that ensures that
[Equation]
then the DR would have small variance.
For equation (2) to hold, a naive method is taking it as a constraint condition when training the error imputation model.
However, the constraint (2) may degrade the accuracy of the imputed errors because it will restrict the hypothesis space of the error imputation model.
Instead of directly estimating ˆ e u,i satisfying the constraint (2), we propose to exploit the extra information on propensities when training the error imputation model.
Without loss of generality, assume the error imputation model can be presented as
[Equation]
where h is an arbitrary function, φ ( · ) is a known function, such as identity, sigmoid, or exponential functions.
The basic idea of TDR consists of the following two steps.
Step 1 (Initialization).
Let ˆ e u,i =
φ { ˆ h ( x u,i ) } be the imputed error obtained by using any of the existing DR methods.
Step 2 (Targeting).
Update ˆ e u,i by fitting an extended one-parameter model as follows:
[Equation]
which includes a single variable 1 / ˆ p u,i − 1 and the offset ˆ h ( x u,i ) .
The parameter η is solved by minimizing the difference (e.g., squared loss and cross entropy loss) between ˜ e u,i ( η ) and e u,i in the exposed events (see Appendix C more details).
Then the proposed TDR estimator is given as
[Equation]
The targeting step enlarges the hypothesis space of ˜ e u,i compared to ˆ e u,i , and does not sacrifice the accuracy of the error imputation model, due to the introduce of an error correction term 1 / ˆ p u,i −to approximate e u,i .
Theorem 1 shows the validity of TDR (see Appendix C for the proof).
Theorem 1.
The imputed error ˜ e u,i obtained with TDR satisfies the following properties: (a) (validity) ˜
e u,i satisfies equation (2), which implies TDR would have smaller bias than EIB and smaller variance than DR based on the initial imputed error ˆ e u,i .
(b) (preservation) ˆ η in the targeting step will converge to 0 and renders ˜ e u,i =
ˆ e u,i when ˆ e u,i already satisfies equation (2).
From Theorem 1, TDR guarantees that equation (2) always holds, regardless of the choice of the initial imputed errors.
In addition, TDR inherits the desirable properties of EIB, such as boundedness and robust to small propensities, since equations (1) and (2) implies that the TDR estimator can be regarded as an EIB estimator.
TDR would reduce the variance of DR as shown in Theorem 1, a further question is whether the variance-reduction will come at the expense of an increase in bias?
Remarkably, TDR has no sacrifice of bias.
Specifically, It can be shown (see Appendix D) that the bias of both L DR and L TDR are composed of the product of the errors of the propensity model and imputation model weighted by 1 / ˆ p u,i .
Therefore, given the same learned propensities, the more accurate the imputed errors are, the smaller the bias is.
Since TDR updates ˆ e u,i by adding an extra term 1 / ˆ p u,i − 1 , so ˜ e u,i is expected to be more accurate than ˆ e u,i , resulting in a smaller bias for L TDR than L DR .
Importantly, the TDR provides a model-agnostic framework due to the free choice of the initial imputed errors in Step 1, which has great potentially strengths for recommendation.
TDR can be assembled into any competing DR estimator (Wang et al., 2019; Guo et al., 2021; Dai et al., 2022), by updating its error imputation model with the targeting step.
This extra targeting step tends to reduce both the bias and variance of the competing DR estimator, resulting in more accurate predictions.
Next, Theorem 2 indicates the double robustness of the TDR estimator (see Appendix D for proofs).
Theorem 2.
The proposed TDR estimator have the following properties:
(a) (unbiasedness under accurate imputed errors)
L TDR is unbiased if ˜ e u,i accurately estimates g u,i .
(b) (unbiasedness under accurate learned propensities) Suppose that ˆ p u,i is an accurate estimate of p u,i and the validity of ˆ e u,i doesn’t hold
, then
L EIB is biased
while L TDR is unbiased.
Besides, Theorem 2(b) reveals that TDR could remove the bias of L EIB even though the initial imputed errors are inaccurate, provided the learned propensities are accurate.
4. S EMI - PARAMETRIC C OLLABORATIVE L EARNING
In this section, we propose a novel TDR-based collaborative learning (TDR-CL) approach, in which the imputed errors ˜ e u,i are decomposed into a parametric error imputation model part ˆ e u,i and a nonparametric targeting part
ω u,i ≜
η (1 / ˆ p u,i − 1) in Section 4.1, with φ being the identify function in equation (3), i.e., ˜ e u,i =
ˆ e u,i + ω u,i .
The latter corrects the residuals of the error imputation model.
By updating both the parametric and nonparametric parts collaboratively, the bias and variance of the TDR estimator can be further reduced, resulting in more accurate predictions.
First, the embedding of each user u and item i is obtained by matrix factorization, and the stack layer gets the embedding x u,i by concatenation.
TDR-based learning methods require estimated propensities for all user-item pairs, thus the Naive Bayes approach is no longer applicable.
To handle this problem, the pre-trained propensities are obtained by conducting logistic regression of o u,i on x u,i , and the model parameters are used as the initialization of p ξ ( x u,i ) in the iterative learning process.
Given both the parametric error imputation model part ˆ e u,i = g ϕ ( x u,i ) and nonparametric targeting part ω u,i , the propensity model p ξ ( x u,i ) and the prediction model f θ ( x u,i ) are updated ∗ min η u e u i − ˆ e u i
− η simultaneously using the training loss
L TDR − CL ( θ, ξ, ϕ ) =
L TDR + |D| − 1 (cid:88) ( u,i ) ∈D [ − o u,i · log ˆ p u,i − (1 − o u,i ) · log(1 − ˆ p u,i )] , where ˆ p u,i = p ξ ( x u,i ) , e u,i =
( f θ ( x u,i ) − r u,i (1)) 2 , ˜ e u,i =
( f θ ( x u,i ) − g ϕ
( x u,i ) − ω u,i − ⊥
( f θ ( x u,i ))) 2 with ⊥ the operator that sets the gradient of the operand to zero thus ∇ θ ⊥ ( f θ ( x u,i ))
= 0 and ⊥ ( f θ ( x u,i ))
=
f θ ( x u,i ) .
Then, unlike traditional alternative learning algorithms that directly use the parametric part g ϕ ( x u,i ) as ˜ e u,i , the proposed collaborative learning additionally uses ω u,i as a non-parametric correction term summed with g ϕ ( x u,i ) to correct the estimation of e u,i .
Specifically, given the prediction model and the propensity model, ˜ e u,i first updates its parametric part g ϕ
( x u,i ) by minimizing
[Equation]
where e u,i = r u,i (1) − f θ ( x u,i ) , ˜ e u,i =
g ϕ
( x u,i ) + ω u,i .
Next, the targeting step described in
Section 4.1 is applied to further update the imputed errors ˜ e u,i .
Through calculating the optimal step size for line search η ∗ = arg min η (cid:80) o u,i ( e u,i ( θ ) − ˆ e u,i ( ϕ ) − η (1 / ˆ p u,i − 1)) 2 , the non parametric targeted error term ω u,i is updated by adding η ∗ (1 / ˆ p u,i − 1) .
In summary, the proposed learning approach collaboratively update the parametric term ˆ e u,i = g ϕ
( x u,i ) and the nonparametric term ω u,i to achieve a better trade-off to estimate e u,i , which can reduce the bias of the existing DR methods such as DR-JL (Wang et al., 2019) and MRDR-DL (Guo et al., 2021), by further modeling for the fitted residuals of the parametric parts ˆ e u,i .
On the other hand, as shown in Theorems 1 and 2, when ˆ e u,i is already an accurate estimate of e u,i , the introduction of the targeted error term ˜ e u,i satisfies no-harm property and the unbiasedness is maintained.
We summarized the proposed TDR-CL approach in Alg. 1.
S EMI - SYNTHETIC E XPERIMENTS
In this section, following the previous studies (Schnabel et al., 2016;
Wang et al., 2019; Saito, 2020;
Guo et al., 2021), we aim to answer the following research question (RQ) on the semi-synthetic dataset:
RQ1.
Does the proposed TDR estimator in estimating the ideal loss have both the statistical properties of lower bias and variance in the presence of selection bias?
[Table]
5. E XPERIMENTAL S ETUP
Dataset and Preprocessing.
MovieLens 100K 1 (ML-100K) is a dataset of 100,000 missing-not-atrandom (MNAR) ratings from 943 users and 1,682 movies collected from movie recommendation ratings.
MovieLens 1M 2 (ML-1M) is a larger dataset of 1,000,209 MNAR ratings from 6,040 users and 3,952 movies.
Following the standard procedure of previous studies (Schnabel et al., 2016;
Wang et al., 2019; Saito, 2020; Guo et al., 2021), the data preprocessing procedure and predicted metrics are provided in Appendix E.
Experimental Details.
For each prediction matrix ˆR = { ˆ r u,i (1) : ( u, i ) ∈ D} , the proposed TDR is compared with Naive (Koren et al., 2009), EIB (Hern ´ andez-Lobato et al., 2014; Steck, 2010),
IPS (Saito et al., 2020; Schnabel et al., 2016), and DR (Wang et al., 2019; Saito, 2020) methods.
We obtain the propensities by 1 / ˆ p u,i = (1 − β ) /p
u,i + β/p e , where p e = |D| − 1 (cid:80) ( u,i ) ∈D
o u,i , and β is randomly sampled from [0 , 1] to introduce noises.
Define ˆ e u,i = CE
( (cid:80) ( u,i ) ∈O r u,i w u,i , ˆ r u,i ) , where w u,i
= (1 / ˆ p u,i ) (cid:14) ( (cid:80) ( u,i ) ∈O 1 / ˆ p u,i ) , CE denotes the cross entropy loss.
For EIB and DR, the imputed error is computed as ˜ e u,i =
ˆ e u,i , For TDR, ˜ e u,i =
ˆ e u,i + η ∗ (1 / ˆ p u,i − 1) , where η ∗ = arg min η (cid:80) ( u,i ) ∈O ( e u,i − ˆ e u,i − η (1 / ˆ p u,i − 1)) 2 .
The performance of the estimators is based on the absolute relative error (RE) of the estimated and ideal loss RE( L estimator ) =
|L ideal ( ˆ R ) −
L estimator ( ˆ R ) | / L ideal ( ˆ R ) , where L estimator denotes the estimator to be compared.
RE evaluates the accuracy of the estimated loss, and a smaller RE value indicates a higher estimation accuracy.
5. E XPERIMENT R ESULTS (RQ1)
In Table 2, we report the means and standard deviations of the RE of the five estimators for each predicted matrix over 20 times of sampling.
On the one hand, the average RE of the IPS, DR and
TDR methods is significantly lower than that of the Naive method, verifying the validity of causalbased debiasing methods.
The proposed TDR achieves the lowest RE in all settings, attributed to the introduced correction term ω u,i for estimating e u,i , that further reduces the bias of DR.
The direct application of the EIB method is even worse than the Naive method, attributed to the challenge to make an accurate estimate of e u,i .
On the other hand, same as the conclusion of Theorem 1, the standard deviation of the EIB method is significantly lower compared to the IPS and DR methods.
The proposed TDR method combines the advantages of the EIB in terms of lower standard deviations than IPS and DR in all settings, reflecting stronger robustness.
It can be concluded that the estimation accuracy and robustness of the proposed method are significantly improved compared to the previous methods.
R EAL - WORLD E XPERIMENTS
In this section, we conduct experiments to evaluate the proposed methods on two real-world benchmark datasets containing missing-at-random (MAR) ratings.
Throughout, our methods are imple- mented without uniform data to estimate the propensities, which differs from the existing Naive
Bayes approach.
We aim to answer the following RQs:
RQ2.
How do the proposed methods compare with the existing models in terms of debiasing performance in practice?
RQ3.
Does the proposed targeting and collaborative learning result in additional time expense?
RQ4.
How does the collaborative learning phase design affect the performance of our methods?
RQ5.
Do our methods stably perform well under different learned propensities?
6. E XPERIMENTAL S ETUP
Dataset and Preprocessing.
MAR ratings are necessary to evaluate the performance of debiasing methods on real-world datasets.
Following previous studies, we take the following two benchmark datasets: Coat Shopping 3 has 4,640 MAR and 6,960 MNAR ratings of 290 users to 300 coats.
Yahoo!
R3 4 has 54,000 MAR and 311,704 MNAR ratings of 15,400 users to 1,000 songs.
Baselines.
We take the widely used Matrix Factorization (MF) as the base model (Koren et al., 2009), and compare the proposed methods with the following baselines: Base Model (Koren et al., 2009), IPS (Schnabel et al., 2016), SNIPS (Swaminathan & Joachims, 2015), IPS with asymmetric training (IPS-AT) (Saito, 2020), CVIB (Wang et al., 2020b), DIB (Liu et al., 2021), DR (Saito, 2020), DR-JL (Wang et al., 2019), DR-CL, MRDR-JL (Guo et al., 2021), MRDR-CL, where DRCL and MRDR-CL are performed using the proposed Alg. 1, but without the targeting step update (lines 9-11), also for comparison purpose.
In addition, the proposed TDR-based methods include
TDR, TDR-JL, and TMRDR-JL executed by a single targeting step, and TDR-CL and TMRDRCL executed by collaborative learning approach as shown in Alg. 1.
The real-world experimental protocols and details are provided in Appendix F.
6. P ERFORMANCE C OMPARISON (RQ2, RQ3)
In Table 3, we report the performance of various debiasing methods using MSE, AUC, NDCG@5, and NDCG@10 as evaluation metrics.
For previous de-biasing methods, propensity-based IPS, SNIPS, IPS-AT, and information bottleneck-based CVIB and DIB all outperform the base model. The doubly robust methods, such as DR-JL, DR-CL, MRDR-JL, and MRDR-CL, using alternating learning and outperforming DR, which are considered as the most competitive baselines.
The proposed TDR estimators are implemented by both single-step and collaborative learning, respectively, based on DR and MRDR as initialized error imputation models, outperforming the baseline methods significantly in all AUCs, NDCG@5, and NDCG@10, attributed to the effectiveness of the introduced nonparametric correction term.
It is noted that the collaborative version of TDR achieves optimal performance both within DR and MRDR, which implements the proposed targeting step repeatedly.
The fact that TDR-JL and TMRDR-JL implemented the targeting step only at the final training of the prediction models outperformed DR-JL and MRDR-JL, respectively, further confirms the effectiveness of the proposed use of the targeting step to correct imputed errors.
Moreover, it is theoretically clear that TDR-JL and TDR-CL do not increase the time complexity compared to DRJL.
We further compared the empirical time expenses when training the prediction model with and without the proposed targeting steps, as shown in Table 4.
It can be summarized that our methods do not have additional empirical time expenses while improving the prediction accuracy.
6. I N - DEPTH A NALYSIS (RQ4, RQ5)
Ablation Study (RQ4).
To illustrate the specific reasons for the effectiveness of the TDR-CL algorithm, we conduct ablation studies on DR-based and MRDR-based methods, respectively.
From Fig.
1, DR-CL and DR-JL perform similarly on MSE, AUC and NDCG@5 metrics, and the MRDR approach has similar findings, which indicates that the directly use of collaborative learning approach without targeting steps cannot improve prediction performance.
However, for the proposed TDR-CL and TMRDR-CL methods, there is a significant performance improvement compared to the DR-CL
[Table]
[Table]
and MRDR-CL methods without targeting steps.
This ablation study confirms that the improvement in the proposed TDR-CL and TMRDR-CL methods originates from the nonparametric correction term of the imputed errors and not from introducing additional model parameters for updating.
Effect on Learned Propensities (RQ5).
An important fact is that the nonparametric correction term in the TDR estimator is based on given learned propensities.
In order to examine whether the proposed targeting steps stably help to improve the prediction accuracy under different learned propensities obtained by setting different clipping threshold, we conducted repeated experiments to quantify the sensitivity of the TDR-CL method to the propensity clipping threshold.
From Fig.
2, the proposed method outperforms the DR-JL and DR-CL methods in terms of AUC, NDCG@5, and NDCG@10 on all clipping thresholds.
The optimal performance is reached when the clipping threshold is equal to 0.15, which is interpreted as achieving the best trade-off between information utilization and robustness.
R ELATED W ORK
Debiasing in Recommendation.
Bias is a common problem inherent in RS (Chen et al., 2020;
Wu et al., 2022), such as popularity bias (Zhang et al., 2021), model selection bias (Yuan et al., 2019), user self-selection bias (Saito, 2020), position bias (Ai et al., 2018), and conformity bias (Liu et al., 2016).
Various methods were proposed for unbiased learning.
For example, Schnabel et al.
(2016) considered the recommendation as treatment and introduced the IPS and self-normalized
IPS (SNIPS) methods to debiasing in explicit feedback data.
Saito et al.
(2020) extended it to the implicit recommendation.
Wang et al.
(2019) proposed a doubly robust joint learning approach that improved the IPS method.
Subsequently, a series of enhanced DR methods were developed, such as MRDR (Guo et al., 2021), Multi-task DR (Zhang et al., 2020), DR-MSE (Dai et al., 2022), and
BRD-DR (Ding et al., 2022).
In addition, several new debiasing algorithm are designed via using an extra small uniform dataset (Bonner & Vasile, 2018; Chen et al., 2021; Liu et al., 2020; Wang et al., 2021).
Chen et al.
(2020) provided a thorough discussion the recent progress on debiasing tasks in RS.
Wu et al.
(2022) established a unified causal analysis framework and gave formal causal definitions of various biases in RS from the perspective of causal inference.
Unlike the existing enhanced DR approaches that purse a better bias-variance trade-off, the proposed TDR reduces both the bias and variance and is theoretically guaranteed.
[Figure]
Targeted Learning.
Targeted learning is a general framework in causal inference (van der Laan &
Rose, 2011) that includes many field-specific approaches to accommodate various scientific problems in different fields, such as survival analysis (Stitelman et al., 2012), genomics (Gruber & van der Laan, 2010), epidemiology (Rose & van der Laan, 2014) and etc.
More application scenarios about targeted learning can refer to the two excellent monographs (van der Laan & Rose, 2011; 2018).
Shi et al.
(2019) proposed adapting neural networks for estimating the average treatment effects based on targeted learning.
Different from the existing literature of targeted learning, this paper deals with the estimator and learning problem simultaneously.
To the best of our knowledge, this is the first paper that extends targeted learning to the field of debiased recommendation.
C ONCLUSION
In this paper, we propose a TDR estimator for debiased recommendation that enjoys the properties of double robustness, boundedness, low variance, and robustness to small propensities simultaneously.
Theoretical analysis shows that TDR can effectively reduce the bias and variance simultaneously for any DR estimator when the error imputation model is less accurate.
In addition, we further propose a novel uniform-data-free TDR-based collaborative learning approach that adaptively executes the targeting step, thus making the prediction model more robust.
We conducted experiments on both semi-synthetic and real-world data.
The superiority of the proposed method is demonstrated when compared with existing debiasing methods.
Throughout, we adopt 1 / ˆ p u,i − 1 as a key choice of targeting step to satisfy equation (2), which is essentially a first-order targeted learning Carone et al.
In future work, we will explore higher-order targeted learning and more effective feature selection in the targeting step.
E THICS S TATEMENT
This work is mostly theoretical and experiments are based on synthetic and public datasets.
We claim that this work does not present any foreseeable negative social impact.
R EPRODUCIBILITY S TATEMENT
Code is provided in Supplementary Materials to reproduce the experimental results.
A D ETAILED COMPARISON OF THE TYPICAL DEBIASING ESTIMATORS
Doubly robust .
It is well known that DR enjoys the property of double robustness, which means that
L DR is an unbiased estimator of L ideal if either ˆ e u,i estimates g u,i accurately or ˆ p u,i estimates p u,i accurately.
In contrast, IPS and EIB do not meet the property of double robustness.
Robust to small propensities.
Both the IPS and DR use 1 / ˆ p u,i as the weight to recover the target distribution (Horvitz & Thompson, 1952).
In the presence of small propensities, the weights will become extremely large and cause instability.
In contrast, EIB does not suffer from such a problem (Imbens & Rubin, 2015; Tan, 2007).
Boundedness.
Both the IPS and DR may lie outside the range of L ideal , i.e., they do not enjoy the property of boundedness.
For example, if we set e u,i ∈ [0 , 1] , then L ideal ∈ [0 , 1] , while L IPS and
L DR may not be within the range.
The EIB can guarantee boundedness property easily if the error imputation model is chosen appropriately (Molenberghs et al., 2015; van der Laan & Rose, 2011).
Without extrapolation.
EIB usually has a large bias, which is a consequence of making implicitly extrapolations.
Specifically, the error imputation model is trained with exposed events while using the predicted values for unexposed events.
This relies heavily on extrapolation since the exposed events are sparse and there may exist a significant difference between the distributions of exposed events and unexposed events.
Thus, it is hard to obtain accurate error imputation and leads to poor performance (Molenberghs et al., 2015).
In comparison, the estimation of propensity score doesn’t rely on extrapolation.
Low variance.
EIB has the smallest variance among these methods, the associated results are presented in Proposition 1 of the main text.
B P ROOF OF P ROPOSITION
Recall that p u,i = P
( o u,i =
| x u,i ) =
E [ o u,i | x u,i ] and g u,i =
E [ e u,i | x u,i ] , both of them are functions of x u,i .
Throughout, we maintain the common unconfoundedness assumption (i.e., r u,i (1) ⊥⊥ o u,i | x u,i ) and the consistency assumption, (i.e., r u,i (1) =
r u,i if o u,i = 1 ).
All the lower-case letters denote random variables for simplification.
Proof of Proposition 1.
The property of unbiasedness is obvious.
Next, we focus on analysing the variance.
Define
[Equation]
then E [ e 2 u,i | x u,i ]
= σ 2 ( x u,i ) + g 2 u,i .
The variance of IPS estimator is given by
Var( L IPS ) = |D| − 1 · Var( o u,i e u,i p u,i ) =
|D| − 1 · (cid:34)
E [ o 2 u
[Equation]
i
[Equation]
2 u
[Equation]
i p 2 u
[Equation]
(cid:110)
[Equation]
(cid:111)
[Equation]
(cid:35)
[Equation]
|D| − 1 · (cid:34)
E (cid:40) E [
[Equation]
u,i |
[Equation]
u,i ] ·
[Equation]
[ e 2 u
[Equation]
i | x u
[Equation]
i ] p
[Equation]
u
[Equation]
(cid:110)
[Equation]
(cid:18)
[Equation]
[ o u
[Equation]
i |
[Equation]
u
[Equation]
i
[Equation]
E
[Equation]
e
[Equation]
i
[Equation]
p
[Equation]
(cid:19) (cid:111)
[Equation]
(cid:35) = |D|
[Equation]
1 · (cid:34)
E [
[Equation]
p u
[Equation]
(cid:110)
[Equation] [Equation]
u
[Equation]
(cid:111)
[Equation]
where the third equation follows by the law of iterated expectations and the unconfoundedness assumption.
The variance of DR estimator is derived by
[Equation]
where the fifth equation holds by noting that
[Equation]
Since E [ o u,i e u,i + (1 − o u,i ) g u,i ] =
E [ g u,i ] =
E [ e u,i ] , we have
[Equation]
C D ETAILS OF THE TARGETING STEP AND THE PROOF OF T HEOREM
This section provides more details about the estimation of η in the targeting step of TDR.
Without loss of generality, we assume error imputation can be presented as
[Equation]
(5) where h ( · ) is an arbitrary function, φ ( · ) is the inverse canonical link function.
The implication of ”canonical” means that the density function of e u,i given x u,i has the following form (similar to generalized linear model)
[Equation]
where ψ is the scale parameter (a nuisance parameter), ϕ is the parameter of interest.
Different density functions have different functions of b ( · ) and c ( · ) .
For example,
• Gaussian distribution: φ ( t ) =
t ,
[Equation]
Thus, ψ = σ
, b
t
=
t
[Equation] [Equation]
Then the maximum likelihood estimation of ϕ is equivalent to the least squares estimation of ϕ .
• Binomial distribution: φ ( t ) = exp(
[Equation] [Equation]
1 + exp
[Equation]
Thus, ψ = 1 , b (
t
=
log {
1 +
exp( t ) } ,
c ( t
, ψ )
= 1 .
Then
the
maximum likelihood estimation of ϕ
is equivalent to the estimation of ϕ trained with cross entropy loss.
Lemma 1.
For model (5), the score function (the first derivative of log likelihood function) of ϕ based on the observed events is given as
[Equation]
Similarly, for model (4), the the score function of η based on the observed events is given as
[Equation]
Proof.
It suffices to show equation (6).
By taking the first-derivative with respect to h ϕ ( x u,i ) of the following equation
[Equation]
it follows that
[Equation]
The likelihood function is given as
[Equation]
Letting ∂ log L ( ϕ, ψ ) /∂ϕ
=
yields that
[Equation]
which implies the equation (6).
Proof of Theorem 1.
According to Lemma 1, taking the first derivative of the loss function of η leads to that
[Equation]
which implies that
[Equation]
namely, equation (2) holds.
This finishes the proof of Theorem 1(a).
If ˆ e u,i already satisfies equation 2), then η = 0 is a solution of equation (7).
Let ˆ η is another solution of equation (7).
Since the solution of equation (7) is unique, then ˆ η will converges to 0.
This proves the conclusion of
Theorem 1(b).
D P ROOF OF T HEOREM
Proof of Theorem 2.
The result of Theorem 2(a) is obvious.
To show Theorem 2(b).
We first claim that if ˆ e u,i is an accurate estimate of g u,i , i.e., ˆ e u,i = g u,i , then it will satisfy equation (2).
It holds immediately from the following calculations
[Equation]
=
[Equation]
Thus, if ˆ e u,i not satisfy equation (2), then ˆ e u,i ̸
= g u,i .
Given ˆ e u,i and ˜ e u,i , the bias of L EIB is
Bias ( L EIB
[Equation]
and the bias of L TDR is
[Equation]
This proves the result of Theorem 2(b).
Biases of DR and TDR.
Given ˆ p u,i and ˜ e u,i for all ( u, i ) ∈ D , the bias of TDR is
Bias ( L TDR ) =
E (cid:18) e u,i + ( o u,i − ˆ p u,i ) ˆ p u,i ( e u,i − ˜ e u,i ) (cid:19) − E
[ e u,i ] =
E (cid:18) E ( o u,i − ˆ p u,i | x u,i ) ˆ p u,
i
E { e u,i − ˜ e u,i | x u,i } (cid:19) =
E (cid:18) ( p u,i − ˆ p u,i ) ˆ p u,i · ( g u,i − ˜ e u,i ) (cid:19) .
Similarly, given ˆ p u,i and ˆ e u,i for all ( u, i ) ∈ D , the bias of DR is
Bias ( L (0) DR ) =
E (cid:18) ( p u,i − ˆ p u,i )
ˆ p u,i · ( g u,i − ˆ e u,i ) (cid:19) .
E S EMI - SYNTHETIC E XPERIMENTAL S ETUP D ETAILS
Data Preprocessing.
(1) Use matrix factorization Koren et al.
(2009) to complete the rating matrix, then all ratings are sorted in ascending order and truncated based on a more realistic rating distribution [ p 1 , p 2 , p 3 , p 4 , p 5 ] Marlin & Zemel (2009), setting the first bottom p 1 fraction to R u,i = 1 , the next p 2 fraction to R u,i = 2 , and so on.
(2) For each predicted ratings R u,i ∈ { 1 , 2 , 3 , 4 , 5 } , assign the CTR p u,i ∈ (0 , 1) with p u,i = pα max(1 , 5 − R u,i ) , where p is set to 1 and α is set to 0 .
5 in our experiments.
(3) Transform the predicted ratings R u,i into true CVR r trueu,i by correspondingly replacing the rating { 1 , 2 , 3 , 4 , 5 } with the conversion rate { 0 . 1 , 0 . 3 , 0 . 5 , 0 . 7 , 0 . 9 } , and sample the binary click indicator and conversion label with the Bernoulli sampling o u,i ∼ Bern( p u,i ) , r u,i ∼ Bern (cid:0)
r trueu,i (cid:1) , ∀ ( u, i ) ∈
D , where Bern( · ) denotes the Bernoulli distribution.
Thereafter, we can derive a fully-observed conversion label matrix R and a click indicator matrix O .
Predicted Metrics.
The following prediction metrics are used to evaluate the debiasing performance under different scenarios.
• ONE: ˆ r u,i is identical to the true CVR r true u,i , except that |{ ( u, i ) | r true u,i = 0 .
9 }| randomly selected true CVR of 0 .
1 are flipped to 0.9.
• THREE:
Same as ONE, but flipping true CVR of 0 .
3 instead.
• FIVE:
Same as ONE, but flipping true CVR of 0 .
5 instead.
• ROTATE: ˆ r u,i =
r u,i − 0 .
2 when r u,i ≥ 0 .
3 , and ˆ r u,i = 0 .
9 when r u,i = 0 .
1 . • SKEW:
ˆ r u,i is sampled from the Gaussian distribution N (cid:0) µ = r true u,i , σ = (1 − r true u,i ) / 2 (cid:1) , and clipped to [0 . 1 , 0 . 9] .
• CRS: ˆ r u,i = 0 .
2 if the true CVR r true u,i ≤ 0 .
Otherwise, ˆ r u,i = 0 .
F R EAL - WORLD E XPERIMENTAL P ROTOCOLS AND D ETAILS .
Experimental protocols and details.
For real-world experiments, the following four metrics were considered as the evaluation metrics: MSE, AUC, NDCG@5, and NDCG@10 .
For fast convergence in the learning phase, Adam is utilized as the optimizer for all models.
We tune the learning rate in { 0 . 001 , 0 . 005 , 0 . 01 , 0 . 05 , 0 . 1 } , weight decay in [1e − 6 , 1e − 2] at 10x multiplicative ratio, and batch size in { 128 , 256 , 512 , 1024 , 2048 } for Coat and { 1024 , 2048 , 4096 , 8192 , 16384 } for Yahoo! R3 .
Specifically for the propensity training, we tune the clipping threshold in { 0 . 05 , 0 . 10 , 0 . 15 , 0 . 20 } .
After finding out the best configuration on the validation set, we evaluate the trained models on the MAR test set.
Experiments are conducted using NVIDIA GeForce RTX 2060 as the compute resources.