Unbiased Recommender Learning from
Biased Graded Implicit Feedback
Yuta Saito

Suguru Yaginuma

Cornell University
ys552@cornell.edu

MC Digital, Inc.
suguru@mcdigital.jp

Taketo Naito

Kazuhide Nakata

SMN Corporation
taketo_naito@so-netmedia.co.jp

Tokyo Institute of Technology
nakata.k.ac@m.titech.ac.jp

ABSTRACT
Binary user-behavior logs such as clicks or views, called implicit
feedback, are often used to build recommender systems becauseof its general availability in real practice. Most existing studies
formulate implicit feedback as binary relevance feedback. However,in numerous applications, implicit feedback is observed not only asa binary indicator but also in a graded form, such as the number ofclicks and the dwell time observed after a click, which we call thegraded implicit feedback. The grade information should appropriately be utilized, as it is considered a more direct relevance data compared to the mere implicit feedback. However, a challenge is thatthe grade information is observed only for the user–item pairs with
implicit feedback, whereas the grade information is unobservablefor the pairs without implicit feedback. Moreover, graded implicit
feedback for some user–item pairs is more likely to be observed
than for others, resulting in the missing-not-at-random (MNAR)
problem. To the best of our knowledge, graded implicit feedback
under the MNAR mechanism has not yet been investigated despite
its prevalence in real-life recommender systems. In response, we
formulate a recommendation with graded implicit feedback as astatistical estimation problem and define an ideal loss function ofinterest, which should ideally be optimized to maximize the user
experience. Subsequently, we propose an unbiased estimator for theideal loss, building on the inverse propensity score estimator. Finally,
we conduct an empirical evaluation of the proposed method on a
public real-world dataset.
ACM Reference Format:
Yuta Saito, Suguru Yaginuma, Taketo Naito, and Kazuhide Nakata. 2020.
Unbiased Recommender Learning from Biased Graded Implicit Feedback. In
Proceedings of ACM Conference (Conference’17). ACM, New York, NY, USA,
5 pages. https://doi.org/10.1145/3336191.3371783

1

INTRODUCTION

Recommender systems constitute a central part of numerous existing online platforms, such as e-commerce (e.g., Amazon and Etsy)
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
Conference’17, July 2017, Washington, DC, USA© 2020 Association for Computing Machinery.
ACM ISBN 978-1-4503-6822-3/20/02. . . $15.00
https://doi.org/10.1145/3336191.3371783

and content recommendation platforms (e.g., Netflix, YouTube, andSpotify). The essential step in building such systems is to accurately
estimate the relevance of user–item pairs. A version of user–item
relevance feedback, called implicit feedback, is often used for solving
this task. Implicit feedback is collected from natural behaviors ofusers in recommender systems, and it is generally available in large
volumes, as expert annotation or crowd sourcing is unnecessary.
Most existing studies focus on learning or evaluating recommenders
using the binary implicit feedback, for example, an indicator ofwhether an item is clicked by a user [5, 10, 11, 14, 15, 18, 23]. However, in numerous application domains, implicit feedback involves
not just binary relevance feedback, but it is observed in a graded
form, such as the number of clicks [7]. The so-called graded implicit
feedback can be considered as a more informative form of relevance
feedback compared to the binary counterpart [7]. This is because
graded implicit feedback contains the level of the relevance, and thegrade information should be useful to better capture the user–item
relevance. However, a difficulty is that the grade information is observable only for the user–item pairs with a click, whereas that forthe pairs without a click is unobservable. Furthermore, the graded
information for several pairs of users and items is more likely tobe observed as compared with others. For example, if we deploy
a popularity-based recommendation policy that recommends popular items with high probabilities in the past, the graded implicit
feedback for the popular items is much easier to be observed compared with that for rare items [2, 23]. Therefore, the graded implicit
feedback is often missing-not-at-random (MNAR), as depicted in
Figure 1, and one has to address this biased missing mechanism toadequately improve the user satisfaction [18, 20].
Related Work. Some works employed graded implicit feedback
to train recommendation models. Lerche and Jannach [9] modified
the loss function of Bayesian personalized ranking (BPR) using
confidence weights, which were estimated based on the differencein the reception times of two messages in a Twitter application.
Another related work is Wang et al. [21], wherein a new set of
triplets that comprised a user and two items was created based on
the graded implicit feedback to train BPR, improving the recommendation accuracy of the vanilla BPR [7, 9]. However, all existingstudies on the graded implicit feedback implicitly assumed themissing-completely-at-random mechanism, where the probability ofobserving the graded implicit feedback is uniform among all theuser–item pairs. This assumption is unrealistic in many real-lifeuser-item pairs
without a click

inference spacetraining spaceuser-item pairs
with a click
and high relevanceuser-item pairs
with a click
and low relevance

Figure 1: Missing-not-at-random (MNAR) problem of thegraded implicit feedback. The training space comprises
user–item pairs with a click (𝑌 = 1), and therefore, does not
represent the entire inference space, which comprises all theuser–item pairs (D).

situations, as implicit feedback is often confounded by various factors, such as the past recommendation policy [2, 13] and position
bias [8, 22]. To the best of our knowledge, no theoretical framework
exists in the literature that is able to handle the MNAR the graded
implicit feedback, thereby limiting the real-world application of themethods leveraging this type of feedback.
Contributions. To better utilize graded implicit feedback, we formulate a recommendation using the MNAR graded implicit feedback from a statistical estimation perspective, which allows us totheoretically characterize the bias in using the graded implicit feedback. Based on our formulation, we first define an ideal loss functionof interest that should be optimized to maximize the relevance inthe platform. We then demonstrate that the naive procedure, which
ignores the MNAR mechanism, is an inappropriate practice and issubject to a statistical bias. Subsequently, we propose an unbiased estimator for the ideal loss function inspired by the inverse propensity
score (IPS) estimator in causal inference [6, 16–19, 23]. The proposed estimator is unbiased for the ideal loss function by weighting
the value of the loss function for each user–item pair using theinverse of the propensity score. Furthermore, we demonstrate thatthe propensity score is estimatable in an unbiased fashion in ourgraded implicit feedback setting, even under the MNAR mechanism.
Finally, we empirically demonstrate that the proposed unbiased
learning procedure outperforms various existing baseline methods
on real-world data.

2 PRELIMINARIES
2.1 Problem Setting
Let 𝑢 ∈ U be a user (|U|= 𝑚), 𝑖 ∈ I be an item (|I|= 𝑛), andD = U × I be the set of all user-item pairs. Let 𝑌𝑢,𝑖 be a Bernoulli
random variable, which represents the binary implicit feedback
observed between 𝑢 and 𝑖. If implicit feedback between 𝑢 and 𝑖 isobserved, then 𝑌𝑢,𝑖 is equal to 1, otherwise, it takes 0. Here, 𝑌𝑢,𝑖 = 1
represents the positive relationship between the user-item pair. In
contrast, 𝑌𝑢,𝑖 = 0 does not always imply a negative relationship.
This is because implicit feedback is never observed when the user
is unaware of the item, despite the fact that the item is potentiallyrelevant to the user. To accurately model this implicit nature, we

introduce two additional random variables. The first is the binary
relevance random variable, denoted by 𝑅𝑢,𝑖 . This random variable
represents the relevance between user 𝑢 and item 𝑖, i.e., if item 𝑖 isrelevant to user 𝑢, 𝑅𝑢,𝑖 is equal to 1, otherwise it is equal to 0. The
second is the binary awareness random variable, denoted by 𝑂𝑢,𝑖 .
This random variable represents whether a user 𝑢 is aware of an
item 𝑖.
Note that, in the binary implicit feedback setting, both the relevance and awareness random variables are unobserved, andonly the binary implicit random variable is observable. Following the implicit feedback generation model used in previous studies [10, 15, 18], we assume the following relationships between theimplicit feedback, relevance, and awareness random variables.
Assumption 2.1. Positive implicit feedback between a user-item
pair is observed if and only if the item is relevant to the user and theuser is aware of the item, i.e., 𝑌𝑢,𝑖 = 𝑂𝑢,𝑖 · 𝑅𝑢,𝑖 .
This assumption implies that the item has to be relevant to theuser (𝑅𝑢,𝑖 = 1), and the user has to be aware of the item (𝑂𝑢,𝑖 = 1)
for the implicit feedback to be observed (𝑌𝑢,𝑖 = 1).
In the binary implicit feedback setting (BI), we can observe only
the binary implicit feedback, and the training dataset for learning
recommendation algorithms can be represented as:
D𝐵𝐼 = {(𝑢, 𝑖, 𝑌𝑢,𝑖 )}(𝑢,𝑖)∈D
In contrast, in our graded implicit feedback setting (GI), we
assume that the confidence score of a user-item pair being relevant
to each other (𝛾𝑢,𝑖 ) is observable as grade information only for thepair with implicit feedback (𝑌𝑢,𝑖 = 1). Therefore, the training dataset
for learning recommendation algorithms in the GI setting can be
represented as:
D𝐺𝐼 = {(𝑢, 𝑖, 𝛾𝑢,𝑖 ) | 𝑌𝑢,𝑖 = 1}(𝑢,𝑖)∈DThis GI setting is general in many application domains. For example, in the music recommendation problem, the dwell or consumption times that are observed after the click of a song are considered
as interest signals with different strengths [4, 24]. Another example
is the repeated consumption of items in an online media service,
which should be interpreted as a stronger signal than a single consumption event [7]. We formulate these types of general real-world
situations by treating the relevance parameter 𝛾𝑢,𝑖 as different levels of relevance signal. This formulation allows us to understandthe bias of the MNAR graded implicit feedback from a theoreticalperspective.

2.2

Ideal Loss Function of Interest

Here, we describe the objective of this study. To evaluate recommendation algorithms in the BI setting, top-N recommendation
metrics such as the mean average precision (MAP), discounted cumulative gain (DCG), and recall are often used [12, 23]. To measure theimprovement in user experience by recommendation algorithms,
we rely on the following top-N recommendation metric definedwith the ground-truth relevance level.
 


X X
b = 1
R𝑟𝑒𝑙 𝒁𝛾𝑢,𝑖· 𝑐 𝑍b𝑢,𝑖(1)|U| 𝑢 ∈U 𝑖 ∈I|{z}relevance level

The focus of this study is to optimize the performance metricin Eq. (1). To achieve this goal, we aim at optimizing the following
ideal pointwise loss function, which is designed to minimize theprediction loss for the ground-truth relevance parameter.

1 X
𝑟𝑒𝑙
L𝑖𝑑𝑒𝑎𝑙(𝑓 ) =
𝛾𝑢,𝑖 𝛿 (1) (𝑓 (𝑢, 𝑖)) + 1 − 𝛾𝑢,𝑖 𝛿 (0) (𝑓 (𝑢, 𝑖)) (2)𝑚𝑛 (𝑢,𝑖)∈ D
where 𝑓 : D → [0, 1] is a relevance predictor and 𝛿 (𝑅) denotes the
local loss for the user-item pair (𝑢, 𝑖). For example, if 𝛿 (𝑅) (𝑓 (𝑢, 𝑖)) :=(𝑅 − 𝑓 (𝑢, 𝑖))2 , then Eq. (2) is referred to as the mean squared loss. In(·)
the following, we use 𝛿 (·) (𝑓 (𝑢, 𝑖)) to denote 𝛿𝑢,𝑖 for simplicity.
A relevance predictor 𝑓 , minimizing the ideal loss defined using
the relevance level in Eq. (2), is expected to lead to the desired
values of the top-N recommendation metric in Eq. (1). However,
we cannot handle the ideal loss function directly in our GI setting,as the ground-truth relevance parameter for the user-item pairwithout implicit feedback (i.e., (𝑢, 𝑖) with 𝑌𝑢,𝑖 = 0) is unobservable.
Therefore, accurately estimating the ideal loss function with only
observable data (i.e., D𝐺𝐼 ) is important to maximize relevance frombiased graded implicit feedback.

3

METHODIn this section, we first show that the naive use of graded implicit
feedback leads to a biased estimation of the ideal loss function.
Then, we propose an approach to unbiasedly estimate the ideal loss
function using only the biased graded implicit feedback.

3.1

Bias of Naive Estimator

To learn an arbitrary recommendation model on the graded implicit
feedback dataset D𝐺𝐼 , one can use the following naive estimator
for the ideal loss function.
Definition 3.1. The naive estimator for the ideal loss function is
X
(1)
(0)
b𝑟𝑒𝑙 (𝑓 ) := 1L𝛾𝑢,𝑖 𝛿𝑢,𝑖 + (1 − 𝛾𝑢,𝑖 )𝛿𝑢,𝑖
𝑁 𝑎𝑖𝑣𝑒|D𝐺𝐼 | (𝑢,𝑖)∈ D𝐺𝐼
1 X 𝑌𝑢,𝑖 (1)(0)
𝛾𝑢,𝑖 𝛿𝑢,𝑖 + (1 − 𝛾𝑢,𝑖 )𝛿𝑢,𝑖(3)=
𝑚𝑛 (𝑢,𝑖)∈ D 𝜋
where 𝜋 = |D𝐺𝐼 |/𝑚𝑛 is the sparsity of the GI training dataset.
The above naive estimator is feasible, as it uses only the grade
information for user-item pairs with observed implicit feedback(𝑌𝑢,𝑖 = 1). However, the following proposition indicates that naive
use of graded implicit feedback can lead to a sub-optimal recommender.
Proposition 3.2. The naive estimator in Eq. (3) is statisticallybiased against the ideal loss function of interest in Eq. (2), i.e., for someb𝑟𝑒𝑙 (𝑓 )] ̸= L𝑟𝑒𝑙 (𝑓 ), where the expectation is
given 𝑓 we have E[ L
𝑁 𝑎𝑖𝑣𝑒𝑖𝑑𝑒𝑎𝑙
taken over the relevance and awareness random variables.
Proof. First, we calculate the expectation of the naive estimator
below.
hi
X E[𝑌𝑢,𝑖 ] (1)(0)
b𝑟𝑒𝑙 (𝑓 ) = 1
E L
𝛾𝑢,𝑖 𝛿𝑢,𝑖 + (1 − 𝛾𝑢,𝑖 )𝛿𝑢,𝑖
𝑁 𝑎𝑖𝑣𝑒𝑚𝑛 (𝑢,𝑖)∈ D 𝜋

1 X 𝜃𝑢,𝑖 𝛾𝑢,𝑖 
(1)
(0)
=
𝛾𝑢,𝑖 𝛿𝑢,𝑖 + (1 − 𝛾𝑢,𝑖 )𝛿𝑢,𝑖𝑚𝑛 (𝑢,𝑖)∈ D 𝜋

Therefore, for the naive estimator to satisfy the unbiasedness, theprobability of observing implicit feedback must be a constant, i.e.,𝜃𝑢,𝑖 𝛾𝑢,𝑖 = 𝜋, ∀(𝑢, 𝑖) ∈ D. However, this necessary condition is violated in the MNAR situation, i.e, 𝜃𝑢,𝑖 𝛾𝑢,𝑖 may not be a constant, andthe naive estimator is subject to bias in the sense that its expectation
is not equal to the ideal loss for some given 𝑓 .
□As shown in Proposition 3.2, the expectation of the naive estimator is not necessarily equal to the ideal loss function. This isbecause the naive estimator ignores the distributional shift between
the user-item pairs with and without implicit feedback. Therefore,
the naive use of graded implicit feedback is an inappropriate approach to approximate the ideal loss function. Instead, one should
rely on an estimator alleviating the bias alternative to using thenaive estimator.

3.2

The Proposed Unbiased Estimator

To alleviate the bias of the graded implicit feedback, we propose
the following unbiased estimator.
Definition 3.3. The unbiased estimator for the ideal loss function
is defined asX1(1) 1 − 𝛾𝑢,𝑖 (0)b𝑟𝑒𝑙 (𝑓 ) := 1
L
𝛿
+
𝛿
𝑈𝐵
𝑚𝑛 (𝑢,𝑖)∈D 𝜃𝑢,𝑖 𝑢,𝑖𝛾𝑢,𝑖 𝑢,𝑖𝐺𝐼
1 X 𝑌𝑢,𝑖 (1) 1 − 𝛾𝑢,𝑖 (0)=𝛿𝑢,𝑖 +
𝛿𝑢,𝑖 ,(4)𝑚𝑛 (𝑢,𝑖)∈D 𝜃𝑢,𝑖
𝛾𝑢,𝑖where the awareness parameter 𝜃𝑢,𝑖 is interpreted as the propensity
score in the GI setting.
The following proposition states that the proposed unbiased
estimator is unbiased for the ideal loss function.
Proposition 3.4. The unbiased estimator in Eq. (4) is statisticallyunbiased against the ideal loss function of interest in Eq. (2), i.e., forb𝑟𝑒𝑙 (𝑓 )] = L𝑟𝑒𝑙 (𝑓 ),any given relevance predictor 𝑓 , we have E[ L𝑈𝐵
𝑖𝑑𝑒𝑎𝑙
where the expectation is taken over the relevance and awarenessrandom variables.
Proof.
"
#hi
1 X 𝑌𝑢,𝑖 (1) 1 − 𝛾𝑢,𝑖 (0)𝑟𝑒𝑙b
E L𝑈 𝐵 (𝑓 ) = E𝛿 +𝛿𝑚𝑛 (𝑢,𝑖)∈D 𝜃𝑢,𝑖 𝑢,𝑖𝛾𝑢,𝑖 𝑢,𝑖
1 X E[𝑌𝑢,𝑖 ] (1) 1 − 𝛾𝑢,𝑖 (0)=𝛿𝑢,𝑖 +𝛿𝑢,𝑖𝑚𝑛 (𝑢,𝑖)∈D 𝜃𝑢,𝑖
𝛾𝑢,𝑖
=

1 X
(1)
(0)
𝛾𝑢,𝑖 𝛿𝑢,𝑖 + (1 − 𝛾𝑢,𝑖 )𝛿𝑢,𝑖𝑚𝑛 (𝑢,𝑖)∈D

𝑟𝑒𝑙= L𝑖𝑑𝑒𝑎𝑙(𝑓 )□Proposition 3.4 suggests that the proposed unbiased estimator is
statistically unbiased for the ideal loss function; one can unbiasedly
approximate the ideal loss function of interest using only biased
graded implicit feedback with this estimator.

3.3

Unbiased Propensity Estimation

The proposed unbiased estimator requires the true value of theawareness parameter 𝜃𝑢,𝑖 . However, in our setting, the ground-truth
awareness parameters are unobserved and need to be estimatedfrom the data. For estimating this parameter, we should optimize
the following loss function.
1 X
𝑠𝑐𝑜𝑟𝑒
L𝑖𝑑𝑒𝑎𝑙(𝑔) :=
𝜃𝑢,𝑖 𝛿 (1) (𝑔(𝑢, 𝑖)) + (1 − 𝜃𝑢,𝑖 )𝛿 (0) (𝑔(𝑢, 𝑖)) (5)𝑚𝑛 (𝑢,𝑖)∈ D
where 𝑔 : D → [0, 1] is an awareness parameter estimator.
However, the realizations of the awareness random variables
are unobserved in D𝐺𝐼 , and the direct optimization of Eq. (5) is
infeasible. Therefore, we propose the following unbiased estimator
to estimate the propensity score with only available data.
Definition 3.5. The unbiased estimator for propensity estimation
is defined as follows

𝑌𝑢,𝑖 (0)1 X 𝑌𝑢,𝑖 (1)𝑠𝑐𝑜𝑟𝑒b
L𝑈 𝐵 (𝑔) =𝛿 + 1−𝛿(6)
𝑚𝑛 (𝑢,𝑖)∈ D 𝛾𝑢,𝑖 𝑢,𝑖𝛾𝑢,𝑖 𝑢,𝑖Note that Eq. (6) depends on only observable data. This is because
the ground-truth relevance parameters can be used for the user-item
pair with implicit feedback (𝑌𝑢,𝑖 = 1), whereas those for the pairwithout implicit feedback (𝑌𝑢,𝑖 = 0) are unnecessary to calculate
Eq. (6).
The following proposition provides a theoretical justification of
the unbiased estimator in Eq. (6).
Proposition 3.6. The unbiased estimator for the propensity estimation in Eq. (6) is statistically unbiased for the loss function in Eq.
b𝑠𝑐𝑜𝑟𝑒 (𝑔)] = L𝑠𝑐𝑜𝑟𝑒 (𝑔).
(5), i.e., for any given 𝑔, we have E[ L𝑈𝐵
𝑖𝑑𝑒𝑎𝑙
Proof.
𝑌𝑢,𝑖 (1)𝑌𝑢,𝑖 (0)E[𝑌𝑢,𝑖 ] (1)E[𝑌𝑢,𝑖 ] (0)E
𝛿𝑢,𝑖 + 1 −𝛿𝑢,𝑖 =
𝛿𝑢,𝑖 + 1 −
𝛿𝑢,𝑖
𝛾𝑢,𝑖
𝛾𝑢,𝑖
𝛾𝑢,𝑖
𝛾𝑢,𝑖


𝛾𝑢,𝑖 𝜃𝑢,𝑖 (1)𝛾𝑢,𝑖 𝜃𝑢,𝑖 (0)=𝛿 + 1−
𝛿𝑢,𝑖
𝛾𝑢,𝑖 𝑢,𝑖
𝛾𝑢,𝑖
 (0)
(1)
= 𝜃𝑢,𝑖 𝛿𝑢,𝑖 + 1 − 𝜃𝑢,𝑖 𝛿𝑢,𝑖
Therefore, we obtain the unbiasedness of the estimator from thelinearity of the expectation operator.
□
Thus, by using the unbiased estimator for the propensity estimation, the unobserved awareness parameters can be estimated in
an unbiased fashion from only observable data.

4

EXPERIMENTAL EVALUATIONIn this section, we empirically evaluate the proposed unbiased
learning framework on a public real-world dataset.

4.1

Experimental Setting

4.1.1 Dataset and Preprocessing. We use the Yahoo! R31 dataset
and employ the following preprocessing procedure.
1 http://webscope.sandbox.yahoo.com/(1) Transform five-star ratings into grade information following
a methodology used in the information retrieval domain[1, 3]:
2𝑟 − 1𝛾𝑢,𝑖 = 𝜖 + (1 − 𝜖) 𝑟
2 𝑚𝑎𝑥 − 1
where 𝑟 ∈ {1, 2, 3, 4, 5} denotes a five-star rating, and 𝑟𝑚𝑎𝑥
is the maximum possible rating, which is 5 in our case. Additionally, 𝜖 ∈ [0, 1] controls the noise level in the grade
information. We apply 𝜖 = 0.1 for the training datasets and𝜖 = 0 for the test datasets to evaluate the recommenders
with the ground-truth (noise-free) relevance in the test sets.
(2) Sample binary relevance variable 𝑅𝑢,𝑖 by performing thefollowing Bernoulli sampling:
𝑅𝑢,𝑖 ∼ 𝐵𝑒𝑟𝑛(𝛾𝑢,𝑖 ), ∀(𝑢, 𝑖) ∈ D
where 𝐵𝑒𝑟𝑛(·) denotes Bernoulli distribution.
(3) Define the awareness variable for a user–item pair as follows:(
1 (if item 𝑖 is rated by user 𝑢)
𝑂𝑢,𝑖 =
0 (if item 𝑖 is not rated by user 𝑢)(4) Define dataset as: D𝐺𝐼 = {(𝑢, 𝑖, 𝛾𝑢,𝑖 ) | 𝑌𝑢,𝑖 = 1}, where 𝑌𝑢,𝑖 =𝑂𝑢,𝑖 · 𝑅𝑢,𝑖 . Note that 𝑅𝑢,𝑖 and 𝑂𝑢,𝑖 are unobservable in our
setting and are not used for training recommenders.
4.1.2 Compared Methods. We compare the following methods: (i)Item popularity model (ItemPop): This method always recommends the 𝑘 most clicked items in the training set; therefore, it
is not personalized. (ii) Naive matrix factorization (NaiveMF):
This model predicts the graded implicit feedback (i.e., 𝛾𝑢,𝑖 ) by optimizing the naive loss in Eq. (3) via MF [5]. It ignores the distributional shift, and thus the loss function is biased. (iii) Exposure
matrix factorization (ExpoMF): This model is a state-of-the-art
method for the MNAR binary implicit feedback [10]. This method
is based on a probabilistic model that uses the exposure variable
and optimizes its parameters using an EM-algorithm-like procedure. (iv) Relevance matrix factorization (Rel-MF) [18]: Thismethod is another state-of-the-art method for the MNAR binary
implicit feedback. It obtains the model parameters by optimizing
the unbiased loss function for the ideal loss function defined with
only the binary implicit feedback signals. (v) Matrix factorization
with unbiased estimator (MF-UB): This is our proposed method,which obtains model parameters by optimizing the proposed unbiased loss function in Eq. (4). We estimate the propensity scorebased on the unbiased propensity estimation procedure describedin the previous section.

4.2

Results

We used DCG@K, Recall@K, and MAP@K to measure the recommendation quality in the test sets. The values of 𝐾 were set to {3, 5}
for all metrics. Table 1 summarizes the ranking performances of all
methods on Yahoo! R3. It shows that the proposed MF-UB method
outperforms other baseline methods in all settings by utilizing thegraded implicit feedback in a theoretically grounded fashion. Specifically, MF-UB outperformed the best baselines by 3.5% for DCG@5,
3.0% for Recall@5, and 3.9% for MAP@5. The results suggest that

Table 1: Evaluating the ranking performance of the proposed method against the baselines
DCG@K

MAP@K

𝐾 =3

𝐾 =5

𝐾 =3

𝐾 =5

𝐾 =3

𝐾 =5

ItemPop
NaiveMF
ExpoMF
Rel-MF

0.26400
0.33509
0.32805
0.335230.35476
0.425960.41591
0.42498

0.30034
0.37393
0.36626
0.37479

0.495390.56866
0.55477
0.567440.30379
0.40950
0.40376
0.409090.50232
0.64196
0.630310.64072

MF-UB (ours)

0.348560.44004

0.388450.58493

0.42872

0.66757

one can significantly improve the recommendation quality by appropriately using the biased graded implicit feedback in the rightmanner.

5

Recall@K

Methods

CONCLUSION

We explored a theoretically sophisticated method for improving
user experience with biased graded implicit feedback. In particular,
we formulated the problem as a statistical estimation problem toaddress the inherent bias of the graded implicit feedback. We thendemonstrated that the naive use of the graded implicit feedback
could result in a sub-optimal recommendation, as it ignores the distributional shift between clicked and unclicked events. To address
the bias of the naive estimator, we proposed an unbiased estimator
for the ideal loss function, which can be estimated using observable
graded implicit feedback. We also experimentally demonstrated
that the proposed unbiased estimation approach outperformed thewidely used baselines by effectively utilizing additional grade information to infer the relevance.

REFERENCES[1] Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W Bruce Croft. 2018. Unbiased learning to rank with unbiased propensity estimation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval.
385–394.
[2] Stephen Bonner and Flavian Vasile. 2018. Causal Embeddings for Recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems(Vancouver, British Columbia, Canada) (RecSys ’18). 104–112.
[3] Olivier Chapelle, Donald Metlzer, Ya Zhang, and Pierre Grinspan. 2009. Expected
reciprocal rank for graded relevance. In Proceedings of the 18th ACM conferenceon Information and knowledge management. 621–630.
[4] Paolo Dragone, Rishabh Mehrotra, and Mounia Lalmas. 2019. Deriving Userand Content-specific Rewards for Contextual Bandits. In The World Wide Web
Conference. 2680–2686.
[5] Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for
implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data
Mining. Ieee, 263–272.
[6] Guido W Imbens and Donald B Rubin. 2015. Causal inference in statistics, social,
and biomedical sciences. Cambridge University Press.
[7] Dietmar Jannach, Lukas Lerche, and Markus Zanker. 2018. Recommending basedon implicit feedback. In Social Information Access. Springer, 510–569.
[8] Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased
learning-to-rank with biased feedback. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM, 781–789.

[9] Lukas Lerche and Dietmar Jannach. 2014. Using graded implicit feedback forbayesian personalized ranking. In Proceedings of the 8th ACM Conference on
Recommender systems. ACM, 353–356.
[10] Dawen Liang, Laurent Charlin, James McInerney, and David M Blei. 2016. Modeling user exposure in recommendation. In Proceedings of the 25th International
Conference on World Wide Web. International World Wide Web Conferences
Steering Committee, 951–961.
[11] Steffen Rendle. 2010. Factorization machines. In 2010 IEEE International Conference on Data Mining. IEEE, 995–1000.
[12] Steffen Rendle. 2019. Evaluation Metrics for Item Recommendation under Sampling. arXiv preprint arXiv:1912.02263 (2019).
[13] Yuta Saito. 2020. Asymmetric Tri-training for Debiasing Missing-Not-At-Random
Explicit Feedback. In Proceedings of the 43rd International ACM SIGIR Conference
on Research and Development in Information Retrieval. 309–318.
[14] Yuta Saito. 2020. Doubly Robust Estimator for Ranking Metrics with Post-Click
Conversions. In Fourteenth ACM Conference on Recommender Systems. 92–100.
[15] Yuta Saito. 2020. Unbiased Pairwise Learning from Biased Implicit Feedback.
In Proceedings of the 2020 ACM SIGIR on International Conference on Theory ofInformation Retrieval. 5–12.
[16] Yuta Saito, Gota Morisihta, and Shota Yasui. 2020. Dual Learning Algorithm
for Delayed Conversions. In Proceedings of the 43rd International ACM SIGIR
Conference on Research and Development in Information Retrieval. 1849–1852.
[17] Yuta Saito, Takuma Udagawa, Haruka Kiyohara, Kazuki Mogi, Yusuke Narita,
and Kei Tateno. 2021. Evaluating the Robustness of Off-Policy Evaluation. arXiv
preprint arXiv:2108.13703 (2021).
[18] Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata.
2020. Unbiased Recommender Learning from Missing-Not-At-Random Implicit
Feedback. In Proceedings of the 13th International Conference on Web Search andData Mining. 501–509.
[19] Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, andThorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learningand Evaluation. In Proceedings of The 33rd International Conference on Machine
Learning, Vol. 48. PMLR, 1670–1679.
[20] Harald Steck. 2010. Training and testing of recommender systems on data missing
not at random. In Proceedings of the 16th ACM SIGKDD international conferenceon Knowledge discovery and data mining. ACM, 713–722.
[21] Sheng Wang, Xiaobo Zhou, Ziqi Wang, and Ming Zhang. 2012. Please spread:
recommending tweets for retweeting with implicit feedback. In Proceedings of the2012 workshop on Data-driven user behavioral modelling and mining from social
media. 19–22.
[22] Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc
Najork. 2018. Position bias estimation for unbiased learning to rank in personalsearch. In Proceedings of the Eleventh ACM International Conference on Web Searchand Data Mining. ACM, 610–618.
[23] Longqi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge Belongie, and Deborah
Estrin. 2018. Unbiased Offline Recommender Evaluation for Missing-not-atrandom Implicit Feedback. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys ’18). 279–287.
[24] Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014.
Beyond clicks: dwell time for personalization. In Proceedings of the 8th ACM
Conference on Recommender systems. 113–120.