Deep candidate generation (DCG) , which narrows down the enormous corpus to a few hundred candidate items via representation learning , is integral to industrial recommender systems. Standard approaches adopt maximum likelihood estimation (MLE) and rely on sampling to ensure scalability , which reduces DCG to a task similar to language modeling. However, live recommender systems face severe unfairness of exposure with a corpus several orders of magnitude larger than that of natural language, which implies that (1) MLE will preserve and even exacerbate the exposure bias in the long run , as it aims to faithfully fit the history records , and (2) suboptimal sampling and inadequate use of item features can lead to inferior representations for the items that are unfairly ignored . In this paper, we introduce CLRec, a Contrastive Learning paradigm  successfully deployed in a real-world massive recommender system, for alleviating exposure unfairness in DCG. We theoretically prove that a popular choice of contrastive loss is equivalent to reducing the exposure bias via inverse propensity scoring, which complements previous understanding of contrastive learning. We further implement a good sampling distribution and reuse most of the computation when encoding rich features for both positive and negative items, by employing a fix-sized queue to store items (and reuse the computed representations ) from previous batches, where the queue serves as the negative sampler . Extensive offline analyses and four-month online A/B tests in Mobile Taobao demonstrate substantial improvement, including a dramatic reduction in the Matthew effect.
Deep candidate generation (DCG) that narrows down the collection of relevant items from billions to hundreds via representation learning is essential to large-scale recommender systems. Standard approaches approximate maximum likelihood estimation (MLE) through sampling for better scalability and address the problem of DCG in a way similar to language modeling. However, live recommender systems face severe unfairness of exposure with a vocabulary several orders of magnitude larger than that of natural language, implying that (1) MLE will preserve and even exacerbate the exposure bias in the long run in order to faithfully fit the observed samples , and (2) suboptimal sampling and inadequate use of item features can lead to inferior representations for the unfairly ignored items . In this paper, we introduce CLRec, a Contrastive Learning paradigm that has been successfully deployed in a real-world massive recommender system, to alleviate exposure bias in DCG. We theoretically prove that a popular choice of contrastive loss is equivalently reducing the exposure bias via inverse propensity scoring, which provides a new perspective on the effectiveness of contrastive learning. We further employ a fix-sized queue to store the items' representations computed in previously processed batches, and use the queue to serve as an effective sampler of negative examples. This queue-based design provides great efficiency in incorporating rich features of the thousand negative items per batch thanks to computation reuse . Extensive offline analyses and four-month online A/B tests in Mobile Taobao demonstrate substantial improvement, including a dramatic reduction in the Matthew effect.
R
, which
that
R
enormous corpus to a few hundred candidate items
collection of relevant items from billions to hundreds
R
, is integral to industrial
is essential to large-scale
R
adopt
approximate
R
and rely on sampling to ensure scalability , which reduces DCG to a task
through sampling for better scalability and address the problem of DCG in a way
R
corpus
vocabulary
R
which implies
implying
R
, as it aims
in order
R
history records
observed samples
R
items that are unfairly ignored
unfairly ignored items
A
that has been
R
for alleviating exposure unfairness
to alleviate exposure bias
R
equivalent to
equivalently
R
complements previous understanding
provides a new perspective on the effectiveness
R
implement a good sampling distribution and reuse most of the computation when encoding rich features for both positive and negative items, by employing a
employ a
D
items (and reuse the computed representations ) from previous batches, where the queue serves as
R
negative sampler
items' representations computed in previously processed batches, and use the queue to serve as an effective sampler of negative examples. This queue-based design provides great efficiency in incorporating rich features of the thousand negative items per batch thanks to computation reuse