Deep candidate generation has become an increasingly popular choice deployed in many industrial search, recommendation and ads systems. Standard deep candidate generation models rely on sophisticated sampling techniques to approximately conduct maximum likelihood estimation of user-item interactions, following the advances in the language modelingcommunity . However, it is unclear whether these sampling strategies are the best choice for candidate generation in recommender system, where we face severe selection bias in the training data with an extremely large candidate set and rich features of various types . In this paper, we propose CLRec, a Contrastive Learning paradigm for large scale candidate generation in Recommender systems. CLRec employs a queue based buffer to hold previous examples or representations as negative labels, within which a contrastive loss is optimized. This framework achieves better performance while requiring no explicit sampling , providing great efficiency in encoding rich types of features on the label side. We analyze both theoretically and empirically that CLRec can in fact alleviate the selection bias, leading to a more diversified and fairer recommendation. We deploy CLRec in Taobao and conduct online A/B test on a traffic-intensive scenario, showing a large margin improvementon both performance and efficiency, as well as a dramatic reduction on the rich-get-richer phenomenon .
Deep candidate generation (DCG), which narrows down the enormous corpus to a few hundred candidate items via representation learning, is integral to industrial recommender systems. Standard approaches adopt maximum likelihood estimation (MLE) and rely on sampling to ensure scalability, which reduces DCG to a task similar to language modeling . However, live recommender systems face severe unfairness of exposure with a corpus several orders of magnitude larger than that of natural language, which implies that (1) MLE will preserve and even exacerbate the exposure bias in the long run, as it aims to faithfully fit the history records, and (2) suboptimal sampling and inadequate use of item features can lead to inferior representations for the items that are unfairly ignored . In this paper, we introduce CLRec, a Contrastive Learning paradigm successfully deployed in a real-world massive recommender system, for alleviating exposure unfairness in DCG. We theoretically prove that a popular choice of contrastive loss is equivalent to reducing the exposure bias via inverse propensity scoring, which complements previous understanding of contrastive learning. We further implement a good sampling distribution and reuse most of the computation when encoding rich features for both positive and negative items, by employing a fix-sized queue to store items (and reuse the computed representations) from previous batches, where the queue serves as the negative sampler. In contrast, previous state-of-art DCG approaches do not use rich features for the negative items due to high computational costs, since the number of negative samples needs to be large. Extensive offline analyses and online A/B tests in Mobile Taobao demonstrate substantial improvement, including a dramatic reduction in the Matthew effect .
R
has become an increasingly popular choice deployed in many industrial search, recommendation and ads
(DCG), which narrows down the enormous corpus to a few hundred candidate items via representation learning, is integral to industrial recommender
R
deep candidate generation models rely on sophisticated sampling techniques to approximately conduct maximum likelihood estimation of user-item interactions, following the advances in the language modelingcommunity
approaches adopt maximum likelihood estimation (MLE) and rely on sampling to ensure scalability, which reduces DCG to a task similar to language modeling
R
it is unclear whether these sampling strategies are the best choice for candidate generation in recommender system, where we face severe selection
live recommender systems face severe unfairness of exposure with a corpus several orders of magnitude larger than that of natural language, which implies that (1) MLE will preserve and even exacerbate the exposure
R
training data with an extremely large candidate set and rich features of various types
long run, as it aims to faithfully fit the history records, and (2) suboptimal sampling and inadequate use of item features can lead to inferior representations for the items that are unfairly ignored
R
propose
introduce
R
for large scale candidate generation in Recommender systems. CLRec employs a queue based buffer to hold previous examples or representations as negative labels, within which a contrastive loss is optimized. This framework achieves better performance while requiring no explicit sampling , providing great efficiency in encoding rich types of features on the label side. We analyze both theoretically and empirically that CLRec can in fact alleviate the selection bias, leading to a more diversified and fairer recommendation. We deploy CLRec in Taobao and conduct
successfully deployed in a real-world massive recommender system, for alleviating exposure unfairness in DCG. We theoretically prove that a popular choice of contrastive loss is equivalent to reducing the exposure bias via inverse propensity scoring, which complements previous understanding of contrastive learning. We further implement a good sampling distribution and reuse most of the computation when encoding rich features for both positive and negative items, by employing a fix-sized queue to store items (and reuse the computed representations) from previous batches, where the queue serves as the negative sampler. In contrast, previous state-of-art DCG approaches do not use rich features for the negative items due to high computational costs, since the number of negative samples needs to be large. Extensive offline analyses and
R
test on a traffic-intensive scenario, showing a large margin improvementon both performance and efficiency, as well as
tests in Mobile Taobao demonstrate substantial improvement, including
R
on the rich-get-richer phenomenon
in the Matthew effect