PURE: An Uncertainty-aware Recommendation Framework for Maximizing Expected Posterior Utility of Platform

1 INTRODUCTION . Commercial recommendation systems have been widely applied among prevalent content distribution platforms such as YouTube , TikTok , Amazon and Taobao . During the interactive process on the recommendation platform , the users may find contents of their interests and avoid the information overload problem with the help of recommendation services . Meanwhile , the platform may gain commercial benefits from user behaviors on the platform such as clicks and purchases . As the platform may serve millions of users and can determine which contents to be recommended , it naturally has some advantages over individual user . Therefore , it would be crucial for the platform to make full use of its advantages in order to maximize the commercial benefits . One typical advantage of the platform is its information advantage , i.e. , they may collect plenty of information over users and items for conducting better recommendation . Typical state-of-the-art recommendation systems ( Covington et al. , 2016 ; Guo et al. , 2017 ; Ren et al. , 2019 ; Zhou et al. , 2019 ) always take these information into consideration including user profiles , item features and historical interactions between users and recommended items . It is worth noting that information over item features is always directly incorporated into the recommendation models without considering that the user may be with different levels of uncertainty over different item dimensions ( which can be regarded as different hidden attributes describing different high-order features of the item ) . For instance , when buying a new coat on the platform , a user may be sure that the logistics is very fast as she ( he ) has bought clothes from the same online store before ( i.e. , the user is with low uncertainty over the logistics ) . But she ( he ) may be uncertain about the quality of the coat since it is of the brand that she ( he ) does not know much about ( i.e. , the user is with high uncertainty over the quality ) . Thus , it would be crucial for the platform to figure out whether it is possible to leverage the user uncertainty over different item dimensions to maximize the platform utility , and if yes , how ? 1Item dimensions : Typical state-of-the-art solutions for recommendation systems always encode each item as an embedding . The item dimensions refer to different dimensions of the item embedding , which can be explained as different high-order features . Actually , with consideration of the user uncertainty over different item dimensions , we would show that more commercial benefits can be gained from the item dimensions with higher uncertainty . Another advantage of the platform is that it owns the capacity of determining which items to display for the users and thus may affect the users ’ behaviors . It has been proved by lots of works ( Kamenica & Gentzkow , 2011 ; Immorlica et al. , 2019 ; Abdollahpouri & Mansoury , 2020 ) that the display signal itself would highly affect users ’ behaviors , and affected behaviors would apparently result in different benefits for the platform . Regarding the recommendation as a game between the platform and the users , it is possible for the platform to achieve more commercial benefits from the game by taking a proper display ( recommendation ) policy . However , though there are works to explore the impact of recommendation policies , it is still not well-studied in recommendation area how to explicitly model and exploit the impact of the display policy over users . In this paper , we propose an uncertainty-aware expected Posterior Utility maximization framework for REcommendation platforms ( denoted as PURE in short ) . We take both the two previously mentioned factors , i.e. , user uncertainty over different item dimensions and influence of display policy over the user , into account and introduce a generic utility function which can be flexibly adjusted for different real-world scenarios . Then , we formulate the problem of maximizing expected posterior utility for the platform as a constrained non-convex optimization problem , and correspondingly propose a solution based on Alternating Direction Method of Multipliers ( ADMM , Boyd et al . ( 2011 ) ) to derive the approximately optimal policy . To verify the effectiveness of the proposed framework , extensive experiments are conducted over data collected from a real-world recommendation platform . Furthermore , we also provide practical insights derived from carefully designed experiments and empirically reveal how the platform utilizes its information advantage to achieve more commercial benefits , which may help to better understand and conduct commercial recommendation . 2 RELATED WORK . Existing state-of-the-art recommendation systems ( Zhou et al. , 2018 ; Pi et al. , 2019 ; Qu et al. , 2016 ) mainly try to make full use of the information advantage of the platform . These works take these information into consideration including user profiles , item features , contextual information and historical interactions between users and recommended items . Typically , some works ( Qu et al. , 2016 ; Zhou et al. , 2018 ; Li et al. , 2019 ) focus on how to achieve better feature interactions or conduct better user interest modeling , while some works ( Ren et al. , 2019 ; Pi et al. , 2019 ) may pay more attention to utilizing extremely long sequential interactive information . However , most of them ignore the existence of user uncertainty over different item dimensions , which might be crucial to conduct better commercial recommendation . In the research area to explore the display influence to the information receiver , Bayesian Persuasion ( Kamenica & Gentzkow , 2011 ) is one of the most crucial works , which theoretically proves that the information sender may benefit from displaying proper information to the receiver . Some works ( Immorlica et al. , 2019 ; Mansour et al. , 2016 ) follow this idea and strive to incentivize exploration via information asymmetry in scenarios such as recommendation . In another research direction that try to develop Reinforcement Learning ( RL ) based solutions for recommendation scenarios , a series of works ( Dulac-Arnold et al. , 2015 ; Zhao et al. , 2018 ; Chen et al. , 2019 ) model the recommendation process as a Markov Decision Process ( MDP ) and maximize the long-term reward via utilizing learned sequential patterns , which can also be regarded as taking the display ( recommendation ) influence into consideration to some extent . 3 METHODOLOGY . 3.1 OPTIMAL POLICY FOR MAXIMIZING PLATFORM ’ S EXPECTED POSTERIOR UTILITY . From the perspective of the platform , the optimal recommendation policy is the one with maximal expected utility ( i.e. , maximal expected commercial benefits ) . As mentioned before , the influence of display policy over users can not be ignored as it would highly affect the commercial benefits of the platform . In this paper , taking the impact of display policy on users into consideration , we formulate the platform ’ s optimal policy πu for user u over a given item set I as follows . πu = argmax π ∑ i∈I πiUu ( i|display ; π ) , s.t. , ∀i ∈ I , πi ≥ 0 and ∑ i∈I πi = 1 , ( 1 ) where Uu ( i|display ; π ) is the posterior utility of recommending item i to user u with consideration of the influence of display policy π . With this formulation , the remaining problem is how to model the posterior utility properly . In the following , we illustrate two reasonable assumptions in detail , which make it possible to model the posterior utility with consideration of the user uncertainty over different item dimensions as well as the influence of display policy over user . As discussed before , it would be crucial to explicitly consider the user uncertainty over different item dimensions to conduct recommendation . For a given user , we assume that the representation for an item is sampled from a multivariate Gaussian distribution and adopt the variances to describe the user uncertainty over different item dimensions , which is formulated as the following assumption . Assumption 1 ( Assumption of uncertainty ( correlation ) over different item dimensions ) . For a user u , the representation of item i is sampled from a n-dimension multivariate Gaussian distribution N ( µu , i , Σu , i ) , i.e. , the probability density function of the representation is : pu , i ( x ) = 1√ ( 2π ) n|Σu , i| e− 1 2 ( x−µu , i ) TΣ−1u , i ( x−µu , i ) , ( 2 ) where x ∈ Rn , µu , i and Σu , i denote the mean vector and the covariance matrix respectively . The covariance matrix can be decomposed as Σu , i = Du , iCu , iDu , i , where Du , i is the diagonal standard deviation matrix andCu , i is the correlation matrix ( see Barnard et al . ( 2000 ) for more information ) . Thus , the covariance matrix can depict the user uncertainty over different item dimensions ( with diagonal standard deviation matrix Du , i ) as well as the correlation between different item dimensions ( with correlation matrix Cu , i ) . Note that we provide a practical method to gain µu , i and Σu , i in Section 4.1 while any other reasonable approach to get µu , i and Σu , i can be applied . From the perspective of users , they may try to understand the display policy of the platform from the interactive process . When an item i is recommended to a user u , the user may consider the corresponding display probability , which could influence his behavior . One reasonable assumption is that the probability of displaying item i from the perspective of user u is linear to the similarity between the item representation x and the representations of historical recommended items . Without loss of generality , we formulate this assumption as follows . Assumption 2 ( Assumption of the influence of the display policy over user ) . Given display policy π and item representation x , the probability of recommending item i to user u from the user ’ s perspective is : pu , i ( display|x ; π ) = Φ ( avTux+ b ) , ( 3 ) where display denotes the event of displaying ( recommending ) the corresponding item to the target user , a and b are scale hyper-parameters , a > 0 , Φ ( z ) = ∫ z −∞ 1√ 2π e− x2 2 dx and vu = Ei∼π ( µu , i ) = ∑ i∈I πiµu , i . Note that Φ is the cumulative distribution function of standard normal distribution , which is widely adopted to map the input into range [ 0 , 1 ] in many models such as probit model , and vu takes the expected value of item representations w.r.t . the display policy π over item set I . Thus , with a > 0 , higher similarity ( calculated by inner product ) between the current item representation x and the expected representation of display items would lead to higher likelihood , which is reasonable as discussed before . So far , we have presented two crucial assumptions adopted in our framework . In the following , we introduce the utility function and derive the formula of posterior utility . To ease the illustration , we denote the utility function of recommending item i to user u given sampled item representation x as f ( wu , x ) , where wu is the embedding of user side typically and f ( wu , x ) depicts the utility ( benefits ) that the platform can gain from recommending item i to user u with item representation x . For instance , when we regard the click trough rate ( CTR ) as the platform ’ s utility , one simple way is to model f as the inner product of wu and x where wu can be regarded as the user preference vector of user u . Note that the function f can be flexibly adjusted to fit different requirements of different scenarios . For example , when we try to maximize the gross merchandise volume ( GMV ) in a recommendation scenario , f can be defined as the product of the corresponding CTR , conversion rate ( CVR ) and the item price . Now we can combine the two assumptions and the utility function f to derive the formula of posterior utility . By adopting the Bayes ’ theorem and the law of total probability , we present the formula of posterior utility of recommending item i to user u as follows . Uu ( i|display ; π ) = ∫ Rn f ( wu , x ) pu , i ( x|display ; π ) dx ( 4 ) = ∫ Rn f ( wu , x ) pu , i ( x ) pu , i ( display|x ; π ) pu , i ( display ; π ) dx ( 5 ) Equation 5 provides a way to model the posterior utility of the platform taking into account both the user uncertainty over different item dimensions and the influence of display policy over user . However , it is still challenging to derive the optimal policy πu for given user u as the right part of Equation 5 is not a close-form expression ( which makes it intractable to calculate the exact value of the posterior utility ) .

Summary: The paper aims to model the posterior utility of showing an item given a static display policy, where the utility function captures both: (a) uncertainty over item dimensions from the user perspective, and (b) influence of the policy on the user. It is motivated by the fact that most recommender systems don't take into account that user may be highly uncertain about value/utility in certain dimensions (e.g., color of a product) while more certain about others. The platform can use this information explicitly while optimizing for what to show.