PURE: An Uncertainty-aware Recommendation Framework for Maximizing Expected Posterior Utility of Platform
Authors: Haokun Chen
Zhaoyang Liu
Chen Xu
Ziqian Chen
Jinyang Gao
Bolin Ding
Venue:  (2021)
Keywords: commercial recommendation
maximizing platform benefits
uncertainty-aware
influence of display policy
non-convex optimization

Abstract: Commercial recommendation can be regarded as an interactive process between the recommendation platform and its target users. One crucial problem for the platform is how to make full use of its advantages so as to maximize its utility, i.e., the commercial benefits from recommendation. In this paper, we propose a novel recommendation framework which effectively utilizes the information of user uncertainty over different item dimensions and explicitly takes into consideration the impact of display policy on user in order to achieve maximal expected posterior utility for the platform. We formulate the problem of deriving optimal policy to achieve maximal expected posterior utility as a constrained non-convex optimization problem and further propose an ADMM-based solution to derive an approximately optimal policy. Extensive experiments are conducted over data collected from a real-world recommendation platform and demonstrate the effectiveness of the proposed framework. Besides, we also adopt the proposed framework to conduct experiments with an intent to reveal how the platform achieves its commercial benefits. The results suggest that the platform should cater to the user's preference for item dimensions that the user prefers, while for item dimensions where the user is with high uncertainty, the platform can achieve more commercial benefits by recommending items with high utilities.

Under review as a conference paper at ICLR 2021
PURE:
AN UNCERTAINTY-AWARE RECOMMENDATION FRAMEWORK FOR MAXIMIZING EXPECTED POSTE- RIOR UTILITY OF PLATFORM
Anonymous authors Paper under double-blind review
ABSTRACT
Commercial recommendation can be regarded as an interactive process between the recommendation platform and its target users. One crucial problem for the platform is how to make full use of its advantages so as to maximize its utility, i.e, the commercial benefits from recommendation. In this paper, we propose a novel recommendation framework which effectively utilizes the information of user uncertainty over different item dimensiong and explicitly takes into consid- eration the impact of display policy on user in order to achieve maximal expected posterior utility for the platform. We formulate the problem of deriving optimal policy to achieve maximal expected posterior utility as a constrained non-convex optimization problem and further propose an ADMM-based solution to derive an approximately optimal policy. Extensive experiments are conducted over data col- lected from a real-world recommendation platform and demonstrate the effective- ness of the proposed framework. Besides, we also adopt the proposed framework to conduct experiments with intent to reveal how the platform achieves its com- mercial benefits. The results suggest that the platform should cater to the user's preference for item dimensions that the user prefers, while for item dimensions where the user is with high uncertainty, the platform can achieve more commer- cial benefits by recommending items with high utilities.
INTRODUCTION
Commercial recommendation systems have been widely applied among prevalent content distribu- tion platforms such as YouTube, TikTok, Amazon and Taobao. During the interactive process on the recommendation platform, the users may find contents of their interests and avoid the information overload problem with the help of recommendation services. Meanwhile, the platform may gain commercial benefits from user behaviors on the platform such as clicks and purchases. As the plat- form may serve millions of users and can determine which contents to be recommended, it naturally has some advantages over individual user. Therefore, it would be crucial for the platform to make full use of its advantages in order maximize the commercial benefits.
One typical advantage of the platform is its information advantage, i.e., they may collect plenty of information over users and items for conducting better recommendation. Typical state-of-the-art recommendation systems (Covington et al. 2016: Guo al. 2017] Ren al.) 2019: Zhou et al. 2019) always take these information into consideration including user profiles, item features and historical interactions between users and recommended items. It is worth noting that information over item features is always directly incorporated into the recommendation models without consid- ering that the user may be with different levels of uncertainty over different item dimensions (which can be regarded as different hidden attributes describing different high-order features of the item). For instance, when buying new coat on the platform, a user may be sure that the logistics is very fast she (he) has bought clothes from the same online store before (i.e., the user is with low un- certainty over the logistics). But she (he) may be uncertain about the quality of the coat since of the brand that she (he) does not know much about (i.e., the user is with high uncertainty over the quality). Thus, it would be crucial for the platform to figure out whether it is possible to leverage the user uncertainty over different item dimensions maximize the platform utility, and yes, how?
Item dimensions: Typical state-of-ther -heee-rt- solutions for recommendation systems always encode each item an embedding. The item dimensions refer different dimensions of the item embedding, which can be explained different high-order features.

---

Under review as a conference paper at ICLR 2021
Actually, with consideration of the user uncertainty over different item dimensions, we would show that more commercial benefits can be gained from the item dimensions with higher uncertainty.
Another advantage of the platform is that it owns the capacity of determining which items to display for the users and thus may affect the users' behaviors. It has been proved by lots of works (Kamenical & Gentzkow] 2011 Immorlica et al.] 2019: Abbollahpouuri & Mansoury (020) that the display signal itself would highly affect users' behaviors, and affected behaviors would apparently result in different benefits for the platform. Regarding the recommendation as game between the platform and the users, it is possible for the platform to achieve more commercial benefits from the game by taking proper display (recommendation) policy. However, though there are works explore the impact of recommendation policies, it is still not well-studied in recommendation area how to explicitly model and exploit the impact of the display policy over users.
In this paper, we propose uncertainty aware expected Posterior Utility maximization framework for REcommendation platforms (denoted as PURE short). We take both the two previously men- tioned factors, i.e., user uncertainty over different item dimensions and influence of display policy over the user, into account and introduce generic utility function which can be flexibly adjusted for different real-world scenarios. Then, we formulate the problem of maximizing expected posterior utility for the platform as a constrained non-conve optimization problem. and correspondingly pro- pose a solution based on Alternating Direction Method of Multipliers (ADMM. [Boyd et al. (2 to derive the approximately optimal policy. To verify the effectiveness of the proposed framework, extensive experiments are conducted over data collected from a real-world recommendation plat- form. Furthermore, we also provide practical insights derived from carefully designed experiments and empirically reveal how the platform utilizes its information advantage to achieve more commer- cial benefits, which may help to better understand and conduct commercial recommendation.
2 RELATED WORK
Existing state-of- oatte-of-thh-art recommendation systems (Zhou et al., 2018 Pi al. 2019 Qu al.) 2016 mainly try to make full use of the information advantage of the platform. These works take these information into consideration including user profiles, item features, contextual information and historical interactions between users and recommended items. Typically, some works (Qu et al., 2016 Zhou et al. 2018 Li et al. 2019) focus on how to achieve better feature interactions or conduct better user interest modeling, while some works (Ren et al. 2019 Pi et al. 2019) may pay more attention to utilizing extremely long sequential interactive information. However, most of them ignore the existence of user uncertainty over different item dimensions, which might be crucial to conduct better commercial recommendation.
In the research area to explore the display influence to the information receiver, Bayesian Persuasion (Kamenica & Gentzkow 2011 is one of the most crucial works, which theoretically proves that the information sender may benefit from displaying proper information to the receiver. Some works [Immorlica et al.) 2019; Mansour et al. 2016) follow this idea and strive to incentivize exploration via information asymmetry scenarios such recommendation. In another research direction that try to develop Reinforcement Learning (RL) based solutions for recommendation scenarios, a series of works Duuac-Arnold et al. 20155| Zhao et al. 2018 Chen et al. 2019) model the recommendation process as a Markov Decision Process (MDP) and maximize the long-term reward via utilizing learned sequential patterns, which can also be regarded as taking the display (recommendation) influence into consideration to some extent.
3
METHODOLOGY
3.1
OPTIMAL POLICY FOR MAXIMIZING PLATFORM's EXPECTED POSTERIOR UTILITY
From the perspective of the platform, the optimal recommendation policy is the one with maximal expected utility (i.e., maximal expected commercial benefits). As mentioned before, the influence of display policy over users can not be ignored as it would highly affect the commercial benefits of the platform. In this paper, taking the impact of display policy on users into consideration, we formulate the platform's 's optimal policy π " for user over given item set II as follows.
π argmar
n: U'U'iiiisspayy TT,, s.L., Vi E TI n, 2 O and Σ TT==1 1, .EI
(1)
where U. (i display; π) is the posterior utility of recommending item i to user u. with consideration of the influence of display policy π. With this formulation, the remaining problem is how to model
2

---

Under review as a conference paper at ICLR 2021
the posterior utility properly. In the following, illustrate two reasonable assumptions in detail, which make it possible to model the posterior utility with consideration of the user uncertainty over different item dimensions as well as the influence of display policy over user.
As discussed before, it would be crucial to explicitly consider the user uncertainty over different item dimensions to conduct recommendation. For a given user, assume that the representation for an item is sampled from a multivariate Gaussian distribution and adopt the variances to describe the user uncertainty over different item dimensions, which is formulated as the following assumption.
Assumption 1 (Assumption of uncertainty (correlation) over different item dimensions). For user u, the representation of item i is sampled from a n-dimension multivariate Gaussian distribution N (puss E. )), i.e., the probability density function of the representation is:
1
(((,)) 2., "i,,,)""",,, (x-p()))
Pu.i (z)
(2T) Lu,i
where I E IR", Mu. and E..; denote the mean vector and the covariance matrix respectively.
The covariance matrix can be decomposed as Σ..; = D Cu. D u.. where D Duii is the diagonal standard deviation matrix and Ca, is the correlation matrix (see Barnard et al. (2000) for more infor- mation). Thus, the covariance matrix can depict the user uncertainty over different item dimensions (with diagonal standard deviation matrix Du,i) as well as the correlation between different item di- mensions (with correlation matrix i. Note that we provide practical method to gain M,, and Eu, in Section 4.1 while any other reasonable approach to get pu,s and Eu,i can be applied.
From the perspective of users, they may try to understand the display policy of the platform from the interactive process. When an item i is recommended to a user u, the user may consider the corresponding display probability, which could influence his behavior. One reasonable assumption that the probability of displaying item from the perspective of user linear to the similarity between the item representation x and the representations of historical recommended items. Without loss of generality, we formulate this assumption as follows.
Assumption (Assumption of the influence of the display policy over user). Given display policy т and item representation æ, the probability of recommending item i to user u from the user's s perspective is:
Pd,(diiplayy):; [=; π) = +bav." x + b).
(3)
where disploy denotes the event of displaying (recommending)) the corresponding item the tar- _ -=002 Le- dx and Vu =
get user, a and b are scale hyper-parameters, a > 0, (22) = Ei-r Ep-n(Hu,.) Eien H.
V2 V2x
Note that & is the cumulative distribution function of standard normal distribution, which is widely adopted to map the input into range [0, 1] in many models such as probit model, and v. takes the expected value of item representations w.r.t. the display policy л over item set I.. Thus, with a >> O, higher similarity (calculated by inner product) between the current item representation æ and the expected representation of display items would lead to higher likelihood, which is reasonable as discussed before.
So far, we have presented two crucial assumptions adopted in our framework. In the following, we introduce the utility function and derive the formula of posterior utility. To ease the illustration, we denote the utility function of recommending item to user u given sampled item representation x as f(w,, x), where w,, is the embedding of user side typically and f(w.. x) depicts the utility (benefits) that the platform can gain from recommending item to user u with item representation at. For instance, when we regard the click trough rate (CTR) the platform's 's utility, one simple way is to model f as the inner product of w,, and xr where w. can be regarded as the user preference vector of user u. Note that the function f can be flexibly adjusted to fit different requirements of different scenarios. For example, when we try to maximize the gross merchandise volume (GMV) in a recommendation scenario. can be defined as the product of the corresponding CTR, conversion rate (CVR) and the item price.
Now we can combine the two assumptions and the utility function f to derive the formula of poste- rior utility. By adopting the Bayes' theorem and the law of total probability, we present the formula of posterior utility of recommending item to user u as follows.
Un (ilddssplay; T) = { f(w,: xx P.. (x) disploy: πT) da
(4)
3

---

Under review as a conference paper at ICLR 2021
= =f f(w,: px) Pa, displdispooyl:: T)
dx Pu.i (display; π))
(5)
Equation 5 provides a way to model the posterior utility of the platform taking into account both the user uncertainty over different item dimensions and the influence of display policy over user. However, it is still challenging derive the optimal policy π for given user the right part of Equation is not a close-form expression (which makes it intractable calculate the exact value of the posterior utility).
3.2 POSTERIOR UTILITY DERIVATION FOR LINEAR AND NON-LINEAR UTILITY FUNCTIONS
In this section, we present how to compute (estimate) the value of posterior utility for both linear and non-linear utility functions. To derive effective calculation formula of U (ilisplay; π for the case with linear utility function, we first introduce a lemma in the following.
Lemma 1. d'dd' E R.
dx d'd e
6/2
+ (( vI+a"
v (2((a241)
The detailed proof of Lemma provided in Appendix A.1 due to page limits. Lemma reveals the fact that the definite integral - (dxx + d + (z)0,, 1) dr can be computed directly, which is a crucial property to derive an effective calculation formula of (, display; π for the case with linear utility function.
By utilizing Lemma I the following corollary shows that Pu.s(display; πτ) can be straightforwardly calculated the covariance matrix is positive definite.
Corollary 1. P πτ) क
av 4u,
), if E. is positive definite.
VI+a"u Eu.
Due to page limits, the detailed proof of Corollary ! is also presented in Appendix A.2 With Lemma [ and Corollary [ we can derive the calculation formula of (i display, π it the utility function is linear, which is illustrated in detail as follows.
(n0 to,
Corollary Uu(ildisppay; πτ) -w" Mu,it
E.,, w.
2(0 Pu,
V2mla2o iu +1 Vall (u
avd Mo
if f(W x) wh and Eu,i is positive definite.
Proof. If Eu,i is positive definite, Dúl is also positive definite. Then we can conduct Cholesky decomposition: E-1 AT YY, where Au,s an upper triangular matrix with real and positive u,i diagonal entries. Thus, we have:
/ f(w,, (z)p xx; πτ) dx
/ wT (av x + ()) Eu,i) dæ
(7)
Rm
[ wt wH((A (A pm,)) (av (A + pua) + (=) d)
(8)
av! Musi
avd T..TE. Eu
(av (b) 2(22 (ic
o Σ u. +1
2n (2 (a2v En +1)
Equation 8 and Equation are derived from Proposition 3 and Proposition 5 respectively. Due to page limits, Proposition and Proposition 5 and their detailed proofs are presented in Appendix. Combing Corollary T and Equation 9, we have:
Un U, dildisplay; π) Pu,i (display; π) [ f(w,, x) (x. x;; πT) dx
(10)
av? E,, ww
(av st,)
w pa. +
2102 Eu,
(a E, En, vw + 1) ov." tu,
(1I)
4

---

Under review as a conference paper at ICLR 2021
Corollary 2 reveals that the posterior utility can be effectively calculated (since it is with a close- form expression and thus can avoid the intractable calculation in Equation [5 when Σ.,i is positive definite and the utility function f is linear, which makes it possible to be effectively utilized in the real-world scenarios.
However, when the utility function f is non-linear, might be challenging or even impossible to calculate the exact value of the posterior utility. To estimate the posterior utility when f is non- linear, we leverage the importance sampling technique approximate the posterior utility. Combing Sorooayyy Assumption] 2 Equation 5 and Corollary]
[
we have:
/
U² (idddiispyay T
s(ww, a)pu (2))((;; πT) da,
(12)
where ((:;; πt)
Dlav xx+b)
Thus, computing Un (ildispaay x) can be regarded caleu-
av. av.. Pu.ith
/o Eu,
lating the expected value of ((,,, w.r.t. the target probability density function q x) (a:)()))((s π). Since Pu,;( the probability density function of a multivariate Gaussian distri- bution from which one can easily conduct sampling, we can adopt (x); as the sampler density and as result, given a sample æ, the corresponding importance weight is I(x;; RT) Thus, given sample set X which consists of samples drawn i.i.d. according distribution N (Hu,,s E,,,), we can approximate the posterior utility as:
Un (ildispplyy; T) E (x:; π
Σ (()x;) DEX
(13)
In this section, we derive an effective calculation formula for posterior utility in the case with linear utility function and provide a sampling- based method to estimate posterior utility in the case with non-linear utility function. Note that the sampling-bast -baeed method can also be applied to estimate the posterior utility for the case with linear utility function. Combining the results in this section (i.e., Equation II and Equation [3 and the formulation of optimal policy (as shown in Equation D notice that the problem of deriving the optimal policy for maximizing platform's 's expected posterior utility, which is formulated in Equation [ is turned into a constrained non-convex optimization problem even for the case with linear utility function. To solve the problem, an ADMM-based solution is proposed in the next section.
3.3 AN ADMM-BASED SOLUTION FOR DERIVING APPROXIMATELY OPTIMAL POLICY
As presented in the previous section, finding the optimal policy to maximize the expected posterior utility of the platform is formulated as a constrained non-convex optimization problem. Noticing that the ADMM technique (Boyd et al. 2011) has been successfully adopted as an effective method to find approximate solution for constraied non-convex optimization problem (see Leng et al. (2017)
for more details), we develop an ADMM-based solution to solve the aforementioned constramed non-convex optimization problem.
Denoting ( -Ete TT, (i) (ildsplay;: T). we rewrite the constrained non-convex optimiza- tion problem (which is formulated in Equation I as follows.
argmin Ju.2 s.t., Vi E II, ग. and
Σ ग 11
(14)
EEll
By introducing an auxiliary parameter vector n², the problem can be reformulated as:
argmin (T. s.t.,, n .,, Wi 2 O and ΣT.=1.
(15)
x
RET
Note that the target function (T') and the target policy is linked via the first constrain, i.e.. ' т. We regard the last two constrains as hard constrains which are required to be satisfied at any optimization step. And the augmented Lagrangian w.r.t. to the first constrain is:
L A) ( + (T'' T)
2 |T'
n 12
(16)
5

---

Under review as a conference paper at ICLR 2021
where À is the Lagrangian multipliers and p > 0 is hyper-parameed To achieve approximate solution, the ADMM method consists of the following three steps for each iteration:
π (k++) argmin (T( TT (k)) > (
(17)
π (k+++) arrgmn p. (k+ n (h))
(18)
A ( = X p( (k+1) - (k+1))
(19)
We can observe that the last two steps do not involve optimization w.r.t π. Thus, the third step can be simply achieved by value updating while the second step can be approximately solved by
adopting gradient-basee solutions such stochastic gradient decent. The optimization problem
w.r.t. the first step can be reformulated as:
argmin [T( T (k) A ( = argmin A (K)T (k) - -x) + 2 111 - π
(20)
"
+ A(6)
argmin
1
(21)
p
st., Vi E I, n 0 and Σ EE
n =1
(22)
which is convex quadratic programming problem and can be solved by convex quadratic program- ming solutions such as active set methods (Wong 2011). It is worth noting that the optimization problem w.r.t. the first step can be regarded as finding the nearest point from a given convex set (which satisfy the two constrains) to given point (i.e., n T<ak) 1
By iteratively conducting the aforementioned three optimization steps until convergence (say, after k' steps), we can acquire an approximately optimal policy (4') for maximizing expected posterior utility of the platform, which demonstrated to be great enough the experiment section.
4 EXPERIMENTS
In this section, we present the experiment setup and the experiment results in detail. First, we present the detailed setup of the experiments including a practical way to gain the mean vector and covari- ance matrix for given pair of user and item, which makes it possible to take the user uncertainty over different item dimensions into consideration. Second, we conduct experiments to verify the effec- tiveness of the proposed technical solutions involved in our framework, i.e., the effectiveness of the ADMM-based solution for maximizing platform expected posterior utility and the sampling-basee technique for posterior utility estimation. Third, to verify the superior of the policy derived from the proposed framework, we conduct experiments and present the results of comparison between our policy and other two heuristic policies. Fourth, to answer the crucial question that how the platform maximizes its commercial benefits, we analyse the relation between the learned policy and two significant factors, the user preference and uncertainty over given item dimension, and provide practical insights to better understand and conduct commercial recommendation.
4.1 EXPERIMENT SETUP
To conduct experiments, we first collect click log data of about 1 million users and million items from the GUESS U LIKE scenario of the Taobao Application. To encode the users and the items, adopt prevalent Embedding&MLP paradigm (He et al. 2017: Guo al. (017) and encode the information of the user and the item sides respectively (Xu et al. 2019). By training on specific task (e.g, click through rate prediction task), we can gain the embedding of the user side as well as that of the item side. Without loss of generality, the model output the inner product of the user and the item embeddings; ii) the model output is achieved by applying a non-linear function over the user and the item embeddings. Note that these two architectures correspond to the cases with linear and non-linear utility functions respectively. For each pair of user group (divided according to age and gender) and item category, the clicked item set is extracted and the mean vector and the covariance matrix of the corresponding item embeddings are calculated. For given pair of user and item, the mean vector and the covariance matrix can be gained by adopting those of the pair of the corresponding user group and item category.
consider the following
model architectures:

---

Under review as a conference paper at ICLR 2021
Without loss of generality, we empirically set the value of most of the values of Pu.(displayl z;; π lie between 0,001 and 0.1 (noticing that other reasonable ranges also lead to similar experimental conclusions). Since is a function in the form of integral which might be intractable to calculate, we adopt an effective approximation (Waissi & Rossin 1996) to ease the calculation. To avoid learning a highly unbalanced policy (with probability I for one item while with 0 for others) due to unbalanced utilities, we also incorporate an entropy-based regularization term into our model. Specifically, instead of directly adopting the optimization step we add extra term to smooth the learned policy and the corresponding
and b to ensure that
as shown in Equation 18 optimization step is:
(k-1) argmin I.p (T (N+1), ((1) +
( SEl
1
(23)
where [I] denotes the number of items in item set I. 1) is the parameter control the smooth level of the learned policy and is set to 10.0 empirically in the experiments. Besides, the optimizer we adopted for gradient decent is the Adam Optimizer (Kingma & Ba, (014)) and the value of p is set as 1.0 empirically, which is proved to be effective in the following experiments.
4.2 EFFECTIVENESS OF THE PROPOSED TECHNICAL SOLUTIONS
linear case with exact U,
case with estimated U,
non-linear case with astimated
expeeted ponterter uthley
sppected posteetoo uniltt
0.22
0.28
posterior
0.26
0.21
0.24
0.21
u
0.22
uder1
0.22
user
User 2
user 2
0.20
User
user3
user3
user
100
20
69
100
optimization iterations
optimization iterations
optimization iterations
Figure I: Expected posterior utility w.r.t. optimization iterations for three cases
optimization curves user
optimization curves for user
optimization curves for user
0.21
acceectdd pontertor unility
0.30-
exppected postertor utility
0.26
postertor
0.25
0.22-
0.24
0.26-
with enact salue
with enodt Volle
0.22
with cnodE value
0.2
oxmple per est mat on
sample per est
cample per est
10 samp en estimatt on
10 samp un per mat
10 sam pi per mat
1.00 samples per estimation
0.22
100 su makka per cstimation
100 samplea per cotimation
1000 samp us per est mat
1003 samp ls ser oi malion
1900 sampless per colimiaiion
0.18
20
40
20
150
20
45
60
60
160
20
d0
60
80
100
optimization ion iterations
optimization iterations
optimization iterations
Figure 2: Optimization curves w.r.t. different numbers of samples per estimation
In this section, we present the experiment results to demonstrate the effectiveness of the two pro- posed technical solutions absorbed into framework, i.e., the effectiveness of the ADMM-based solution for maximizing expected posterior utility and the importance sampling technique for pos- terior utility estimation.
To demonstrate the effectiveness of the proposed ADMM-based solution for maximizing expected posterior utility, we record the optimization curves (i.e., expected posterior utility w.r.t. the number of optimization iterations) and present the results in Figure []] According to Section 3.2 there are three cases to be considered: i) linear utility function with exact posterior utility (calculated by adopting Equation 1D; ii) linear utility function with estimated posterior utility: iii) non-linear utility function with estimated posterior utility. We randomly sample 3 users and 100 items and utilize the proposed ADMM- the expected posterior utility of the platform. As shown in Figure D the expected posterior utility first increases and then converges to its maximum with the increase of the number of optimization iterations. These results demonstrate that the proposed ADMM- based solution can effectively learn a policy to maximize the expected posterior utility. Besides, comparing the optimization curves of the three cases, we can observe that the variances of the expected posterior utility in the cases with
optimization method to learn the approximately optimal policy to maximize

---

Under review as a conference paper at ICLR 2021
estimated posterior utility Ü. (Eldisplay: TT) (the last two cases) are higher than that in the case with exact calculated posterior utility Uu(ildisplay; π (the first case). The reason that extra random noise is induced by importance sampling when we conduct posterior utility estimation and as a result, expected posterior utility is with higher variance in the case with estimated posterior utility.
In order to verify the effectiveness of the adopted importance sampling technique for posterior timation, we record the optimization curves w.r.t. different numbers of samples for estimating pos- terior utility for three randomly sampled users and present the results in Figure [2] The linear utility function is adopted to conduct the experiments which makes it possible to make comparison be- tween the results of utilizing estimated posterior utility (estimated according to Equation [3) and exact posterior utility (calculated according to Equation IT]. From Figure 2 we can observe that even in the case with 1 sample per estimation, the tendency of the optimization curve is similar to that of the case with exact posterior utility, which verifies that the adopted sampling-based tech- nique for posterior utility estimation is effective. Besides, when varying the number of samples per estimation from I to 1000, we observe that the optimization curves become more and more stable. It is because that larger number of samples per estimation leads to more accurate approximation of posterior utility, and thus results in more stable optimization curve. More details including the Root Mean Square Error (RMSE) between the estimated values and the exact values of the posterior utilities for each sampling setting are provided in Appendix AerendixxA.4ddue A.4due to page limits.
4.3 EXPERIMENTAL. COMPARISON WITH HEURISTIC POLICIES
In this section, we present the experiment details and the corresponding results to verify the effec- tiveness of the policy derived from the proposed framework. To the best of our knowledge, the presented framework is the first complete work that proposes to maximize the expected posterior utility (which explicitly takes the impact of display policy into consideration) and develops cor- responding solution to derive the approximately optimal policy. Thus, there may lack solutions for conducting straightforward comparison and we intent to verify the effectiveness of the learned pol- icy by comparing it with other heuristic policies. One simple policy for comparison is the random policy that recommends each item with same probability. Thus, the random policy can be regarded as an uniform probability distribution over the candidate items. We also incorporate a heuristic prior policy for comparison where the recommendation probability is higher for item with higher prior utility] Specifically, the recommendation probability of item is given by
pop
(24)
Ejer
where 7 the temperature parameter (Hinton et al. 2015) to control the smooth level of the heuristic prior policy. For conducting fair comparison, we adjust the value of T to ensure that the smooth levels of the heuristic prior policy and our learned policy are comparable.
Table 1: Expected posterior utilities w.r.t different policies
random policy prior policy
our policy
linear case non-linear case
0.1593 0.1609
0.1922
0.2171
0.1936
0.2156
We randomly sample 1000 users, calculate the mean of expected posterior utilities for each policy and present the results in Table [ As shown in Table [ the policy derived from the proposed framework achieves the highest expected posterior utility compared to the random and the heuristic prior policies in both linear and non-linear cases, which demonstrates the superior of the proposed framework.
4.4 HOw DOES THE PLATFORM ACHIEVE ITS COMMERCIAL BENEFITS
In this section, we intent to adopt the proposed framework to reveal how the platform achieves its commercial benefits taking both the user preference and the user uncertainty over different item dimensions into account. In the experiments, the utility function realized by applying the sigmoid
2 Similar to the derivation of posterior utility. we can derive that i) for linear case, U.( wh pu ii) for non-linear case, (i) Sg f(w,, x (m)) da.
8

---

Under review as a conference paper at ICLR 2021
influence of user preference
influence of item variance
0.555
0250-
0:240
0.024
30240
Ieamed
0.033
0247
044
Pearson's r: 0.9995
Pearson's 0.9931
0.311
0244
-1.0
-0.5
0.0
0.
10
15
0.005
0.010
0.015
3.503
mean user preference of dimension
the mean item varlance dimension
Figure 3: Influence of user preference and item variance
activation function over the inner product of item representation and user representation (i.e., W..) Each element of w,, can be explained the user preference over the corresponding item dimension. For randomly sampled user, we denote the mean vector of the user preference vectors over items as w and denote its first element as Wy, whose value indicates the user's 's preference of item dimension O. We vary the value of tig and record the learned expected value (w.r.t the learned policy) of the elements with dimension 0 of the mean vectors, which is denoted vn. As shown in the left part of Figure 3 with the increase of wo, the value of vo increases approximately linearly (noting that the Pearson correlation coefficient is 0.9995 which indicates strong linear correlation). According to its definition, details about how the policy adjust to different levels of user preference or uncertainty are provided in Appendix A.5). Thus, the result suggests that the policy would tend to adjust more for dimensions with higher value. In other words, the platform can achieve more commercial benefits from the item dimension with stronger user preference. We also vary the mean of item variance of dimension 0, which indicates the user uncertainty over item dimension O, and record vo as presented in the right part of Figure 3 By similar analysis, the result indicates that the platform can achieve more commercial benefits from the item dimension that the user is with higher uncertainty.
can reflect to what extent the policy has adjusted to fit item dimension 0 (more
Therefore, the results in Figure indicate that commercial benefits mainly come from the item di- mensions with either strong user preference or high user uncertainty when we take the impact of display policy on user into consideration. Note that this result may contribute to better understand how commercial benefits stance, from the perspective of the platform, one effective way to improve its commercial benefits is to better balance the user experience and its commercial actions, i.c., for item dimensions where the user is with high preference, the platform should cater his preference, while for flexible item dimensions where the user is with high uncertainty, the platform may achieve more benefits by recommending items with high platform utilities.
achieved and better conduct commercial recommendation. For in-
5 CONCLUSION
In the paper, we propose a novel recommendation framework and take into account the user uncer- tainty over different item dimensions and the influence of display policy over user, both of which could highly affect the commercial benefits of the platform. We derive the calculation formula of the posterior utility in the case with linear utility function and provide sampling- -based method to estimate the posterior utility in the case with non-linear utility function. Based on these works, we formulate the problem of deriving optimal policy for maximizing the expected posterior utility of the platform constrained non-convex optimization problem and further propose ADMM-basee solution to derive an approximately optimal policy. To demonstrate the effectiveness the pro- posed technical solutions absorbed into our framework, extensive experiments are conducted over data collected from a real-word world recommendation scenario. Furthermore, we also provide some practical insights about how to achieve commercial benefits for the platform taking both the user preference and the user uncertainty over different item dimensions into consideration, which might contribute to better understand and conduct commercial recommendation.
REFERENCES
Himan Abdollahpouri and Masoud Mansoury. Multi-sided exposure bias in recommendation. arXiv preprint arXiv:2006.15771 2020.

---

Under review as a conference paper at ICLR 2021
John Barnard, Robert McCulloch, and Xiao-Li Meng. Modeling covariance matrices in terms of standard deviations and correlations, with application shrinkage. Statistica Sinica, pp. 1281- 1311, 2000.
2000.
Stephen Boyd, Neal Parikh, and Eric Chu. Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc, 2011.
Haokun Chen, Xinyi Dai, Han Cai, Weinan Zhang, Xuejian Wang, Ruiming Tang, Yuzhou Zhang, and Yong Yu. Large-scale interactive recommendation with tree-structured policy gradient. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3312-3320, 2019.
Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, pp. 191-198, 2016.
Gabriel Amnod, Richard Evans, Hado van Hasselt, Peter Sunehag. Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weher, Thomas Degris, and Ben Coppin. Deep rein- forcement learning in large discrete action spaces. arXiv preprint arXiv: 1512.07679, 2015.
Huifeng Guo, Ruiming Tang. Yumming Ye, Zhenguo Li, and Xiuqiang He. Deepfin: factorization- machine based neural network for ctr prediction. arXiv preprint arXiv: 1703.04247, 2017.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural col- laborative filtering. In Proceedings of the 26th international conference on world wide web, pp- 173-182, 2017.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint arXiv:1503.02533 2015.
Nicole Immorlica, Jieming Mao, Aleksandrs Slivkins, and Zhiwei Steven Wu. Bayesian exploration with heterogeneous agents. In The World Wide Web Conference, pp. 751-761, 2019.
Emir Kamenica and Matthew Gentzkow. Bayesian persuasion. American Economic Review, 101 (6):2590-2615, 2011.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1x12v:/412.69000 2014.
Cong Leng, Hao Li, Shenghuo Zhu, and Rong Jin. Extremely low bit neural network: Squeeze the last bit out with admm. arXiv preprint arXiv:1707.09877 2017.
Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Huan Zhao, Pipei Huang, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. Multi-interest network with dynamic routing for recommen- dation at tmall. In Proceedings of the 28th ACM International Conference Information and Knowledge Management, pp. 2615-2623, 2019.
Yishay Mansour, Alcksandrs Slivkins, Vasilis Syrgkanis, and Zhiwei Steven Wu. Bayesian explo- ration: Incentivizing exploration in bayesian games. arXiv preprint arXiv:1602.07570. 2016.
Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. Practice long sequential user behavior modeling for click-through rate prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pp. 267 1-679, 2019.
Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. Product-based neural networks for user response prediction. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 1149-1154. IEEE, 2016.
Kan Ren, Jiarui Qin, Yuchen Fang, Weinan Zhang, Lei Zheng, Weijie Bian, Guorui Zhou, Jian Xu, Yong Yu, Xiaogiang Zhu, et al. Lifelong sequential modeling with personalized memorization for user response prediction. In Proceedings the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.
2019.
Gary R Waissi and Donald F Rossin. A sigmoid approximation of the standard normal integral. Applied Mathematics and Computation, 77(1):11955 1996.
Elizabeth Lai Sum Wong. Active-set methods for quadratic programming. PhD thesis, UC San Diego, 2011.
10

---

Under review as a conference paper at ICLR 2021
Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang. Changhua Pei, Fei Sun, Jian Wu, Hanxiao Sun, and Wenwu Ou. Privileged features distillation at taobao recommendations, arXiv preprint orXhv:197.00517,, 2019.
Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. Deep rein- forcement learning for page-wise recommendations, In Proceedings of the 12th ACM Conference on Recommender Systems, pp. 95-103, 2018.
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp- 1059-1068, 2018
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for thrc-th rate prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 5941-5948, 2019.
A APPENDIX
A.I PROOF OF LEMMA II
To prove Lemma
I
we first prove the following two propositions.
Proposition 1. (a'x (z) of) dr = $ a po ). Va', No,
V VItaBag
No, Jg E R.
Proof. By introducing two independent variables and which satisty X ~ Nopo, σo) and Y ~ NO0, 1), we have
(('' + (bl'+ (z) ExxNNio, +b)
(25)
P(Y < <d'X +b¹)
(26)
P(Y - a'X < <b)
(27)
P( Y - X'X + d'po
8' + d'po
(28)
VI + 22o0
v + 2200
Φ a' Po +
(29)
Note that Equation 26 is gained by the definition of function 4, i.c., (22) is the probability of sampling from the standard normal distribution with the sampled value less than z. Since
X NoHo on) and Y -N(0, 1). we have V=-a'ta'pp NN 1). Thus, Equation 29 can V1+a2az
be straightforwardly derived by definition of $ which concludes the proof.
Proposition 2. Denoting the error function by er which satisfies er f() 2 Soe e-e-t dt, have: 1 xe 4 er fll' B') dx Va
2n'
y2
Va' D' R.
Proof. For case with a' = 0, we have:
[ xe 1 2 er erf(a'x b)) dx
0
22 dr
/ 2 erf((') er dr
ve 2 er fr((())
(30)
J0
+ er f(b)) do+ 1 I er 1(6 dr
(31)
Jn
0
(32)
2d
122
e
2
(33)
Va²2
aa2
11

---

Under review as a conference paper at ICLR 2021
For case with a' # 0, we have:
De - +)) = = or err(('=tb) -
0
2a
-(d'x+8/) dx
-
e
π
(34)
2d'
e
-
+a 2)= 2
- 22'b''
62 dr
(35)
VT
2a'
-aa
+ 222 8
5
57
dx
(36)
VT
2a'
22
-la" dz
(37)
VT
2a
222
62
(38)
a
+ 1
2
2a
2u
(39)
Va²
Note that Equation 34 is derived by applying Newton-Leibhiz formula.
Thus, combining the above two cases, we have: L xe erf(d'x + d)
W22 VaL.VE
20
+
With Proposition and Proposition P now we prove Lemma] as follows.
By definition, 4(2) [+erl( Thus, have:
Proof of Lemmal 7
(c'x + ('a'' +oClx++)J[C ())
(40)
8
to0
a'z +
+20 dx + S
(cxx) er
1)e
(41)
2n
V2
a'c'
b'
24 d' ((
(42)
12m(aa+1)
The last equation is derived by utilizing Proposition and Proposition 2] which completes the proof.
A.2 PROOF OF COROLLARY
In order to prove Corollary I we first prove the following two propositions.
Proposition 3. If Eu,i is positive definite, then SEn (x)) Eu,i) dx S h( M(A + Hw,:)N(2 00 I) dz, where Au,i is the upper triangular matrix gained by Cholesky decomposition over Σ D-1 iee. D-1 AT An,i
Proof. If Eu,i positive definite, ET is also positive definite. Then we can conduct Cholesky decomposition:
Σ An,i
(43)
where A is upper triangular matrix with real and positive diagonal entries.
Thus, by adopting integration by substitution, we have:
/ gm
(xx En,)) dx
(44)
12

---

Under review as a conference paper at ICLR 2021
mRm -/o
=
h(x) ((2T)" (2
((a-)u.))* Du,bla puu))) dx
(45)
Eunil
= =f( h(A - (ays) + Mu,s) ((2T))n
e- 2 (Au,il (uuee)))))))) ( Au,la (astaa-uree |uus | dæ (46)
-f( h(Am + Mu,i) 1
e f=== dz
(47)
Jg"
((2z)
=/( Jgn h(A (2.00
(48)
Thus, if Σn,i is positive definite and An,i is the upper triangular matrix gained by Cholesky decomposition over En then we have SR ((xx)(a mu,s) dæ Sz. h h(A (z,)) I) dz, which concludes the proof.
Proposition 4. Sax- $(v + a') (ar I) de D1 {( (((TrT))) Vo E RD and a' R.
Proof. Denoting ú; = d' + Ej=i+1 Vyzg, we have:
4 T x + a (z) 1) dx
(49)
Rh
( + Σ
vΙ Wi d...)) .N(( 00, 1) dxn
(50)
00
1=1
Φ11101 + गn (<> 00, dr, .. -N 1) dx,
(51)
(
W( 00 1) dxy.- -<,, |0, 1) dx.,
(52)
D2 + 02
W( 00, 1) dæy .. (a 1) dar
(53)
VI
of
/1
④
WW (0. 1) d. NN(, 0, 1) dxn
(54)
I+
W(xz) 0, -Na,--,, 1) dx
(55)
1
8 02
a'
(56)
8 4
Equation 52 and Equation 54 are derived from Lemma T (or Proposition T. Similarly, Equation 56 is achieved by applying Lemma (or Proposition iteratively, which completes the proof.
With Proposition 3 and Proposition now we can prove Corollary D as follows.
Proof of Corollary 7 Eu, is positive definite, E is also positive definite. Then we can conduct
Cholesky decomposition: E,, AT Au,i, where A,, is an upper triangular matrix with real and positive diagonal entries. Thus, we have:
/ Rn
Pa. Pu.i( (displayy TT) =
Pu, (display [x;; π (x)) da
(57)
D(a x)) b) (x) (o E.,i) dx
(58)
[
Φlavz A + ax Mu,i + (z(00 I) dz
(59)
g"
13

---

Under review as a conference paper at ICLR 2021
av Muri +
(60)
+ a'uj V
Uu
Equation 59 and Equation 60 are derived from Proposition B and Proposition 4 respectively, which concludes the proof.
A.3 PROPOSITION ADOPTED FOR PROVING COROLLARY 2
Proposition 5. x + U) (z) da
1 n V2x(v=0+11
2(0 u+1]
+
Vu, v E RR" and d' R. ( u+11
Proof.
Denoting ú; =V Lj=it1 uj and i; d' + Ez=i+1 Myay, we have:
[u x 1 x (==(00 1) da
(61)
400
(1/ +
ug ) (a + u ( da. ... ..,, 1) dæ,
(62)
U1 01 + u1 ) (
+ 03 (<> 00, 1) dx ... (N 10, 1) dæn
(63)
100
of z +i + 인
Ui 11
e
(2 0, 1) dx² NN 1) dæ
2n of +
v1 + o
00
(64)
+00
+00
(wgag ( og) 2(07
U2 + 02
+ (ug022 + t
)) (xx220 1) dx
50
1Tm(2t (of +1)
+ d
.N 00, dx,
(65)
+00
+00
Keaia o,12
16:1
81
2(1 111
V22l22
2(oftg+1)
2 ( { + 03 +1)
2 (म) + 0z +
Uza'3 + 03
(uzaz + uz) (
)) (0, 1) daz
Nax 10, da
(66)
V + of U/
100
V1 1 U2
Joyes oز -of-
Uz + 03
(uzx3 + (2)
2m (uf + v2 11)
V + 0f + 02
W(xx:00, (0, 1) dxy ... N(xx 10, 1) dæy
(67)
JTu u
2( 111) + b' 4/
(68)
(2T(v''-1)
Equation 64 is derived from Lemma]] Applying Lemma I and conducting definite integration over the exponential function, iteratively to derive Equation (68, which completes the proof.
derive Equation 666 Similar calculation step be conducted
A.4 ADDITIONAL EXPERIMENTS TO VERIFY THE EFFECTIVENESS OF THE IMPORTANCE SAMPLING BASED POSTERIOR UTILITY ESTIMATION
Table 2: Statistics of posterior utilities w.r.t samples per estimation
exact value
samples per estimation
100
1000
RMSF of posterior utilities
0.0000
0.0369
0.0125
0.0038
0.0011
mean expected posterior utilitics
0.2681
0.2660
0.2677
0.2681
0.2681
variance of expected posterior utilities
3.4917e 9
3.5331e:5
3.0590e-6
2.9973e.7
8
14

---

Under review as a conference paper at ICLR 2021
importance sampling technique can be adopted to estimate the value of posterior utility. However, it apparent that the number of samples for each posterior utility estima- tion would highly affect the results. Extra experiments are conducted to verify the effectiveness of the importance sampling based posterior utility estimation as shown in Table 2 When we vary the number of samples per estimation from 1 to 1000, the Root Mean Square Error (RMSE) between the estimated values and the exact values of the posterior utilities decreases from 0.0369 to 0.001 I, which indicates that larger number of samples per estimation leads to more accurate approximation. solution to maximize the expected posterior utility and
As described in Section 3.2
We also utilize the proposed ADMM-basee record the mean and variance of the expected posterior utilities of the last 50 optimization iterations for each sampling setting. For the case with the number of samples per estimation set to 100 (or 1000), the variance of the expected posterior utilities is very small while the learned final value of expected posterior utility around 0.2681, which similar the result of optimization with exact value of posterior utility and thus verifies the effectiveness of the proposed importance sampling based posterior utility estimation.
A.5 ADDITIONAL, EXPERIMENTS TO ILLUSTRATE HOW THE POLICY ADJUST W.R.T. DIFFERENT LEVELS OF USER PREFERENCE OR UNCERTAINTY
12
F5
14
Fs
F7
1s
1
f10
41
112
f15
F14
f15
155
0.020
0.018
@ommmenddaa probabillty
Wo
0.5
0.0043
0.021
6433
00002
6990
C.2244
6233
8355
0072
0.016
49 0.0
0.011
0.031
613
K0054
0.011
89M
8359
3 833
0.755
E175
0.014
R9 0.5
6.030
0,0222
6.039
0,053
6C200
C.007
8357
0:444
8320
0.012
0.010
W
0.008
3 1.5
0:012
02109
EUTTER
0037%
00214
0.006
Figure 4: Learned policy w.r.t. different levels of user preference
F'a
f'22
Fi3
F'15
0.018
Xg.a = 0.005
2.07
5/113
A0081
0231>
$32.0
3:113
52113
5.331
02137
121.1
0013
3.574
02131
0.016
1o.3 m 0.010
5,077
40112
0487
000G
6317
53211
1.1.0
0013
1200
C211
0.014
1 = 0.015
0.0.0
0.012
3 0.020
00010
0.008
A
3 0.025
Figure 5: Learned policy w.r.t. different levels of user uncertainty
We also provide analysis about how different levels of user preference would affect the final learned policy over items. For a randomly sampled user, we vary wo from -1.0 to 1.5 and keep the values of other dimensions unchanged. We also randomly pick 16 items from candidate item set, sort these items according the value of dimension 0 of the item mean vector and denote the sorted items as i1, .,16- For each case (with specific value for wh)), we record the final learned policy (recommendation probabilities) over the picked items and present the heat map in Figure 4 where darker color represents larger recommendation probability. As shown in Figure 4 with the increase of the value of Wo. the learned recommendation probabilities of items with large value of dimension 0 (c.g.. f15 and (16) increase while those with small value decrease. Thus, to maximize the platform's 's commercial benefits, the platform would tend to recommend items that match user's 's preference over item dimension with high user preference. Similar results can be derived when we analyse how the learned policy adjust to different levels of user uncertainty over item dimensions. Thus, the proposed framework reveals that the policy would adjust more for item dimensions with higher preference or uncertainty so as to maximize the commercial benefits of the platform.
15


Reviews:
[{"review_id": "yCSjYzvvtV", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the problem of system utility optimization by explicitly exploiting user uncertainty regarding a specific item’s properties (e.g., certain dimensions of item representations) and the impact of the display policy. The basic idea is that when a user is uncertain about some properties of a particular item (e.g., quality of the item) or he/she might be more (or less) influenced by the system’s past recommendations, the system would have room to recommend items that maximize system utility. The idea sounds interesting and feasible to me, but the presented technical solution is unfortunately opaque and hard to follow. I do have a few of questions regarding the two assumptions and main results in this work.\n\nFirst, what is the meaning of $p_{u,i}(x)$? Based on the description, it should be the representation of item $i$ from user $u$’s perspective. However, the way to estimate the parameters of this distribution is based on the system learnt embeddings in user groups and item categories. Why would those learnt embeddings or such an estimation method correspond to a particular user’s assessment on an item? For example, the user group is constructed by age and gender, does it mean all users with the same gender and age would have the same uncertainty on items, disregarding to their specific experiences of those items? I believe this item uncertainty could be better modeled with respect to individual user’s interaction history with the item (e.g., non-durable consumer goods or service might be a better target for this study). \n\nThe second assumption is even harder to understand. Should this $p_{u,i}(display|x;\\pi)$ be understood as a user’s belief of why item $x$ is recommended to him/her? But how would this estimate on the user side affect his/her behavior? The higher the probability, the more likely the user would click on the result? Based on Eq (5), it seems to be the case; but why would it be in practice? For example, when the user realizes the system is taking advantage of him/her for profit, would the user go against the recommendations? This actually leads to a bigger question regarding the problem setup in this paper: throughout the paper, the tone is the system takes advantage of users to maximize its “commercial benefits”, which is very shortsighted. This is based on a static assumption about users’ behaviors and their trust on the system, as required by Eq (5). However, once the user realizes the system’s strategy and loses his/her trust on the system, the system will lose further opportunities for making recommendations (or gaining its utilities). \n\nThe reported experiment results are also not easy to follow. For example, Figure 1 and 2 showed the obtained utility from the learnt policy, which illustrate the learning is converging. But how good is the learnt policy? There should be some baselines for comparison purpose, e.g., at least against a random policy? Figure 3’s result isn’t a surprise: the utility function is a sigmoid function, which is a generalized linear function, over the item representation and user preference. As a result, how the learnt policy reacts to different user preference can be suggested by the linear case’s analysis. Moreover, I do not quite follow the definition of $v_0$: “record the learned expected value (w.r.t. the learned policy) of the elements with dimension 0 of the mean vectors, which is denoted as $v_0$.”   \n\nBy the way, I could not derive Eq (5) from Eq (4), and this is what I have: \n$$\np_{u,i}(x|display,\\pi)=\\frac{ p_{u,i}(x,display,\\pi)}{ p_{u,i}(display,\\pi)} =\\frac{ p_{u,i}(x) p_{u,i}(display,\\pi|x)}{ p_{u,i}(display,\\pi)}\n$$\n\n**Acknowledgement of author rebuttal** I appreciate the detailed discussions that the authors have provided. However, my concerns still remain, and I am not convinced to improve my overall evaluation. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "tB8WETxh21q", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper aims to model the posterior utility of showing an item given a static display policy, where the utility function captures both: (a) uncertainty over item dimensions from the user perspective, and (b) influence of the policy on the user. It is motivated by the fact that most recommender systems don't take into account that user may be highly uncertain about value/utility in certain dimensions (e.g., color of a product) while more certain about others. The platform can use this information explicitly while optimizing for what to show.\n\nA question that needs empirical evidence is whether uncertainty in certain dimensions have affect on recommendation outcomes. A priori it seems unclear which type of an effect this has. For instance, if a user is more uncertain about some dimensions for item 1 vs item 2, how can the platform exploit it? \n\n\n - Isn't the below incorrect as in the very next line you point out works that capture this:\n\n> may affect the users’ behaviors, which is crucial but always ignored.\n\n - The following seems untrue. There are several papers at RecSys (and workshops that focus on explicit modeling of user behavior and their response to displays, see https://recsys.acm.org/recsys20/workshops/ )\n\n>  However, most existing works in recommendation area do not explicitly model the impact of the display policy\non the user, which might be self-limiting and incomplete.\n\n\n\nIn eq (1),  the decision variable seems to be a prob. mass function over the items. How is the recommendation performed is unclear at this point.\n\nIt is unclear what x represents in Assumption 1/eq (2). The papers say its the representation of the item. So is x a realization of item attributes from the user's perspective? Can they be interpreted as utilities/values per unit?\n\nThe normality assumptions in eq (2) and (3): what is the price of reality deviation from these?\n\nIt says v_u is the expected item representation given a display policy \\pi, but unclear how this can be computed. Assumption 1 does not have any influence of display policy on it.\n\nIn eq (3), what is the random variable 'display' indicating? Is it just user imagining that the item i will be displayed? This is very unclear.\n\nThe dependence of the utility on display policy is through the vector v_u which is unfortunately not explained fully.\n\nLemma 1, Corollary 1 and 2 seem to follow in a straightforward manner from standard properties of Gaussians. \nThe use of importance sampling to deal with a nonlinear f (which is still unclear how we get this in the first place? domain knowledge?) seems straightforward as well.\n\nThe use of ADMM procedure, while good, would also be considered standard at this point.\n\nOverall, the paper is well written, although the ideas are relatively less novel.\n\n\n - [Minor] Rephrase below:\n\n> .. serving as the role of user aggregation", "extra_scores": {}, "extra_text": {}}, {"review_id": "t0bRkp7ItcC", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \nThis paper studies an interesting problem and proposes a novel recommendation framework to utilize user uncertainty over different item dimensions and the impact of display policy over users. To achieve the maximized expected posterior utility, it also provides a technically sound solution to derive the approximately optimal policy based on ADMM, and achieve provides insights for the commercial recommendation based on the experiments with real-world data.\n\nReasons for score: \nI like the idea of taking into consideration user uncertainty and display policy. My major concern is about experiments (see cons below). Hope the authors can address my concern in the rebuttal period.\n \nPros: \n1. The paper studies the most important problem for recommendation platforms: commercial benefits. The problem itself will have great impacts on real-world scenarios.\n2. The proposed recommendation framework is novel for taking into consideration user uncertainty and display policy for maximizing commercial benefits. Deriving the problem into maximizing the posterior utility, this paper also proposes the ADMM-based solution to derive the approximately optimal policy. \n3. Experiments on the real-world dataset provide some practical insights about how to achieve commercial benefits for the recommendation platform.\n \nCons: \n1. Although several insightful experiments are provided, I still have several suggestions on the experiments to enhance the quality of the paper: \n(1) Although the data is from a real-world scenario, there are only results on expected utility. It might be valuable to investigate the real-world beneficial increases (if it is sensitive for companies to share the exact value, maybe it can provide the value of the increasing percentage).\n(2) There is no baseline. It might because this paper utilizes expected posterior utility to evaluate the effectiveness,  this metric may not be suitable for other recommendation baselines. Thus, is it possible to select another metric as a supplement and have several comparison experiments?\n2. In the introduction, it would be better to provide more details about “item dimensions” and “user uncertainty”, which seems not very clear to me. \n\nQuestions during the rebuttal period: \nPlease address and clarify the cons above ", "extra_scores": {}, "extra_text": {}}, {"review_id": "5VXircYlENe", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper proposes a recommendation approach by optimising expected utility over item display policies. The objective also accounts for the uncertainty of item representations which is particularly interesting. The ADMM method is adopted to search for an approximately optimal item display policy for both linear and nonlinear utility functions. Experiments are conducted and results presented.\n\n\nStrengths:\n\nThe proposed approach incorporates the uncertainty of item representations in recommendation systems which is very interesting in my opinion.\n\nThe paper is well organised and it would be even better if lemma 1 and the proof of corollary 2 were moved to Appendix.\n\n\nConcerns:\n\nI was unable to find any empirical evaluation on test data for the proposed approach, in addition, lacking experiments that compare the proposed method with competing methods is another major concern.\n\nHow efficient is the proposed method on large item set (e.g., millions of items)? It seems very challenging to optimise the constrained quadratic program (Eq 21-22) in that case.\n\n\nMinor comments:\n\nSec 1 first sentence: \nTicTok -> TikTok\n\nIn Eq (1) and many other equations: \nWhat is the \"display\" random variable?\n\nAssumption 1: \nthe probability of being x -> the density of being x\n\nPage 5 first sentence: \n\"Corollary 2 reveals that the posterior utility can be effectively calculated within constant time …\"\nI disagree with this.\n\nSec 4.1 second paragraph: \n\"we also incorporate an entropy-based regularization term into our model.\"\nCould this be precise?\n\n\nUpdates: I would like to thank the authors for their response and the updated draft. Unfortunately, I believe the above major concerns are still valid and therefore retain my original rating.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper received borderline negative scores. The reviewers all agree that the proposed approach is interesting. However, there are also common concerns around the clarity of the paper, as well as lacking sufficient empirical evaluation. One reviewer also argues that technical contribution is relatively limited. The author responses were taken into account but it didn't manage to swing the reviews. Therefore, I recommend reject and wish the authors can incorporate the feedback in the revision. 
Decision: Reject