Original: B EYOND G REEDY R ANKING : S LATE O PTIMIZATION VIA L IST -CVAE
Revised: B EYOND G REEDY R ANKING : S LATE O PTIMIZATION VIA L IST -CVAE

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The conventional approach to solving the recommendation problem greedily ranks individual document candidates by prediction scores.
Revised: The conventional approach to solving the recommendation problem greedily ranks individual document candidates by prediction scores.

Original: However, this method fails to optimize the slate as a whole, and hence, often struggles to capture biases caused by the page layout and document interdepedencies.
Revised: However, this method fails to optimize the slate as a whole, and hence, often struggles to capture biases caused by the page layout and document interdepedencies.

Original: The slate recommendation problem aims to directly ﬁnd the optimally ordered subset of documents ( i.e. slates ) that best serve users’ interests.
Revised: The slate recommendation problem aims to directly ﬁnd the optimally ordered subset of documents ( i.e. slates ) that best serve users’ interests.

Original: Solving this problem is hard due to the combinatorial explosion of document candidates and their display positions on the page.
Revised: Solving this problem is hard due to the combinatorial explosion of document candidates and their display positions on the page.

Revised: Therefore we propose a paradigm shift from the traditional viewpoint of solving a ranking problem to a direct slate generation framework.

Original: In this paper, we introduce List Conditional Variational Auto-Encoders (ListCVAE) , which learn the joint distribution of documents on the slate conditioned on user responses, and directly generate full slates.
Revised: In this paper, we introduce List Conditional Variational Auto-Encoders (List-CVAE) , which learn the joint distribution of documents on the slate conditioned on user responses, and directly generate full slates.

Original: Experiments on simulated and real-world data show that List-CVAE outperforms greedy ranking methods consistently on various scales of documents corpora.
Revised: Experiments on simulated and real-world data show that List-CVAE outperforms greedy ranking methods consistently on various scales of documents corpora.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Recommendation systems modeling is an important machine learning area in the IT industry, powering online advertisement, social networks and various content recommendation services (Schafer et al., 2001; Lu et al., 2015).
Revised: Recommendation systems modeling is an important machine learning area in the IT industry, powering online advertisement, social networks and various content recommendation services (Schafer et al., 2001; Lu et al., 2015).

Original: In the context of document recommendation, its aim is to generate and display an ordered list of “documents” (called a “slate” in (Swaminathan et al., 2017; Sunehag et al., 2015)) to users, based on both user preferences and documents content.
Revised: In the context of document recommendation, its aim is to generate and display an ordered list of “documents” (called a “slate” in (Swaminathan et al., 2017; Sunehag et al., 2015)) to users, based on both user preferences and documents content.

Original: For large scale recommender systems, a common scalable approach at inference time is to ﬁrst select a small subset of candidate documents S out of the entire document pool D .
Revised: For large scale recommender systems, a common scalable approach at inference time is to ﬁrst select a small subset of candidate documents S out of the entire document pool D .

Original: This step is called “candidate generation”.
Revised: This step is called “candidate generation”.

Original: Then a function approximator such as a neural network (e.g., a Multi-Layer Perceptron or MLP) called the “ranking model” is used to predict probabilities of user engagements for each document in the small subset S and greedily generates a slate by sorting the top documents from S based on estimated prediction scores (Covington et al., 2016).
Revised: Then a function approximator such as a neural network (e.g., a Multi-Layer Perceptron or MLP) called the “ranking model” is used to predict probabilities of user engagements for each document in the small subset S and greedily generates a slate by sorting the top documents from S based on estimated prediction scores (Covington et al., 2016).

Original: This two-step process is widely popular to solve large scale recommender problems due to its scalability and fast inference at serving time.
Revised: This two-step process is widely popular to solve large scale recommender problems due to its scalability and fast inference at serving time.

Original: The candidate generation step can decrease the number of candidates from millions to hundreds or less, effectively dealing with scalability when faced with a large corpus of documents D .
Revised: The candidate generation step can decrease the number of candidates from millions to hundreds or less, effectively dealing with scalability when faced with a large corpus of documents D .

Original: Since |S| is much smaller than |D| , the ranking model can be reasonably complicated without increasing latency.
Revised: Since |S| is much smaller than |D| , the ranking model can be reasonably complicated without increasing latency.

Original: However, there are two main problems with this approach.
Revised: However, there are two main problems with this approach.

Original: First the candidate generation and the ranking models are not trained jointly, which can lead to having candidates in S that are not the highest scoring documents of the ranking model.
Revised: First the candidate generation and the ranking models are not trained jointly, which can lead to having candidates in S that are not the highest scoring documents of the ranking model.

Original: Second and most importantly, the greedy ranking of documents on the slate suffers from numerous biases that come with the visual presentation of the slate and context in which documents are presented.
Revised: Second and most importantly, the greedy ranking of documents on the slate suffers from numerous biases that come with the visual presentation of the slate and context in which documents are presented.

Original: For example, there exists positional biases caused by users paying more attention to prominent slate positions (Joachims et al., 2005), and contextual biases, due to interactions between documents presented together in the same slate, such as competition and complementarity, relative attractiveness, etc.
Revised: For example, there exists positional biases caused by users paying more attention to prominent slate positions (Joachims et al., 2005), and contextual biases, due to interactions between documents presented together in the same slate, such as competition and complementarity, relative attractiveness, etc.

Original: (Yue et al., 2010).
Revised: (Yue et al., 2010).

Original: In this paper, we study the slate recommendation problem beyond the greedy ranking paradigm.
Revised: In this paper, we propose a paradigm shift from the traditional viewpoint of solving a ranking problem to a direct slate generation framework.

Original: We consider a slate “optimal” when it maximizes some type of user engagement feedback, a typical desired scenario in recommendation systems.
Revised: We consider a slate “optimal” when it maximizes some type of user engagement feedback, a typical desired scenario in recommendation systems.

Original: For example, given a database of song tracks, the optimal slate can be an ordered list (in time or space) of k songs such that the user ideally likes every song in that list (in the order they are presented).
Revised: For example, given a database of song tracks, the optimal slate can be an ordered list (in time or space) of k songs such that the user ideally likes every song in that list (in the order they are presented).

Original: Another example considers news articles, the optimal slate has k ordered articles such that every article is read by the user.
Revised: Another example considers news articles, the optimal slate has k ordered articles such that every article is read by the user.

Original: In general, optimality can be deﬁned as a desired user response vector on the slate and the proposed model should be agnostic to these problem-speciﬁc deﬁnitions.
Revised: In general, optimality can be deﬁned as a desired user response vector on the slate and the proposed model should be agnostic to these problem-speciﬁc deﬁnitions.

Original: Solving the slate recommendation problem directly differs from ranking in that ﬁrst, it does not assume that more relevant documents should necessarily be put in earlier positions in the slate.
Revised: Solving the slate recommendation problem directly differs from ranking in that ﬁrst, it does not assume that more relevant documents should necessarily be put in earlier positions in the slate.

Original: Second, the entire slate is used as a training example instead of single documents, preserving numerous biases encoded into the slate that might inﬂuence user responses.
Revised: Second, the entire slate is used as a training example instead of single documents, preserving numerous biases encoded into the slate that might inﬂuence user responses.

Original: Finally, our model directly generates slates, again taking into account all the inﬂuential biases the model learned through training.
Revised: Finally, our model directly generates slates, again taking into account all the inﬂuential biases the model learned through training.

Original: In this paper, we apply Conditional Variational Auto-Encoders (CVAEs) (Kingma et al., 2014; Kingma and Welling, 2013; Jimenez Rezende et al., 2014) to model the distributions of all documents in the same slate conditioned on the user response.
Revised: In this paper, we apply Conditional Variational Auto-Encoders (CVAEs) (Kingma et al., 2014; Kingma and Welling, 2013; Jimenez Rezende et al., 2014) to model the distributions of all documents in the same slate conditioned on the user response.

Original: All documents in a slate along with their positional, contextual biases are jointly encoded into the latent space, which is then sampled and combined with desired conditioning for direct slate generation, i.e. sampling from the learned conditional joint distribution.
Revised: All documents in a slate along with their positional, contextual biases are jointly encoded into the latent space, which is then sampled and combined with desired conditioning for direct slate generation, i.e. sampling from the learned conditional joint distribution.

Original: Therefore, the model ﬁrst learns which slates give which type of responses and then directly generates similar slates given a desired response vector as the conditioning at inference time.
Revised: Therefore, the model ﬁrst learns which slates give which type of responses and then directly generates similar slates given a desired response vector as the conditioning at inference time.

Original: We call our proposed model List-CVAE .
Revised: We call our proposed model List-CVAE .

Original: The key contributions of our work are:
Revised: The key contributions of our work are:

Original: To the best of our knowledge, this is the ﬁrst model that provides a conditional generative modeling framework for slate recommendation by direct generation.
Revised: To the best of our knowledge, this is the ﬁrst model that provides a conditional generative modeling framework for slate recommendation by direct generation.

Original: It does not necessarily require a candidate generator at inference time and is ﬂexible enough to work with any visual presentation of the slate as long as the ordering of display positions is ﬁxed throughout training and inference times.
Revised: It does not necessarily require a candidate generator at inference time and is ﬂexible enough to work with any visual presentation of the slate as long as the ordering of display positions is ﬁxed throughout training and inference times.

Original: To deal with the problem at scale, we introduce an architecture that uses pretrained document embeddings combined with a negatively downsampled k -head softmax layer within the List-CVAE model, where k is the slate size.
Revised: To deal with the problem at scale, we introduce an architecture that uses pretrained document embeddings combined with a negatively downsampled k -head softmax layer within the List-CVAE model, where k is the slate size.

Original: The structure of this paper is the following.
Revised: The structure of this paper is the following.

Original: First we introduce related work using various CVAE-type models as well as other approaches to solve the slate generation problem.
Revised: First we introduce related work using various CVAE-type models as well as other approaches to solve the slate generation problem.

Original: Next we introduce our
Revised: Next we introduce our

Original: List-CVAE modeling approach.
Revised: List-CVAE modeling approach.

Original: The last part of the paper is devoted to experiments on both simulated and the real-world datasets.
Revised: The last part of the paper is devoted to experiments on both simulated and the real-world datasets.

Original: R ELATED W ORK

Original: [Figure]
Revised: [Figure]

Original: Traditional matrix factorization techniques have been applied to recommender systems with success in modeling competitions such as the Netﬂix Prize (Koren et al., 2009).
Revised: Traditional matrix factorization techniques have been applied to recommender systems with success in modeling competitions such as the Netﬂix Prize (Koren et al., 2009).

Original: Later research emerged on using autoencoders to improve on the results of matrix factorization (Wu et al., 2016; Wang et al., 2015) (CDAE, CDL).
Revised: Later research emerged on using autoencoders to improve on the results of matrix factorization (Wu et al., 2016; Wang et al., 2015) (CDAE, CDL).

Original: More recently several works use Boltzmann Machines (Abdollahi and
Revised: More recently several works use Boltzmann Machines (Abdollahi and

Original: Nasraoui, 2016) and variants of VAE models in the Collaborative Filtering (CF) paradigm to model recommender systems (Li and She, 2017; Lee et al., 2017; Liang et al., 2018)
Revised: Nasraoui, 2016) and variants of VAE models in the Collaborative Filtering (CF) paradigm to model recommender systems (Li and She, 2017; Lee et al., 2017; Liang et al., 2018)

Original: (Collaborative VAE,
Revised: (Collaborative VAE,

Original: JMVAE, CVAE-CF, JVAE-CF).
Revised: JMVAE, CVAE-CF, JVAE-CF).

Original: See Figure 1 for model comparisons.
Revised: See Figure 1 for model comparisons.

Original: In this paper, unless speciﬁed otherwise, the user features and any context are routinely considered part of the conditioning variables (in Section 4.1 Personalization Test, we test List-CVAE generating personalized slates for different users).
Revised: In this paper, unless speciﬁed otherwise, the user features and any context are routinely considered part of the conditioning variables (in Appendix A Personalization Test, we test List-CVAE generating personalized slates for different users).

Original: These models have primarily focused on the greedy approach of modeling independently each document or pairs of documents in the slate and applying greedy ordering at inference time.
Revised: These models have primarily focused on the greedy approach of modeling independently each document or pairs of documents in the slate and applying greedy ordering at inference time.

Original: Our model is also using a VAE type structure and in particular, is closely related to the Joint
Revised: Our model is also using a VAE type structure and in particular, is closely related to the Joint

Original: Multimodel Variational Auto-Encoder (JMVAE) architecture (Figure 1d).
Revised: Multimodel Variational Auto-Encoder (JMVAE) architecture (Figure 1d).

Original: However, we use whole slates as input instead of single documents, and directly generate slates instead of using greedy ranking by prediction scores.
Revised: However, we use whole slates as input instead of single documents, and directly generate slates instead of using greedy ranking by prediction scores.

Original: Other relevant work from the Information Retrieval (IR) literature are listwise ranking methods, e.g.
Revised: Other relevant work from the Information Retrieval (IR) literature are listwise ranking methods, e.g.

Original: (Cao et al., 2007; Xia et al., 2008; Shi et al., 2010; Huang et al., 2015).
Revised: (Cao et al., 2007; Xia et al., 2008; Shi et al., 2010; Huang et al., 2015; Ai et al., 2018).

Original: These methods use a listwise loss function that takes context and position into account.
Revised: These methods use a listwise loss function that takes context and position into account.

Original: They also eventually assign a prediction score for each document and greedily rank them at inference time.
Revised: They also eventually assign a prediction score for each document and greedily rank them at inference time.

Original: In the Reinforcement Learning (RL) literature, Sunehag et al.
Revised: In the Reinforcement Learning (RL) literature, Sunehag et al.

Original: (2015) view the whole slates as actions and use a deterministic policy gradient update to learn a policy that generates these actions, given concatenated document features as input.
Revised: (2015) view the whole slates as actions and use a deterministic policy gradient update to learn a policy that generates these actions, given concatenated document features as input.

Original: Finally, the framework proposed by (Wang et al., 2016) predicts user engagement for document and position pairs.
Revised: Finally, the framework proposed by (Wang et al., 2016) predicts user engagement for document and position pairs.

Original: It optimizes whole page layouts but may suffer from poor scalability due to combinatorial explosion.
Revised: It optimizes whole page layouts but may suffer from poor scalability due to combinatorial explosion.

Original:  T HEORY 3. P ROBLEM S ETUP
Revised:  M ETHOD 3. P ROBLEM S ETUP

Original: We formally deﬁne the slate recommendation problem as follows.
Revised: We formally deﬁne the slate recommendation problem as follows.

Original: Let D denote a corpus of documents and let k be the slate size.
Revised: Let D denote a corpus of documents and let k be the slate size.

Original: Then let r =
Revised: Then let r =

Original: ( r 1 , . . .
Revised: ( r 1 , . . .

Original: , r k ) be the engagement response vector from users where r i ∈ R is the user response for document d i .
Revised: , r k ) be the engagement response vector from users where r i ∈ R is the user response for document d i .

Original: For example, if the problem is to maximize the number of clicks on a slate, then r i ∈ { 0 , 1 } denotes whether the document d i is clicked, and an optimal slate s = ( d 1 , d 2 , . . .
Revised: For example, if the problem is to maximize the number of clicks on a slate, then r i ∈ { 0 , 1 } denotes whether the document d i is clicked, and an optimal slate s = ( d 1 , d 2 , . . .

Original: , d k )
Revised: , d k )

Original: where
Revised: where

Original: d i ∈
Revised: d i ∈

Original: D is such that s maximizes E [ (cid:80) ki =1 r i ] .
Revised: D is such that s maximizes E [ (cid:80) ki =1 r i ] .

Original: 3. V ARIATIONAL A UTO -E NCODERS
Revised: 3. V ARIATIONAL A UTO -E NCODERS

Original: Variational Auto-Encoders (VAEs) are latent-variable models that deﬁne a joint density P θ ( x, z ) between observed variables x and latent variables z parametrized by a vector θ .
Revised: Variational Auto-Encoders (VAEs) are latent-variable models that deﬁne a joint density P θ ( x, z ) between observed variables x and latent variables z parametrized by a vector θ .

Original: Training such models requires marginalizing the latent variables in order to maximize the data likelihood
Revised: Training such models requires marginalizing the latent variables in order to maximize the data likelihood

Original: P θ ( x ) = (cid:82) P θ ( x, z ) dz .
Revised: P θ ( x ) = (cid:82) P θ ( x, z ) dz .

Original: Since we cannot solve this marginalization explicitly, we resort to a variational approximation.
Revised: Since we cannot solve this marginalization explicitly, we resort to a variational approximation.

Original: For this, a variational posterior density Q φ ( z | x ) parametrized by a vector φ is introduced and we optimize the variational Evidence Lower-Bound (ELBO) on the data loglikelihood:
Revised: For this, a variational posterior density Q φ ( z | x ) parametrized by a vector φ is introduced and we optimize the variational Evidence Lower-Bound (ELBO) on the data loglikelihood:

Original: [Equation]
Revised: [Equation]

Original: where KL is the Kullback–Leibler divergence and where P θ ( z ) is a prior distribution over latent variables.
Revised: where KL is the Kullback–Leibler divergence and where P θ ( z ) is a prior distribution over latent variables.

Original: In a Conditional VAE (CVAE) we extend the distributions P θ ( x, z ) and Q φ ( z | x ) to also depend on an external condition c .
Revised: In a Conditional VAE (CVAE) we extend the distributions P θ ( x, z ) and Q φ ( z | x ) to also depend on an external condition c .

Original: The corresponding distributions are indicated by P θ ( x, z | c ) and
Revised: The corresponding distributions are indicated by P θ ( x, z | c ) and

Original: Q φ ( z | x, c ) .
Revised: Q φ ( z | x, c ) .

Original: Taking the conditioning c into account, we can write the variational loss to minimize as
Revised: Taking the conditioning c into account, we can write the variational loss to minimize as

Original: [Equation]
Revised: [Equation]

Original: 3. O UR M ODEL
Revised: 3. O UR M ODEL

Original: We assume that the slates s = ( d 1 , d 2 , . . .
Revised: We assume that the slates s = ( d 1 , d 2 , . . .

Original: d k ) and the user response vectors r are jointly drawn from a distribution P D k ×R k .
Revised: d k ) and the user response vectors r are jointly drawn from a distribution P D k ×R k .

Original: In this paper, we use a CVAE to model the joint distribution of all documents in the slate conditioned on the user responses r , i.e. P ( d 1 , d 2 , . . .
Revised: In this paper, we use a CVAE to model the joint distribution of all documents in the slate conditioned on the user responses r , i.e. P ( d 1 , d 2 , . . .

Original: d k | r ) .
Revised: d k | r ) .

Original: At inference time, the List-CVAE model attempts to generate an optimal slate by conditioning on the ideal user response r (cid:63) .
Revised: At inference time, the List-CVAE model attempts to generate an optimal slate by conditioning on the ideal user response r (cid:63) .

Original: As we explained in Section 1, “optimality” of a slate depends on the task.

Original: With that in mind, we deﬁne the mapping Φ : R k (cid:55)→ C .

Original: It transforms a user response r into a vector in the conditioning

Original: MLP Encoder
Revised: MLP Encoder

Original: Embedding Ψ Embedding Ψ
Revised: Embedding Ψ Embedding Ψ

Original: Embedding Ψ c =
Revised: Embedding Ψ c =

Original: Φ( r ) s ( µ , σ ) z ∼ Q φ ( z | s , c ) =
Revised: Φ( r ) s ( µ , σ ) z ∼ Q φ ( z | s , c ) =

Original: N ( µ , σ )
Revised: N ( µ , σ )

Original: MLP µ 0 , σ
Revised: MLP µ 0 , σ

Original: P θ ( z | c ) =
Revised: P θ ( z | c ) =

Original: N ( µ 0 , σ 0 ) f prior
Revised: N ( µ 0 , σ 0 ) f prior

Original: MLP x 1 , x 2 ..., x k
Revised: MLP x 1 , x 2 ..., x k

Original: Dot-productDot-product
Revised: Dot-productDot-product

Original: SoftmaxSoftmaxSoftmax
Revised: SoftmaxSoftmaxSoftmax

Original: Decoder s ∼ P θ ( s | z , c )
Revised: Decoder s ∼ P θ ( s | z , c )

Original: Dot-product
Revised: Dot-product

Original: Embedding Ψ Embedding Ψ
Revised: Embedding Ψ Embedding Ψ

Original: Embedding Ψ
Revised: Embedding Ψ

Original: D = { d 1 ,...,d n } (a) Training
Revised: D = { d 1 ,...,d n } (a) Training

Original: MLP z ∼ N ( µ (cid:63) , σ (cid:63) ) c (cid:63) =
Revised: MLP z ∼ N ( µ (cid:63) , σ (cid:63) ) c (cid:63) =

Original: Φ( r (cid:63) )
Revised: Φ( r (cid:63) )

Original: MLP f prior ( µ (cid:63) , σ (cid:63) ) x 1 , x 2 ..., x k
Revised: MLP f prior ( µ (cid:63) , σ (cid:63) ) x 1 , x 2 ..., x k

Original: Dot-productDot-product
Revised: Dot-productDot-product

Original: MaxMaxMax
Revised: MaxMaxMax

Original: Decoder s (cid:63)
Revised: Decoder s (cid:63)

Original: Dot-product
Revised: Dot-product

Original: Embedding Ψ Embedding Ψ
Revised: Embedding Ψ Embedding Ψ

Original: Embedding Ψ
Revised: Embedding Ψ

Original: D = { d 1 ,...,d n }
Revised: D = { d 1 ,...,d n }

Original: [Figure]
Revised: [Figure]

Original: ) Inference is the conditioning where r = ( r 1 , r 2 , . . .
Revised: ) Inference is the conditioning where r = ( r 1 , r 2 , . . .

Original: , r k ) is the user responses on the slate s .
Revised: , r k ) is the user responses on the slate s .

Original: The concatenation of s and c makes the input vector to the encoder.
Revised: The concatenation of s and c makes the input vector to the encoder.

Original: z ∈ R m is the latent variable with a learned prior distribution N ( µ 0 , σ 0 ) .
Revised: z ∈ R m is the latent variable with a learned prior distribution N ( µ 0 , σ 0 ) .

Original: The raw output from the decoder are k vectors x 1 , x 2 . . .
Revised: The raw output from the decoder are k vectors x 1 , x 2 . . .

Original: , x k , each of which is mapped to a real document through taking the dot product with the matrix Φ containing all document embeddings.
Revised: , x k , each of which is mapped to a real document through taking the dot product with the matrix Φ containing all document embeddings.

Original: Thus produced k vectors of logits are then passed to the negatively downsampled k -head softmax operation.
Revised: Thus produced k vectors of logits are then passed to the negatively downsampled k -head softmax operation.

Original: c (cid:63) is the ideal condition whose concatenation with sampled z is the input to the decoder at inference time.
Revised: c (cid:63) is the ideal condition whose concatenation with sampled z is the input to the decoder at inference time.

Revised: As we explained in Section 1, “optimality” of a slate depends on the task.

Revised: With that in mind, we deﬁne the mapping Φ : R k (cid:55)→ C .

Original:  space C that encodes the user engagement metric we wish to optimize for.
Revised: It transforms a user response r into a vector in the conditioning space C that encodes the user engagement metric we wish to optimize for.

Original: For instance, if we want to maximize clicks on the slate, we can use the binary click response vectors and set the conditioning to c =
Revised: For instance, if we want to maximize clicks on the slate, we can use the binary click response vectors and set the conditioning to c =

Original: Φ( r ) :
Revised: Φ( r ) :

Original: = (cid:80) ki =

Revised: = (cid:80) ki =

Original: r i .
Revised: r i .

Original: Then at inference time, the corresponding ideal user response r (cid:63) would be (1 , 1 , . . .
Revised: Then at inference time, the corresponding ideal user response r (cid:63) would be (1 , 1 , . . .

Original: , 1) , and thus the ideal conditioning would be c (cid:63) =
Revised: , 1) , and thus the ideal conditioning would be c (cid:63) =

Original: Φ( r (cid:63) ) =
Revised: Φ( r (cid:63) ) =

Original: (cid:80) k i =0 1 = k .
Revised: (cid:80) ki =0 1 = k .

Original: As usual with CVAEs, the decoder models a distribution P θ ( s | z , c ) that, conditioned on z, is easy to represent.
Revised: As usual with CVAEs, the decoder models a distribution P θ ( s | z , c ) that, conditioned on z, is easy to represent.

Original: In our case, P θ ( s | z , c ) models an independent probability for each document on the slate, represented by a softmax distribution.
Revised: In our case, P θ ( s | z , c ) models an independent probability for each document on the slate, represented by a softmax distribution.

Original: Note that the documents are only independent to each other conditional on z .
Revised: Note that the documents are only independent to each other conditional on z .

Original: In fact, the marginalized posterior P θ ( s | c ) =
Revised: In fact, the marginalized posterior P θ ( s | c ) =

Original: (cid:82) z P θ ( s | z , c ) P θ ( z | c ) d z can be arbitrarily complex.
Revised: (cid:82) z P θ ( s | z , c ) P θ ( z | c ) d z can be arbitrarily complex.

Original: When the encoder encodes the input slate s into the latent space, it learns the joint distribution of the k documents in a ﬁxed order, and thus also encodes any contextual, positional biases between documents and their respective positions into the latent variable z .
Revised: When the encoder encodes the input slate s into the latent space, it learns the joint distribution of the k documents in a ﬁxed order, and thus also encodes any contextual, positional biases between documents and their respective positions into the latent variable z .

Original: The decoder learns these biases through reconstruction of the input slates from latent variables z with conditions.
Revised: The decoder learns these biases through reconstruction of the input slates from latent variables z with conditions.

Original: At inference time, the decoder reproduces the input slate distribution from the latent variable z with the ideal conditioning, taking into account all the biases learned during training time.
Revised: At inference time, the decoder reproduces the input slate distribution from the latent variable z with the ideal conditioning, taking into account all the biases learned during training time.

Original: To shed light onto what is encoded in the latent space, we simplify the prior distribution of z to be a ﬁxed Gaussian distribution N ( 0 , I ) in R 2 .
Revised: To shed light onto what is encoded in the latent space, we simplify the prior distribution of z to be a ﬁxed Gaussian distribution N ( 0 , I ) in R 2 .

Original: We train List-CVAE and plot the predictive prior z .
Revised: We train List-CVAE and plot the predictive prior z .

Original: As training evolves, generated output slates with low total responses are pushed towards the edge of the latent space while high response slates cluster towards a larger, centered area (Figure 3).
Revised: As training evolves, generated output slates with low total responses are pushed towards the edge of the latent space while high response slates cluster towards a larger, centered area (Figure 3).

Original: Thereforeafter training, if we sample z from its prior distribution N ( 0 , I ) and generate the corresponding output slates, they more likely to have high total responses.
Revised: Therefore

Original: [Figure]
Revised: [Figure]

Revised: after training, if we sample z from its prior distribution N ( 0 , I ) and generate the corresponding output slates, they more likely to have high total responses.

Original: Since the number of documents in D can be large, we ﬁrst embed them into a low dimensional space.
Revised: Since the number of documents in D can be large, we ﬁrst embed them into a low dimensional space.

Original: Let Ψ : D (cid:55)→ S q − 1 be that normalized embedding where S q − 1 denotes the unit sphere in R q .
Revised: Let Ψ : D (cid:55)→ S q − 1 be that normalized embedding where S q − 1 denotes the unit sphere in R q .

Original: Ψ can easily be pretrained using a standard supervised model that predicts user responses from documents or through a standard auto-encoder technique.
Revised: Ψ can easily be pretrained using a standard supervised model that predicts user responses from documents or through a standard auto-encoder technique.

Original: For the i -th document in the slate, our model produces a vector x
Revised: For the i -th document in the slate, our model produces a vector x

Original: i in R q that is then matched to each document from D via a dot-product.
Revised: i in R q that is then matched to each document from D via a dot-product.

Original: This operation produces k vectors of logits for k softmaxes, i.e. k -head softmax.
Revised: This operation produces k vectors of logits for k softmaxes, i.e. k -head softmax.

Original: At training time, for large document corpora D , we uniformly randomly downsample negative documents and compute only a small subset of the logits for every training example, therefore efﬁciently scaling the nearest neighbor search to millions of documents with minimal model quality loss.
Revised: At training time, for large document corpora D , we uniformly randomly downsample negative documents and compute only a small subset of the logits for every training example, therefore efﬁciently scaling the nearest neighbor search to millions of documents with minimal model quality loss.

Original: We train this model as a CVAE by minimizing the sum of the reconstruction loss and the KLdivergence term:
Revised: We train this model as a CVAE by minimizing the sum of the reconstruction loss and the KLdivergence term:

Original: [Equation]
Revised: [Equation]

Original:  where β is a function of the current training step (Higgins et al., 2017).
Revised: [log P θ ( s | z , c )] , (4) where β is a function of the current training step (Higgins et al., 2017).

Original: During inference, output slates are generated by ﬁrst sampling z from the conditionally learned prior distribution N ( µ (cid:63) , σ (cid:63) ) , then concatenating with the ideal condition c (cid:63) =
Revised: During inference, output slates are generated by ﬁrst sampling z from the conditionally learned prior distribution N ( µ (cid:63) , σ (cid:63) ) , then concatenating with the ideal condition c (cid:63) =

Original: Φ( r (cid:63) ) , thus passed into the decoder generating ( x 1 , . . .
Revised: Φ( r (cid:63) ) , thus passed into the decoder generating ( x 1 , . . .

Original: , x k ) from the learned P θ ( s | z , c (cid:63) ) , and ﬁnally picking the argmax over the dot-products with the full embedding matrix independently for each i = 1 , . . .
Revised: , x k ) from the learned P θ ( s | z , c (cid:63) ) , and ﬁnally picking the argmax over the dot-products with the full embedding matrix independently for each i = 1 , . . .

Original: , k .
Revised: , k .

Original:  E XPERIMENTS 4. S IMULATION D ATA
Revised:  E XPERIMENTS 4. S IMULATION D ATA

Original: Setup:
Revised: Setup:

Original: The simulator generates a random matrix W ∼ N ( µ, σ ) k × n × k × n where each element
Revised: The simulator generates a random matrix W ∼ N ( µ, σ ) k × n × k × n where each element

Original: W i,d i ,j,d j represents the interaction between document d i at position i and document d j at position j , and n = |D| .
Revised: W i,d i ,j,d j represents the interaction between document d i at position i and document d j at position j , and n = |D| .

Original: It simulates biases caused by layouts of documents on the slate (below, we set µ =and σ = 0 . 5 ).
Revised: It simulates biases caused by layouts of documents on the slate (below, we set µ =and σ = 0 . 5 ).

Original: Every document d i ∈ D has a probability of engagement A i ∼ U (0 , 1) representing its innate attractiveness.
Revised: Every document d i ∈ D has a probability of engagement A i ∼ U (0 , 1) representing its innate attractiveness.

Original: User responses are computed by multiplying A i with interaction multipliers
Revised: User responses are computed by multiplying A i with interaction multipliers

Original: W ( i, d i , j, d j ) for each document presented before d i on the slate.
Revised: W ( i, d i , j, d j ) for each document presented before d i on the slate.

Original: Thus the user response
Revised: Thus the user response

Original: [Equation]
Revised: [Equation]

Revised: (cid:18)

Revised: [Equation]

Revised: (cid:89)

Original: for i = 1 , . . .
Revised: j [Equation] W [Equation]

Revised: d

Revised: [Equation]

Revised: (cid:19) (cid:12)(cid:12) (cid:12)(cid:12) [

Revised: [Equation]

Revised:   (5) for i = 1 , . . .

Original: , k , where B represents the Bernoulli distribution.
Revised: , k , where B represents the Bernoulli distribution.

Original: During training, all models see uniformly randomly generated slates s ∼ U{ 1 , n } k and their generated responses r .
Revised: During training, all models see uniformly randomly generated slates s ∼ U{ 1 , n } k and their generated responses r .

Original: During inference time, we generate slates s by conditioning on c (cid:63) =
Revised: During inference time, we generate slates s by conditioning on c (cid:63) =

Original: We do not require document de-duplication since repetition may be desired in certain applications (e.g. in an online advertisement session).
Revised: We do not require document de-duplication since repetition may be desired in certain applications (e.g. in an online advertisement session).

Original: Moreover List-CVAE should learn to produce the optimal slates whether those slates contain duplication or not from learning the training data distribution.
Revised: Moreover List-CVAE should learn to produce the optimal slates whether those slates contain duplication or not from learning the training data distribution.

Original: Evaluation: For evaluation, we cannot use ofﬂine ranking evaluation metrics such as Normalized
Revised: Evaluation: For evaluation, we cannot use ofﬂine ranking evaluation metrics such as Normalized

Original: Discounted Cumulative Gain (NDCG) (Järvelin and Kekäläinen, 2000), Mean Average Precision (MAP) (Baeza-Yates and Ribeiro-Neto, 1999) or Inverse Propensity Score (IPS) (Little and Rubin, 2002), etc.
Revised: Discounted Cumulative Gain (NDCG) (Järvelin and Kekäläinen, 2000), Mean Average Precision (MAP) (Baeza-Yates and Ribeiro-Neto, 1999) or Inverse Propensity Score (IPS) (Little and Rubin, 2002), etc.

Original: These metrics either requires prediction scores for individual documents or assumes that more relevant documents should appear in earlier ranking positions, unfairly favoring greedy ranking methods.
Revised: These metrics either requires prediction scores for individual documents or assumes that more relevant documents should appear in earlier ranking positions, unfairly favoring greedy ranking methods.

Revised: Moreover, we ﬁnd it limiting to use various diversity metrics since it is not always the case that a higher diversity-inclusive score gives better slates measured by user’s total responses.

Revised: Even though these metrics may be more transparent, they do not measure our end goal, which is to maximize the expected number of total responses on the generated slates.

Original: Instead, we evaluate the expected number of clicks over the distribution of generated slates and over the distribution of clicks on each document:
Revised: Instead, we evaluate the expected number of clicks over the distribution of generated slates and over the distribution of clicks on each document:

Original: [Equation]
Revised: [Equation]

Original: In practice, we distill the simulated environment of Eq.
Revised: In practice, we distill the simulated environment of Eq.

Original: 5 using cross-entropy loss onto a neural network model that ofﬁciates as our new simulation environment.
Revised: 5 using cross-entropy loss onto a neural network model that ofﬁciates as our new simulation environment.

Original: The model consists of an embedding layer, which encodes documents into 8-dimensional embeddings.
Revised: The model consists of an embedding layer, which encodes documents into 8-dimensional embeddings.

Original: It then concatenates the embeddings of all the documents that form a slate and follows this concatenation with two hidden layers and a ﬁnal softmax layer that predicts the slate response amongst the 2 k possible responses.
Revised: It then concatenates the embeddings of all the documents that form a slate and follows this concatenation with two hidden layers and a ﬁnal softmax layer that predicts the slate response amongst the 2 k possible responses.

Original: Thus we call it the “response model”.
Revised: Thus we call it the “response model”.

Original: We use the response model to predict user responses on 100, sampled output slates for evaluation purposes.
Revised: We use the response model to predict user responses on 100, sampled output slates for evaluation purposes.

Original: This allows us to accurately evaluate our output slates by List-CVAE and all other baseline models.
Revised: This allows us to accurately evaluate our output slates by List-CVAE and all other baseline models.

Original: Baselines: Our experiments compare List-CVAE with several greedy ranking baselines that are popularly deployed in industry productions, namely Greedy MLP,Position MLP, Pairwise MLP, 
Revised: Baselines: Our experiments compare List-CVAE with several greedy ranking baselines that are often deployed in industry productions, namely Greedy MLP, Pairwise MLP, Position MLP and

Original: Greedy Long Short-Term Memory (LSTM) models as well as the randomly generated slates  as a sanity check.
Revised: Greedy Long Short-Term Memory (LSTM) models. In addition to these greedy baselines, we also compare against auto-regressive (AR) versions of Position MLP and LSTM, as well as randomly-selected slates from the training set as a sanity check.

Original: List-CVAE generates slates s = arg max s ∈{ 1 ,...,n } k P θ ( s | z , c (cid:63) ) .
Revised: List-CVAE generates slates s = arg max s ∈{ 1 ,...,n } k P θ ( s | z , c (cid:63) ) .

Original: The encoder and decoder of List-CVAE, as well as all thegreedy MLP-type models consist of two fully-connected neural network layers of the same size.
Revised: The encoder and decoder of List-CVAE, as well as all the MLP-type models consist of two fully-connected neural network layers of the same size.

Original: Greedy MLP trains on ( d i , r i ) pairs and outputs the greedy slate consisting of the top k
Revised: Greedy MLP trains on ( d i , r i ) pairs and outputs the greedy slate consisting of the top k

Original: highest ˆ P ( r | d ) scoring documents.
Revised: highest ˆ P ( r | d ) scoring documents.

Original: Position MLP uses position in the slate as a feature during training time and sets it simply to 0 for fast performance in inference time.

Original: Pairwise MLP is an MLP model with a pairwise ranking loss L = α L x + (1 − α )
Revised: Pairwise MLP is an MLP model with a pairwise ranking loss L = α L x + (1 − α )

Original: L ( ˆ P ( x 1 ) − ˆ P ( x 2 ) + η ) where L x is the cross entropy loss and ( x 1 , x 2 )’s are pairs of documents randomly selected with different user responses from the same slate.
Revised: L ( ˆ P ( x 1 ) − ˆ P ( x 2 ) + η ) where L x is the cross entropy loss and ( x 1 , x 2 ) are pairs of documents randomly selected with different user responses from the same slate.

Original: We sweep on hyperparameters α and η in addition to the shared MLP model structure sweep.
Revised: We sweep on hyperparameters α and η in addition to the shared MLP model structure sweep.

Revised: Position MLP uses position in the slate as a feature during training time and sets it simply to 0 for fast performance at inference time.

Revised: AR Position MLP is identical to Position MLP with the exception that position used at inference to different for each slate position (as such it is slower than Position MLP).

Original: Greedy LSTM is an LSTM model with fully-connect layers before and after the recurrent middle layers.
Revised: Greedy LSTM is an LSTM model with fully-connect layers before and after the recurrent middle layers.

Original: We tune the hyperparams on the number of layers and their respective widths.
Revised: We tune the hyperparameters corresponding to the number of layers and their respective widths.

Original: We use sequences of documents that form slates as the input at training time, and uses single examples as inputs with sequence length 1 at inference time, which is similar to scoring documents as if they are in the ﬁrst position of a slate of size 1.
Revised: We use sequences of documents that form slates as the input at training time, and use single examples as inputs with sequence length 1 at inference time, which is similar to scoring documents as if they are in the ﬁrst position of a slate of size 1.

Original: Then we greedily rank the documents based on their prediction scores.
Revised: Then we greedily rank the documents based on their prediction scores.

Revised: AR LSTM is identical to Greedy LSTM during training.

Revised: During inference, however, it selects documents sequentially by ﬁrst selecting the best document for position 1, then unrolling the LSTM for 2 steps to select the best document for position 2, and so on.

Revised: Random selects slates uniformly randomly from the training set.

Original: Small-scale experiment ( n = 100 , 1000 , k = 10 ):
Revised: Small-scale experiment ( n = 100 , 1000 , k = 10 ):

Original: We use the trained document embeddings from the response model for List-CVAE and all the baseline models.
Revised: We use the trained document embeddings from the response model for List-CVAE and all the baseline models.

Original: For List-CVAE, we also use trained priors P θ ( z | c )
Revised: For List-CVAE, we also use trained priors P θ ( z | c )

Original: =
Revised: =

Original: N ( µ (cid:63) , σ (cid:63) ) where µ (cid:63) , σ (cid:63) =
Revised: N ( µ (cid:63) , σ (cid:63) ) where µ (cid:63) , σ (cid:63) =

Original: f prior ( c (cid:63) ) and f prior is modeled by a small MLP (16, 32).
Revised: f prior ( c (cid:63) ) and f prior is modeled by a small MLP (16, 32).

Original: Additionally, since we found little difference between different hyperparameters, we ﬁxed the width of all hidden layers to 128, the learning rate to 10 −and the number of latent dimensions to 16.
Revised: Additionally, since we found little difference between different hyperparameters, we ﬁxed the width of all hidden layers to 128, the learning rate to 10 −and the number of latent dimensions to 16.

Original: For all other baseline models, we sweep on the learningrates, model structures and any model speciﬁc hyperparameters such as α, η for Position MLP and the forget bias for the LSTM model. Figure 4a, 4b shows the performance comparison when the number of documents n = 100 , 1000 and slate size to k = 10 . While List-CVAE is not quite capable of reaching a perfect performance ofclicks (which is probably even above the optimal upper bound), it easily outperforms all other greedy ranking baselines after only a few training steps. Personalization test ( |U| = 50 , n = 100 , k = 10 ):
Revised: For all other baseline models, we sweep on the learning

Original: We add user features into the conditioning c , by adding a set U of 50 different users to the simulation engine and permuting the innate attractiveness of documents and their interactions matrix W by a

Original: [Figure]
Revised: [Figure]

Original: user-speciﬁc function π u . Let [Equation] [Equation]
Revised: rates, model structures and any model speciﬁc hyperparameters such as α, η for Position MLP and the forget bias for the LSTM model.

Original: be the response of the user u on the document d i . During training, the condition c is a concatenation of 16 dimensional user embeddings Θ( u ) obtained from the response model [Figure] and responses r .
Revised: Figures 4a and 4b show the performance comparison when the number of documents n = 100 ,and slate size to k = 10 .

Original: At inference time, the model conditions on c (cid:63) =

Original: ( r (cid:63) , Θ( u )) for each randomly generated test user u . We sweep over hidden layers of 512 or 1024 units in List-CVAE, and all baseline MLP structures. Figure 4c show that slates generated by List-CVAE have on average higher clicks than those produced by the greedy baseline models although the convergence took much longer to reach.
Revised: While List-CVAE is not quite capable of reaching a perfect performance of 10 clicks (which is probably even above the optimal upper bound), it easily outperforms all other ranking baselines after only a few training steps. Appendix A includes an additional personalization test.

Original: 4. R EAL - WORLD D ATA
Revised: 4. R EAL - WORLD D ATA

Original: Due to a lack of publicly available large scale slate datasets, we use the data provided by the RecSys 2015 YOOCHOOSE Challenge (Ben-Shimon et al., 2015).
Revised: Due to a lack of publicly available large scale slate datasets, we use the data provided by the RecSys 2015 YOOCHOOSE Challenge (Ben-Shimon et al., 2015).

Original: This dataset consists of 9.2M user purchase sessions around 53K products.
Revised: This dataset consists of 9.2M user purchase sessions around 53K products.

Original: Each user session contains an ordered list of products on which the user clicked, and whether they decided to buy them.
Revised: Each user session contains an ordered list of products on which the user clicked, and whether they decided to buy them.

Original: The List-CVAE model can be used on slates with temporal ordering.
Revised: The List-CVAE model can be used on slates with temporal ordering.

Original: Thus we form slates of size 5 by taking consecutive clicked products.
Revised: Thus we form slates of size 5 by taking consecutive clicked products.

Original: We then build user responses from whether the user bought them.
Revised: We then build user responses from whether the user bought them.

Original: We remove slates with no positive responses such that after removal they only account for 50% of the total number of slates.
Revised: We remove slates with no positive responses such that after removal they only account for 50% of the total number of slates.

Original: After ﬁltering out products that are rarely bought, we get 375K slates of size 5 and a corpus of 10, [Figure] [Figure] [Figure]
Revised: After ﬁltering out products that are rarely bought, we get 375K slates of size 5 and a corpus of 10, candidate documents.

Original: [Figure]

Original: candidate documents.

Original: Figure 5a shows the user response distribution of the training data.
Revised: Figure 5a shows the user response distribution of the training data.

Original: Notice that in the response vector, 0 denotes a click and 1 denotes a buying action.
Revised: Notice that in the response vector, 0 denotes a click and 1 denotes a buying action.

Original: For example, (1,0,0,0,1) means the user clicked on all ﬁve products but bought only the ﬁrst and the last products.
Revised: For example, (1,0,0,0,1) means the user clicked on all ﬁve products but bought only the ﬁrst and the last products.

Original: Medium-scale experiment ( n = 10 , 000 , k = 5 ):
Revised: Medium-scale experiment ( n = 10 , 000 , k = 5 ):

Original: Similarly to the previous section, we train a two-layer response model that ofﬁciates as a new semisynthetic simulation environment.
Revised: Similarly to the previous section, we train a two-layer response model that ofﬁciates as a new semisynthetic simulation environment.

Original: We use the same hyperparameters used previously.
Revised: We use the same hyperparameters used previously.

Original: Figure 5b shows that List-CVAE outperforms all greedy baseline models within 500 training steps, which corresponds to having seen less than 10 − 11 % of all possible slates.
Revised: Figure 5b shows that List-CVAE outperforms all other baseline models within 500 training steps, which corresponds to having seen less than 10 − 11 % of all possible slates.

Original: Large-scale experiment ( n = 1 million, 2 millions, k = 5 ):
Revised: Large-scale experiment ( n = 1 million, 2 millions, k = 5 ):

Original: We synthesize 1,990k documents by adding independent Gaussian noise N (0 , 0 . 01)
Revised: We synthesize 1,990k documents by adding independent Gaussian noise N (0 , 0 . 01)

Original: k to the original 10k documents and label the synthetic documents by predicted responses from the response model.
Revised: k to the original 10k documents and label the synthetic documents by predicted responses from the response model.

Original: The new pool of candidate documents consists of 10k original documents and 1,990k synthetic ones, totaling 2 million documents.
Revised: The new pool of candidate documents consists of 10k original documents and 1,990k synthetic ones, totaling 2 million documents.

Original: To match each of the k decoder outputs ( x 1 , x 2 , . .
Revised: To match each of the k decoder outputs ( x 1 , x 2 , . .

Original: , x k ) with real documents, we uniformly randomly downsample the negative document examples keeping in total only 1000 logits (the dot product outputs in the decoder).
Revised: , x k ) with real documents, we uniformly randomly downsample the negative document examples keeping in total only 1000 logits (the dot product outputs in the decoder).

Original: At inference time, we pick the argmax for each of k dot product outputs with the full embedding matrix without sampling.
Revised: At inference time, we pick the argmax for each of k dot product outputs with the full embedding matrix without sampling.

Original: This technique speeds up the total training and inference time for 2 million documents to merely 4 minutes on1 GPU
Revised: This technique speeds up the total training and inference time for 2 million documents to merely 4 minutes on

Original: [Figure]

Original:  for both the response model with 40k training steps and List-CVAE with 5k training steps.
Revised: GPU for both the response model with 40k training steps and List-CVAE with 5k training steps.

Original: To order to stress test our model
Revised: We

Original: [Figure]
Revised: [Figure]

Original: we used embedding dimension 16 and masked out 50% of embedding features to mimic the real world scenario where we observe only a subset of all features.
Revised: [Figure] [Figure] [Figure] [Figure]

Original: We ran 2 experiments with 1 million and 2 millions documents respectively.
Revised: ran 2 experiments with 1 million and 2 millions document respectively.

Original: From the results shown inFigure 5c and 5d,
Revised: From the results shown in

Original:  List-CVAE steadily outperforms all othergreedy baselines again.
Revised: Figure 5c and 5d, List-CVAE steadily outperforms all other baselines again.

Original: The greatly increased number of training examples helped List-CVAE really learn all the interactions between documents and their respective positional biases.
Revised: The greatly increased number of training examples helped List-CVAE really learn all the interactions between documents and their respective positional biases.

Original: The resulting slates were able to receive close to 5 buyings on average due to the limited complexity provided by the response model.
Revised: The resulting slates were able to receive close to 5 purchases on average due to the limited complexity provided by the response model.

Original: Generalization test: In practice, we may not have any close-to-optimal slates in the training data.
Revised: Generalization test: In practice, we may not have any close-to-optimal slates in the training data.

Original: Hence it is crucial that List-CVAE is able to generalize to unseen optimal conditions.
Revised: Hence it is crucial that List-CVAE is able to generalize to unseen optimal conditions.

Original: To test its generalization capacity, we use the medium-scale experiment setup on RecSys 2015 dataset and eliminate from the training data all slates where the total user response exceed some ratio h of the slate size k = 5 , i.e. (cid:80) ki
Revised: To test its generalization capacity, we use the medium-scale experiment setup on RecSys 2015 dataset and eliminate from the training data all slates where the total user response exceed some ratio h of the slate size k = 5 , i [Figure]

Original:  =1 r i ≥ hk for h = 40% , 60% , 80% , 100% .
Revised: e. (cid:80) ki =1 r i ≥ hk for h = 40% , 60% , 80% , 100% .

Original: Figure 6 shows test results on increasingly difﬁcult training sets from which to infer on the optimal slates.
Revised: Figure 6 shows test results on increasingly difﬁcult training sets from which to infer on the optimal slates.

Original: Without seeing any optimal slates (Figure 6a) or slates with 4 or 5 clicks (Figure 6b), List-CVAE can still produce close to optimal slates.
Revised: Without seeing any optimal slates (Figure 6a) or slates with 4 or 5 clicks (Figure 6b), List-CVAE can still produce close to optimal slates.

Original: Even training on slates with only 1 or 2 total clicks ( h = 60% ), List-CVAE still surpasses the performance of all greedy baselines within 1000 steps (Figure 6c).
Revised: Even training on slates with only 1 or 2 total clicks ( h = 60% ), List-CVAE still surpasses the performance of all greedy baselines within 1000 steps (Figure 6c).

Original: Thus demonstrating the strong generalization power of the model.
Revised: Thus demonstrating the strong generalization power of the model.

Revised: List-CVAE cannot learn much about the interactions between documents given only 0 or 1 positive response per slate (Figure 6d), whereas the MLPtype models learn click probabilities of single documents in the same way as in slates with higher responses.

Original: D ISCUSSION
Revised: [Figure] Although evaluation of our model requires choosing the ideal conditioning c (cid:63) at or near the edge of the support of P ( c ) [Figure] we can always compromise generalization vs. performance by controlling c (cid:63) in practice.

Revised: Moreover, interactions between documents are from similar mechanisms whether they are from the optimal or sub-optimal slates and the model can learn these mechanisms from suboptimal slates and generalize to optimal slates as the experiment results indicate.

Revised:  D ISCUSSION

Original: The List-CVAE model moves away from the conventional greedy ranking paradigm and provides the ﬁrst conditional generative modeling framework that approaches slate recommendation problem using direct slate generation.
Revised: The List-CVAE model moves away from the conventional greedy ranking paradigm and provides the ﬁrst conditional generative modeling framework that approaches slate recommendation problem using direct slate generation.

Original: By modeling the conditional probability distribution of documents in a slate directly, this approach not only picks up automatically the positional and contextual biases between documents but also gracefully avoids the problem of combinatorial explosion of possible slates when the candidate set is large.
Revised: By modeling the conditional probability distribution of documents in a slate directly, this approach not only picks up automatically the positional and contextual biases between documents but also gracefully avoids the problem of combinatorial explosion of possible slates when the candidate set is large.

Original: The framework is ﬂexible and can incorporate different types of conditional generative models.
Revised: The framework is ﬂexible and can incorporate different types of conditional generative models.

Original: In this paper we showed its superior performance over greedy baseline models with a conditional VAE model.
Revised: In this paper we showed its superior performance over greedy baseline models with a conditional VAE model.

Original: In addition, the List-CVAE model has good scalability.
Revised: In addition, the List-CVAE model has good scalability.

Original: We designed an architecture that uses pretrained document embeddings combined with a negatively downsampled k -head softmax layer that greatly speeds up the training and scales easily to millions of documents. 
Revised: We designed an architecture that uses pretrained document embeddings combined with a negatively downsampled k -head softmax layer that greatly speeds up the training and scales easily to millions of documents. A P ERSONALIZATION TEST

Revised: This test complements the small-scale experiment.

Revised: To the 100 documents with slate size 10, we add user features into the conditioning c , by adding a set U of 50 different users to the simulation engine ( |U| = 50 , n = 100 , k = 10 ) and permuting the innate attractiveness of documents and their interactions matrix W by a user-speciﬁc function π u .

Revised: Let

Revised: [Equation]

Revised: [Equation]

Revised: be the response of the user u on the document d i .

Revised: During training, the condition c is a concatenation of 16 dimensional user embeddings Θ( u ) obtained from the response model, and responses r .

Revised: At inference time, the model conditions on c (cid:63) =

Revised: ( r (cid:63) , Θ( u )) for each randomly generated test user u .

Revised: We sweep over hidden layers of 512 or 1024 units in List-CVAE, and all baseline MLP structures.

Revised: Figure 7a show that slates generated by List-CVAE have on average higher clicks than those produced by the greedy baseline models although the convergence took longer to reach.

Revised: [Figure]