Original: Z ERO -S HOT R ECOMMENDER S YSTEMS
Revised: Z ERO -S HOT R ECOMMENDER S YSTEMS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Performance of recommender systems (RecSys) relies heavily on the amount of training data available.
Revised: Performance of recommender systems (RecSys) relies heavily on the amount of training data available.

Original: This poses a chicken-and-egg problem for early-stage products, whose amount of data, in turn, relies on the performance of their RecSys.
Revised: This poses a chicken-and-egg problem for early-stage products, whose amount of data, in turn, relies on the performance of their RecSys.

Original: In this paper, we explore the possibility of zero-shot learning in RecSys, to enable generalization from an old dataset to an entirely new dataset.
Revised: In this paper, we explore the possibility of zero-shot learning in RecSys, to enable generalization from an old dataset to an entirely new dataset.

Original: We develop an algorithm, dubbed ZEro-Shot Recommenders (ZESR EC ), that is trained on an old dataset and generalize to a new one where there are neither overlapping users nor overlapping items , a setting that contrasts typical cross-domain RecSys that has either overlapping users or items.
Revised: We develop an algorithm, dubbed ZEro-Shot Recommenders (ZESR EC ), that is trained on an old dataset and generalize to a new one where there are neither overlapping users nor overlapping items , a setting that contrasts typical cross-domain RecSys that has either overlapping users or items.

Original: Different from previous methods that use categorical item indices (i.e., item ID), ZESR EC uses items’ generic features, such as natural-language descriptions, product images, and videos, as their continuous indices, and therefore naturally generalizes to any unseen items.
Revised: Different from previous methods that use categorical item indices (i.e., item ID), ZESR EC uses items’ generic features, such as natural-language descriptions, product images, and videos, as their continuous indices, and therefore naturally generalizes to any unseen items.

Original: In terms of users,
Revised: In terms of users,

Original: ZESR EC builds upon recent advances on sequential RecSys to represent users using their interactions with items, thereby generalizing to unseen users as well.
Revised: ZESR EC builds upon recent advances on sequential RecSys to represent users using their interactions with items, thereby generalizing to unseen users as well.

Original: We study three pairs of real-world RecSys datasets and demonstrate that ZESR EC can successfully enable recommendations in such a zero-shot setting, opening up new opportunities for resolving the chicken-and-egg problem for data-scarce startups or early-stage products.
Revised: We study three pairs of real-world RecSys datasets and demonstrate that ZESR EC can successfully enable recommendations in such a zero-shot setting, opening up new opportunities for resolving the chicken-and-egg problem for data-scarce startups or early-stage products.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Many large scale e-commerce platforms (such as Etsy, Overstock, etc.) and online content platforms (such as Spotify, Overstock, Disney+, Netﬂix, etc) have such a large inventory of items that showcasing all of them in front of their users is simply not practical.
Revised: Many large scale e-commerce platforms (such as Etsy, Overstock, etc.) and online content platforms (such as Spotify, Overstock, Disney+, Netﬂix, etc) have such a large inventory of items that showcasing all of them in front of their users is simply not practical.

Original: In particular, in the online content category of businesses, it is often seen that users of their service do not have a crisp intent in mind unlike in the retail shopping experience where the users often have a clear intent of purchasing something.
Revised: In particular, in the online content category of businesses, it is often seen that users of their service do not have a crisp intent in mind unlike in the retail shopping experience where the users often have a clear intent of purchasing something.

Original: The need for personalized recommendations therefore arises from the fact that not only it is impractical to show all the items in the catalogue but often times users of such services need help discovering the next best thing — be it the new and exciting movie or be it a new music album or even a piece of merchandise that they may want to consider for future buying if not immediately.
Revised: The need for personalized recommendations therefore arises from the fact that not only it is impractical to show all the items in the catalogue but often times users of such services need help discovering the next best thing — be it the new and exciting movie or be it a new music album or even a piece of merchandise that they may want to consider for future buying if not immediately.

Original: Modern personalized recommendation models of users and items have often relied on the idea of extrapolating preferences from similar users.
Revised: Modern personalized recommendation models of users and items have often relied on the idea of extrapolating preferences from similar users.

Original: Different machine learning models deﬁne the notion of similarity differently.
Revised: Different machine learning models deﬁne the notion of similarity differently.

Original: Classical bi-linear Matrix Factorization (MF) approaches model users and items via their identiﬁers and represent them as vectors in the latent space [11; 26].
Revised: Classical bi-linear Matrix Factorization (MF) approaches model users and items via their identiﬁers and represent them as vectors in the latent space [13; 28].

Original: Modern deep-learningbased recommender systems [31; 10; 23], which are also used for predicting top- k items given an item, learn the user-to-item propensities from large amounts of training data containing many (user, item) tuples, optionally with available item content information (e.g., product descriptions) and user metadata.
Revised: Modern deep-learningbased recommender systems [34; 12; 25], which are also used for predicting top- k items given an item, learn the user-to-item propensities from large amounts of training data containing many (user, item) tuples, optionally with available item content information (e.g., product descriptions) and user metadata.

Original: As machine learning models, the performance of RecSys relies heavily on the amount of training data available.
Revised: As machine learning models, the performance of RecSys relies heavily on the amount of training data available.

Original: This might be feasible for large e-commerce or content delivery websites such as Overstock and Netﬂix, but poses a serious chicken-and-egg problem for small startups, whose amount of data, in turn, relies on the performance of their RecSys.
Revised: This might be feasible for large e-commerce or content delivery websites such as Overstock and Netﬂix, but poses a serious chicken-and-egg problem for small startups, whose amount of data, in turn, relies on the performance of their RecSys.

Original: On the other hand, zero-shot learning promises some degree of generalization from an old dataset to an entirely new dataset.
Revised: On the other hand, zero-shot learning promises some degree of generalization from an old dataset to an entirely new dataset.

Original: In this paper, we explore the possibility of zero-shot learning in RecSys.
Revised: In this paper, we explore the possibility of zero-shot learning in RecSys.

Original: We develop an algorithm, dubbed ZEro-Shot
Revised: We develop an algorithm, dubbed ZEro-Shot

Original: Recommenders (ZESR EC ), that is trained on an old dataset and generalize to a new one where there are neither overlapping users nor overlapping items , a setting that contrasts typical cross-domain
Revised: Recommenders (ZESR EC ), that is trained on an old dataset and generalize to a new one where there are neither overlapping users nor overlapping items , a setting that contrasts typical cross-domain

Original: RecSys that has either overlapping users or items [36; 34; 2; 14].
Revised: RecSys that has either overlapping users or items [40; 38; 3; 16].

Original: Naturally, generalization of RecSys to unseen users and unseen items becomes the two major challenges for developing zero-shot RecSys.
Revised: Naturally, generalization of RecSys to unseen users and unseen items becomes the two major challenges for developing zero-shot RecSys.

Original: For the ﬁrst challenge on unseen users, we build on a rich body of literature on sequential recommendation models [10; 25; 23; 12; 15; 19].
Revised: For the ﬁrst challenge on unseen users, we build on a rich body of literature on sequential recommendation models [12; 27; 25; 14; 17; 21].

Original: These models are built with sequential structure to encode temporal ordering of items in user’s item interaction history.
Revised: These models are built with sequential structure to encode temporal ordering of items in user’s item interaction history.

Original: Representing users by the items they have consumed in the past allows the model to extrapolate the preference learning to even novel users who the model did not see during training, as long as the items these unseen users have interacted with have been seen during training.
Revised: Representing users by the items they have consumed in the past allows the model to extrapolate the preference learning to even novel users who the model did not see during training, as long as the items these unseen users have interacted with have been seen during training.

Original: However, such deep learning models encode item via its categorical item index, i.e., the item ID, and therefore fall short in predicting a likely relevant but brand-new item not previously seen during training.
Revised: However, such deep learning models encode item via its categorical item index, i.e., the item ID, and therefore fall short in predicting a likely relevant but brand-new item not previously seen during training.

Original: This brings us to the second challenge of developing zero-shot recommender systems, i.e., dealing with unseen items.
Revised: This brings us to the second challenge of developing zero-shot recommender systems, i.e., dealing with unseen items.

Original: To address this challenge, ZESR EC goes beyond traditional categorical item indices and uses items’ generic features such as natural-language descriptions, product images, and videos as their continuous indices, thereby naturally generalizing to any unseen items.
Revised: To address this challenge, ZESR EC goes beyond traditional categorical item indices and uses items’ generic features such as natural-language descriptions, product images, and videos as their continuous indices, thereby naturally generalizing to any unseen items.

Original: Take natural-language (NL) descriptions as an example.
Revised: Take natural-language (NL) descriptions as an example.

Original: One can think of NL descriptions as a system of universal identiﬁers that indexes items from arbitrary domains.
Revised: One can think of NL descriptions as a system of universal identiﬁers that indexes items from arbitrary domains.

Original: Therefore as long as one model is trained on a dataset with NL descriptions, it can generalize to a completely different dataset with a similar NL vocabulary.
Revised: Therefore as long as one model is trained on a dataset with NL descriptions, it can generalize to a completely different dataset with a similar NL vocabulary.

Original: In ZESR EC we build on state-of-the-art pretrained NL embedding models such as BERT [6] to extract NL embeddings from raw NL descriptions, leading to an item ID system in the continuous space that is generalizable across arbitrary domains.
Revised: In ZESR EC we build on state-of-the-art pretrained NL embedding models such as BERT [8] to extract NL embeddings from raw NL descriptions, leading to an item ID system in the continuous space that is generalizable across arbitrary domains.

Original: For instance, in e-commerce platform, one could use items’ description text; and similarly in the online content platforms, one could use movie synopsis or music track descriptions to represent an item.
Revised: For instance, in e-commerce platform, one could use items’ description text; and similarly in the online content platforms, one could use movie synopsis or music track descriptions to represent an item.

Original: Combining the merits of sequential RecSys and the idea of universal continuous ID space,
Revised: Combining the merits of sequential RecSys and the idea of universal continuous ID space,

Original: our
Revised: our

Original: ZESR EC successfully enables recommendation in an extreme cold-start setting, i.e., the zero-shot setting where all users and items in the target domain are unseen during training.
Revised: ZESR EC successfully enables recommendation in an extreme cold-start setting, i.e., the zero-shot setting where all users and items in the target domain are unseen during training.

Original: Essentially ZESR EC tries to learn transferable user behavioral patterns in a universal continuous embedding space.
Revised: Essentially ZESR EC tries to learn transferable user behavioral patterns in a universal continuous embedding space.

Original: For example, in the source domain, ZESR EC can learn that if users purchase snacks or drinks (e.g., ‘Vita
Revised: For example, in the source domain, ZESR EC can learn that if users purchase snacks or drinks (e.g., ‘Vita

Original: Coconut Water’ with a lemonade ﬂavor) that they like, they may purchase similar snacks or drinks with different ﬂavors (e.g., ‘Vita Coconut Water’ with a pineapple ﬂavor), as shown in the case study.
Revised: Coconut Water’ with a lemonade ﬂavor) that they like, they may purchase similar snacks or drinks with different ﬂavors (e.g., ‘Vita Coconut Water’ with a pineapple ﬂavor), as shown in the case study.

Original: Later in the target domain, if one user purchase ‘V8 Splash’ with a tropical ﬂavor, ZESR EC can recommend ‘V8 Splash’ with a berry ﬂavor to the user (see Fig.
Revised: Later in the target domain, if one user purchase ‘V8 Splash’ with a tropical ﬂavor, ZESR EC can recommend ‘V8 Splash’ with a berry ﬂavor to the user (see Fig.

Original: 4 of Sec.
Revised: 4 of Sec.

Original: 4.6 for details).
Revised: 4.6 for details).

Original: Such generalization is possible due to the use of the NL descriptions as universal identiﬁers, based on which ZESR EC could easily identify similar products of the same brand with different ﬂavors.
Revised: Such generalization is possible due to the use of the NL descriptions as universal identiﬁers, based on which ZESR EC could easily identify similar products of the same brand with different ﬂavors.

Original: To summarize our contributions:
Revised: To summarize our contributions:

Original: • We identify the problem of zero-shot recommender systems and propose ZESR EC as the ﬁrst hierarchical Bayesian model for addressing this problem.
Revised: • We identify the problem of zero-shot recommender systems and propose ZESR EC as the ﬁrst hierarchical Bayesian model for addressing this problem.

Original: • We introduce the notion of universal continuous identiﬁers that makes recommendation in a zero-shot setting possible.
Revised: • We introduce the notion of universal continuous identiﬁers that makes recommendation in a zero-shot setting possible.

Original: • We provide empirical results which show that ZESR EC can successfully recommend items in the zero-shot setting.
Revised: • We provide empirical results which show that ZESR EC can successfully recommend items in the zero-shot setting.

Original: • We conduct case studies demonstrating that ZESR EC can learn interpretable user behavioral patterns that can generalize across datasets.
Revised: • We conduct case studies demonstrating that ZESR EC can learn interpretable user behavioral patterns that can generalize across datasets.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Deep Learning for RecSys.
Revised: Deep Learning for RecSys.

Original: Deep learning has been prevalent in modern recommender systems [27; 30; 31; 17; 3; 8; 29] due to its scalability and superior performance.
Revised: Deep learning has been prevalent in modern recommender systems [29; 33; 34; 19; 4; 32] due to its scalability and superior performance.

Original: As a pioneer work, [27] uses restricted Boltzmann machine (RBM) to perform collaborative ﬁltering in recommender systems, however the system is a single-layer RBM.
Revised: As a pioneer work, [29] uses restricted Boltzmann machine (RBM) to perform collaborative ﬁltering in recommender systems, however the system is a single-layer RBM.

Original: Later, [31] and [17] build upon Bayesian deep learning to develop hierarchical Bayesian models that tightly integrate content information and user-item rating information, thereby signiﬁcantly improving recommendation performance.
Revised: Later, [34] and [19] build upon Bayesian deep learning to develop hierarchical Bayesian models that tightly integrate content information and user-item rating information, thereby signiﬁcantly improving recommendation performance.

Original: After that, there are also various proposed sequential (or session-based) recommender systems [10; 25; 1; 16; 20; 35; 23], GRU4Rec [10] was ﬁrst proposed to use gated recurrent units (GRU) [4] for recommender systems.
Revised: After that, there are also various proposed sequential (or session-based) recommender systems [12; 27; 2; 18; 22; 39; 14; 31; 41; 25], GRU4Rec [12] was ﬁrst proposed to use gated recurrent units (GRU) [6] for recommender systems.

Original: Since then, follow-up works such as hierarchical GRU [25], temporal convolutional networks (TCN) [1], and hierarchical RNN (HRNN) [23] have achieved improvement in terms of accuracy by utilizing cross-session information [25], causal convolutions [1], as well as meta data and control signals [23].
Revised: Since then, follow-up works such as hierarchical GRU [27], temporal convolutional networks (TCN) [2], and hierarchical RNN (HRNN) [25] have achieved improvement in terms of accuracy by utilizing cross-session information [27], causal convolutions [2], as well as meta data and control signals [25].

Revised: Another line of work focusing on building self-attention based sequential models such as SASRec [14], BERT4Rec [31], and S3Rec [41].

Original: In this paper we build on such sequential RecSys and note that our ZESR EC is model agnostic, that is, it is compatible with any sequential RecSys.
Revised: In this paper we build on such sequential RecSys and note that our ZESR EC is model agnostic, that is, it is compatible with any sequential RecSys.

Original: Cross-Domain and Cold-Start RecSys.
Revised: Cross-Domain and Cold-Start RecSys.

Original: There is a rich literature on cross-domain RecSys focusing on training a recommender system in the source domain and deploying it in the target domain where there exist either common users or items [36; 34; 2; 14].
Revised: There is a rich literature on cross-domain RecSys focusing on training a recommender system in the source domain and deploying it in the target domain where there exist either common users or items [40; 38; 3; 16].

Original: These works are also related to the problem of recommendation for cold-start users and items, i.e., users and items with few interactions (or ratings) available during training [9; 18; 37; 21].
Revised: These works are also related to the problem of recommendation for cold-start users and items, i.e., users and items with few interactions (or ratings) available during training [11; 20; 42; 23].

Original: There are also works [22; 7] handling cold start on both user and item with meta-learning, however they cannot generalize across domains.
Revised: There are also works [24; 9] handling cold start on both user and item with meta-learning, however they cannot generalize across domains.

Original: In summary, prior systems are either (1) not sufﬁcient to address our zero-shot setting where there are neither common users nor common items in the target domain or (2) unable to learn user behavior patterns that are transferable across datasets/domains.
Revised: In summary, prior systems are either (1) not sufﬁcient to address our zero-shot setting where there are neither common users nor common items in the target domain or (2) unable to learn user behavior patterns that are transferable across datasets/domains.

Original: Therefore, they are not applicable to our problem of zero-shot recommendations.
Revised: Therefore, they are not applicable to our problem of zero-shot recommendations.

Original:  Z ERO -S HOT R ECOMMENDER S YSTEMS
Revised:  Z ERO -S HOT R ECOMMENDER S YSTEMS

Original: In this section we introduce our ZESR EC which is compatible with any sequential model.
Revised: In this section we introduce our ZESR EC which is compatible with any sequential model.

Original: Without loss of generality, here we focus on NL descriptions as a possible instantiation of universal identiﬁers, but note that our method is general enough to use as identiﬁers other content information such as items’ images and videos.
Revised: Without loss of generality, here we focus on NL descriptions as a possible instantiation of universal identiﬁers, but note that our method is general enough to use as identiﬁers other content information such as items’ images and videos.

Original: We leave exploration for other potential modalities to future work.
Revised: We leave exploration for other potential modalities to future work.

Original: Notation.
Revised: Notation.

Original: We focus on the setting of zero-shot recommendation where there are neither overlapping users nor overlapping items between a source domain and a target domain.
Revised: We focus on the setting of zero-shot recommendation where there are neither overlapping users nor overlapping items between a source domain and a target domain.

Original: We assume a set V s of J s items and a set U s of I s users in the source domain, as well as a set V t of J t items and a set
Revised: We assume a set V s of J s items and a set U s of I s users in the source domain, as well as a set V t of J t items and a set

Original: U t of I t users in the target domain.
Revised: U t of I t users in the target domain.

Original: We let I =
Revised: We let I =

Original: I s
Revised: I s

Original: + I t and J =
Revised: + I t and J =

Original: J s + J t ; we use j ∈ V s ∪ V t to index items and i ∈
Revised: J s + J t ; we use j ∈ V s ∪ V t to index items and i ∈

Original: U s ∪ U t to index users.
Revised: U s ∪ U t to index users.

Original: The zero-shot setting dictates that V s ∩ V t = ∅ and that U s ∩ U t = ∅ .
Revised: The zero-shot setting dictates that V s ∩ V t = ∅ and that U s ∩ U t = ∅ .

Original: We denote the collection of all users’ interactions as a 3 D tensor (with necessary zero-padding)
Revised: We denote the collection of all users’ interactions as a 3 D tensor (with necessary zero-padding)

Original: R ∈
Revised: R ∈

Original: R
Revised: R

Original: I × N max × J , where N max is the maximum number of interactions among all users.
Revised: I × N max × J , where N max is the maximum number of interactions among all users.

Original: We use the subscript ‘ ∗ ’ to represent the collection of all elements in a certain dimension.
Revised: We use the subscript ‘ ∗ ’ to represent the collection of all elements in a certain dimension.

Original: Speciﬁcally, each user i has a sequence of N i interactions (e.g., purchase history) with various items denoted as R i ∗∗ =
Revised: Speciﬁcally, each user i has a sequence of N i interactions (e.g., purchase history) with various items denoted as R i ∗∗ =

Original: [ R it ∗ ]
Revised: [ R it ∗ ]

Original: N i t =1 , where R it ∗ ∈ { 0 , 1 } J is one-hot vector denoting the t -th item user i interacted with.
Revised: N i t =1 , where R it ∗ ∈ { 0 , 1 } J is one-hot vector denoting the t -th item user i interacted with.

Original: The same user i has different user embeddings at different time t , reﬂecting dynamics in user interests; here we denote as u it ∈
Revised: The same user i has different user embeddings at different time t , reﬂecting dynamics in user interests; here we denote as u it ∈

Original: R D the latent user vector when user i interacts with the t -th item in her history, and we use U =
Revised: R D the latent user vector when user i interacts with the t -th item in her history, and we use U =

Original: [ u it ] I,N max i =1 ,t =1 ∈ R
Revised: [ u it ] I,N max i =1 ,t =1 ∈ R

Original: I × N max × D (with necessary zero-padding) to denote the collection of user latent vectors.
Revised: I × N max × D (with necessary zero-padding) to denote the collection of user latent vectors.

Original: We denote as v j ∈ R D the item latent vector and
Revised: We denote as v j ∈ R D the item latent vector and

Original: V =
Revised: V =

Original: [ v j ] Jj =1 R J × D as the collection.
Revised: [ v j ] Jj =1 R J × D as the collection.

Original: For simplicity and without loss of generality, in this paper we focus on using NL descriptions (i.e., a sequence of words) to describe items.
Revised: For simplicity and without loss of generality, in this paper we focus on using NL descriptions (i.e., a sequence of words) to describe items.

Original: For item j we denote its
Revised: For item j we denote its

Original: NL description as x j , and the number of words as M j ; similar to V , we let X =
Revised: NL description as x j , and the number of words as M j ; similar to V , we let X =

Original: [ x j ] Jj =1 .
Revised: [ x j ] Jj =1 .

Original: With slight notation overload on t , we denote as R ( s ) and R ( t ) the sub-tensor of R that corresponds to the source and target domains, respectively.
Revised: With slight notation overload on t , we denote as R ( s ) and R ( t ) the sub-tensor of R that corresponds to the source and target domains, respectively.

Original: Similarly, we also have U ( s ) , U ( t ) , V ( s ) , V ( t ) , X ( s ) , and X ( t ) .
Revised: Similarly, we also have U ( s ) , U ( t ) , V ( s ) , V ( t ) , X ( s ) , and X ( t ) .

Original: Problem Setup.
Revised: Problem Setup.

Original: A model is trained using all users’ interaction sequences from the source domain, i.e., { R i ∗∗ } i ∈U s , and then deployed to recommend items for any user ι ∈ U t in the target domain, given user ι ’s previous history R ι ∗∗ , which can be empty.
Revised: A model is trained using all users’ interaction sequences from the source domain, i.e., { R i ∗∗ } i ∈U s , and then deployed to recommend items for any user ι ∈ U t in the target domain, given user ι ’s previous history R ι ∗∗ , which can be empty.

Original: In practice we append a dummy item at the beginning of each user session, so that during inference we could conduct recommend even for users without any history by ingesting the dummy item as context to infer the user latent vector.
Revised: In practice we append a dummy item at the beginning of each user session, so that during inference we could conduct recommend even for users without any history by ingesting the dummy item as context to infer the user latent vector.

Original: In our zero-shot setting, the model is not allowed to ﬁne-tune or retrain on any data from the target domain.
Revised: In our zero-shot setting, the model is not allowed to ﬁne-tune or retrain on any data from the target domain.

Revised: Deﬁnition of Zero-shot Learning in RecSys.

Revised: Our zero-shot setting includes three unique properties: (1) cold users, (2) cold items, and (3) domain gap.

Revised: It is fundamentally different from previous content-based cold start as the latter setting usually satisﬁes either (1) or (2), but not often both.

Original: 3. F ROM C ATEGORICAL D OMAIN -S PECIFIC I TEM ID TO C ONTINUOUS U NIVERSAL I TEM ID
Revised: 3. F ROM C ATEGORICAL D OMAIN -S PECIFIC I TEM ID TO C ONTINUOUS U NIVERSAL I TEM ID

Original: Most models in recommender systems learn item embeddings through interactions.
Revised: Most models in recommender systems learn item embeddings through interactions.

Original: These embeddings are indexed by categorical domain-speciﬁc item ID , which is transductive and cannot be generalized to unseen items.
Revised: These embeddings are indexed by categorical domain-speciﬁc item ID , which is transductive and cannot be generalized to unseen items.

Original: In this paper, we propose to use item generic content information such as NL descriptions and image to produce item embeddings, which can be used as continuous universal item ID .
Revised: In this paper, we propose to use item generic content information such as NL descriptions and image to produce item embeddings, which can be used as continuous universal item ID .

Original: Since such content information is domain agnostic, the model trained on top of it can be transferable from one domain to another, therefore making zero-shot recommender systems feasible.
Revised: Since such content information is domain agnostic, the model trained on top of it can be transferable from one domain to another, therefore making zero-shot recommender systems feasible.

Original: Based on the universal item embeddings, one can then build sequential models to obtain user embeddings by aggregating embeddings of items in the user history.
Revised: Based on the universal item embeddings, one can then build sequential models to obtain user embeddings by aggregating embeddings of items in the user history.

Original: Here we introduce the notion of universal embedding networks (UEN) that use continuous universal embeddings to index items (and therefore users) rather than categorical ID that is not transferable across domains.
Revised: Here we introduce the notion of universal embedding networks (UEN) that use continuous universal embeddings to index items (and therefore users) rather than categorical ID that is not transferable across domains.

Original: We call the UEN generating item and user universal embeddings item UEN and user
Revised: We call the UEN generating item and user universal embeddings item UEN and user

Original: UEN, respectively.
Revised: UEN, respectively.

Original: 3. M ODEL O VERVIEW
Revised: 3. M ODEL O VERVIEW

Original: We propose a hierarchical Bayesian model with a probabilistic encoder-decoder architecture.
Revised: We propose a hierarchical Bayesian model with a probabilistic encoder-decoder architecture.

Original: The encoder ingests items from user history to yield the user embedding, while decoder computes recommendation scores based on similarity between user embeddings and item embeddings.
Revised: The encoder ingests items from user history to yield the user embedding, while decoder computes recommendation scores based on similarity between user embeddings and item embeddings.

Original: Generative Process.
Revised: Generative Process.

Original: The generative process of ZESR EC (in the source domain) is as follows:
Revised: The generative process of ZESR EC (in the source domain) is as follows:

Original: For each item j : • Compute the item universal embedding: m j = f e ( x j ) .
Revised: For each item j : • Compute the item universal embedding: m j = f e ( x j ) .

Original: • Draw a latent item offset vector (cid:15)
Revised: • Draw a latent item offset vector (cid:15)

Original: j ∼ N (cid:0) 0 , λ − 1 v I D (cid:1) .
Revised: j ∼ N (cid:0) 0 , λ − 1 v I D (cid:1) .

Original: • Obtain the item latent vector: v j = (cid:15) j
Revised: • Obtain the item latent vector: v j = (cid:15) j

Original: + m j . 2.
Revised: + m j . 2.

Original: For each user i : • For each time step t : – Compute the user universal embedding: n it = f seq ([ v i τ ] t − 1 τ =1 ) .
Revised: For each user i : • For each time step t : – Compute the user universal embedding: n it = f seq ([ v i τ ] t − 1 τ =1 ) .

Original: – Draw a latent user offset vector ξ it ∼ N (cid:0) 0 , λ − 1 u I D (cid:1) .
Revised: – Draw a latent user offset vector ξ it ∼ N (cid:0) 0 , λ − 1 u I D (cid:1) .

Original: – Obtain the latent user vector: u it = ξ it + n it .
Revised: – Obtain the latent user vector: u it = ξ it + n it .

Original: – Compute recommendation score S itj for each user-interaction-item tuple ( i, k, j ) , S itj = f softmax ( u (cid:62) it v j ) and draw the t -th item for user i : R it ∗ ∼ Cat ([ S itj ]
Revised: – Compute recommendation score S itj for each user-interaction-item tuple ( i, k, j ) , S itj = f softmax ( u (cid:62) it v j ) and draw the t -th item for user i : R it ∗ ∼ Cat ([ S itj ]

Original: Jj =1 ) .
Revised: Jj =1 ) .

Original: 𝜆 ! 𝒗 " 𝒙 " 𝑹 !"
Revised: 𝜆 ! 𝒗 " 𝒙 " 𝑹 !"

Original: # 𝒖 #$ 𝜆 % 𝒗 # $ j ∈
Revised: # 𝒖 #$ 𝜆 % 𝒗 # $ j ∈

Original: [1, J] i ∈ [
Revised: [1, J] i ∈ [1, I] t ∈ [1, 𝑁 ! ]

Original: [Figure]
Revised: User UEN 𝒙 # $ 𝜆 ! Item UEN 𝜏 ∈ [0, t-1]

Original: Here f softmax ( · ) is the softmax function: 
Revised: Here f softmax ( · ) is the softmax function: f softmax ( u (cid:62) it v j ) =

Original: [Equation]
Revised: exp( u (cid:62) it v j ) / (cid:80) j exp( u (cid:62) it v j ) .

Original: Cat ( · ) is a categorical distribution.
Revised: Cat ( · ) is a categorical distribution.

Original: f e ( · ) is item UEN (see Sec. 3.3), f seq ( · ) is user UEN (see Sec. 3.4).
Revised: f e ( · ) is item UEN (see Sec. 3.3), f seq ( · ) is user UEN (see Sec. 3.4).

Original: i τ in v
Revised: i τ in v

Original: i τ indexes the τ -th item that user i interacts with.
Revised: i τ indexes the τ -th item that user i interacts with.

Original: λ
Revised: λ

Original: u and λ v are hyperparameters.
Revised: u and λ v are hyperparameters.

Original: The latent item offset (cid:15)
Revised: The latent item offset (cid:15)

Original: j = v j − m j provides the ﬁnal latent item vector v j with the ﬂexibility to slightly deviate from the contentbased item universal embedding m j .
Revised: j = v j − m j provides the ﬁnal latent item vector v j with the ﬂexibility to slightly deviate from the contentbased item universal embedding m j .

Original: Similarly, the latent user offset ξ it =
Revised: Similarly, the latent user offset ξ it =

Original: u it − n it provides the ﬁnal latent user vector u it with the ﬂexibility to slightly deviate from the user universal embedding n it .
Revised: u it − n it provides the ﬁnal latent user vector u it with the ﬂexibility to slightly deviate from the user universal embedding n it .

Original: Intuitively, (cid:15) j and ξ it provide domain-speciﬁc information on top of the domain-agnostic information from m j and n it .
Revised: Intuitively, (cid:15) j and ξ it provide domain-speciﬁc information on top of the domain-agnostic information from m j and n it .

Original: In the target domain we will remove (cid:15) j from v j , and ξ it from u it , which can be seen as an attempt to remove the bias learned from the source domain.
Revised: In the target domain we will remove (cid:15) j from v j , and ξ it from u it , which can be seen as an attempt to remove the bias learned from the source domain.

Original: Training.
Revised: Training.

Original: The MAP estimation in the source domain can be decomposed as following:
Revised: The MAP estimation in the source domain can be decomposed as following:

Original: [Equation]
Revised: [Equation]

Original: X ( s ) , λ − 1 v ) . Maximizing the posterior probability is equivalent to minimizing the joint Negative Log-Likelihood (NLL) of U ( s ) and V ( s ) given R ( s ) , X ( s ) , λ − 1 u , and λ − 1 v :L =
Revised: Maximizing the posterior probability is equivalent to minimizing the joint Negative Log-Likelihood (NLL) of U ( s ) and V ( s ) given R ( s ) , X ( s ) , λ − 1 u , and λ − 1 v :

Original: [Equation]
Revised: [Equation]

Original: where n it = f seq ( { v i τ } t − 1 τ =1 ) and m j = f e ( x j ) .
Revised: where n it = f seq ( { v i τ } t − 1 τ =1 ) and m j = f e ( x j ) .

Original: See the Appendix for a full Bayesian treatment of
Revised: See the Appendix for a full Bayesian treatment of

Original: ZESR EC using inference networks and generative networks [13].
Revised: ZESR EC using inference networks and generative networks [15].

Original: Inference and Recommendation in the Target Domain.
Revised: Inference and Recommendation in the Target Domain.

Original: Once the model is trained using sourcedomain data, it can recommend unseen items j ∈ V t (where V t ∩ V s = ∅ ) for any unseen user i ∈
Revised: Once the model is trained using sourcedomain data, it can recommend unseen items j ∈ V t (where V t ∩ V s = ∅ ) for any unseen user i ∈

Original: U t (where U t ∩ U s = ∅ ) from the target domain based on the approximate MAP inference below:
Revised: U t (where U t ∩ U s = ∅ ) from the target domain based on the approximate MAP inference below:

Original: [Equation]
Revised: [Equation]

Original: where δ ( · ) denotes a Dirac delta distribution.
Revised: where δ ( · ) denotes a Dirac delta distribution.

Original: U ( t ) MAP and V ( t ) MAP are the MAP estimate of U ( t ) and
Revised: U ( t ) MAP and V ( t ) MAP are the MAP estimate of U ( t ) and

Original: V ( t ) given X ( t ) , which we approximate as:
Revised: V ( t ) given X ( t ) , which we approximate as:

Original: [Equation]
Revised: [Equation]

Original: The reason for the approximation is that ZESR EC has no access to interactions R ( t ) in the target domain, making the cross-entropy loss in the Eqn. 4 disappear.
Revised: The reason for the approximation is that ZESR EC has no access to interactions R ( t ) in the target domain, making the cross-entropy loss in the Eqn. 4 disappear.

Original: The user and item latent matrices
Revised: The user and item latent matrices

Original: U ( t ) MAP , V ( t ) MAP in the target domain enable us to perform zero-shot recommendation by computing recommendation scores based on inner products and recommend item argmax j f softmax ( u (cid:62) it v j ) .
Revised: U ( t ) MAP , V ( t ) MAP in the target domain enable us to perform zero-shot recommendation by computing recommendation scores based on inner products and recommend item argmax j f softmax ( u (cid:62) it v j ) .

Original: As long as the unseen items’ NL descriptions are available, ZESR EC could obtain both the unseen users’ latent vectors and the unseen items’ latent vectors based on the item universal embedding network.
Revised: As long as the unseen items’ NL descriptions are available, ZESR EC could obtain both the unseen users’ latent vectors and the unseen items’ latent vectors based on the item universal embedding network.

Original: This is in contrast to previous methods that rely on catogorical domain-speciﬁc item ID, which is not transferable as unseen items have completely different ID from items in the training set.
Revised: This is in contrast to previous methods that rely on catogorical domain-speciﬁc item ID, which is not transferable as unseen items have completely different ID from items in the training set.

Original: Note that ZESR EC is general enough to adapt to other data modalities such as images and videos.
Revised: Note that ZESR EC is general enough to adapt to other data modalities such as images and videos.

Original: 3. I TEM U NIVERSAL E MBEDDING N ETWORK : m j = f e ( x j )
Revised: 3. I TEM U NIVERSAL E MBEDDING N ETWORK : m j = f e ( x j )

Original: The purpose of the item universal embedding network, denoted as f e ( · ) , is to extract item embeddings that are universal across domains.
Revised: The purpose of the item universal embedding network, denoted as f e ( · ) , is to extract item embeddings that are universal across domains.

Original: The network consists of a pretrained BERT network [6], denoted as f BERT , followed by a single-layer neural network, denoted by f NN ( · ) .
Revised: The network consists of a pretrained BERT network [8], denoted as f BERT , followed by a single-layer neural network, denoted by f NN ( · ) .

Original: Formally we have
Revised: Formally we have

Original: [Equation]
Revised: [Equation]

Original: (3) where m j is the universal embedding for item j and x j is the NL description for item j .
Revised: where m j is the universal embedding for item j and x j is the NL description for item j .

Original: We use the embedding for the ‘CLS’ token from the last layer of BERT as the output of f BERT ( x i ) .
Revised: We use the embedding for the ‘CLS’ token from the last layer of BERT as the output of f BERT ( x i ) .

Original: This UEN is jointly trained with the sequential model using the objective function in Eqn. 4.
Revised: This UEN is jointly trained with the sequential model using the objective function in Eqn. 4.

Original: Note that f NN ( · ) is necessary as we need to adapt the pre-trained BERT for recommendation tasks.
Revised: Note that f NN ( · ) is necessary as we need to adapt the pre-trained BERT for recommendation tasks.

Original: 3. U SER U NIVERSAL E MBEDDING N ETWORK : n it = f seq([ v i τ ] t − 1 τ =1 )
Revised: 3. U SER U NIVERSAL E MBEDDING N ETWORK : n it = f seq

Revised: [Equation]

Original: The user UEN f seq ( · ) is built on top of the item UEN in Eqn. 3.
Revised: The user UEN f seq ( · ) is built on top of the item UEN in Eqn. 3.

Original: Speciﬁcally, given user i ’s interaction sequence until time t , R i ∗∗ =
Revised: Speciﬁcally, given user i ’s interaction sequence until time t , R i ∗∗ =

Original: [ R iτ ∗ ] t − 1 τ =1 , we ﬁrst replace it the corresponding NL descriptions l ( x ) i =
Revised: [ R iτ ∗ ] t − 1 τ =1 , we ﬁrst replace it the corresponding NL descriptions l ( x ) i =

Original: [ x i τ ] t − 1 τ =1 , and then fed it into item UEN in Eqn.
Revised: [ x i τ ] t − 1 τ =1 , and then fed it into item UEN in Eqn.

Original: 3 to obtain l ( m )i =
Revised: 3 to obtain l ( m )

Original: [ m iτ ] t − 1 τ =1 =
Revised: i

Original: [ f e ( x i τ )]
Revised: [Equation]

Original: t − 1 τ = where item i t is user i ’s t -th interaction.
Revised: where item i t is user i ’s t -th interaction.

Original: Each vector in the sequence, m j , is the universal embedding for item j .
Revised: Each vector in the sequence, m j , is the universal embedding for item j .

Original: During training, we obtain item latent vector v j based on m j (see Eqn. 4), while during inference v j = m j
Revised: During training, we obtain item latent vector v j based on m j (see Eqn. 4), while during inference v j = m j

Original: (see Eqn. 2).
Revised: (see Eqn. 2).

Original: We can then treat each v j as the input at each time step of the sequential model, which gives us the ﬁnal user universal embedding n it = f seq ([ v i τ ] t − 1 τ =1 ) .
Revised: We can then treat each v j as the input at each time step of the sequential model, which gives us the ﬁnal user universal embedding n it = f seq ([ v i τ ] t − 1 τ =1 ) .

Original: Note that this user UEN is used during both training (Eqn. 4) and inference (Eqn. 2).
Revised: Note that this user UEN is used during both training (Eqn. 4) and inference (Eqn. 2).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we evaluate ourproposed ZESR EC against a number of in-domain and zero-shot baselines on three source-target dataset pairs, with the major goals of addressing the following questions:
Revised: In this section, we evaluate our ZESR EC against various in-domain and zero-shot baselines on three source-target dataset pairs, with the major goals of addressing the following questions:

Original: Q1 How accurate (effective) is ZESR EC compared to the baselines?
Revised: Q1 How accurate (effective) is ZESR EC compared to the baselines?

Original: Q

Revised: Q

Original: If one allows training models using target-domain data, how long does it take for non zero-shot models to outperform zero-shot recommenders?
Revised: If one allows training models using target-domain data, how long does it take for non zero-shot models to outperform zero-shot recommenders?

Original: Q3 Does ZESR EC yield meaningful recommendations for users with similar behavioral patterns in the source domain and target domain?
Revised: Q3 Does ZESR EC yield meaningful recommendations for users with similar behavioral patterns in the source domain and target domain?

Original: [Table]
Revised: [Table]

Original: 4. D ATASETS
Revised: 4. D ATASETS

Original: We use three different real-world dataset pairs, one from Amazon [24] and two from MIND [33]:
Revised: We use three different real-world dataset pairs, one from Amazon [26] and two from MIND [37]:

Original: • Amazon [24]: A publicly available dataset collection which contains a group of datasets in different categories with abundant item metadata such as item description, product images, etc.
Revised: • Amazon [26]: A publicly available dataset collection which contains a group of datasets in different categories with abundant item metadata such as item description, product images, etc.

Original: In our experiments, we consider two datasets: (1) ‘Prime Pantry’, which contains 300K interactions, 10K items, and 76K users, and (2) ‘Grocery and Gourmet Food’, which contains 2.3M interactions, 213K items, and 739K users.
Revised: In our experiments, we consider two datasets: (1) ‘Prime Pantry’, which contains 300K interactions, 10K items, and 76K users, and (2) ‘Grocery and Gourmet Food’, which contains 2.3M interactions, 213K items, and 739K users.

Original: • MIND [33]: A large-scale news recommendation dataset collected from the user click logs of Microsoft News.
Revised: • MIND [37]: A large-scale news recommendation dataset collected from the user click logs of Microsoft News.

Original: It includes 4-week user history and 5th-week interactions.
Revised: It includes 4-week user history and 5th-week interactions.

Original: In our experiments, we simulate zero-shot learning by leave-one-out splitting on subcategories under the largest category in MIND: ‘sports’.
Revised: In our experiments, we simulate zero-shot learning by leave-one-out splitting on subcategories under the largest category in MIND: ‘sports’.

Original: We consider two pairs
Revised: We consider two pairs: (1) ‘Others’ to ’NFL’ , where ‘Others’ contains all

Revised: the other

Original: : (1) ‘Others’ to ’NFL’, where ‘Others’ contains all the other subcategories except ‘NFL’.
Revised: subcategories except ‘NFL’ ‘Others’

Original: ‘Others’ contains 169K interactions, 8K items, and 57K users,while ‘NFL’ contains 1M interactions, 11K items, and 203K users.
Revised: contains 169K interactions , 8K items, and 57K users,

Original: (2) ‘Others’ to ‘NCAA’, where ‘Others’ includes all the subcategories except ‘NCAA’.
Revised: while ‘NFL’ contains 1M interactions

Revised: 11K items,

Revised: and 203K

Revised: users

Revised: ‘Others’

Revised: to

Revised: ‘NCAA’, where ‘Others’

Revised: includes

Revised: all the subcategories

Revised: except

Revised: ‘NCAA’

Original: ‘Others’ contains 797K interacions, 17K items, and 206K users, while‘NCAA’ contains 238K interactions, 31K items, and 56K users.
Revised: ‘Others’ contains 797K interacions, 17K items and 206K users, while

Revised: ‘NCAA’

Revised: contains 238K interactions, 31K

Original:  We adopted a rigorous experimental setup for zero-shot learning to ensure (1) no overlapping users and items and (2) no temporal leakage .
Revised: items, and 56K users. We adopted a rigorous experimental setup for zero-shot learning to ensure (1) no overlapping users and items and (2) no temporal leakage .

Original: See the Appendix for details.
Revised: See the Appendix for details.

Original: Datasets in Amazon pair are divided into training (80%) and test (20%) sets.
Revised: Datasets in Amazon pair are divided into training (80%) and test (20%) sets.

Original: For datasets in the two MIND pairs, we follow the ofﬁcial splitting [33] and use the four-week user history as the training set and interactions in the ﬁfth week as the test set.
Revised: For datasets in the two MIND pairs, we follow the ofﬁcial splitting [37] and use the four-week user history as the training set and interactions in the ﬁfth week as the test set.

Original: Due to strong recency bias in the news recommendation [33], for MIND we run experiments under two settings to measure temporal invariance of the model: (1) evaluate on interactions in the ﬁrst day of the week ( 1st-day setting ), and (2) evaluate on interactions of the whole week , i.e., the full test set ( whole-week setting ).
Revised: Due to strong recency bias in the news recommendation [37], for MIND we run experiments under two settings to measure temporal invariance of the model: (1) evaluate on interactions in the ﬁrst day of the week ( 1st-day setting ), and (2) evaluate on interactions of the whole week , i.e., the full test set ( whole-week setting ).

Original: For all datasets, We further split 10% of the training set by user as validation set.
Revised: For all datasets, We further split 10% of the training set by user as validation set.

Original: 4. B ASELINES AND ZESR EC V ARIANTS
Revised: 4. B ASELINES AND ZESR EC V ARIANTS

Original: To demonstrate the effectiveness of our model, we compare ZESR EC against two groups of baselines: in-domain methods and zero-shot methods.
Revised: To demonstrate the effectiveness of our model, we compare ZESR EC against two groups of baselines: in-domain methods and zero-shot methods.

Original: In-Domain Methods.
Revised: In-Domain Methods.

Original: We compare variants of our model ZESR EC against a variety of state-of-theart session-based recommendation models including GRU4Rec [10], TCN [1], and HRNN [23].
Revised: We compare variants of our model ZESR EC against a variety of state-of-theart session-based recommendation models including GRU4Rec [12], TCN [2], and HRNN [25].

Original: We also consider their extensions, HRNN-Meta , GRU4Rec-Meta , and TCN-Meta , which use items’ NL description embeddings to replace item ID hidden embeddings.
Revised: We also consider their extensions, HRNN-Meta , GRU4Rec-Meta , and TCN-Meta , which use items’ NL description embeddings to replace item ID hidden embeddings.

Original: Besides aforementioned sophisticated models, we introduce POP which is a simple baseline conducting recommendation only based on item global popularity.
Revised: Besides aforementioned sophisticated models, we introduce POP which is a simple baseline conducting recommendation only based on item global popularity.

Original: It can be a strong baseline in certain domains.
Revised: It can be a strong baseline in certain domains.

Original: All the abovemethods are trained directly on target-domain data and therefore are considered ‘oracle’ methods.
Revised: All the abovemethods are trained directly on target-domain data and therefore are considered ‘oracle’ methods.

Original: [Figure]
Revised: [Figure]

Original: advanced Zero-Shot Methods. Since no previous work has been done on this thread, we consider two intuitive zero-shot models (1) EmbeddingKNN : a K-nearest-neighbors algorithm based on inner product between the user embedding and item embedding.
Revised: user embedding and item embedding.

Original: The item embedding is the BERT embedding from
Revised: The item embedding is the BERT embedding from

Original: NL description, while the user embedding is an average over embeddings of the user’s interacted items, and (2) Random : recommending items by random selection from the whole item catalogue without replacement.
Revised: NL description, while the user embedding is an average over embeddings of the user’s interacted items, and (2) Random : recommending items by random selection from the whole item catalogue without replacement.

Original: ZESR EC Variants.
Revised: ZESR EC Variants.

Original: We evaluate three variants of our ZESR EC , including ZESR EC -G , ZESR ECT , and ZESR EC -H which use GRU4Rec, TCN and HRNN as base models, respectively.
Revised: We evaluate three variants of our ZESR EC , including ZESR EC -G , ZESR ECT , and ZESR EC -H which use GRU4Rec, TCN and HRNN as base models, respectively.

Original: 4. I MPLEMENTATION D ETAILS
Revised: 4. I MPLEMENTATION D ETAILS

Original: We adopted pre-trained google/bert_uncased_L-12_H-768_A-12 BERT model from Huggingface [32] to process item description and generate item embedding.
Revised: We adopted pre-trained google/bert_uncased_L-12_H-768_A-12 BERT model from Huggingface [36] to process item description and generate item embedding.

Original: The dimension of BERT embedding is 768.
Revised: The dimension of BERT embedding is 768.

Original: For the model architecture, we use BERT embedding as input to a single-layer neural netwrok (i.e., f NN ( · ) ).
Revised: For the model architecture, we use BERT embedding as input to a single-layer neural netwrok (i.e., f NN ( · ) ).

Original: The output dimension for the NN is set to D (see the Appendix for more details).
Revised: The output dimension for the NN is set to D (see the Appendix for more details).

Original: 4. E XPERIMENT S ETUP AND E VALUATION M ETRICS
Revised: 4. E XPERIMENT S ETUP AND E VALUATION M ETRICS

Original: We conducted three sets of experiments to answer questions proposed at the begining part of the experiment section, speciﬁcally:
Revised: We conducted three sets of experiments to answer questions proposed earlier in this section.

Original: Zero-Shot Experiments.
Revised: Zero-Shot Experiments.

Original: We trained in-domain baselines on target domain training set, while our
Revised: We trained in-domain baselines on target domain training set, while our

Original: ZESR EC is trained on source domain.
Revised: ZESR EC is trained on source domain.

Original: All models are tested on the testing set of the target domain for an apples to apples comparison.
Revised: All models are tested on the testing set of the target domain for an apples to apples comparison.

Original: Incremental Training Experiments.
Revised: Incremental Training Experiments.

Original: To measure how long it takes for non-zero-shot models to outperform zero-shot recommenders, we conducted incremental training experiments on in-domain base models GRU4Rec, TCN, HRNN as well as GRU4Rec-Meta, TCN-Meta, HRNN-Meta.
Revised: To measure how long it takes for non-zero-shot models to outperform zero-shot recommenders, we conducted incremental training experiments on in-domain base models GRU4Rec, TCN, HRNN as well as GRU4Rec-Meta, TCN-Meta, HRNN-Meta.

Original: Note that the variants of our ZESR EC are NOT retrained on target domain.
Revised: Note that the variants of our ZESR EC are NOT retrained on target domain.

Original: It is also inevitable that non-zero-shot models eventually outperform ZESR EC because ZESR EC does not have access to target-domain data.
Revised: It is also inevitable that non-zero-shot models eventually outperform ZESR EC because ZESR EC does not have access to target-domain data.

Original: Due to space constraint, we only reported results for incremental training experiments on Amazon pair and MIND NCAA pair under the 1st-day setting.
Revised: Due to space constraint, we only reported results for incremental training experiments on Amazon pair and MIND NCAA pair under the 1st-day setting.

Original: Importantly, under the whole-week setting for the MIND dataset, the ZESRec variants already signiﬁcantly outperform in-domain competitors which are trained on the full target training set; therefore it is meaningless to compare incremental training results under this setting, as in-domain competitors will never beat
Revised: Importantly, under the whole-week setting for the MIND dataset, the ZESRec variants already signiﬁcantly outperform in-domain competitors which are trained on the full target training set; therefore it is meaningless to compare incremental training results under this setting, as in-domain competitors will never beat

Original: ZESRec variants.
Revised: ZESRec variants.

Original: For all the source-target dataset pairs, we group the interactions by user and sort interactions within each user based on interaction timestamps.
Revised: For all the source-target dataset pairs, we group the interactions by user and sort interactions within each user based on interaction timestamps.

Original: We randomly select users until we get enough interactions and build three datasets which contain 2.5K, 5K, and 10K interactions, respectively.
Revised: We randomly select users until we get enough interactions and build three datasets containing 2.5K, 5K, and 10K interactions, respectively.

Original: Case Studies.
Revised: Case Studies.

Original: To gain more insight what ZESR EC learns, we perform several case studies.
Revised: To gain more insight what ZESR EC learns, we perform several case studies.

Original: Speciﬁcally, we randomly select users from the test set of the target domain Amazon Prime Pantry (where we evaluate ZESR EC ) and only keep users for whom ZESR EC correctly predicts the 6-th items in the sequence given the ﬁrst 5 items as context, as we want to focus on sequences where ZESR EC works.
Revised: Speciﬁcally, we randomly select users from the test set of the target domain Amazon Prime Pantry (where we evaluate ZESR EC ) and only keep users for whom ZESR EC correctly predicts the 6-th items in the sequence given the ﬁrst 5 items as context, as we want to focus on sequences where ZESR EC works.

Original: We use these users as queries to ﬁnd users with similar behavioral patterns from the source domain Amazon Grocery and Gourmet Food (where we train ZESR EC ) based on user embeddings from ZESR EC .
Revised: We use these users as queries to ﬁnd users with similar behavioral patterns from the source domain Amazon Grocery and Gourmet Food (where we train ZESR EC ) based on user embeddings from ZESR EC .

Original: User embeddings are generated based on the ﬁrst 5 items of the sequence.
Revised: User embeddings are generated based on the ﬁrst 5 items of the sequence.

Revised: Simulated Online Scenarios.

Revised: In the experiments, ZESRec only accesses target domain data during inference to simulate online scenarios, where new businesses just open and the customers are using

Original: [Figure]
Revised: [Figure]

Original: We select two users with similar universal embeddings according to Sec. 4.4.
Revised: We select two users with similar universal embeddings according to Sec. 4.4.

Original: This  demonstrates ZESR EC can learn the user behavioral pattern that ‘users who bought sugary snacks and tea tend to buy caffeine-free herbal tea later’. 
Revised: This case study demonstrates ZESR EC can learn the user behavioral pattern that ‘users who bought sugary snacks and tea tend to buy caffeine-free herbal tea later’. service in real-time.

Revised: This real-time online access setting is substantially different from batch access ahead of time as it prevents us from training a recommender in the target domain before serving.

Original: Evaluation Protocol.
Revised: Evaluation Protocol.

Original: For evaluation, we adopted Recall (R@20) and the ranking metric Normalized
Revised: For evaluation, we adopted Recall (R@20) and the ranking metric Normalized

Original: Discounted Cumulative Gain (NDCG) [28] (N@20).
Revised: Discounted Cumulative Gain (NDCG) [30] (N@20).

Original: We removed all the repetitive interactions (e.g., user A clicked item B two times in a row) to only focus on evaluating the model’s capability of capturing the transition between user history to the next item.
Revised: We removed all the repetitive interactions (e.g., user A clicked item B two times in a row) to only focus on evaluating the model’s capability of capturing the transition between user history to the next item.

Original: 4. E XPERIMENTAL R ESULTS
Revised: 4. E XPERIMENTAL R ESULTS

Original: Zero-Shot Experiments.
Revised: Zero-Shot Experiments.

Original: The experimental results on three dataset pairs are in Table 1.
Revised: The experimental results on three dataset pairs are in Table 1.

Original: Overall, our ZESR EC outperforms zero-shot baselines Embedding-KNN and Random by a large margin in most cases; it can also achieve performance comparable to in-domain baselines.
Revised: Overall, our ZESR EC outperforms zero-shot baselines Embedding-KNN and Random by a large margin in most cases; it can also achieve performance comparable to in-domain baselines.

Original: On the Amazon pair, ZESR EC beats POP and achieves comparable performance with HRNNMeta and HRNN-Interactions, suggesting the existence of shared recommendation patterns in two domains.
Revised: On the Amazon pair, ZESR EC beats POP and achieves comparable performance with HRNN-Meta and HRNNInteractions, suggesting the existence of shared recommendation patterns in two domains.

Original: For the two MIND pairs, we made the following observations: (1) Under the whole-week setting, ZESR EC consistently outperforms HRNN-Meta and HRNN-Interactions by a large margin.
Revised: For the two MIND pairs, we made the following observations: (1) Under the whole-week setting, ZESR EC consistently outperforms HRNN-Meta and HRNN-Interactions by a large margin.

Original: Comparing the ﬁrst-day results and the whole-week results, it is obvious that the in-domain models overﬁt the history and fail to generalize well on latest user-item interactions.
Revised: Comparing the ﬁrst-day results and the whole-week results, it is obvious that the in-domain models overﬁt the history and fail to generalize well on latest user-item interactions.

Original: This reﬂects strong recency bias in news recommendation.
Revised: This reﬂects strong recency bias in news recommendation.

Original: In contrast, our zero-shot learning naturally comes with an inductive bias to only model the transition from user history to the next item.
Revised: In contrast, our zero-shot learning naturally comes with an inductive bias to only model the transition from user history to the next item.

Original: (2) Under the ﬁrst-day setting, ZESR EC still achieves reasonable performance comparing with in-domain models on both pairs.
Revised: (2) Under the ﬁrst-day setting, ZESR EC still achieves reasonable performance comparing with in-domain models on both pairs.

Original: (3) Surprisingly in the MIND NFL pair, the ZESRec variants perform well under the whole-week setting even when the source domain contains much fewer interactions than the target domain (169K VS 1M).
Revised: (3) Surprisingly in the MIND NFL pair, the ZESRec variants perform well under the whole-week setting even when the source domain contains much fewer interactions than the target domain (169K VS 1M).

Original: Incremental Training Experiments.
Revised: Incremental Training Experiments.

Original: Incremental training results are in Fig.
Revised: Incremental training results are in Fig.

Original: Overall, almost all the in-domain baselines are not able to outperform ZESR EC by retraining on at most 10K interactions in target domain, which shows the vast importance of conducting zero-shot learning in recommender domain.
Revised: Overall, almost all the in-domain baselines are not able to outperform ZESR EC by retraining on at most 10K interactions in target domain, which shows the vast importance of conducting zero-shot learning in recommender domain.

Original: For new business operating an early-stage RecSys, it’s hard to train a good RecSys with limited interactions.
Revised: For new business operating an early-stage RecSys, it’s hard to train a good RecSys with limited interactions.

Original: This is a chicken-and-egg problem, as training good RecSys requires sufﬁcient interactions, while in turn, collecting sufﬁcient interactions requires a satisfactory RecSys to attract users.
Revised: This is a chicken-and-egg problem, as training good RecSys requires sufﬁcient interactions, while in turn, collecting sufﬁcient interactions requires a satisfactory RecSys to attract users.

Original: Therefore the ﬁrst 10K interactions are crucial to get the RecSys started.
Revised: Therefore the ﬁrst 10K interactions are crucial to get the RecSys started.

Original: 4. C ASE S TUDIES
Revised: 4. C ASE S TUDIES

Original: To gain more insight, we randomly select pairs of user purchase histories from the source and target domains in the Amazon pair as case studies according to the procedure in Sec. 4.4.
Revised: To gain more insight, we randomly select pairs of user purchase histories from the source and target domains in the Amazon pair as case studies according to the procedure in Sec. 4.4.

Original: The goal is
Revised: The goal is

Original: [Figure]
Revised: [Figure]

Original: We select two users with similar universal embeddings according to Sec. 4.4.
Revised: We select two users with similar universal embeddings according to Sec. 4.4.

Original: This case study demonstrates ZESR EC can learn the user behavioral pattern that ‘if users bought snacks or drinks that they like, they may later purchase similar snacks or drinks with different ﬂavors’.
Revised: This case study demonstrates ZESR EC can learn the user behavioral pattern that ‘if users bought snacks or drinks that they like, they may later purchase similar snacks or drinks with different ﬂavors’.

Original: to demonstrate our ZSR could learn relevant dynamics of users’ purchase history from the source domain and successfully recommend unseen products to an unseen user in the target domain.
Revised: to demonstrate our ZSR could learn relevant dynamics of users’ purchase history from the source domain and successfully recommend unseen products to an unseen user in the target domain.

Original: Fig.
Revised: Fig.

Original: 3 shows the purchase history of a user in the source domain ( top ), ‘Amazon Grocery and Gourmet
Revised: 3 shows the purchase history of a user in the source domain ( top ), ‘Amazon Grocery and Gourmet

Original: Food’, and the purchase history of an unseen user in the target domain ( bottom ), ‘Amazon Prime
Revised: Food’, and the purchase history of an unseen user in the target domain ( bottom ), ‘Amazon Prime

Original: Pantry’, where all items are unseen during training.
Revised: Pantry’, where all items are unseen during training.

Original: The user in the source domain bought ‘Tension
Revised: The user in the source domain bought ‘Tension

Original: Tamer Tea’, which is a type of herbal tea, after buying some sugary snacks (KitKat) and other tea.
Revised: Tamer Tea’, which is a type of herbal tea, after buying some sugary snacks (KitKat) and other tea.

Original: Such a pattern is captured by ZESR EC , which then recommended ‘Lemon Zinger Herbal Tea’ to an unseen user after she bought some sugary snacks (‘Energy Bars from Clif Bar’) and some black tea.
Revised: Such a pattern is captured by ZESR EC , which then recommended ‘Lemon Zinger Herbal Tea’ to an unseen user after she bought some sugary snacks (‘Energy Bars from Clif Bar’) and some black tea.

Original: This case study demonstrates ZESR EC can learn the user behavioral pattern that ‘users who bought sugary snacks and tea tend to buy caffeine-free herbal tea later’.
Revised: This case study demonstrates ZESR EC can learn the user behavioral pattern that ‘users who bought sugary snacks and tea tend to buy caffeine-free herbal tea later’.

Original: More interestingly, another case study in Fig.
Revised: More interestingly, another case study in Fig.

Original: 4 demonstrates that ZESR EC can learn the user behavioral pattern that ‘if users bought snacks or drinks that they like, they may later purchase similar snacks or drinks with different ﬂavors’.
Revised: 4 demonstrates that ZESR EC can learn the user behavioral pattern that ‘if users bought snacks or drinks that they like, they may later purchase similar snacks or drinks with different ﬂavors’.

Original: Speciﬁcally, in the source domain, the user purchased ‘Vita Coconut Water’ with four different ﬂavors; such a pattern is captured by ZESR EC .
Revised: Speciﬁcally, in the source domain, the user purchased ‘Vita Coconut Water’ with four different ﬂavors; such a pattern is captured by ZESR EC .

Original: Later in the target domain, an unseen user purchase ‘V8 Splash’ with a tropical ﬂavor, ZESR EC then successfully recommends ‘V8 Splash’ with a berry ﬂavor to the user.
Revised: Later in the target domain, an unseen user purchase ‘V8 Splash’ with a tropical ﬂavor, ZESR EC then successfully recommends ‘V8 Splash’ with a berry ﬂavor to the user.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we identify the problem of zero-shot recommender systems where a model is trained in the source domain and deployed in a target domain where all the users and items are unseen during training.
Revised: In this paper, we identify the problem of zero-shot recommender systems where a model is trained in the source domain and deployed in a target domain where all the users and items are unseen during training.

Original: We propose ZESR EC as the ﬁrst general framework for addressing this problem.
Revised: We propose ZESR EC as the ﬁrst general framework for addressing this problem.

Original: We introduce the notion of universal continuous identiﬁers leveraging the fact that item ID can be grounded in natural-language descriptions.
Revised: We introduce the notion of universal continuous identiﬁers leveraging the fact that item ID can be grounded in natural-language descriptions.

Original: We provide empirical results, both quantitatively and qualitatively, to demonstrate the effectiveness of our proposed ZESR EC and verify that ZESR EC can successfully learn user behavioral patterns that generalize across datasets (domains).
Revised: We provide empirical results, both quantitatively and qualitatively, to demonstrate the effectiveness of our proposed ZESR EC and verify that ZESR EC can successfully learn user behavioral patterns that generalize across datasets (domains).

Revised: The main limitation of this work is that commonality between domains is crucial for ZESRec, as is the case for most other works in transfer learning.

Original: Future work includes  exploring other modalities, e.g., images and videos, as alternative universal identiﬁers.
Revised: Future work includes quantiﬁcation of such commonality and exploring other modalities, e.g., images and videos, as alternative universal identiﬁers.

Original: It would also be interesting to investigate the interpretability provided by the pretrained BERT model and to incorporate additional auxiliary information to further improve the zero-shot performance.
Revised: It would also be interesting to investigate the interpretability provided by the pretrained BERT model and to incorporate additional auxiliary information to further improve the zero-shot performance.

Revised: A A PPENDIX A. D ATA U SAGE AND P RIVACY D ISCUSSIONS

Revised: Data Usage: For real-world scenarios, new business owners who hope to do zero-shot recommendation need to check the dataset policy before usage.

Revised: On the other hand, if multinational corporations hope to establish a branch in a new region they could use their own data from other existing regions.

Revised: Privacy: To protect the privacy of user data, we encourage people who want to adopt our methods to train the model using source-domain data with differential privacy embedded.

Revised: Some related references include [1; 5; 35].

Original: A A PPENDIX A. B AYESIAN T REATMENT WITH I NFERENCE N ETWORKS
Revised: A. B AYESIAN T REATMENT WITH I NFERENCE N ETWORKS

Original: Training.
Revised: Training.

Original: Using Jensen’s inequality we have the following evidence lower bound (ELBO):
Revised: Using Jensen’s inequality we have the following evidence lower bound (ELBO):

Original: [Equation]
Revised: [Equation]

Original: , where the expectation is over q ( U , V | X ) .
Revised: where the expectation is over q ( U , V | X ) .

Original: We have
Revised: We have

Original: [Equation] =
Revised: L =

Original: [Equation]
Revised: [Equation]

Original: For the variational distribution q ( U , V | X ) we have the following factorization:q
Revised: For the variational distribution q ( U , V | X ) we have the following factorization:

Original: [Equation]
Revised: [Equation]

Original:  U  , V| X ) =
Revised: , where U = [ u i ] Ii =1 , V =

Original: q
Revised: [ v j ] Jj =1 , and X =

Original: [Equation] V | X ) q ( U | V
Revised: [ x j ] Jj = q ( v j | X ) =

Original: [Equation] X ) = q ( V | X )  q ( U | V) , where U =
Revised: q ( v j | x j ) = N (cid:16) v j ; f e,µ ( x j ) , f e,σ 2 ( x j ) (cid:17) , q ( u i | V , X ) =

Original: [ u i ] Ii =1 , V = [ v j ] Jj =1  , and X = [ x j ] Jj = [Equation]
Revised: N (cid:16) u i ; f seq ,µ ( { v j k } n i k =1 ) , f seq ,σ 2 ( { v j k } n i k =1 ) (cid:17)

Original: Overall,
Revised: Overall,

Original: [Equation]
Revised: [Equation]

Original: where the expectation is over q ( U , V | X ) .
Revised: where the expectation is over q ( U , V | X ) .

Original: Below we discuss several important terms including − E q [log q ( U , V | X )] and E q ( U , V | X ) [log p ( U | V )] in the ELBO.
Revised: Below we discuss several important terms including − E q [log q ( U , V | X )] and E q ( U , V | X ) [log p ( U | V )] in the ELBO.

Original: The term − E q [log q ( U , V | X )] is the entropy of q ( U , V | X ) . 
Revised: The term − E q [log q ( U , V | X )] is the entropy of q ( U , V | X ) . −

Original: [Equation]
Revised: [Equation]

Original: U

Original: [Equation]
Revised: [Table]

Original: where V is sampled for N v times to get a Monte Carlo estimate of E q ( V | X ) [log f seq ,σ 2 ( v i τ } t − 1 τ =1 )] .
Revised: where V is sampled for N v times to get a Monte Carlo estimate of E q ( V | X ) [log f seq ,σ 2 ( v i τ } t − 1 τ =1 )] .

Original: In practice, it is found that one sample is usually sufﬁcient due to the use of SGD-based optimization process.
Revised: In practice, it is found that one sample is usually sufﬁcient due to the use of SGD-based optimization process.

Original: Similarly the term
Revised: Similarly the term

Original: [Equation]
Revised: [Equation]

Original: where V is sampled for N v times from q ( V | X ) to get a Monte Carlo estimate of
Revised: where V is sampled for N v times from q ( V | X ) to get a Monte Carlo estimate of

Original: E q ( V | X ) [ (cid:107) f seq ,σ 2 ( v i k } n i k =1 ) (cid:107) 22 ] .
Revised: E q ( V | X ) [ (cid:107) f seq ,σ 2 ( v i k } n i k =1 ) (cid:107) 22 ] .

Original: In practice, it is found that one sample is usually sufﬁcient due to the use of SGD-based optimization process.
Revised: In practice, it is found that one sample is usually sufﬁcient due to the use of SGD-based optimization process.

Original: Inference.
Revised: Inference.

Original: Inference can be done via Monte Carlo estimates of p ( R |
Revised: Inference can be done via Monte Carlo estimates of p ( R |

Original: X , λ u , λ v ) .
Revised: X , λ u , λ v ) .

Original: Speciﬁcally,
Revised: Speciﬁcally,

Original: [Equation]
Revised: [Equation]

Original: V ( n )

Original: [Equation]

Original: , where V ( n ) ∼ q ( V | X ) and U ( n ) ∼ q ( U | V ( n ) , X ) .
Revised: where V ( n ) ∼ q ( V | X ) and U ( n ) ∼ q ( U | V ( n ) , X ) .

Original: One could also use MAP inference to trade accuracy for speed.
Revised: One could also use MAP inference to trade accuracy for speed.

Original: [Equation]
Revised: [Equation]

Original: where δ ( · ) denotes a Dirac delta distribution.
Revised: where δ ( · ) denotes a Dirac delta distribution.

Original: U MAP and V MAP are the MAP estimate of U and V given X :
Revised: U MAP and V MAP are the MAP estimate of U and V given X :

Original: [Equation]
Revised: [Equation]

Original: A. I MPLEMENTATION D ETAILS
Revised: A. I MPLEMENTATION D ETAILS

Original: We use pre-trained google/bert_uncased_L-12_H-768_A-12 BERT model from Huggingface [32] to process item description and generate item embedding.
Revised: We use pre-trained google/bert_uncased_L-12_H-768_A-12 BERT model from Huggingface [36] to process item description and generate item embedding.

Original: The dimension of BERT embedding is 768.
Revised: The dimension of BERT embedding is 768.

Original: We use BERT embedding as input to a single-layer neural netwrok (NN) and the output dimension for the NN is set to D , which equals to the hidden dimension of the sequential model.
Revised: We use BERT embedding as input to a single-layer neural netwrok (NN) and the output dimension for the NN is set to D , which equals to the hidden dimension of the sequential model.

Original: For ZESR EC variants we use the default optimal setting: we set the hidden dimension D as 300, the dropout rate as 0.2, and the number of training epochs as 20.
Revised: For ZESR EC variants we use the default optimal setting: we set the hidden dimension D as 300, the dropout rate as 0.2, and the number of training epochs as 20.

Original: We use Adagrad [5] as the optimizer with a learning rate of 0.1, and train ZESR EC variants in the source domain with early stopping based on validation loss.
Revised: We use Adagrad [7] as the optimizer with a learning rate of 0.1, and train ZESR EC variants in the source domain with early stopping based on validation loss.

Original: We set the hyperparameter λ v as a relatively large value 100 to restrain the variance of the item offset vector (cid:15) j .
Revised: We set the hyperparameter λ v as a relatively large value 100 to restrain the variance of the item offset vector (cid:15) j .

Original: For base models (HRNN, TCN, GRU4Rec) and corresponding base-meta models (HRNN-Meta,
Revised: For base models (HRNN, TCN, GRU4Rec) and corresponding base-meta models (HRNN-Meta,

Original: TCN-Meta, GRU4Rec-Meta), we set the dropout rate as 0.2 and the number of training epochs as 20; we choose Adagrad [5] as the optimizer.
Revised: TCN-Meta, GRU4Rec-Meta), we set the dropout rate as 0.2 and the number of training epochs as 20; we choose Adagrad [7] as the optimizer.

Original: We train base models and base-meta models in the target domain with early stopping and perform hyperparameter tuning, both are based on the validation loss.
Revised: We train base models and base-meta models in the target domain with early stopping and perform hyperparameter tuning, both are based on the validation loss.

Original: We tried the hidden dimension D in { 128 , 300 } and the learning rate η in { 0 . 01 , 0 . 1 , 1 } , and choose to use the conﬁgurations { D : 128 , η : 1 } for base models and { D : 128 , η : 0 . 1 } for base-meta models.
Revised: We tried the hidden dimension D in { 128 , 300 } and the learning rate η in { 0 . 01 , 0 . 1 , 1 } , and choose to use the conﬁgurations { D : 128 , η : 1 } for base models and { D : 128 , η : 0 . 1 } for base-meta models.

Original: [Figure]
Revised: [Figure]

Original: For all datasets, we treat the rating as implicit feedback (interactions between user and item).
Revised: For all datasets, we treat the rating as implicit feedback (interactions between user and item).

Original: Since we are considering session-based recommendation and using sequential model, we ﬁlter out users with only 1 interaction as the sequential model need to ingest at least one item from user history as context to perform next-step prediction.
Revised: Since we are considering session-based recommendation and using sequential model, we ﬁlter out users with only 1 interaction as the sequential model need to ingest at least one item from user history as context to perform next-step prediction.

Original: No Temporal Leakage.
Revised: No Temporal Leakage.

Original: We adopted a rigorous experimental setup for zero-shot learning:
Revised: We adopted a rigorous experimental setup for zero-shot learning:

Original: Firstly, we makes sure that there are no overlapping users and items between the source domain and the target domain.
Revised: Firstly, we makes sure that there are no overlapping users and items between the source domain and the target domain.

Original: Secondly, we temporally split the two domains, meaning that all the training interactions in the source domain must be happened before all the testing interactions in the target domain.
Revised: Secondly, we temporally split the two domains, meaning that all the training interactions in the source domain must be happened before all the testing interactions in the target domain.

Original: All experiments were ran on a GPU machine with Nvidia Tesla V100 16G memory GPU.
Revised: All experiments were ran on a GPU machine with Nvidia Tesla V100 16G memory GPU.

Original: [Figure] A 
Revised: A. M ODEL A RCHITECTURE

Original: [Figure]
Revised: [Figure]

Original: M ODEL A RCHITECTURE

Original: The graphical model for ZESR EC is shown in Fig.
Revised: The graphical model for ZESR EC is shown in Fig.

Original: In the MAP estimation version of ZESR EC , we set λ u → ∞ to remove latent user offset vector, thereby preventing ZESR EC from over-parameterization.
Revised: In the MAP estimation version of ZESR EC , we set λ u → ∞ to remove latent user offset vector, thereby preventing ZESR EC from over-parameterization.

Original: Fig.
Revised: Fig.

Original: 6 shows a simpliﬁed deterministic model architecture from the neural network point of view.
Revised: 6 shows a simpliﬁed deterministic model architecture from the neural network point of view.

Original: Below we elaborate on the process in terms of two stages: training in source domain and inference in target domain.
Revised: Below we elaborate on the process in terms of two stages: training in source domain and inference in target domain.

Original: Training in Source Domain.
Revised: Training in Source Domain.

Original: During training, on the encoder side, we generate BERT embeddings from items’ NL descriptions as item universal embeddings and then add the learnable item offset vectors to them, which yield the ﬁnal item embeddings (item latent vectors).
Revised: During training, on the encoder side, we generate BERT embeddings from items’ NL descriptions as item universal embeddings and then add the learnable item offset vectors to them, which yield the ﬁnal item embeddings (item latent vectors).

Original: The sequential model will aggregate item embeddings of items in user history to generate user embeddings.
Revised: The sequential model will aggregate item embeddings of items in user history to generate user embeddings.

Original: On the decoder side, we compute item latent vectors the same way we do on the encoder side.
Revised: On the decoder side, we compute item latent vectors the same way we do on the encoder side.

Original: Here we share item latent vectors on both encoder and decoder sides to reduce number of parameters.
Revised: Here we share item latent vectors on both encoder and decoder sides to reduce number of parameters.

Original: Empirically we ﬁnd this prevent overﬁtting and improve performance.
Revised: Empirically we ﬁnd this prevent overﬁtting and improve performance.

Original: Inference in Target Domain.
Revised: Inference in Target Domain.

Original: In this phase, we will remove the item offset vectors on both the encoder and decoder sides.
Revised: In this phase, we will remove the item offset vectors on both the encoder and decoder sides.

Original: We use item universal embeddings directly instead as the ﬁnal item latent vectors since we have no access to interactions in the target domain, and therefore the model is unable to estimate the item offset vectors.
Revised: We use item universal embeddings directly instead as the ﬁnal item latent vectors since we have no access to interactions in the target domain, and therefore the model is unable to estimate the item offset vectors.