  Word embeddings (e.g., word2vec) have been applied successfully to eCommerce
products through prod2vec. Inspired by the recent performance improvements on
several NLP tasks brought by contextualized embeddings, we propose to transfer
BERT-like architectures to eCommerce: our model -- ProdBERT -- is trained to
generate representations of products through masked session modeling. Through
extensive experiments over multiple shops, different tasks, and a range of
design choices, we systematically compare the accuracy of ProdBERT and prod2vec
embeddings: while ProdBERT is found to be superior to traditional methods in
several scenarios, we highlight the importance of resources and hyperparameters
in the best performing models. Finally, we conclude by providing guidelines for
training embeddings under a variety of computational and data constraints.
