Q uery2 P rod2 V ec: Grounded Word Embeddings for e C ommerce (2021)

Abstract: We present Query2Prod2Vec, a model that grounds lexical representations for product search in product embeddings: in our model, meaning is a mapping between words and a latent space of products in a digital shop. We leverage shopping sessions to learn the underlying space and use merchandising annotations to build lexical analogies for evaluation: our experiments show that our model is more accurate than known techniques from the NLP and IR literature. Finally, we stress the importance of data efficiency for product search outside of retail giants, and highlight how Query2Prod2Vec fits with practical constraints faced by most practitioners.

## introduction the ecommerce market reached in recent years an unprecedented scale: in 2020, 3.9 trillion dollars were spent globally in online retail @xcite . while shoppers make significant use of search functionalities, improving their experience is a never-ending quest @xcite , as outside of few retail giants users complain about sub-optimal performances (baymard institute, 2020). as the technology behind the industry increases in sophistication, neural architectures are gradually becoming more common @xcite and, with them, the need for accurate word embeddings for information retrieval (ir) and downstream natural language processing (nlp) tasks @xcite . unfortunately, the success of standard and contextual embeddings from the nlp literature @xcite could not be immediately translated to the product search scenario, due to some peculiar challenges @xcite , such as short text, industry-specific jargon @xcite , lowresource languages; moreover, specific embedding strategies have often been developed in the context of high-traffic websites @xcite , which limit their applicability in many practical scenarios. in this work, we propose a sample efficient word embedding method for ir in ecommerce, and benchmark it against sota models over industry data provided by partnering shops. we summarize our contributions as follows: while perhaps not fundamental to its industry significance, it is important to remark that grounded lexical learning is well aligned with theoretical considerations on meaning in recent (and less recent) literature @xcite @xcite . ## embeddings for product search: an industry perspective in product search, when the shopper issues a query (e.g. "sneakers") on a shop, the shop search engine returns a list of k products matching the query intent and possibly some contextual factor -the shopper at that point may either leave the website, or click on n products to further explore the offering and eventually make a purchase. unlike web search, which is exclusively performed at massive scale, product search is a problem that both big and small retailers have to solve: while word embeddings have revolutionized many areas of nlp @xcite , word embeddings for product queries are especially challenging to obtain at scale, when considering the huge variety of use cases in the overall ecommerce industry. in particular, based on industry data and first-hand experience with dozens of shops in our network, we identify four constraints for effective word embeddings in ecommerce: ## related work the literature on learning representations for lexical items in nlp is vast and growing fast; as an overview of classical methods, @xcite benchmarks several count-based and neural techniques @xcite ; recently, context-aware embeddings @xcite have demonstrated state-of-the-art performances in several semantic tasks @xcite , including document-based search @xcite , in which target entities are long documents, instead of product @xcite . to address ir-specific challenges, other embedding strategies have been proposed: search2vec @xcite uses interactions with ads and pages as context in the typical context-target setting of skip-gram models @xcite ; queryngram2vec @xcite additionally learns embeddings for ngrams of word appearing in queries to better cover the long tail. the idea of using vectors (from images) as an aid to query representation has also been suggested as a heuristic device by @xcite , in the context of personalized language models; this work is the first to our knowledge to benchmark embeddings on lexical semantics (not tuned for domain-specific tasks), and investigate sample efficiency for small-data contexts. ## query2prod2vec in query2prod2vec, the representation for a query q is built through the representation of the objects that q refers to. consider a typical shopperengine interaction in the context of product search: the shopper issues a query, e.g. "shoes", the engine replies with a noisy set of potential referents, e.g. pairs of shoes from the shop inventory, among which the shopper may select relevant items. hence, this dynamics is reminiscent of a cooperative language game @xcite , in which shoppers give noisy feedback to the search engine on the meaning of the queries. a full specification of query2prod2vec therefore involves a representation of the target domain of reference (i.e. products in a digital shop) and a denotation function. ## solving analogies in ecommerce as discussed in section 2, we consider evaluation tasks focused on word meaning, without using product-based similarity (as that would implicitly and unfairly favor referential embeddings). analogy-based tasks @xcite are a popular choice to measure semantic accuracy of embeddings, where a model is asked to fill templates like man : @xmath0: ?; however, preparing analogies for digital shops presents non trivial challenges for human annotators: these would in fact need to know both the language and the underlying space ("air max" is closer to "nike" than to "adidas"), with the additional complication that many candidates may not have "determinate" answers (e.g. if adidas is to gazelle, then nike is to what exactly?). in building our testing framework, we keep the intuition that analogies are an effective way to test for lexical meaning and the assumption that human-level concepts should be our ground truth: in particular, we programmatically produce analogies by leveraging existing human labelling, as indirectly provided by the merchandisers who built product catalogs 10 . ## conclusion and future work in this work, we learned reference-based word embeddings for product search: query2prod2vec significantly outperforms other embedding strategies on lexical tasks, and consistently provides good performance in small-data and zero-data scenarios, with the help of synthetic embeddings. in future work, we will extend our analysis to i) specific ir tasks, within the recent paradigm of the dual encoder model @xcite , and ii) compositional tasks, trying a systematic replication of the practical success obtained by @xcite through image-based heuristics. when looking at models like query2prod2vec in the larger industry landscape, we hope our methodology can help the field broaden its horizons: while retail giants indubitably played a major role in moving ecommerce use cases to the center of nlp research, finding solutions that address a larger portion of the market is not just practically important, but also an exciting agenda of its own foot_10 .