Graph-based Multilingual Product Retrieval in E -Commerce Search (2021)

Abstract: Nowadays, with many e-commerce platforms conducting global business, e-commerce search systems are required to handle product retrieval under multilingual scenarios. Moreover, comparing with maintaining per-country specific e-commerce search systems, having an universal system across countries can further reduce the operational and computational costs, and facilitate business expansion to new countries. In this paper, we introduce an universal end-to-end multilingual retrieval system, and discuss our learnings and technical details when training and deploying the system to serve billion-scale product retrieval for e-commerce search. In particular, we propose a multilingual graph attention based retrieval network by leveraging recent advances in transformer-based multilingual language models and graph neural network architectures to capture the interactions between search queries and items in e-commerce search. Offline experiments on five countries data show that our algorithm outperforms the state-of-the-art baselines by 35% recall and 25% mAP on average. Moreover, the proposed model shows significant increase of conversion/revenue in online A/B experiments and has been deployed in production for multiple countries.

## introduction modern e-commerce search engines @xcite typically consist of a retrieval stage and a ranking stage. the retrieval stage is responsible for collecting a set of relevant products with minimum computational resources. the ranking stage then applies sophisticated machine learning (ml) algorithms to determine their impression positions. traditional retrieval models rely on keyword matching @xcite , which may lead to poor results when the exact term match is unavailable. recently, semantic matching models @xcite have been adopted to improve retrieval performance @xcite . these models are trained using click/purchase logs and typically separated by countries @xcite . however, such per-country specific training schema exposes three major drawbacks. first, maintaining countryspecific models increase both operational burden and model iteration risks among countries. second, the small amount of training data in low traffic countries may limit the ml model performance and this can also block the business expansion to new countries. third, such models can not handle second language searches well. for example, the training data in us are dominated by english, which produces a model that cannot handle spanish searches well. to solve above issues, ideally, a multilingual semantic retrieval model should be considered over monolingual retrieval models. however, how to design an effective and scalable multilingual semantic retrieval model for industry grade e-commerce search engine remains unsolved. built upon the success of pre-trained transformer-based models @xcite @xcite such as bidirectional encoder representations from transformers (bert) for natural language processing, multilingual bert (m-bert) has also demonstrated success for multilingual tasks @xcite . though the techniques are promising, it is not straightforward to directly apply them to our problem due to the vocabulary gap issue @xcite , i.e., customer searched queries are often short and from spoken input (e.g. 'fancy clothes') but product descriptions are usually in formal written style (e.g. 'formal attire'). there lacks a well established practice for fine-tuning multilingual bert models on product search retrieval tasks. in this work, we address the vocabulary gap by sharing information between queries and products in the model via graph convolution networks (gcn). the query-to-product purchase/click logs naturally form a bipartite graph where each clicked product links with searched queries as neighbors. we expect to improve the product representation for retrieval tasks by incorporating information from its neighbor queries. for example, it is difficult for neural networks to directly match the query 'great gifts for child' to the product 'disney puzzles for kids' given the vocabulary gap. but using the information that the product is connected with query 'children gifts', we can incorporate this information in its final representation, and the product will have a higher chance to be matched with the given query. this paper presents an end-to-end multilingual retrieval system for e-commerce search engine. our contributions are three-fold. 1. model: we present a general framework that is compatible with any transformer-based models and any gcn architectures to capture interactions between products and search queries; 2. practice: we provide a principled and practical guide of how to train the proposed model for large-scale product retrieval problem, e.g., how to define effective loss functions, how to feed online model-based hard negative samples to train the model and how to train the multilingual model with a novel one-languageat-a-batch (sec 2.2) approach; 3. system: we discuss how to deploy the model to support product retrievals in multiple countries for e-commerce search. to validate the effectiveness of our proposed method, we take offline experiments on billionscale data across five languages and conduct online a/b testing experiments to measure the real traffic impacts. through experimental results, our model outperforms state-of-the-art baselines by more than 25% and increases revenue and conversion over the current production system. ## methodology we formulate the search retrieval task as follows. supposed that we have a set of products @xmath0, ..., p n }. each product p i has a number of neighbor queries q @xmath1,1 , ..., q i,t } where (q i,j , p i ) appears in the search logs (customers search for q i,j and purchase p i ). for an arbitrary query q, we want to find the top-k relevant products from p . note that q is not in @xmath2, ..., q n } when our retrieval system handles unseen queries. ## deployment the deployment of the proposed model has two parts: a query encoder and pre-computed product embeddings. as the query encoder is a standard transformer-based model and many papers have talked about the serving of it, this part can be easily deployed online. for the pre-computed product embeddings, we first compute the embeddings for products in our catalog, and incrementally update the product embedding periodically. to avoid repeated computations during the inference time (multiple products might have the same neighbor query), we first use the transformer-based encoder to compute the embeddings for all queries and products in the graph, and then join the products' embeddings with their neighbor queries' embeddings. lastly, we pass these intermediate embeddings through the gcn layer to generate the final product embeddings. during the search retrieval step, we simply use the query encoder to extract the embedding for an input query. then, we find the knearest neighbors (k-nn) products by calculating the cosine-distance between the query embedding and the pre-computed product embeddings. the k-nn products are used to augment the matchset of the given query. ## experiments we collect the data from a large e-commerce platform that has business in multiple countries. to provide a comprehensive understanding on the role of multilingual queries in a real-world product search system, we select five countries: united states (us), spain (es), france (fr), italy (it) and germany (de). we subsample our data from one year of search log in each country. we organize the collected search log into query-product pairs with different customer behavior signals, e.g. click/purchase. for model offline testing, we first randomly sample 20k queries from each country. we then use our algorithm to rank a subcorpus of 100k products (in each country) for those queries. the 100k product corpus consists of purchased products for those 20k queries and additional random negatives. since our work focuses on the retrieval part of a product search engine, we adopt two matching metrics to summarize our results: recall@10 (recall) and mean average precision (map). we employ the multilingual distilbert @xcite with 6 layers and 768 hidden units as our encoder. we set the batch size to 640 and use adam optimizer (kingma and ba, 2014) with α = 0.0001, β 1 = 0.9, and β 2 = 0.999. we run all the experiments on an aws p3dn.24xlarge instance with 768gb memory and 8 nvidia v100 gpus. we train the model on 8 gpus in a distributed fashion. the model is trained for 140k batches, where the 28k 'warm up' batches are trained with random negatives and remaining batches are trained with online modelbased hard negatives. ## conclusion our paper present a multilingual graph convolution networks model for language-agonistic semantic retrieval in product search engine. our method not only can handle multilingual text data, but also addresses the vocabulary gap issues between queries and product descriptions. we also provide a practical guide of fine-tuning the proposed model on retrieval tasks. we conduct various experiments including offline evaluation on 5 languages and online a/b test in three countries. in all experiments, our model consistently beats the baselines and demonstrates improved product discoverability.