Deep Metric Learning to Hierarchically Rank - An Application in Product Retrieval (2023)

Abstract: Most e-commerce search engines use customer behavior signals to augment lexical matching and improve search relevance. Many e-commerce companies like Amazon, Alibaba, Ebay etc. operate in multiple countries with country specific stores. However, customer behavior data is sparse in newer stores. To compensate for sparsity of behavioral data in low traffic stores, search engines often use cross-listed products in some form. However, cross-listing across stores is not uniform and in many cases itself sparse. In this paper, we develop a model to identify duplicate and near-duplicate products across stores. Such a model can be used to unify product catalogs worldwide, improve product meta-data or as in our case, use near-duplicate products across multiple to improve search relevance. To capture the product similarity hierarchy, we develop an approach that integrates retrieval and ranking tasks across multiple languages in a single step based on a novel Hierarchical Ranked Multi Similarity (HRMS) Loss that combines Multi-Similarity (MS) loss and Hierarchical Triplet Loss to learn a hierarchical metric space. Our method outperforms strong baselines in terms of catalog coverage and precision of the mappings. We also show via online A/B tests that the product mappings found by our method are successful at improving search quality in low traffic stores, measured in rate of searches with at least one click, significantly by 0.8% and improving cold start product engagement measured as new product clicks significantly by 1.72% in established stores.

## introduction modern search engines utilize two types of information associated with products in their matching and ranking stages: 1) semantic information in terms of product attributes (e.g. title, description, brand, color etc.) provided by the sellers, and 2) behavioral information derived from the customer's interaction with the product (e.g. clicks, adds-tocart, purchases, reviews & ratings etc.). behavioral features are critical for both matching and ranking stages and play an important role in improving the quality of search results. large amount of customer behavior data is required for building high quality behavioral features. the difference in query volume across high-traffic established stores versus a new store is often staggering, with newer stores receiving orders of magnitude lower traffic. lower traffic in newer stores lowers the quality of behavioral features and consequently the quality of search results. also, many major e-commerce companies operate a multilingual search system where each country has a primary language and possibly multiple secondary languages. however, not all secondary languages have rich behavior data, resulting in poorer search quality compared to primary language queries. one practical way to mitigate this sparsity in large scale commercial search engines is integrative knowledge transfer (ikt) @xcite . as part of transfer learning, ikt is a technique used to infuse knowledge from one domain into a different domain similar to feature engineering and data integration methods @xcite . in e-commerce, ikt is used to synthetically associate behavioral data for a product from higherresource source store with the cross-listed product in a lower-resource target store. despite its success, ikt based methods suffer from selection bias: owing to more (transferred) behavioral data, crosslisted products dominate search results while products exclusive to a specific store get pushed down in relevance. this negative transfer can overwhelm local preferences and results in bad customer experience. in this paper, we reduce the selection bias by identifying products, across multiple stores, with near-identical shopping intents and apply ikt techniques to these product mappings. we refer to such products as substitutes. the problem of identifying substitutes across stores with different languages can be formulated as cross-lingual information retrieval (clir). existing works in clir focus on either query/document translation using machine translation @xcite or expanding queries with translations @xcite . nguyen et al. (2008) uses wikipedia as source for cross-lingual document sets, tigrine et al. (2015) constructs cross-lingual ontologies from existing knowledge bases. all these works suffer from poor generalization to new domains such as product search. most works in clir don't use behavioral features which are important for product search. gupta et al. ( 2020 ) addresses some of these challenges for product search by estimating prior values of behavioral features for cold start products. however, their approach is neither cross-locale nor easily extendable to a cross-lingual setting. @xcite addresses the problem of product search in multiple languages with limited amount of data per language. they achieve this by learning language independent query and product representations with an end-to-end (query, product) relevance ranking model. clir approaches, however, ignore hierarchical information associated with most product catalogs. product catalogs of all major e-commerce businesses are organized in a product taxonomy @xcite . a typical example is 'shoes->mens sports shoes->running shoes'. it follows then that cross-listed products exist in a cross-lingual product hierarchy, even though the taxonomy mapping may not be one-to-one. product taxonomy or other forms of hierarchy can still be leveraged when retrieving substitute products from catalogs of different stores. hierarchical ranked loss functions can learn the characteristics of the product taxonomy and score exact crosslisted cross-lingual products higher than substitutes, and substitutes higher than dissimilar products in other categories (figure 1 ). in this paper, we propose an approach that combines retrieval and ranking across multiple languages in a single step. our solution makes use of product hierarchy by combining multi-similarity (ms) loss @xcite and hierarchical triplet loss @xcite into hierarchical ranked multi similarity (hrms) loss. while there exist loss functions which optimize for output rank figure 1 : the diversity of a cross-listed cross-lingual multi-store product catalog means that for a given product there exist multiple possible exact and substitutes matches. a learned metric space should reflect the nature of the data set by ranking exact matches closer than substitutes and far from other levels of product categorization. @xcite or hierarchical objectives @xcite independently, none learns both together. using hrms, we learn a ranked metric space by generating informative pairs based on the hierarchical nature of the data. by varying the margin based on the hierarchical label, we are able to improve the ranked query output compared to vanilla multi-similarity and ranking based loss functions. we demonstrate the use of hrms by training a multi-modal cross-lingual model nicknamed productsim based on a hierarchical cross-lingual product catalog. finally we demonstrate the business use of productsim by using its output to improve search results for new stores and products as measured by significant improvement in business metrics like search-ratewith-clicks (percent of searches with at least one clicks) and new product impressions and clicks. the main contribution of this paper is the hierarchical ranked multi similarity (hrms) loss, built by extending multi similarity loss for hierarchical ranking in metric space. ## method in this section, we review the challenges of a hierarchical product data set and limitations of proxy based loss functions @xcite applied to sparse data. we then introduce our hierarchical ranked multi similarity loss (hrms), an adaption from multi similarly (ms) loss. ## product dml model -productsim productsim is a light weight multi-modal dml model whose goal is to output a trained set of embeddings such that similar products are positioned close together in euclidean space. productsim is built using language-agnostic bert sentence enbedding (labse), shift vision transformers (shiftvit) as language and image encoders respectively. the embeddings from respective encoders are then fed through a fusion encoder to capture inter-modality interaction in a late fusion manner @xcite . details on productsim can be found in appendix a. ## offline evaluations we validate productsim on a holdout set from the training data. we select the holdout set such that each product and its substitutes exist in at least two languages across multiple stores. the catalog items from the same products are exact matches and should be ranked highest in the retrieved ranked list. different products within a single product family are considered substitutes and should be ranked lower than exact matches. all other products are irrelevant and should not be retrieved. we selected an evaluation test set of 33k product families (substitutes) containing an average of 6.7 products in 11.2 languages on average per family. each product contains an average of 3.7 catalog items. for a given catalog item of a product, we generate the productsim embedding and retrieve the top-k neighbors by euclidean distance. our goal is to retrieve all the other catalog items of the input product in the top-k neighbors ranked such that exact matches are ranked ahead of substitutes. we use the embeddings from the test set and built a flat faiss index @xcite for the purpose of performing exhaustive search for each query. in order to measure the ranked nature of the results, we adopted the implementation of weighted ndcg similar to @xcite . in our eval- ## impact on search results all search engines use behavioral data to match and rank products to queries. low traffic stores and new products lack such behavioral data and thus suffer from less relevant search results. we used integrative knowledge transfer (ikt) @xcite to boost the behavioral data associated with the new stores and products by synthetically transferring query-product associations from hightraffic stores to low-traffic stores. in addition to cross-listed products we used product substitutes identified by productsim to identify like products to transfer behavior. the following experiments show the impact of expanded ikt coverage on search results. low-traffic store: we used productsim to map products from a low-traffic store to those in multiple high-traffic stores. we then boosted the behavior associated with the products in the lowtraffic store by using the behavior associated with the mapped products from the high-traffic store(s). we did this by treating the traffic associated with a product in the established store as if it occurred in the low-traffic store albeit in the context of the product identified by productsim. in our experiment, we used a store where roughly 50% of the query traffic was in english and 50% in a single non-english language. using productsim, we boosted the behavior associated with 21% of the active catalog products, thereby boosting their rank in the search results. this improved customer engage-ment, in an online a/b test using as measured by search click rate defined as number of searches with at least one click over total number of searches by 0.8% (p-@xmath0.026) new products in a high-traffic store: cold start is a known problem in product search @xcite . this is specially true in high-traffic stores where established products dominate search impressions. we used productsim to map new products in a high-traffic store to established products in the same store when possible, and a different store otherwise. boosting behavior associated with the new products using the mapping, we were able to increase customer engagement with these products. in particular in an online a/b test, we boosted new product impressions by 1.76% (p-@xmath1.0) and new product clicks by 1.72% (p-@xmath2.0). the strong correlation between impressions and clicks also tells us that we boosted products that customers desired. more recently, attempts are made to learn hierarchical metric learning representation by adapting proxy-based methods @xcite . while such methods work well for dense data sets (small number of classes and large number of samples per class), it is challenging to scale to granular settings during training (e.g., e-commerce product data sets), where classes could be defined at the product level and a small number of examples are available per class. further existing methods do not optimize for the ranked output hierarchically instead focus either on recall or mean average precision @xcite in this paper we combine ideas from both paradigm to propose hrms. by mirroring the characteristics of a real life hierarchical product data set and without the complexity of a graph neural network, we show that it is possible to induce a model to learn both retrieval and ranking simultaneously. ## conclusion and future work in this paper we propose a metric learning loss function which is suitable for use on hierarchical data sets similar to amazon catalog. we extend the existing multi similarity loss with adaptive margin for a hierarchical data set. hierarchical ranked multi similarity loss (hrms) works by optimizing for ranked retrieval instead of multi-similarity or ranked loss individually. future work could consider the multi-aspect dimension of similarity @xcite . similarity is context specific, for example one could look for products with similar brand.