S iamese Neural Networks for Detecting Complementary Products (2021)

Abstract: Recommender systems play an important role in e-commerce websites as they improve the customer journey by helping the users find what they want at the right moment. In this paper, we focus on identifying a complementary relationship between the products of an e-commerce company. We propose a content-based recommender system for detecting complementary products, using Siamese Neural Networks (SNN). To this end, we implement and compare two different models: Siamese Convolutional Neural Network (CNN) and Siamese Long Short-Term Memory (LSTM). Moreover, we propose an extension of the SNN approach to handling millions of products in a matter of seconds, and we reduce the training time complexity by half. In the experiments, we show that Siamese LSTM can predict complementary products with an accuracy of ~85% using only the product titles.

## introduction as much as the diverse and rich offers on ecommerce websites help the users find what they need at one market place, the online catalogs are sometimes too overwhelming. recommender systems play a significant role in making this process convenient for users. a specific case for recommender systems is complementary products (also known as add-ons), which are the products that are sold separately but are used together, each creating a demand for the other. figure 1 shows some examples of complementary products. detecting complementary products in many of the current platforms is mainly based on copurchase history and business rules. in this approach, if two items have been bought together more than a certain number of times, they are assumed to complement one another with a high probability. however, complementarity among products cannot be accurately detected using only the purchase history because (i) identical items having different sizes or colors are likely to be bought together and are pure substitutes instead of complementary products (e.g., a user buys three flower vases in different sizes), and (ii) if there are no purchases made yet, the ground truth is missing (it is known as the cold-start problem). one solution to overcome these problems is to introduce human labeling for accurate validation. the problem of this approach lies in the time and scalability limitations. moreover, these approaches focus on popular items; thus, unpopular (less frequently bought) products will stay undiscovered. to address the aforementioned problems, we propose a supervised deep learning approach based on siamese neural networks (snn) @xcite , and in particular siamese convolutional neural network (cnn) and siamese long short-term memory (lstm). to train the model, we give the input dataset in the format of mainproduct, addonproduct, and label(y/n) that identifies if two products are complementary. using this data, the snn creates embeddings and generates vector outputs that show the actual distance between the two given products in terms of their complementarity. for each product, we consider the title, the description, and the brand as its attributes. figure 2 shows the pipeline of the proposed model. the objectives of this work are three-fold: our work builds upon the siamese cnn introduced by @xcite by comparing siamese cnn and siamese lstm models and showing how siamese architecture can be transformed to handle massive data. through the experiments we show that siamese lstm outperforms siamese cnn in predicting complementary products using textual product attributes with an accuracy of ∼ 85%. we also observe that among different attributes of products, the product title produces results with a higher accuracy. moreover, we show that we can extend the proposed siamese lstm approach to a knn problem that reduces the time complexity by half. the source code of our model is available on github foot_0 . ## preliminary in this section, we briefly present the snn architecture and explain how it works. snn (chicco, 2021) is a twin neural network (nn) composed of two separate nns sharing the same architecture and the same weights, with no limitation on the nn architecture (figure 2 ). in other words, snn is an nn architecture capable of learning similarity between data samples by receiving pairs of samples and analyzing the differences between their features to map them to a multidimensional feature space @xcite . by receiving two different inputs, the main goal of such networks is to develop similarity knowledge between the two produced outputs. @xcite categorize snns in three groups based on the time of merging the layers: late merge (lm), intermediate merge (im), and early merge (em), which are shown in figure 3 . in lm, the output vectors of each network are merged at the last dense layer. im suggests to merge the outputs of the two networks in the middle of the network and process them as one output in the last layers. in em, the two inputs are merged right before the actual network, resulting in a single-like nn architecture. one of the benefits of using snn is its scalability. it processes each data sample once and then computes each pair's compatibility score, which results in a significantly lower complexity than iterating through the whole model for each pair of products. in a real-life scenario, we are usually given target products set @xmath0, q 2 , • • • , q n } and candidate set for the add-ons @xmath1, c 2 , • • • , c m } where n and m have values larger than 10 6 , indicating a few millions of products. thus, to train a nn, we need to create n × m pairs of products to make input samples to the nn. however, usually, we are interested in the top k candidate add-ons for a given target product, and snns enable us to do so. ## method we now discuss the network architectures used in our siamese cnn and siamese lstm models. the input to these networks is the pair of any two product attributes (e.g., title and description). ## experiments in this section, we first explain the dataset and the preprocessing step to make it ready to be given to the models, and then present the experiments. ## related work we split the available methods for measuring similarity and complementarity into two groups: unsupervised and supervised learning approaches. one of the most common unsupervised learning methods using co-purchase history is the frequent pattern (fp) growth @xcite algorithm. other groups of research focus on using the paradigm of word2vec @xcite . @xcite propose a prod2vec model that learns product representations from sequences of past orders by considering the purchase sequence as a sentence and products within the sequence as words. the meta-prod2vec model by @xcite extends the prod2vec model by taking into account products' metadata. the bb2vec model @xcite eliminates the cold-start problem by using browsing and purchase session data, and is a combination of several prod2vec models. @xcite introduce the siamese cnn approach, which this work is based on. other supervised learning approaches focus on image data, text attributes, or both. sceptre is a model introduced by @xcite , and its main goal is topic modeling using latent dirichlet allocation (lda) @xcite and edge detection of related topics. @xcite suggest encore, a three-step algorithm: (i) detecting the complementarity among products based on their embedding distances of image and text attributes, (ii) taking into account user preferences for detecting validity of each complementarity distance, and (iii) training a nn with the outcomes of the previous two steps @xcite . @xcite explore dynamic cnn (dcnn) for semantic modeling of sentences using cnns. ## conclusion in this paper, we present a supervised learning approach for complementary product recommendations. we take manually labelled pairs of complementary products from an e-commerce company and propose a scalable solution. to this end, we design and compare siamese cnn and siamese lstm architectures to create embeddings for products' features and compute a complementarity score for a given pair of products. we conclude that siamese lstm outperforms siamese cnn and its baselines. we show that the product title is the most valuable attribute. lastly, we show that our model can be transformed into a knn solution to handle big data scenarios. this work can be extended by introducing user click history to include items that have been viewed in the same session (items which are very similar) in the negative training sample, thus teaching the model the difference between substitute and complementary relationships. furthermore, including more product attributes (such as the price or subcategory) could improve the model's performance.