Personalization for Search
By: Sonu Sharma
Tags: Personalization, Data Science, Deep Learning, Search, Reinforcement Learning

Personalization! Everyone’s doing it, so is it even special anymore? Personalization has been a very hot topic these days which is being blended everywhere including search results as well. It means the search results are ordered to meet the individual needs of a consumer. Here, I will be taking you through the application of personalization for search in the eCommerce domain, also known as personalized search.

There are too many choices, you end up not knowing which one you want — Black Mirror

Search

Search services have to go through broadly three stages before the results are served, namely query understanding (pre-processing), retrieval (processing), ranking (post-processing). In a pre-processing stage, the received query is structured, labeled and intent is extracted and then the simplified query is passed to the search engine (solr, elastic, etc.) in the processing stage. In the last stage i.e. post-processing stage, the raw response is re-ranked based on several features/signals based on the user feedback.

Personalized Search

Here, personalization for search is proposed as a post-processing step where the raw response from the search engine (recall set) can be re-ranked as per the user preference by leveraging deep reinforcement learning. Before we dive into the modeling aspect, let’s understand the requirement for the model. For text embedding, we have used BERT from the TensorFlow hub. In order to compress the large size concatenated embeddings, Autoencoder can be used.

Autoencoder

Autoencoders are self-supervised dimensionality reduction technique with an encoder-decoder architecture which is trained to reconstruct its input. It has a bottleneck layer in the middle that contains the embedding of the D-dimensional input vector; the embedding layer usually has much fewer units than D. The goal of the decoder is to reconstruct the input feature vector from this embedding. Autoencoders can be built of several types, two of them are used for creating the embedding — feed-forward fully-connected and deep convolutional fully-connected autoencoder. Watch out this blog for a separate blog on autoencoders.

autoencoders

Three embeddings for input to the model are -

Query Embedding — This is simple sentence embedding for which we can get the pooled embedding output of every single term in the sequence of keywords. Let’s say it gives an embedding E(Q) of shape (1,768). Product Embedding — Based on the several features like display name, brand, category, price, nutritional features, rating, and reviews, etc. of the product and concatenate it together, we can generate the product embedding. The numerical features from the catalog like price, rating, etc. can be scaled before concatenation, and individual text features can be encoded using word embedding. But, combining it together the embedding dimension increase. Now, to reduce the dimension, we have to use the deep feed forward auto-encoder as stated above to compress and get the required reduced dimension of product embedding at the end. Also, the similar products will be near each other in 2D space which can be analyzed through PCA followed by tSNE. Let’s say this process gives an embedding E(P) of shape (1,512). Customer Embedding — We can use the order details and profile details of customers to generate their individual embedding where every customer can be represented through a floating-point vector. All the features and behavior of a customer can be captured in a single embedding. The order details contain the history of items ordered in the past and top-N of them can be chosen. The embeddings for all of the N products have already been generated through E(P) and this is used here for fetching N products embedding. Since each E(P) has a shape(1,512), the N products together have a shape(100,512) which is quite large. So, we need a deep convolutional autoencoder for compressing them to have an embedding E(C) of shape(1,512) again. This embedding can be concatenated with the profile-related details of the user such as gender, age, location, etc. after encoding them to get the final E(C) of shape > (1,512). Again, the customer having similar behavior can be spotted near each other in 2D space which can be analyzed through PCA followed by tSNE.

The Dataset

The idea of creating a dataset is inspired by openai gym. Since the interaction of a particular user while searching for keywords through the shopping journey is considered as an environment, the entire session of the user is taken as an episode. The observation should be query wise interaction which consists of triplets of embedding discussed above. If positive feedback is received from the user in terms of click/add to cart, then we give a reward near 1 otherwise near 0. Sample dataset can be -

Raw customer engagement feedback data

Here, the user has searched for four keywords in a session and didn’t find anything to add for keyword onions . An episode of the dataset contains the observations of all keywords interaction. Let’s create an observation dataset based on the first keyword yoghurt.

Observation for a query

E(X) stands for the embedding of X.

The Model

For modeling, we need to have a model based on Deep Q-Network (DQN), where a deep learning model is built to find the actions an agent can take at each state. The problem can be formulated as MDP and as far as reinforcement learning is concerned, the search result page is assumed as the environment, the model as an agent, and the proper ranking for each query as an action. The action for each query is to surface the most preferred product at the top by giving a high score for that product. The reward will be the feedback that is received from the end-user after displaying the result set. The idea is to maximize the overall basket size in a search session. The model takes the embedding of triplet {E(P), E(C), E(Q)} as input. The reward can be discounted based on the future outcome.

Discounted reward

Below is some snippet/skeleton of code that explains the training sequence of the model:

Model skeleton to train the deep DQN for personalized search

Based on the loss function for the model, we can have two model -

model based on point-wise Loss function — The easiest method is pointwise learning. In this kind of method, mapping takes place between each triplet of user-document-query pairs and their relationship. Here, each action is a selecting document for the position by giving a score between 0 and 1. model based on list-wise loss function — The model obtain, as input, the multi-features of all documents for a query and a user, and learn to forecast the scores for all documents in a single inference. Two features of user_context, search_term in a dense layer with the addition of all product features in the LSTM network in a time-distributed manner can be passed as input to the model. The final layer having the probability corresponding to each of the products. Custom NDCG can be used here as the loss function. Further references.

Sample DQN deep learning model in Keras here.

Inference

Most of the deep learning models are not served realtime in production because it becomes too heavy. But here, the model is very light and all we have to do is to store the pre-generated embeddings in a fastly accessible datastore/cache. The model can be exposed over a REST API protocol and served using TensorFlow serving. For the pointwise loss method, the scores for the entire recall set has to be predicted, and hence, the call to the model will be made parallelly (asynchronously) for all the products to get the scores. Based on the scores received, the recall-set are re-ranked and presented to the user. For the list-wise loss method, since the scores for all the products are fetched in a single call to the model, we just need to re-rank the output and present it to the user.

Please do read my other stories and watch out this space for more stuff! Feel free to reach out to me in case of any help.

References