How to build Content Recommendation Using Elastic Search
Tags: beginners, tutorial, elasticsearch, developer
Published: 2020-08-31 14:41:00

---
title: How to build Content Recommendation Using Elastic Search
published: true
tags: beginners,tutorial,elasticsearch,developer
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/prmv133gq9ja6s1p2c8n.jpeg
---

When we read an article in any news website, medium, dev.to etc.. we generally see additional sections like *Recommended Articles, Similar Articles, etc..* where we see a few more articles matching the content of the article you are reading or may be based on your previous read history you get a few more recommendations.

Basically, Articles recommendation can be done in two ways.

- Collaborative Filtering
- Content similarity

#### Collaborative Filtering:

- Collaborative filtering is a method of making automatic predictions (*filtering*) about the interests of a user by collecting preferences or taste information from many users (*collaborating*).

- This can be implemented using Machine Learning techniques by identifying the group of users having similar behaviour.

- Another way of identifying a similar group of users is by using [Graph Databases](https://en.wikipedia.org/wiki/Graph_database) like [Neo4j](https://neo4j.com/), [ArangoDB](https://www.arangodb.com/) etc.. where you can build a graph of users connected via their interests, activities on the website, purchase patterns etc.. and identify similar correlated user groups.

#### Content Similarity:

- The content similarity is the degree of *similarity* between two articles, based on the textual *content* (terms appearing in them) of the two articles.

- This can be implemented using information retrieval techniques like [Bow (Bag Of Words)](https://en.wikipedia.org/wiki/Bag-of-words_model), [TF-IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf), etc..

In this blog post, I will explain more about implementing **_Content Similarity_** using Elastic Search, which internally uses TF-IDF for calculating the relevant articles for the given search query. I took sample articles dataset from [kaggle](https://www.kaggle.com/asad1m9a9h6mood/news-articles#Articles.csv) (dataset from [thenews](https://www.thenews.com.pk/) website) for this activity.

This dataset contains 2692 articles, out of which 1408 are **_sports_** related articles and remaining 1284 articles are **_business_** related articles. Will explore **sports** articles in this post.

#### Let’s look into the interesting part… The implementation

| ![Let's go](https://dev-to-uploads.s3.amazonaws.com/i/5xk1sr4s6yqhnro40ji3.jpg) |
|:--:| 
| *Lets go ..!* |


Below is the sample article stored in Elastic Search which talks about  **Cricket**.

| ![Image from Firstpost](https://images.firstpost.com/wp-content/uploads/2019/05/South-Africa-cricket-team-Reuters-380_opt.jpg) |
|:--:| 
| *Image from Firstpost* |


{% gist https://gist.github.com/vishnuchilamakuru/fd19ba2eb88e3a9f1c4da4d03e400f53.js %}

Above article talks more about _Hashim Amla, Temba Bavuma, South Africa, England, Test Cricket ..._ Now, let’s see the top 4 articles matching this content (article id — 1817 as mentioned above)

Below are the recommended articles which have similar content 
{% gist https://gist.github.com/vishnuchilamakuru/7db4a2be34581af4156c8e07a8750c13.js %}


- “title”: “Injured Amla stands firm as South Africa build lead”
- “title”: “Amla makes century De Villiers falls for 88”
- “title”: “Amla and Stephen Cook lead South Africa to 329/5”
- “title”: “Root Stokes fire up England ” (Eng Vs SA Test match)

So, If u see most of the recommended articles are in context with the article (id- 1121).

I used Elastic search’s [More Like This Query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-mlt-query.html) to identify the similar articles matching the content of the current article.

Sample Elastic Search More Like This Query:

{% gist https://gist.github.com/vishnuchilamakuru/783cfca2f7546c421b3f25b1f4e12492.js %}

Let’s see one more similar example, this time for the **Football news**.

| ![Image from Business Insider](https://i.insider.com/5ace331a146e7126008b456e?width=1100&format=jpeg&auto=webp) |
|:--:| 
| *Image from Business Insider* |

{% gist https://gist.github.com/vishnuchilamakuru/9a2666e6f5b14fded96af7895122bb35.js %}

Above article talks more about _Lionel Messi and Argentina._ Now, let’s see the top 4 articles matching this content (article id — 1817 as mentioned above)

Below are the recommended articles which have similar content 
{% gist https://gist.github.com/vishnuchilamakuru/4f1c6bad86d6867e1de4caef4396f336.js %}

- “title”: “Messi record as Argentina thrash Venezuela”
- “title”: “Magical Messi grabs hat trick as Argentina romp into quarter”
- “title”: “Messi scores 50th Argentina goal in 2 0 wins over Bolivia”
- “title”: “Messi primed to end Argentina drought Copa fi”

Almost all the recommended articles are in context with the article id- 1817.

###**Conclusion**:

Overall, ElasticSearch MoreLikeThis query will help you in identifying similar content articles in a fast and efficient manner which can give you decent recommendations based on text content.


Thank you for your time! 
If you like the article keep sharing it and you can follow me on [Twitter](https://twitter.com/vishnuchi) for more posts on technology, startups & leadership