The beginners’ guide to Elasticsearch
By: Chetan Ambi
Tags: Data Science, Data Analytics, Elasticsearch, Machine Learning

What is Elasticsearch

As mentioned on the official page, Elasticsearch — is a distributed, open-source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on Apache Lucene.

So, what does this mean?

Distributed — a distributed system contains multiple nodes/machines which are spread geographically but connected together so that they can communicate and coordinate to achieve a common goal.

Open-source — the original source code is freely available and can be modified.

Search & analytics engine — a system capable of searching a given text/keyword in the database and display relevant data to the end-users. This is much like Google or Bing search engine.

Data — Elasticsearch works with both structured and unstructured data such as textual, numerical, geospatial, etc.

Apache Lucene — Apache Lucene is a free and open-source search engine written in Java. It is supported by the Apache Software Foundation and Elasticsearch is built on top of Lucene.

Why use Elasticsearch

Distributed : Elasticsearch is a distributed document store. The documents are stored in the nodes which are distributed across the cluster and can be accessed from any node.

: Elasticsearch is a distributed document store. The documents are stored in the nodes which are distributed across the cluster and can be accessed from any node. Scalable : Elasticsearch provides the ability to quickly increase or decrease the server (nodes). Elasticsearch automatically distributes the data and query load across all of the available nodes.

: Elasticsearch provides the ability to quickly increase or decrease the server (nodes). Elasticsearch automatically distributes the data and query load across all of the available nodes. Fast : Elasticsearch is blazing fast. It can search the document in near real-time usually less than a second. Note that there is little latency between the time when the documents are indexed and when the documents are ready for search. Elasticsearch uses a data structure called ‘inverted index’ which helps in very fast full-text searches.

: Elasticsearch is blazing fast. It can search the document in near real-time usually less than a second. Note that there is little latency between the time when the documents are indexed and when the documents are ready for search. Elasticsearch uses a data structure called ‘inverted index’ which helps in very fast full-text searches. Schema-less : Elasticsearch is schema-less. This means that we don’t have to necessarily specify the data type of each of the fields in the document. If the dynamic mapping is enabled, Elasticsearch is smart enough to identify the data type when indexing the document.

: Elasticsearch is schema-less. This means that we don’t have to necessarily specify the data type of each of the fields in the document. If the dynamic mapping is enabled, Elasticsearch is smart enough to identify the data type when indexing the document. Rich set of features: The Elasticsearch comes with powerful built-in features that make storing and searching data even more efficient. The Elastic Stack Logstash, Kibana, and Beats makes data ingest, visualization, and reporting easier.

Elasticsearch Vs RDBMS

Relational Database Systems (RDBMS) are not suitable for full-text search, synonym search, phonetic search, log analysis, etc. Elasticsearch is specifically designed for enormous text searches and it is really quick!! The more data we want to search in the more relevant Elasticsearch becomes.

Elasticsearch is not meant to be a primary data store so the expert’s recommendation is to use a simple RDBMS like PostgreSQL and Elasticsearch for search functionality.

What is Elasticsearch used for?

The full-text search is the core functionality of the Elasticsearch. Some of the use cases include:

Enterprise search

E-commerce search

Logging and log analytics

Analyze application logs and system metrics (Application Performance Management)

Forecast future values using Machine Learning

Scraping remote data from multiple sources

Geospatial data analysis and visualization

Competitors

These are some of the competitors to Elasticsearch. Currently, Elasticsearch is in the top 5 w.r.t. market share.

The Elastic Stack

As mentioned earlier, the full-text search is the core functionality of the Elasticsearch. It also provides other functionalities such as logging, analytics, visualization, etc. by software that are part of The Elastic Stack. Earlier Elasticsearch , Logstash and Kibana together was referred to as ELK stack . Recently Beats got added to it and currently, this combination is called The Elastic Stack .

Kibana

Kibana is one of the core products of the Elastic stack. It is used as an analytics & visualization tool for creating real-time histograms, line charts, pie-charts, etc. It can also manage some of the functionalities of Elasticsearch and Logstash via its web interface.

Logstash

Logstash was designed to process logs from applications and send them Elasticsearch for further processing. However, in recent years it many more advanced functionalities are added to Logstash.

Beats

This is a recent entry to the Elastic Stack. It includes a collection of lightweight shipping agents known as Beats that are used to send data to Elasticsearch.

Conclusion

Hope you got a good understanding of what is Elasticsearch. I am sure you will be interested in learning more about Elasticsearch. Do check this Part 2, where I have covered key concepts of Elasticsearch.

References