Basic Operations of Elasticsearch in Spring Boot with RestHighLevelClient
By: Luthfi Hariz
Tags: Spring Boot, Elasticsearch, Full Text Search

In this post, we are going to learn how to do index, bulk index, and search Elasticsearch documents within your Spring Boot application using RestHighLevelClient.

I am using Elasticsearch 7.9.3. If you haven’t, you can install it using Docker, and the Docker image is available here.

Let’s jump into the code and include this dependency in your project.

<dependency>

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-starter-data-elasticsearch</artifactId>

</dependency>

Document Model

We are going to store e-commerce like Product items in the ElasticSearch. Let’s make a simple model for that.

import lombok.Data;



@Data

public class Product {

private Integer id;

private String name;

private String description;

private Double price;

private String category;

}

Before we actually do some operations, we should create a @Configuration class that manages the connections to our Elasticsearch and a REST controller as our HTTP interface to make our test easier.

Elasticsearch Config

@Configuration

public class ElasticSearchConfig { @Value("${spring.data.elasticsearch.cluster-nodes}")

String elasticHost; @Bean

public RestHighLevelClient client() {

return new RestHighLevelClient(RestClient.builder(HttpHost.create(elasticHost)));

}

}

Add this line to your application.properties

# ES

spring.data.elasticsearch.cluster-nodes=${ELASTICSEARCH_HOST:localhost:9200}

This property use the system environment to retrieve the value of the ElasticSearch node host, however, it also has a default value of localhost:9200

Product Controller

@RestController

@RequestMapping("/product")

public class ProductController { }

Let’s keep it as simple as this, we are going to modify it later.

Product Repository

Usually, we want to have a service that intermediates a controller to a repository. But for the sake of simplicity, let allow the controller to talk directly to a repository.

So right now, we are going to create a ProductRepository that has the responsibility to manage the operations of our Elasticsearch.

public interface ProductRepository {

SearchResponse search(SearchQueryDto searchQueryDto) throws IOException;



IndexResponse save(Product product) throws IOException;



BulkResponse saveAll(List<Product> products) throws IOException;

}

Next, we are going to implement each method and actually use the RestHighLevelClient. We also define our Index name where we will read and write our document to.

@Repository

public class ProductRepositoryImpl implements ProductRepository {



@Autowired

RestHighLevelClient restHighLevelClient;



private static final String INDEX_NAME = "product"; }

Writing Document (Indexing)

Indexing is a process where we store a document in the Elasticsearch node. IndexingRequest is provided to help us with this.

@Override

public IndexResponse save(Product product) throws IOException {



IndexRequest indexRequest = Requests.indexRequest(INDEX_NAME)

.id(product.getId().toString())

.source(convertProductToMap(product));



RequestOptions options = RequestOptions.DEFAULT;

return restHighLevelClient.index(indexRequest, options);

}

Pay attention to convertProductToMap() function, it will convert our Product object to a Map<String, Object>.

Here is the implementation.

@Autowired

ObjectMapper objectMapper; private Map<String, Object> convertProductToMap(Product product) throws JsonProcessingException { String json = objectMapper.writeValueAsString(product);

return objectMapper.readValue(json, Map.class); }

I am using Gson to encode it to JSON, then decode it to a Map class.

Then, we need to update our ProductController to call this save() function in our repository.

@Autowired

ProductRepository productRepository; @PostMapping

public IndexResponse addProduct(@RequestBody Product product) throws IOException {

return productRepository.save(product);

}

I’m going to use the return object from the RestHighLevelClient class as our HTTP response for the sake of this tutorial. We may want a different response in a real case.

Finally, we can test our POST /product API by using Postman or curl command.

curl --location --request POST 'localhost:8080/product' \

--header 'Content-Type: application/json' \

--data-raw '{

"id" : 1,

"name" : "Apple Bookmac",

"description" : "Next generation of Book",

"price" : 2000.0,

"category" : "Electronic"

}'

Bulk Indexing

Sometimes we want to store a lot of documents at the same time. In order to do that, we can use BulkRequest class.

BulkRequest has an add() function that receives IndexRequest as a parameter. We are going to loop over the products, create an IndexRequest each and call the add() function.

@Override

public BulkResponse saveAll(List<Product> products) throws IOException {



BulkRequest bulkRequest = Requests.bulkRequest();

products.forEach(product -> {

try {

IndexRequest indexRequest = Requests

.indexRequest(INDEX_NAME)

.source(convertProductToMap(product));

bulkRequest.add(indexRequest);

} catch (JsonProcessingException e) {

// log error

}

});



RequestOptions options = RequestOptions.DEFAULT;

return restHighLevelClient.bulk(bulkRequest, options);

}

Last, modify our ProductController.

@PostMapping("/bulk")

public BulkResponse addProducts(@RequestBody List<Product> products) throws IOException {

return productRepository.saveAll(products);

}

Finally, we can test it by hitting POST /product/bulk.

Searching

Search is probably, the most advanced operations in Elasticsearch. And we are not going to cover everything here. At least, we want our search to be able to handle a request like this :

{

"query": "covid",

"filter": {

"match" : {

"category" : "vaccine"

},

"range" : {

"price" : {

"lte" : 50,

"gte" : 20

}

}

},

"page": 0,

"size": 20

}

query will find the value that should match with either name, description, or category.

filter will only select Products that match the exact value and within the range

lte is lower than equal (≤)

gte is greater than equal (≥)

gt is greater than (>)

lt is lower than (<)

page and size will responsible for handling pagination.

In order to fulfill that, we need to create our request object.

@Getter

@Setter

@AllArgsConstructor

@NoArgsConstructor

public class SearchQueryDto {

private String query;

private FilterRequestDto filter;

private Integer page;

private Integer size;

} @Getter

@Setter

@NoArgsConstructor

public class FilterRequestDto {

private Map<String, Object> match;

private Map<String, RangeFilterDto> range;

} @Getter

@Setter

@NoArgsConstructor

public class RangeFilterDto {

private Double lte;

private Double gte;

private Double lt;

private Double gt;

}

Next, we will implement our search() method in ProductRepository.

@Override

public SearchResponse search(SearchQueryDto searchQuery) throws IOException {

SearchRequest searchRequest = Requests.searchRequest(INDEX_NAME);



// bool query

BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()

.should(QueryBuilders.matchQuery("name", searchQuery.getQuery()))

.should(QueryBuilders.matchQuery("description", searchQuery.getQuery()))

.should(QueryBuilders.matchQuery("category", searchQuery.getQuery()));





// facet query

if (searchQuery.getFilter() != null) {

FilterRequestDto filter = searchQuery.getFilter();

if (filter.getRange() != null) {

for (String keyToFilter : filter.getRange().keySet()) {

RangeFilterDto valueToFilter = filter.getRange().get(keyToFilter);



RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery(keyToFilter);



if (valueToFilter.getLte() != null) {

rangeQueryBuilder.lte(valueToFilter.getLte());

}



if (valueToFilter.getLt() != null) {

rangeQueryBuilder.lt(valueToFilter.getLt());

}



if (valueToFilter.getGt() != null) {

rangeQueryBuilder.gt(valueToFilter.getGt());

}



if (valueToFilter.getGte() != null) {

rangeQueryBuilder.gte(valueToFilter.getGte());

}



boolQueryBuilder.filter(rangeQueryBuilder);

}



} else if (filter.getMatch() != null) {

for (String keyToFilter : filter.getMatch().keySet()) {

Object valueToFilter = filter.getMatch().get(keyToFilter).toString().toLowerCase();



boolQueryBuilder.filter(QueryBuilders.termQuery(keyToFilter, valueToFilter));

}

}

}



// pagination

SearchSourceBuilder searchSourceBuilder = SearchSourceBuilder.searchSource()

.from(searchQuery.getPage() * searchQuery.getSize())

.size(searchQuery.getSize())

.query(boolQueryBuilder);



searchRequest.source(searchSourceBuilder);

return restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

}

I know It’s a long snippet. Here’s come the brief.

Using SearchRequest as our parameter to RestHightLevelClient

as our parameter to Construct a bool query by using BoolQueryBuilder

query by using Using RangeQueryBuilder and termQuery to filter the results.

and to filter the results. Create a SearchSourceBuilder to apply query and pagination.

to apply query and pagination. Apply SearchSourceBuilder to SearchRequest

A bool query is one of the Compound query clauses that is provided by Elasticsearch. In our case, it will combine should and filter .

You can read more about Query DSL provided by Elasticsearch here.

Last, add a function to our ProductController.

@PostMapping("/search")

public SearchResponse searchProduct(@RequestBody SearchQueryDto searchQueryDto) throws IOException {

return productRepository.search(searchQueryDto);

}

POST /product/search is now ready to test, but let’s add documents first by POST /product/bulk.

[

{

"id" : 1,

"name" : "Lenovo Thinkbook 14 Core i3 16GB",

"description" : "Lenovo thinkbook laptop",

"price" : 22000.0,

"category" : "Laptop"

},

{

"id" : 6,

"name" : "Macbook Air M1 8GB 256GB 2020",

"description" : "The latest laptop from Apple",

"price" : 50000.0,

"category" : "Laptop"

},

{

"id" : 3,

"name" : "Macbook Air M1 16GB 512GB 2020",

"description" : "The latest laptop from Apple",

"price" : 60000.0,

"category" : "Laptop"

},

{

"id" : 2,

"name" : "Samsung Galaxy S20 Ultra 16GB",

"description" : "Flagship Android phone from Samsung",

"price" : 10000.0,

"category" : "Smartphone"

},

{

"id" : 10,

"name" : "Xiaomi Mi 10T 8GB",

"description" : "Flagship killer smartphone",

"price" : 8000.0,

"category" : "Smartphone"

}

]

After it’s a success, let search it by:

{

"query": "16gb laptop",

"filter": {

"match" : {

"category" : "laptop"

},

"range" : {

"price" : {

"lte" : 80000,

"gte" : 20000

}

}

},

"page": 0,

"size": 20

}

And it will give us a response similar to this:

{

...

"hits": {

"hits": [

{

"score": 2.4307795,

"id": "75pDH3cBI0wGBJ9KPRfJ",

"type": "_doc",

"sourceAsMap": {

"price": 22000.0,

"name": "Laptop Lenovo Thinkbook 14 Core i3",

"description": "Lenovo thinkbook laptop",

"id": 1,

"category": "Laptop"

}

},

{

"score": 1.0390294,

"id": "8JpDH3cBI0wGBJ9KPRfO",

"type": "_doc",

"sourceAsMap": {

"price": 50000.0,

"name": "Macbook Air M1 8GB 256GB 2020",

"description": "The latest laptop from Apple",

"id": 6,

"category": "Laptop"

}

},

{

"score": 1.0390294,

"id": "8ZpDH3cBI0wGBJ9KPRfO",

"type": "_doc",

"sourceAsMap": {

"price": 60000.0,

"name": "Macbook Air M1 16GB 512GB 2020",

"description": "The latest laptop from Apple",

"id": 3,

"category": "Laptop"

}

}

],

"maxScore": 2.4307795

},

...

}

In each hit result, it will have its own score based on the similarity of query value to price, name, and description.

The response is actually longer than this, but I don’t think that we need to display everything here.

So that’s it, you can visit my Github for full project source code. If you found this tutorial is helpful, you can Clap on the story and share it with others!