# Unlocking Semantic Search: Building a Java-Powered Movie Search Engine with Atlas Vector Search and Spring Boot

In the rapidly evolving world of technology, the quest to deliver more relevant, personalized, and intuitive search results has led to the rise in popularity of semantic search. 

MongoDB's Vector Search allows you to search your data related semantically, making it possible to search your data by meaning, not just keyword matching.

In this tutorial, we'll delve into how we can build a Spring Boot application that can perform a semantic search on a collection of movies by their plot descriptions.

## What we'll need

Before you get started, there are a few things you'll need.

- Java 11 or higher
- Maven or Gradle, but this tutorial will reference Maven
- Your own MongoDB Atlas account
- An OpenAI account, to generate our embeddings

## Set up your MongoDB cluster

Visit the MongoDB Atlas dashboard and set up your cluster. In order to take advantage of the `$vectorSearch` operator in an aggregation pipeline, you need to run MongoDB Atlas 6.0.11 or higher.

Selecting your MongoDB Atlas version is available at the bottom of the screen when configuring your cluster under "Additional Settings."

.

For this project, we're going to use the sample data MongoDB provides. When you first log into the dashboard, you will see an option to load sample data into your database. 

 in your database to automatically embed your data.

## Create a Vector Search Index

In order to use the `$vectorSearch` operator on our data, we need to set up an appropriate search index. Select the "Search" tab on your cluster and click the "Create Search Index."

 for more information on these configuration settings.

## Setting up a Spring Boot project

To set up our project, let's use the Spring Initializr. This will generate our **pom.xml** file which will contain our dependencies for our project.

For this project, you want to select the options in the screenshot below, and create a JAR:

. Feel free to use a more up to date version in order to make use of some of the most up to date features, such as the `vectorSearch()` method. You will also notice that throughout this application we use the MongoDB Java Reactive Streams. This is because we are creating an asynchronous API. AI operations like generating embeddings can be compute-intensive and time-consuming. An asynchronous API allows these tasks to be processed in the background, freeing up the system to handle other requests or operations simultaneously. Now, let’s get to coding!

To represent our document in Java, we will use Plain Old Java Objects (POJOs). The data we're going to handle are the documents from the sample data you just loaded into your cluster. For each document and subdocument, we need a POJO. MongoDB documents bear a lot of resemblance to POJOs already and are straightforward to set up using the MongoDB driver.

In the main document, we have three subdocuments: `Imdb`, `Tomatoes`, and `Viewer`. Thus, we will need four POJOs for our `Movie` document.

We first need to create a package called `com.example.mdbvectorsearch.model` and add our class `Movie.java`. 

We use the `@BsonProperty("_id")` to assign our `_id` field in JSON to be mapped to our `Id` field in Java, so as to not violate Java naming conventions.

```java
public class Movie {

    @BsonProperty("_id")
    private ObjectId Id;
    private String title;
    private int year;
    private int runtime;
    private Date released;
    private String poster;
    private String plot;
    private String fullplot;
    private String lastupdated;
    private String type;
    private List directors;
    private Imdb imdb;
    private List cast;
    private List countries;
    private List genres;
    private Tomatoes tomatoes;
    private int num_mflix_comments;
    private String plot_embeddings;

    // Getters and setters for Movie fields

}

```

Add another class called `Imdb`.

```java
public static class Imdb {

    private double rating;
    private int votes;
    private int id;

    // Getters and setters for Imdb fields

}

```

Yet another called `Tomatoes`.

```java
public static class Tomatoes {

    private Viewer viewer;
    private Date lastUpdated;

    // Getters and setters for Tomatoes fields

}
```

And finally, `Viewer`.

```java
public static class Viewer {

    private double rating;
    private int numReviews; 

    // Getters and setters for Viewer fields

}
```

> Tip: For creating the getters and setters, many IDEs have shortcuts.

### Connect to your database

In your main file, set up a package `com.example.mdbvectorsearch.config` and add a class, `MongodbConfig.java`. This is where we will connect to our database, and create and configure our client. If you're used to using Spring Data MongoDB, a lot of this is usually obfuscated. We are doing it this way to take advantage of some of the latest features of the MongoDB Java driver to support vectors.

From the MongoDB Atlas interface, we'll get our connection string and add this to our `application.properties` file. We'll also specify the name of our database here.

```
mongodb.uri=mongodb+srv://:@.mongodb.net/
mongodb.database=sample_mflix
```

Now, in your `MongodbConfig` class, import these values, and denote this as a configuration class with the annotation `@Configuration`.

```java
@Configuration
public class MongodbConfig {

    @Value("${mongodb.uri}")
    private String MONGODB_URI;

    @Value("${mongodb.database}")
    private String MONGODB_DATABASE;
```

Next, we need to create a Client and configure it to handle the translation to and from BSON for our POJOs. Here we configure a `CodecRegistry` to handle these conversions, and use a default codec as they are capable of handling the major Java data types. We then wrap these in a `MongoClientSettings` and create our `MongoClient`.

```java
    @Bean
    public MongoClient mongoClient() {
        CodecRegistry pojoCodecRegistry = CodecRegistries.fromRegistries(
            MongoClientSettings.getDefaultCodecRegistry(),
            CodecRegistries.fromProviders(
                PojoCodecProvider.builder().automatic(true).build()
            )
        );

        MongoClientSettings settings = MongoClientSettings.builder()
            .applyConnectionString(new ConnectionString(MONGODB_URI))
            .codecRegistry(pojoCodecRegistry)
            .build();

        return MongoClients.create(settings);
    }
```

Our last step will then be to get our database, and we're done with this class.

```java
    @Bean
    public MongoDatabase mongoDatabase(MongoClient mongoClient) {
        return mongoClient.getDatabase(MONGODB_DATABASE); 
    }
}
```

### Embed your data with the OpenAI API

We are going to send the prompt given from the user to the OpenAI API to be embedded.
An embedding is a series (vector) of floating point numbers. The distance between two vectors measures their relatedness. Small distances suggest high relatedness and large distances suggest low relatedness.

This will transform our natural language prompt, such as `"Toys that come to life when no one is looking"`, to a large array of floating point numbers that will look something like this `-0.012670076, -0.008900887, ..., 0.0060262447, -0.031987168]`.

In order to do this, we need to create a few files. All of our code to interact with OpenAI will be contained in our `OpenAIService.java` class and go to `com.example.mdbvectorsearch.service`. The `@Service` at the top of our class dictates to Spring Boot that this belongs to this service layer and contains business logic. 

```java
@Service
public class OpenAIService {

private static final String OPENAI_API_URL = "https://api.openai.com";

@Value("${openai.api.key}")

private String OPENAI_API_KEY;

private WebClient webClient;

@PostConstruct
void init() {
this.webClient = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector())
.baseUrl(OPENAI_API_URL)
.defaultHeader("Content-Type", MediaType.APPLICATION_JSON_VALUE)
.defaultHeader("Authorization", "Bearer " + OPENAI_API_KEY)
.build();
}

public Mono> createEmbedding(String text) {
Map body = Map.of(
"model", "text-embedding-ada-002",
"input", text
);

return webClient.post()
.uri("/v1/embeddings")
.bodyValue(body)
.retrieve()
.bodyToMono(EmbeddingResponse.class)
.map(EmbeddingResponse::getEmbedding);
}
}
```

We use the Spring WebClient to make the calls to the OpenAI API. We then create the embeddings. To do this, we pass in our text and specify our embedding model (e.g., `text-embedding-ada-002`). You can read more about the OpenAI API parameter options [in their docs.

To pass in and receive the data from the Open AI API, we need to specify our models for the data being received. We're going to add two models to our `com.example.mdbvectorsearch.model` package, `EmbeddingData.java` and `EmbeddingResponse.java`.

```java
public class EmbeddingData {
private List embedding;

public List getEmbedding() {
return embedding;
}

public void setEmbedding(List embedding) {
this.embedding = embedding;
}
}
```

```java
public class EmbeddingResponse {
private List data;

public List getEmbedding() {
return data.get(0).getEmbedding();
}

public List getData() {
return data;
}

public void setData(List data) {
this.data = data;
}
}
```

### Your vector search aggregation pipeline in Spring Boot

We have our database. We are able to embed our data. We are ready to send and receive our movie documents. How do we actually perform our semantic search?

The data access layer of our API implementation takes place in the repository. Create a package `com.example.mdbvectorsearch.repository` and add the interface `MovieRepository.java`.

```java
public interface MovieRepository {
    Flux findMoviesByVector(List embedding);
}
```

Now, we implement the logic for our `findMoviesByVector` method in the implementation of this interface. Add a class `MovieRepositoryImpl.java` to the package. This method implements the data logic for our application and takes the embedding of user's inputted text, embedded using the OpenAI API, then uses the `$vectorSearch` aggregation stage against our `embedded_movies` collection, using the index we set up earlier.

```java
@Repository
public class MovieRepositoryImpl implements MovieRepository {

    private final MongoDatabase mongoDatabase;

    public MovieRepositoryImpl(MongoDatabase mongoDatabase) {
        this.mongoDatabase = mongoDatabase;
    }

    private MongoCollection getMovieCollection() {
        return mongoDatabase.getCollection("embedded_movies", Movie.class);
    }

    @Override
    public Flux findMoviesByVector(List embedding) {
        String indexName = "PlotVectorSearch";
        int numCandidates = 100;
        int limit = 5;

        List pipeline = asList(
            vectorSearch(
                fieldPath("plot_embedding"),
                embedding,
                indexName,
                numCandidates,
                limit));

        return Flux.from(getMovieCollection().aggregate(pipeline, Movie.class));
    }
}

```

For the business logic of our application, we need to create a service class. Create a class called `MovieService.java` in our `service` package.

```java
@Service
public class MovieService {

    private final MovieRepository movieRepository;
    private final OpenAIService embedder;

    @Autowired
    public MovieService(MovieRepository movieRepository, OpenAIService embedder) {
        this.movieRepository = movieRepository;
        this.embedder = embedder;
    }

    public Mono> getMoviesSemanticSearch(String plotDescription) {
        return embedder.createEmbedding(plotDescription)
                .flatMapMany(movieRepository::findMoviesByVector)
                .collectList();
    }
}
```

The `getMoviesSemanticSearch` method will take in the user's natural language plot description, embed it using the OpenAI API, perform a vector search on our `embedded_movies` collection, and return the top five most similar results.

This service will take the user's inputted text, embed it using the OpenAI API, then use the `$vectorSearch` aggregation stage against our `embedded_movies` collection, using the index we set up earlier.

This returns a `Mono` wrapping our  list of `Movie` objects. All that's left now is to actually pass in some data and call our function. 

We’ve got the logic in our application. Now, let’s make it an API! First, we need to set up our controller. This will allow us to take in the user input for our application. Let's set up an endpoint to take in the users plot description and return our semantic search results. Create a `com.example.mdbvectorsearch.service` package and add the class `MovieController.java`.

```java
@RestController
public class MovieController {

private final MovieService movieService;

@Autowired
public MovieController(MovieService movieService) {
this.movieService = movieService;
}

@GetMapping("/movies/semantic-search")
public Mono> performSemanticSearch(@RequestParam("plotDescription") String plotDescription) {
return movieService.getMoviesSemanticSearch(plotDescription);
}
}
```
We define an endpoint `/movies/semantic-search` that handles get requests, captures the `plotDescription` as a query parameter, and delegates the search operation to the `MovieService`.

You can use your favorite tool to test the API endpoints but I'm just going to send a cURL command. 

```console

curl -X GET "http://localhost:8080/movies/semantic-search?plotDescription=A%20cop%20from%20china%20and%20cop%20from%20america%20save%20kidnapped%20girl"

```
>Note: We use `%20` to indicate spaces in our URL.

Here we call our API with the query, `"A cop from China and a cop from America save a kidnapped girl"`. There's no title in there but I think it's a fairly good description of a particular action/comedy movie starring Jackie Chan and Chris Tucker. Here's a slightly abbreviated version of my output. Let's check our results!

```markdown
Movie title: Rush Hour
Plot: Two cops team up to get back a kidnapped daughter.

Movie title: Police Story 3: Supercop
Plot: A Hong Kong detective teams up with his female Red Chinese counterpart to stop a Chinese drug czar.
  
Movie title: Fuk sing go jiu
Plot: Two Hong-Kong cops are sent to Tokyo to catch an ex-cop who stole a large amount of money in diamonds. After one is captured by the Ninja-gang protecting the rogue cop, the other one gets ...
  
Movie title: Motorway
Plot: A rookie cop takes on a veteran escape driver in a death defying final showdown on the streets of Hong Kong.
  
Movie title: The Corruptor
Plot: With the aid from a NYC policeman, a top immigrant cop tries to stop drug-trafficking and corruption by immigrant Chinese Triads, but things complicate when the Triads try to bribe the policeman.
```

We found *Rush Hour* to be our top match. Just what I had in mind! If its premise resonates with you, there are a few other films you might enjoy.

You can test this yourself by changing the `plotDescription` we have in the cURL command.

## Conclusion

This tutorial walked through the comprehensive steps of creating a semantic search application using MongoDB Atlas, OpenAI, and Spring Boot. 

Semantic search offers a plethora of applications, ranging from sophisticated product queries on e-commerce sites to tailored movie recommendations. This guide is designed to equip you with the essentials, paving the way for your upcoming project. 

Thinking about integrating vector search into your next project? Check out this article — How to Model Your Documents for Vector Search — to learn how to design your documents for vector search.

  [1]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltda8a1bd484272d2c/656d98d6d28c5a166c3e1879/image2.png
  [2]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt9570de3c5dcf3c0f/656d98d6ec7994571696ad1d/image6.png
  [3]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt531721a1672757f9/656d98d6d595490c07b6840b/image4.png
  [4]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt63feb14bcd48bc33/656d98d65af539247a5a12e5/image3.png
  [5]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/blt29c9e89933337056/656d98d6d28c5a4acb3e1875/image1.png
  [6]: https://images.contentstack.io/v3/assets/blt39790b633ee0d5a7/bltfcb95ac6cfc4ef2b/656d98d68d1092ce5f56dd73/image5.png