Building An Open Source Search App With Appwrite And Meilisearch
Published: 2022-08-04 18:46:00

Imagine you are building an inventory or an ecommerce website or a social media of your own, what are the two major things you will have to deal with? A. Database  B. Search engine

This article deals with building a simple app where we search for movies, to achieve this we use:
**1. Meilisearch**-  for building the search and 
**2. Appwrite** - for the backend to store data and run functions to automate the search

---
# What is Appwrite?
[Appwrite](https://appwrite.io/) is a self-hosted backend-as-a-service platform that provides developers with all the core APIs required to build any application. Appwrite provides you with a set of APIs, tools, and a management console UI to help you build your apps a lot faster and in a much more secure way. Appwrite takes the heavy lifting for developers and handles user authentication and authorization, databases, file storage, cloud functions, webhooks, and much more!

---
# What is Meilisearch?
[Meilisearch](https://www.meilisearch.com/) is a powerful, fast, open-source, easy to use and deploy search engine. Both searching and indexing are highly customizable. Features such as typo-tolerance, filters, and synonyms are provided out-of-the-box. For more information about features go to our documentation

---
# Prerequisites
Now that you have been introduced to Appwrite and Meilisearch, let’s start building with it. Before that, let’s make sure you have everything we need. Here’s the prerequisites:

1. An IDE of your choice
2. Latest version of [Node.js](https://nodejs.org/en/download/)
3. [Docker](https://docs.docker.com/get-docker/) 
4. [Appwrite installed](https://appwrite.io/docs/installation)
5. [Meilisearch installed](https://docs.meilisearch.com/learn/cookbooks/running_production.html#step-1-install-meilisearch)

---

# Building The Application
 Let's get started building! Are you ready?

## Getting Started With Meilisearch
For this demo, we used the [meilisearch quickstart tutorial](https://docs.meilisearch.com/learn/getting_started/quick_start.html) as a reference.

To install Meilisearch on your machine you can do it using Docker or a package manager depending on your OS.

Alternatively, you can also deploy Meillisearch on a [cloud service](https://docs.meilisearch.com/learn/getting_started/quick_start.html#setup-and-installation)

_Note:_ _Since I am on a Mac machine I used homebrew to install Meilisearch with the following command:_

```
brew update && brew install meilisearch
```

---

## Getting Started With Appwrite

To install Appwrite you need to have Docker installed and then run the following command to install:
```
docker run -it --rm \ --volume /var/run/docker.sock:/var/run/docker.sock \ --volume "$(pwd)"/appwrite:/usr/src/code/appwrite:rw \ --entrypoint="install" \ appwrite/appwrite:0.15.3
```
Alternatively, you can also go for [one-click setups](https://appwrite.io/docs/installation#one-click-setups) using Gitpod or Digital Ocean.

---

## Creating a Project
- After installing, Appwrite will be running on localhost and you should see a screen like this:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wv2ap347oros3jtykhci.png)
 

- Clicking on `Create Project`, enter `Demo` as the name and click on `Create`.

This is how your Appwrite console will look:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e3hdgyksdz03l63qftpk.png)
 

Next, let's set up the database and collection for the search application.

---

### Creating a Database and Collection

A database in Appwrite is a group of collections for managing data. To create a database, visit the Database section:

- Click on `Create Database`
- Enter `Movies` as the name
- Click Create
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mi4kztyinegexs6vi223.png)

**For the collection:**
- Click on Create Collection
- Enter `Movie details` as the name
- Click Create

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k5s9q494ubhm2chf50j6.png)
 

We also want to configure permissions for the collection for read/write access. For `Movie details`, you’ll choose Document Level permissions. You can choose more granular permissions depending on your use case. The [permissions page](https://appwrite.io/docs/permissions) has more details on permissions so the user keeps ownership of their data.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h58pv6iimvmk9v40s8iv.png)
 

---

## Creating Document Attributes
Attributes can be defined as strings, numbers, emails, and more. 
To create an attribute:
- Click on `Create Attribute`.
- Select the type of Attribute to create.
- Use the table below to create the necessary attributes for chat.

| Attribute id | Size | 
|------------|----------|
| title      | 255      |
| id         | 255      | 
| overview   | 10000    | 
| genres     | 255      | 


_Note- The attribute type will be string for all the attributes_

---

## Uploading Data to Appwrite

The movie data that we will use for this example is [movies.json](https://github.com/Haimantika/Meili-integration/blob/master/movies.json). 

To upload it in Appwrite database we will be using a script.

Make a new `dataWriter.js` file and paste the following code:
```js
const { Client, Databases } = require("node-appwrite");
const fs = require("fs");
const movies = require("./movies.json");
const dotenv = require('dotenv');
 
// Init SDK
let client = new Client();
dotenv.config();
 
client
 .setEndpoint(process.env.APPWRITE_ENDPOINT) // Your API Endpoint
 .setProject(process.env.APPWRITE_PROJECTID) // Your project ID
 .setKey(process.env.APPWRITE_KEY) // Your secret API key
 .setSelfSigned(); // Use only on dev mode with a self-signed SSL cert
 
let databases = new Databases(client, "YOUR_DATABASE_ID");
 
//let promise = database.getDocument('627e9b5e008ffd8382ff', '6283ccb7773a9474319d')
// Run this until you are happy with the number of documents
const movieWrite = async () => {
 console.log('starting addition of documents to db')
 
 for (const movie of movies) {
   const response = await databases.createDocument("YOUR_COLLECTION_ID", String(Math.random()), {
     title: String(movie.title || Math.random()),
     id: String(movie.id || Math.random()),
     overview: String(movie.overview || Math.random()),
     genres: String(movie.genres || Math.random()),
   })
   console.log(`Added ${movie.title}`, response);
   await new Promise(resolve => setTimeout(resolve, 100));
 }
}
 
movieWrite();
```

---

### Storing your Appwrite secrets

Make a .env file as the following and put in your Appwrite secrets there, make sure to `git ignore` this file while committing :

```
APPWRITE_ENDPOINT=http://localhost/v1
APPWRITE_PROJECTID=meili-example
APPWRITE_KEY=find-this-in-project-settings
```

_Note - You will get Appwrite secrets from `Settings` in the left pane and you need to generate an API KEY from the `API Keys` option on the left._

---

## Creating an Appwrite Function

To create Appwrite function, we use the [Appwrite CLI](https://appwrite.io/docs/command-line) for easy deployment.

- Login to your Appwrite account using the command `appwrite login`
- Enter your existing Appwrite project using the command `appwrite init project` and choose the project you want to.
- Create a new [node.js function](https://appwrite.io/docs/functions#writingYourOwnFunction) or [fork our repository](https://github.com/Haimantika/Meili-integration) and use the command `appwrite deploy function`
- Choose the `add-to-meilisearch` function and it will be added to your project

Head on to your Appwrite console and click on `Functions`, this is what you will see:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4agmsnmh6bef1odrummj.png)

---

## Configuring the Appwrite Function

Now that the function is ready on Appwrite console, to execute it, we need to add some variable.

- Go to `Settings`
- Scroll down to `Variables` and fill in data according to the below table:

| Key| Value| 
|------------|----------|
| APPWRITE_FUNCTION_ENDPOINT     | http://yourip/v1     |
| APPWRITE_FUNCTION_PROJECT_ID       | Put in your function id   | 
| APPWRITE_FUNCTION_API_KEY   | Put in your API KEY  | 
| APPWRITE_MEILISEARCH_HOST    | http://yourip:7700   | 
| APPWRITE_MEILISEARCH_KEY    | Put in the ADMIN API KEY that you get  from [here](https://docs.meilisearch.com/reference/api/keys.html#get-all-keys)     | 

- Once the above variables are set, your function is ready to run

---

## Building The Search App

To build the frontend of the app we use the following code, in an `index.html` file:

```html
<!DOCTYPE html>
<html lang="en">
 <head>
   <meta charset="utf-8" />
   <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@meilisearch/instant-meilisearch/templates/basic_search.css" />
 </head>
 <body>
   <div class="wrapper">
     <div id="searchbox" focus></div>
     <div id="hits"></div>
   </div>
 </body>
 <script src="https://cdn.jsdelivr.net/npm/@meilisearch/instant-meilisearch@0.8.1/dist/instant-meilisearch.umd.min.js"></script>
 <script src="https://cdn.jsdelivr.net/npm/instantsearch.js@4"></script>
 <script>
   const search = instantsearch({
     indexName: "movie",
     searchClient: instantMeiliSearch(
       "http://206.189.206.205",
       "3ca53aec3885eb48b4d430607dabc5515ea10e4d025319a94eccb6f8d6931823"
     )
     });
     search.addWidgets([
       instantsearch.widgets.searchBox({
         container: "#searchbox"
       }),
       instantsearch.widgets.configure({ hitsPerPage: 8 }),
       instantsearch.widgets.hits({
         container: "#hits",
         templates: {
         item: `
           <div>
           <div class="hit-name">
                 {{#helpers.highlight}}{ "attribute": "title" }{{/helpers.highlight}}
           </div>
           </div>
         `
         }
       })
     ]);
     search.start();
 </script>
</html>
```

--- 

## Fetching Data From Appwrite To MeiliSearch

This the final step of the demo, where we bring in all the bits and pieces from above. So far, we have:

1. A database up and running in Appwrite - it contains the data that we will use to search the contents
2. A meilisearch instance running, ready to search
3. A simple frontend UI
4. An Appwrite Function running in background which we use to search for movies

In this step, we will trigger the Appwrite function which will fetch data from Appwrite and Meilisearch will use this data to search in our simple application that we built in the above step.

To able able to do that, an additional step will be required in the function:
- Adding `database.create` event, and adding it to your [json](https://github.com/Haimantika/Meili-integration/blob/master/appwrite.json) file, so that every new data added shows up in the search result

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tsd8780dwo1gwlqmuejj.png)
 
 
---


## Testing the Application

Before testing the application, make sure you have the following things ready:

1. Appwrite up and running
2. Meilisearch running
3. The frontend built using the code in `index.html`
4. The `dataWrite.js` file ready to upload data to Appwrite
5. An .env file containing all Appwrite secrets
6. Appwrite function ready 

Now you need to run the ‘dataWriter.js` file and the Appwrite function running in the background will be pushing the data, which Meilisearch will use to search.

You should see the result on: http://localhost:7700


---

## Ending Notes
If you want to build your own search app, you can find the entire code [here](https://github.com/Haimantika/Meili-integration)

Liked what we built? Here's some resources:
- [Appwrite Docs](https://appwrite.io/docs)
- [Appwrite GitHub](https://github.com/appwrite/appwrite)
- [Meilisearch Docs](https://docs.meilisearch.com/)
- [Meilisearch GitHub](https://github.com/meilisearch/meilisearch)