Improve Your MongoDB Performance Using Index Selectivity
By: Tek Loon
Tags: Programming, Mongodb, Software Engineering, DevOps, Database

Performance Experiment

Before we start any experiment, let’s ensure the setup is correct. There are no indexes created for the collection yet except the default _id field.

The experiments I would like to perform here are:

Experiment 1. Evaluate query performance using destination and stop index

and index Experiment 2. How index selectivity affects compound indexes

Experiment 1. Evaluate query performance with created indexes

Before we start the query and evaluate the performance, let’s create an index for the stop field.

Use the command below to create an index for the stop field.

Next, we will see the performance where we query the bookings with destination “Gerlachmouth” and more than one stop. From the screenshot below, we can see that the query performance is not efficient, as we are scanning through 262K index keys and documents, and in the end, it only returned 12K documents.

We only need 4% of what we examined, and this is not cool.

Now let’s try indexing the destination field. Use the command below to create an index for the destination field.

Query Performance using Destination Index

The query performance where we use the destination indexes is way more performant compared to using the stop index we created above. Refer to the screenshot above. Now, we're only examing 25K index keys and documents, which is almost 10 times less than the using stop index.

If you think about it, it is acceptable, and pretty common sense too, when we query using destination. The destination is way more specific compared to whether the number of stops is more than one. This is what we call index selectivity in MongoDB.

It means the higher the index selectivity, the easier it is for MongoDB to narrow down the query results equal to a significant improvement to MongoDB performance. With this example, the execution time for this query is 7.6 times faster compared to using the stop index.

Experiment 2. How index selectivity affects compound indexes

Until this stage, you might be thinking that we can resolve this index selectivity by using a compound index. We can create a compound index with both the stop and destination fields.

Let’s try it out by creating a compound index using the command below.

Query Performance Using Compound Index stop_destination

From the screenshot above, the performance and execution time seems excellent compared to the index we created in Experiment 1. However, it is pretty weird that we’re examing four more keys compared to the total documents returned.

You might be thinking it’s only four more keys. But I have seen a scenario where 90K index keys were examined but only 10K documents returned. Thus, this isn’t very nice, and we can fix it using the index selectivity theory.

Let’s move on by creating a compound index based on the strength index selectivity. We start with the strongest selectivity.

We can create index starting with destination then followed by stop using the command below.

From the screenshot below, we achieved the result that index keys examined is equal to the number of returned documents. Although it is just a very small improvement that seems negligible in this example, it’s good practice to create an index according to the index selectivity.