<figure><img src="image_1.png" /></figure>

HAL
open science

# Performance Evaluation of MongoDB I/O access patterns

Abdulqawi Saif, Lucas Nussbaum, Ye-Qiong Song

► To cite this version:

Abdulqawi Saif, Lucas Nussbaum, Ye-Qiong Song. Performance Evaluation of MongoDB I/O access patterns. CLOUD DAYS'2017, Action transverse Virtualisation et Cloud du GdR RSD Sep 2017, Nancy, France. hal-01793479

HAL Id: hal-01793479

https://inria.hal.science/hal-01793479

Submitted on 16 May 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
Performance Evaluation of MongoDB I/O access patterns

Abdulqawi Saif¹,²,³,⁴

abdulqawi.saif@loria.fr

Lucas Nussbaum¹,²,³

lucas.nussbaum@loria.fr

Ye-Qiong Song¹,²,³

ye-qion song@loria.fr

¹Inria, Villers-les-Nancy, F-54600, France

²Université de Lorraine, LORIA, F-54500, France

³CNRS, LORIA - UMR 7503, F-54500, France

⁴Xilopix SAS, F-88000, France

September 07, 2017

CLOUD DAYS'2017 - Nancy, France
## NoSQL databases

* Principle category of Big Data Storage

* Overcome the scalability issues of SQL DBs

* Almost Cloud-friendly systems

* Different architectures and data-models

<figure><img src="image_2.png" /></figure>

## MongoDB

* A document-based NoSQL database (JSON documents)

* It uses sharding for distributing data

* Different tools for easy-deployment and migration
  (Ex. mongorestore, mongodump)

* Simple to use and fit well with various kinds of apps.
Querying data

■ NoSQL flexibility: store data; think how to use after!

■ No schema ⇒ not only one way to query the data

frequent need to **create indexes on demand, on large-scale data!**
Querying data

■ NoSQL flexibility: store data; think how to use after!

■ No schema ⇒ not only one way to query the data

frequent need to **create indexes on demand, on large-scale data!**

Problem statement

■ MongoDB takes unjustified amount of time for making indexes on a pre-existed data

this problem could affect other NoSQL databases!
Benchmarking?

⇒ Tools such as *Yahoo! Cloud Serving Benchmarks (YCSB)* could:

- report high level metrics (Ex. throughput, latency, ...) ✓

- reduce the effort for comparing different systems ✓

But

- could not reflect the complexity of production-like data 😂

- are not suitable for explaining performance issues 😂

Research objectives

- Identifying the performance issue(s) of MongoDB while indexing data

- Introducing new experimental tools for explaining the causes of performance issue(s)
Datasets

<table><thead><tr><td>Dataset type</td><td>(min, avrg, max) in Kb</td><td>N. of data units</td><td>Size (Gb)</td></tr></thead><tbody><tr><td>SDU<sub>1</sub></td><td>(1, 3, 6)</td><td>20,000,000 docs</td><td>71</td></tr></tbody></table>

* Every doc x (int, date, 2 x string(min, max), array[1..4] x string(min, max))

MongoDB's storage engine: Wiredtiger

* The default storage engine for persistent data.

* `_ids` for holding the collection's records

* collection is a separate file

* Btree data structure on disks

* Contiguous allocation on disk, but **unspanned**

* Default page size is 4 KB

<figure><img src="image_3.png" /><figcaption>FIGURE - a) Spanned & b) Unspanned allocation</figcaption></figure>
## Experimental setup

* Experiments are performed on Grid'5000 testbed
Machines x 2 CPUs Intel Xeon E5-2630 v3, 8 cores/CPU, 128GB
RAM, 2x558GB HDD, 10Gbps ethernet

* Ubuntu 14.04, Linux 4.9.13, MongoDB v3.4

* wiredTiger cache = 2Gb, replicas are disabled, sharding key is `_id<sub>s</sub>`

* Considering only **hash sharding** in this talk ⇒ load balancing

<figure><img src="image_4.png" /><figcaption>FIGURE - Experimental scenarios</figcaption></figure>
<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>
Experiments : Performance results with SDU₁

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>
Experiments : Performance results with SDU₁

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>
Experiments : Performance results with SDU1

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

⇒ Indexing is a 100% read workload

⇒ I/O bounded operations

⇒ Data distribution issue

⇒ Benchmarking is not sufficient to look beyond!
Experiments : Performance results with SDU₁

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

⇒ Indexing is a 100% read workload

⇒ I/O bounded operations

⇒ Data distribution issue

⇒ Benchmarking is not sufficient to look beyond!
Experiments : Performance results with SDU1

<figure><img src="image_25.png" /></figure>

⇒ How to get the main reason behind these results ?
## extended Berkeley Packet Filter (eBPF)

* Connect to all Linux data sources: (Kprobes, Uprobes, tracepoints, ...)

* Almost no overhead (4 ns per syscall) [A. Starovoitov]

* In-kernel verifier, JIT-compilation, mapping (kernel-userspace exchange)

* Processing, tracing, filtering inside Linux kernel.

* Suitable to work with system in-production & real data.

## BPF compiler collection (BCC)

* Like-standard frontend project for eBPF tools

* No more byte code! ⇒ write your BPF code in restricted C

* Write your frontend code using Python or Lua

* Towards script-driven tracing
## An eBPF tool is built to evaluate the I/O patterns

* I/O access are no longer considered as black boxes!

* A generic tool, works on VFS layer

* It traces all I/O read requests

* It reports all files offsets, req. latencies & data size

* Could filter the results by file

<figure><img src="image_26.png" /><figcaption>FIGURE - Linux abstracted I/O stack</figcaption></figure>

## Tracing impact on our experiments

* Overhead cost is lower than 0.8% of execution time in all Experiments. 😊

* Basically, it depends on the number issued I/O requests.
Experiments : eBPF results with SDU1

<figure><img src="image_27.png" /><figcaption>I/O access pattern and latency on standalone config.</figcaption></figure>
Experiments : eBPF results with SDU₁

<figure><img src="image_28.png" /></figure>

I/O access pattern and latency on standalone config.
Experiments : eBPF results with SDU₁

<figure><img src="image_29.png" /><figcaption>I/O access pattern and latency on standalone config.</figcaption></figure>

<figure><img src="image_30.png" /><figcaption>First shard results in two-shards config (50.9% of data)</figcaption></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

→ Acceptable access pattern on standalone

→ Almost **random** access on shards!

→ Data distribution is poorly done

<figure><img src="image_33.png" /><figcaption>Second shard results in two-shards config (49.1% of data)</figcaption></figure>

<figure><img src="image_34.png" /></figure>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>

FIGURE - MongoDB scanning table Vs records order on the disk
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><thead><tr><th>_id 1</th><th>_id 2</th><th>_id 3</th><th>_id 5</th></tr></thead><tbody><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></tbody></table>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><thead><tr><th>_id 1</th><th>_id 3</th><th>_id 7</th><th>_id 5</th><th>_id 8</th><th>_id 10</th><th>_id 2</th><th>_id 19</th><th>_id 11</th><th>_id 20</th><th>_id 30</th><th>_id 27</th></tr></thead><tbody><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></tbody></table>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><thead><tr><th>_id 1</th><th>_id 2</th><th>_id 3</th><th>_id 5</th></tr></thead><tbody><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></tbody></table>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><thead><tr><th>_id 1</th><th>_id 3</th><th>_id 7</th><th>_id 5</th><th>_id 8</th><th>_id 10</th><th>_id 2</th><th>_id 19</th><th>_id 11</th><th>_id 20</th><th>_id 30</th><th>_id 27</th></tr></thead><tbody><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></tbody></table>

FIGURE - MongoDB scanning table Vs records order on the disk
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_35.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_36.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_37.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_38.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_39.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_42.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_43.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<table><tr><td>_id 1</td><td>_id 2</td><td>_id 3</td><td>_id 5</td></tr><tr><td>_id 7</td><td>_id 8</td><td>_id 10</td><td>_id 11</td></tr><tr><td>_id 19</td><td>_id 20</td><td>_id 27</td><td>...</td></tr></table>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<table><caption>FIGURE - MongoDB scanning table Vs records order on the disk</caption><tr><td>_id 1</td><td>_id 3</td><td>_id 7</td><td>_id 5</td><td>_id 8</td><td>_id 10</td><td>_id 2</td><td>_id 19</td><td>_id 11</td><td>_id 20</td><td>_id 30</td><td>_id 27</td></tr><tr><td>rec 1</td><td>rec 2</td><td>rec 3</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>rec(n-1)</td><td>rec (n)</td></tr></table>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_44.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_45.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_46.png" /></figure>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_47.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_48.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_49.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_50.png" /></figure>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_51.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

* Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_52.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_53.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_54.png" /></figure>

1) Get this doc

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_55.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
## The exact issue

- Mismatch between the scanning table vs data stored on disk

Collection_ids:

used by MongoDB process and WiredTiger

<figure><img src="image_56.png" /></figure>

1) Get this doc

2) Which doc is next?

Collection file on disk:

allocated regarding the key sharding (hashed_id)

<figure><img src="image_57.png" /><figcaption>FIGURE - MongoDB scanning table Vs records order on the disk</figcaption></figure>
MongoDB integrated tools?

■ RepairDatabase ⇒ verifying the coherence of data. 😊

■ Rebuilding the `_id` index ⇒ consider the pre-existed ordering. 😊
MongoDB integrated tools?

■ RepairDatabase ⇒ verifying the coherence of data. 😤

■ Rebuilding the `_id` index ⇒ consider the pre-existed ordering. 😤

Our ad-hoc solution

■ recreating the `_id` index regarding how the data is stored on disk.
MongoDB integrated tools?

■ RepairDatabase ⇒ verifying the coherence of data. 😤

■ Rebuilding the `_id` index ⇒ consider the pre-existed ordering. 😤

Our ad-hoc solution

■ recreating the `_id` index regarding how the data is stored on disk.

The worst case example!

■ Data on one shard in a *four conf.* experiment (1/4 out of data)

■ indexing time : Doubling the indexing time of the *whole data* on standalone config (3000 sec).

■ indexing time after : six time faster! (about 500 sec)

<figure><img src="image_58.png" /></figure>
MongoDB integrated tools?

■ RepairDatabase ⇒ verifying the coherence of data. 😤

■ Rebuilding the `_id` index ⇒ consider the pre-existed ordering. 😤

Our ad-hoc solution

■ recreating the `_id` index regarding how the data is stored on disk.

The worst case example!

■ Data on one shard in a **four conf. experiment** (1/4 out of data)

■ indexing time : Doubling the indexing time of the **whole data** on standalone config (3000 sec).

■ indexing time after : six time faster! (about 500 sec)

<figure><img src="image_59.png" /><figcaption>FIGURE - I/O access pattern a) before & b) after applying the solution</figcaption></figure>
## Conclusion

* A performance study on MongoDB I/O access pattern is done

* A generic tool for testing I/O access patterns is introduced

* An ad hoc solution is proposed to correct MongoDB I/O access patterns

* We showed how a trivial data management issue could affect the performance

* We demonstrated how it is worthy to use dynamic tracing to go beyond benchmarking results

## Future works

* Including other solutions like Cassandra in those tests.

* Trying to enrich eBPF tools which explain specific issues.
## Conclusion

* A performance study on MongoDB I/O access pattern is done

* A generic tool for testing I/O access patterns is introduced

* An ad hoc solution is proposed to correct MongoDB I/O access patterns

* We showed how a trivial data management issue could affect the performance

* We demonstrated how it is worthy to use dynamic tracing to go beyond benchmarking results

## Future works

* Including other solutions like Cassandra in those tests.

* Trying to enrich eBPF tools which explain specific issues.

Questions are welcome!