MongoDB Write Concern: 3 Must-Know Caveats
Tags: mongodb, writeconcern, nosql, replicas
Published: 2018-07-25 18:02:26

---
title: MongoDB Write Concern: 3 Must-Know Caveats
published: true
description: Write Concern in MongoDB describes the level of write acknowledgment you can expect from a MongoDB database instance. It's a rather important setting to remember in your write operations and useful 
tags: MongoDB, Write Concern, NoSQL, Replicas,
canonical_url: https://scalegrid.io/blog/mongodb-write-concern-3-must-know-caveats/
---
<link rel="canonical" href="https://scalegrid.io/blog/mongodb-write-concern-3-must-know-caveats/" />

<img src="https://uploads.disquscdn.com/images/125e5ddc69932d0f2b8972bd59962a00965e6cb6ed359e27007570ff88239555.png" width="100%" height="auto">
<p style="text-align: justify;"><a href="https://docs.mongodb.com/manual/reference/write-concern/" target="_blank">'Write concern'</a> in <a href="https://scalegrid.io/mongodb.html" target="_blank">MongoDB</a> describes the level of write acknowledgment you can expect from it. It's a rather important setting to remember in your write operations and its behavior is useful to understand, especially in distributed MongoDB deployments (i.e. <a href="https://docs.mongodb.com/manual/reference/glossary/#term-replica-set" target="_blank">replica sets</a> and <a href="https://docs.mongodb.com/manual/reference/glossary/#term-sharded-cluster" target="_blank">sharded clusters</a>). In this post, we discuss 3 gotchas when using MongoDB write concern.</p>

<p style="text-align: justify;">Read the original: <a href="https://scalegrid.io/blog/mongodb-write-concern-3-must-know-caveats/" target="_blank">MongoDB Write Concern: 3 Must-Know Caveats</a>

<h2>MongoDB Write Concern</h2>
<p style="text-align: justify;"><a href="https://docs.mongodb.com/manual/reference/write-concern/" target="_blank">MongoDB's documentation</a> defines write concern as "<em>the level of acknowledgment requested from MongoDB for write operations to a standalone mongod or to replica sets or to sharded clusters.</em>"</p>
<p style="text-align: justify;">Simply put, a write concern is an indication of 'durability' passed along with write operations to MongoDB. To clarify, let us look at the syntax:</p>

<pre>{ w: &lt;value&gt;, j: &lt;boolean&gt;, wtimeout: &lt;number&gt; }
Where*,
 <strong>w</strong> can be an integer | "majority" | , it represents the number of members that must acknowledge the write. <strong>Default value is 1.</strong>
 <strong>j</strong> Requests that a write be acknowledged after it is written to the on-disk journal as opposed to just the system memory. Unspecified by default.
<strong>wtimeout</strong> specifies timeout for the applying the write concern. Unspecified by default.</pre>
<p style="text-align: justify;">* You can find the detailed syntax in the <a href="https://docs.mongodb.com/manual/reference/write-concern/#write-concern-specification" target="_blank">Write Concern Specification documentation</a>.</p>
<p style="text-align: justify;">* Learn more about the different "tags" you can use for common write concern values in our <a href="https://scalegrid.io/blog/understanding-durability-write-safety-in-mongodb/" target="_blank">Understanding Durability &amp; Write Safety in MongoDB</a> blog.</p>
Example:
<pre>db.inventory.insert(
    { sku: "abcdxyz", qty : 100, category: "Clothing" },
    { writeConcern: { w: 2, j: true, wtimeout: 5000 } }
)</pre>
<p style="text-align: justify;">The above insert's write concern can be read as follows:  acknowledge this write when 'at least 2 members of the replica set have written it to their journals within 5000 msecs or return an error'. A write concern value for option was <a href="https://docs.mongodb.com/manual/reference/write-concern/#writeconcern._dq_majority_dq_" target="_blank">majority, </a>meaning "r<em>equests acknowledgment that write operations have propagated to the majority of <a href="https://docs.mongodb.com/manual/reference/write-concern/#majority-definition" target="_blank">voting nodes</a>, including the primary.</em>"</p>

<p style="text-align: justify;">The importance of write concern is apparent. Increasing values of w increases the latency of writes while also decreasing their probability of <a href="https://docs.mongodb.com/manual/core/replica-set-rollbacks/" target="_blank">getting lost.</a> Choosing the correct values for write concern depends on the latency and durability requirements of writes being performed.</p>

<p style="text-align: justify;">With that as the background on what a write concern is, let's move on to the three caveats to remember when using write concern.</p>


<h3 style="padding:20px 0px 10px 0px;line-height:1.5em;"><span style="color: #2da964;"><span style="text-decoration: underline;">CAVEAT 1</span>: </span>Setting write concern on replica sets without a wtimeout can cause writes to block indefinitely</h3>
<p style="text-align: justify;">The <a href="https://docs.mongodb.com/manual/reference/write-concern/#majority-definition" target="_blank">majority definition</a> (applicable MongoDB 3.0 onwards) above states that acknowledgment is requested from a majority of the "voting nodes". <a href="https://docs.mongodb.com/manual/reference/write-concern/#wtimeout" target="_blank">Note</a> that <em>"If you do not specify the <code class="docutils literal"><span class="pre">wtimeout</span></code> option and the level of write concern is unachievable, the write operation will block indefinitely. "</em></p>
<p style="text-align: justify;">This can have unexpected consequences, for example, consider a 2+1 replica set (i.e. a primary, a secondary and an arbiter). If your sole read replica goes down, then all writes with a write concern w option of "majority" will block indefinitely.  The same will happen if the w option is set to 2. Another extreme example is in the case of a 3+2 replica set (primary, 2 secondaries and 2 arbiters, <a href="https://docs.mongodb.com/manual/tutorial/add-replica-set-arbiter/#add-an-arbiter">not </a>a recommended configuration). All "majority" writes will block even if a single data node is unavailable as the majority number, in this case, is 3.</p>
<p style="text-align: justify;">The simplest way to alleviate this issue is to always specify a wtimeout value so the query can timeout if the write concern can't be enforced. However, in case of such timeout errors, MongoDB <a href="https://docs.mongodb.com/manual/reference/write-concern/#wtimeout" target="_blank">doesn't undo</a> already successful writes made to some of the members before the timeout occurred.</p>
<p style="text-align: justify;">There is also currently no setting to ensure a write reaches the majority of nodes that are currently reachable, so be careful about setting the value of write concern w based on the topology, desired durability, and availability.</p>

<h3 style="padding:20px 0px 10px 0px;line-height:1.5em;"><span style="color: #2da964;"><span style="text-decoration: underline;">CAVEAT 2</span>: </span>You might lose data even with w: majority</h3>
<p style="text-align: justify;">It seems intuitive that once a write has been acknowledged by the majority of voting members, its durability is guaranteed. However, that isn't the case! Remember that when the j option is unspecified, a write is acknowledged right after it has been written to memory.</p>
<p style="text-align: justify;">So, such a write can be lost if a freak power outage takes out the majority of the nodes to which the write had propagated (and before <a href="https://docs.mongodb.com/manual/reference/configuration-options/#storage.syncPeriodSecs" target="_blank">syncPeriodSecs</a> i.e. before it could be flushed to disk).</p>
<p style="text-align: justify;">In order to ensure the durability of writes, it's best not to turn off <a href="https://docs.mongodb.com/manual/core/journaling/" target="_blank">journaling</a> on your database and set the j option to true. In fact, starting MongoDB 3.6, the <tt>--nojournal</tt> flag has been <a href="https://docs.mongodb.com/manual/reference/program/mongod/#cmdoption-mongod-nojournal">deprecated</a> for replica set members using the WiredTiger storage engine.</p>
<p style="text-align: justify;">With a w value of "majority" and the j option unspecified on a replica set, the exact durability behavior depends on the value of the replica set configuration <a href="https://docs.mongodb.com/manual/reference/replica-configuration/#rsconf.writeConcernMajorityJournalDefault" target="_blank">writeConcernMajorityJournalDefault</a>. When set to true (and when <a href="https://docs.mongodb.com/manual/core/journaling/" target="_blank">journaling</a> is enabled), it acknowledges writes after they have been written to the journals of a majority of voting members.</p>
<p style="text-align: justify;">Aside: Even with <a href="https://scalegrid.io/blog/should-you-enable-mongodb-journalling/">journaling turned on, </a>your writes might still get lost on the <a href="https://docs.mongodb.com/manual/core/journaling/#journaling-and-the-mmapv1-storage-engine" target="_blank">MMAPv1 storage engine</a> if an outage occurs within <a href="https://docs.mongodb.com/manual/reference/configuration-options/#storage.journal.commitIntervalMs" target="_blank">commitIntervalMs</a> duration. The WiredTiger storage engine, on the other hand, <a href="https://docs.mongodb.com/manual/core/journaling/#journaling-process" target="_blank">forces a sync of journal files</a> when it receives a write with j option set to true. And, even with j set to false, an acknowledged "majority" write to a <a href="https://docs.mongodb.com/manual/core/journaling/#journaling-process" target="_blank">latest WiredTiger based deployment </a>can be lost only when majority of the data nodes crash simultaneously.</p>

<h3 style="padding:20px 0px 10px 0px;line-height:1.5em;"><span style="color: #2da964;"><span style="text-decoration: underline;">CAVEAT 3</span>: </span>w: 0 while setting j: true doesn't improve write performance</h3>
<p style="text-align: justify;">This is easy enough to reason once you think about it, but equally easy to forget. Setting w option to 0 is usually done to write to the database in a "fire-and-forget" fashion - when you have a fair amount of confidence on the database infrastructure and care more about latency than the durability of every write. However, if you set the j option to true, your w option will effectively be overridden as the database will ensure that the write is written to the on-disk journal before returning.</p>
<p style="text-align: justify;">If you're using write concerns to guarantee the success of your write operations, make sure that you remember these three crucial caveats! We're here to help, so feel free to connect with any questions through <a href="https://twitter.com/scalegridio" target="_blank">Twitter</a> or by <a href="mailto:support@scalegrid.io" target="_blank">email</a>.</p>
&nbsp;