MySQL Tutorial – Understanding The Seconds Behind Master Value
Tags: mysql, tutorial, database, replication
Published: 2019-01-30 18:57:34

---
title: MySQL Tutorial – Understanding The Seconds Behind Master Value
published: true
description: In a MySQL hosting replication setup, Seconds Behind Master (SBM) indicates the slave replication lag. Learn how to interpret this value in various situations.
tags: MySQL, Tutorial, Database, Replication,
canonical_url: https://scalegrid.io/blog/mysql-tutorial-understanding-the-seconds-behind-master-value/
---
<link rel="canonical" href="https://scalegrid.io/blog/mysql-tutorial-understanding-the-seconds-behind-master-value/" />

<a href="https://scalegrid.io/blog/mysql-tutorial-understanding-the-seconds-behind-master-value/"><img class="alignnone size-full wp-image-3938" src="https://scalegrid.io/blog/wp-content/uploads/2019/01/MySQL-Tutorial-Understanding-The-Seconds-Behind-Master-Value-ScaleGrid-Blog.jpg" alt="MySQL Tutorial – Understanding The Seconds Behind Master Value" width="100%" height="auto" /></a>
<p style="text-align: justify;">In a <a style="color: #2da964;" title="MySQL Hosting" href="https://scalegrid.io/mysql.html" target="_blank">MySQL hosting</a> replication setup, the parameter Seconds_Behind_Master (SBM), as displayed by the SHOW SLAVE STATUS command, is commonly used as an indication of the current replication lag of the slave. In this blog post, we examine how to understand and interpret the <a style="color: #2da964;" title="MySQL Hosting" href="https://scalegrid.io/blog/mysql-tutorial-understanding-the-seconds-behind-master-value/">MySQL Seconds Behind Master value</a> in various situations.</p>

<h2 style="padding-top: 15px;">Possible Values of  Seconds Behind Master</h2>
<p style="text-align: justify;">The value of SBM, as explained in the  <a style="color: #2da964;" href="https://dev.mysql.com/doc/refman/5.7/en/show-slave-status.html" target="_blank">MySQL documentation</a>, depends on the state of the MySQL slave in general, and the states of MySQL slave SQL_THREAD and IO_THREAD in particular. While IO_THREAD connects with the master and reads the updates, SQL_THREAD applies these updates on the slave. Let’s examine the possible values of SBM during different states of the MySQL Slave.</p>

<h3 style="padding:10px 0px;">When SBM Value is Null</h3>
<ul>
	<li style="text-align: justify;">SBM is always NULL if your slave is stopped, or your SQL Thread is stopped (or not running).</li>
	<li style="text-align: justify;">SBM will also be NULL if the IO Thread is stopped, provided the SQL Thread has already processed all events from the relay log. A sample output of SHOW SLAVE STATUS (trimmed to show only values of interest) demonstrates this:</li>
</ul>
<p style="padding-left: 40px; padding-top: 15px;">Slave_IO_State:</p>
<p style="padding-left: 60px;">Master_Host: 172.19.0.13</p>
<p style="padding-left: 60px;"><span style="color: #e97b00;">Slave_IO_Running: No</span></p>
<p style="padding-left: 60px;">Slave_SQL_Running: Yes</p>
<p style="padding-left: 60px;"><span style="color: #e97b00;">Seconds_Behind_Master: NULL</span></p>
<p style="padding-left: 60px;">Master_UUID: 23b326b1-a452-11e8-91ca-000d3a065e8e</p>
<p style="padding-left: 60px;"><span style="color: #e97b00;">Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates</span></p>
<p style="padding-left: 60px;">Retrieved_Gtid_Set: 23b326b1-a452-11e8-91ca-000d3a065e8e:818-389213</p>
<p style="padding-left: 60px; padding-bottom: 15px;">Executed_Gtid_Set: 23b326b1-a452-11e8-91ca-000d3a065e8e:1-389213</p>

<h3 style="padding:10px 0px;">When SBM Value is Zero or Positive</h3>
<ul>
	<li style="text-align: justify;">SBM is going to reflect a valid value (&gt;= 0)  when the SQL Thread is actively processing events. This is true irrespective of the IO Thread state. For instance:</li>
</ul>
<p style="padding-left: 40px; padding-top: 15px;">Slave_IO_State:</p>
<p style="padding-left: 60px;">Master_Host: 172.19.0.13</p>
<p style="padding-left: 60px;">Slave_IO_Running: No</p>
<p style="padding-left: 60px;">Slave_SQL_Running: Yes</p>
<p style="padding-left: 60px;">Seconds_Behind_Master: 3399</p>
<p style="padding-left: 60px;">Master_UUID: 23b326b1-a452-11e8-91ca-000d3a065e8e</p>
<p style="padding-left: 60px;">Slave_SQL_Running_State: Waiting for slave workers to process their queues</p>
<p style="padding-left: 60px;"><span style="color: #e97b00;">Retrieved_Gtid_Set: 23b326b1-a452-11e8-91ca-000d3a065e8e:818-389213</span></p>
<p style="padding-left: 60px; padding-bottom: 15px;"><span style="color: #e97b00;">Executed_Gtid_Set: 23b326b1-a452-11e8-91ca-000d3a065e8e:1-118774</span></p>
<p style="text-align: justify;">In the above example, we can see that slave is behind the master by comparing the Retrieved_GTID_Set and the Executed_GTID_Set. In such cases, Seconds_Behind_Master will represent the difference between the timestamp of the latest transaction processed by the SQL Thread and the timestamp of the same transaction when it was processed on the master. This transaction timestamp of the master is preserved through replication and hence the slave will be able to compute the SBM locally.</p>
<p style="text-align: justify;">Also, once the slave fully catches up with all the relay logs, (i.e. executed GTID becomes 23b326b1-a452-11e8-91ca-000d3a065e8e:1-389213/), Seconds_Behind_Master will turn to ‘0’ if the IO Thread is running, or to ‘NULL’ if IO Thread is not running.</p>

<h2 style="padding-top: 15px;">Understanding Execution Speed of the MySQL Slave</h2>
<p style="text-align: justify;">Assuming that the SQL Thread and IO Thread on the slave are in running states, it’s possible to understand the relative execution speeds of the master and the slave by monitoring the SBM value. A consistent ‘0’ value or a constant value indicates that the slave is executing at the same speed as the master.  On the other hand, an upward slope for Seconds_Behind_Master indicates that the slave is performing slower than the master.</p>
<p style="text-align: justify;">ScaleGrid’s Monitoring Console for <a style="color: #2da964;" href="https://scalegrid.io/mysql/azure.html" target="_blank">MySQL on Azure</a> plots the values of SBM over time for the slave nodes.</p>

<h3 style="padding:10px 0px;">Zero Or Constant Value of SBM</h3>
<a href="https://scalegrid.io/blog/wp-content/uploads/2019/01/sbm-02.jpg"><img class="alignnone size-full wp-image-3938" src="https://scalegrid.io/blog/wp-content/uploads/2019/01/sbm-02.jpg" alt="MySQL Monitoring Console - Seconds Behind Master Zero or Constant Value" width="100%" height="auto" /></a>
<p style="text-align: justify;">In the above example, the slave was started about 40 hours after the master had active writes.  Once started, the slave began replicating that data, and we see the SBM was pretty flat indicating the slave executed at the same speed as the master. Also note the fall of SBM to ‘0’ is steep, which really means that though the last transaction the slave ran was executed about 40 hours before on the master, once we have caught up, there is ‘0’ delay.</p>

<h3 style="padding:10px 0px;">Increasing Values of SBM</h3>
<p style="text-align: justify;">In the graph below, we can see that SBM is constantly increasing, which means that the slave’s execution speed is less compared to that of the master. This is actually a case where we are running 20 threads doing continuous writes on the master and a single-threaded slave is not able to keep pace with it.</p>
<a href="https://scalegrid.io/blog/wp-content/uploads/2019/01/sbm-1.jpg"><img class="alignnone size-full wp-image-3939" src="https://scalegrid.io/blog/wp-content/uploads/2019/01/sbm-1.jpg" alt="MySQL Monitoring Console - Seconds Behind Master Increasing Value" width="100%" height="auto" /></a>
<p style="text-align: justify;">Lastly, it’s important to note that in our discussions so far, we have not assumed any network bottlenecks. In case of slow networks, the slave IO Thread itself will be lagging behind the master, and if the SQL Thread is fast enough, the SBM will be oscillating between ‘0’ and a positive number. In such cases, SBM will not be a useful parameter to understand the real lag with the master.</p>
<p style="text-align: justify; padding-bottom:15px;">If your MySQL deployments are setup using <a style="color: #2da964;" href="https://scalegrid.io/blog/data-integrity-and-performance-considerations-in-mysql-semisynchronous-replication/" target="_blank">semisynchronous replication</a>, you can also check out our <a style="color: #2da964;" href="https://scalegrid.io/blog/mysql-high-availability-framework-explained-part-2/">MySQL High Availability Framework Explained - Part II</a> post to learn more about managing the execution speed of your slaves.</p>