Send messages when you're back online with Service Workers and Background Sync
Tags: javascript, serviceworker, offlinefirst, twilio
Published: 2017-02-22 02:01:37

---
title: Send messages when you're back online with Service Workers and Background Sync
published: true
description: Use the service worker to make a network resilient messaging application with background sync
cover_image: https://www.twilio.com/blog/wp-content/uploads/2017/02/header-1.gif
tags: javascript, service worker, offline first, twilio
canonical_url: https://www.twilio.com/blog/2017/02/send-messages-when-youre-back-online-with-service-workers-and-background-sync.html
---

When you send an SMS message but your phone doesn't have great signal it will continue to try to send the message in the background, even if you close the app. Pretty useful right?

In this post we're going to see how to replicate this behaviour in a web application using the <a href="https://developers.google.com/web/updates/2015/12/background-sync">Background Sync API</a> from the <a href="https://developers.google.com/web/fundamentals/getting-started/primers/service-workers">Service Worker</a>.

## What we need

To demonstrate the Background Sync API let's build a feature into an existing application, a web based SMS inbox for a Twilio number. You may have already seen this app when we implemented <a href="https://www.twilio.com/blog/2016/02/web-powered-sms-inbox-with-service-worker-push-notifications.html">web push notifications</a> for incoming text messages.

To run the application, you will need:

* A Twilio account (<a href="https://www.twilio.com/try-twilio">sign up for a free account here</a>)
* A Twilio number that can send and receive SMS messages (<a href="https://www.twilio.com/user/account/phone-numbers/incoming">buy your number in your account portal here</a>)
* <a href="https://nodejs.org/">Node.js</a> to run the application (check out our <a href="https://www.twilio.com/docs/guides/how-to-set-up-your-node-js-and-express-development-environment">Node.js setup guide here</a>)
* Chrome to test this in (support is coming in other browsers, but is only live in Chrome so far)

Once you have those bits sorted, you can either <a href="https://github.com/philnash/sms-messages-app">download the application from GitHub</a> or clone it using git.

<pre class="lang:bash decode:true">$ git clone https://github.com/philnash/sms-messages-app.git</pre>

Change into the directory and install the dependencies:

<pre class="lang:bash decode:true">$ cd sms-messages-app
$ npm install</pre>

Copy the `.env.example` file to a file called `.env` and fill it in with your Twilio Account SID, Auth Token and your Twilio phone number.

You should now be able to start the application and test it out. Start it with the following command:

<pre class="lang:bash decode:true ">$ node index.js</pre>

Open the application in your browser at <a href="http://localhost:3000">http://localhost:3000</a>. You can send yourself an SMS message using the <a href="http://localhost:3000/messages/new">new message form</a>.

<img class="aligncenter size-full wp-image-19625" src="https://twilioinc.wpengine.com/wp-content/uploads/2017/02/working.gif" alt="When you send a message from the application it arrives on the phone" width="643" height="540" />

If you were to try to send a message with this application without a network connection then you don't get quite the same experience.

<img class="aligncenter size-full" src="https://twilioinc.wpengine.com/wp-content/uploads/2017/02/offline-1.gif" alt="When offline and you try to send a message you just get Chrome's offline screen" width="643" height="540" />

In fact, the experience is awful, but the Background Sync API can fix this. If we have a Service Worker and register for a sync event then the browser will only fire the event when it believes it has a good network connection. This allows us to store the messages we want to send while we are offline and only send them when the browser has a connection again.

## What we are going to do

There are a few steps ahead of us to complete this task, so let's break them down. On the page we need to:

1. Register a Service Worker
2. Intercept the "submit" event for our message form
3. Place the message details into <a href="https://developer.mozilla.org/en/docs/Web/API/IndexedDB_API">IndexedDB</a>, an in browser database.
4. Register the Service Worker to receive a "sync" event

Then, in the Service Worker we need to:

1. Listen for sync events
2. When a sync event is received, retrieve the messages from IndexedDB
3. For each message, send a request to our server to send the message</li>
4. If the message is sent successfully, then remove the message from IndexedDB

And that's it. We'll get started with Service Worker registration.

## Register a Service Worker

Create a new file called `app.js` in the `public/js` directory. Include it on the page by adding the following to the bottom of `views/layout.hbs`:

<pre class="lang:html decode:true mark:2">  &lt;script type="text/javascript" src="/js/material.min.js"&gt;&lt;/script&gt;
  &lt;script type="text/javascript" src="/js/app.js"&gt;&lt;/script&gt;
&lt;/body&gt;
&lt;/html&gt;</pre>

Open `public/js/app.js` and start with the following:

<pre class="lang:javascript decode:true ">document.addEventListener('DOMContentLoaded', function(event) {
  if ('serviceWorker' in navigator) {
    navigator.serviceWorker.register('/sw.js').then(function(reg) {
      if ('sync' in reg) {
        // do stuff here
      }
    }).catch(function(err) { 
      console.error(err); // the Service Worker didn't install correctly
    });
  }
});</pre>

Once the content of the DOM has fully loaded this code will check whether the browser supports Service Workers as there's no point carrying on if it doesn't. Then it registers the Service Worker that will live in the file `sw.js`. When the Promise resolves and the Service Worker is successfully registered we then check to see if sync is supported too. Then we can really start to get to work.

## Intercept form submit events

We need to get references to the form and the fields that we will read values from later. Then we listen for the `submit` event on the form so that when it triggers we can prevent it from actually submitting. Instead we'll create a `message` object that contains the phone number and the body of the message that we can save to IndexedDB.

<pre class="lang:javascript decode:true mark:2-14">if ('sync' in reg) {
  var form = document.querySelector('.js-background-sync');
  var phoneNumberField = form.querySelector('#phoneNumber');
  var bodyField = form.querySelector('#body');

  form.addEventListener('submit', function(event) {
    event.preventDefault();
    var message = {
      phoneNumber: phoneNumberField.value,
      body: bodyField.value
    };

    // do more stuff here
  });
}</pre>

## Saving the message to IndexedDB

Now that we have our message as an object we need to save it to <a href="https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API">IndexedDB</a> so that the Service Worker can get it again later. We're using IndexedDB to do this because the <a href="https://developer.mozilla.org/en/docs/Web/API/Window/localStorage">localStorage</a> API, while simpler, doesn't work in a Service Worker.

To make working with IndexedDB a little easier, I'm going to include <a href="https://github.com/jakearchibald/idb">Jake Archibald's idb library</a> which replaces the callback system with Promises. This has two benefits, the code to use the database is slightly less complicated and Service Workers expect to work with Promises so this prepares us for easier work later. If you aren't familiar with Promises, check out <a href="https://www.twilio.com/blog/2016/10/guide-to-javascript-promises.html">Dominik's guide to Promises</a>.

I've already included the library in the project, so just add the following script tag to the layout before `app.js`.

<pre class="lang:html decode:true mark:2">  &lt;script type="text/javascript" src="/js/material.min.js"&gt;&lt;/script&gt;
  &lt;script type="text/javascript" src="/js/idb.js"&gt;&lt;/script&gt;
  &lt;script type="text/javascript" src="/js/app.js"&gt;&lt;/script&gt;
&lt;/body&gt;
&lt;/html&gt;</pre>

Next we open up a database that we'll call "messages", in the callback we create an object store called "outbox" and tell it to use an auto-incrementing key which we'll call `id`. When that Promise resolves we will have access to our database. We can then use it to start a transaction in our object store and `put` the message object into the store.

<pre class="lang:javascript decode:true mark:6-13">    var message = {
      phoneNumber: phoneNumberField.value,
      body: bodyField.value
    };

    idb.open('messages', 1, function(upgradeDb) {
      upgradeDb.createObjectStore('outbox', { autoIncrement : true, keyPath: 'id' });
    }).then(function(db) {
      var transaction = db.transaction('outbox', 'readwrite');
      return transaction.objectStore('outbox').put(message);
    }).then(function() {
      // register for sync and clean up the form
    });
  });
}</pre>

Once that database transaction completes successfully all we need to do is register for the sync event that we want the Service Worker to receive and tidy up our form.

<pre class="lang:javascript decode:true mark:3-12">      return transaction.objectStore('outbox').put(message);
    }).then(function() {
      bodyField.value = '';
      if (phoneNumberField.getAttribute('type') !== 'hidden') {
        phoneNumberField.value = '';
      }
      return reg.sync.register('outbox');
    }).catch(function(err) {
      // something went wrong with the database or the sync registration, log and submit the form
      console.error(err); 
      form.submit();
    });
  });
}</pre>

Before we test this first part out we need to create a file for the Service Worker otherwise our initial install will fail and none of our code will run. Just create a blank file in the `public` directory called `sw.js`. Now, open up the message form, fill in a message and your phone number and click send. The form will clear but no message will get sent. If you open up dev tools and inspect IndexedDB from the Application tab you will find the message stored there safely.

<img class="aligncenter size-full" src="https://twilioinc.wpengine.com/wp-content/uploads/2017/02/devtools-indexeddb.png" alt="In dev tools you can see the message has been stored in IndexedDB in the messageOutbox." width="842" height="619" />

To make development with Service Workers easier for us, check the "Update on reload" checkbox under the Application tab of dev tools. This will save us having to deal with the Service Worker life cycle while in development.

<img class="aligncenter size-full" src="https://twilioinc.wpengine.com/wp-content/uploads/2017/02/devtools-sw.png" alt="When in development, check the &quot;update on reload&quot; box under the Service Workers section of dev tools." width="842" height="619" />

Now we need to implement the actual sync event in the Service Worker.

## Implementing the sync event

Open up `public/sw.js` and start by <a href="https://developer.mozilla.org/en-US/docs/Web/API/WorkerGlobalScope/importScripts">importing</a> the idb library and listening for the sync event.

<pre class="lang:javascript decode:true ">importScripts('/js/idb.js');

self.addEventListener('sync', function(event) {

});</pre>

Within the event listener we need to tell the browser to wait for the Service Worker to complete the asynchronous tasks of connecting to IndexedDB, retrieving any messages that need to be sent and sending them.

To do this we use the <a href="https://developer.mozilla.org/en-US/docs/Web/API/ExtendableEvent/waitUntil">`waitUntil`</a> method of the event, which takes a Promise as an argument and only completes when the Promise is resolved.

<pre class="lang:javascript decode:true mark:4-6">importScripts('/js/idb.js');

self.addEventListener('sync', function(event) {
  event.waitUntil(
    // do asynchronous tasks here
  );
});</pre>

### Time for a refactor

We need to connect to our IndexedDB database, but we already wrote the code to do that in our `app.js` file. This connection code can be refactored and shared between the Service Worker and the page.

Create a file called `store.js` in the `public/js` directory and enter the following:

<pre class="lang:javascript decode:true ">var store = {
  db: null,

  init: function() {
    if (store.db) { return Promise.resolve(store.db); }
    return idb.open('messages', 1, function(upgradeDb) {
      upgradeDb.createObjectStore('outbox', { autoIncrement : true, keyPath: 'id' });
    }).then(function(db) {
      return store.db = db;
    });
  },

  outbox: function(mode) {
    return store.init().then(function(db) {
      return db.transaction('outbox', mode).objectStore('outbox');
    })
  }
}</pre>

The `init` function opens and upgrades the database if it needs it and returns a Promise that resolves with the database object itself. The `outbox` function returns a Promise that resolves with a reference to the object store for the outbox in the chosen mode ("readonly" or "readwrite") within a transaction.

Add this script to the `views/layouts.hbs`.

<pre class="lang:html decode:true mark:3">  &lt;script type="text/javascript" src="/js/material.min.js"&gt;&lt;/script&gt;
  &lt;script type="text/javascript" src="/js/idb.js"&gt;&lt;/script&gt;
  &lt;script type="text/javascript" src="/js/store.js"&gt;&lt;/script&gt;
  &lt;script type="text/javascript" src="/js/app.js"&gt;&lt;/script&gt;
&lt;/body&gt;</pre>

Replace the use of idb in `app.js` with our new store.

<pre class="lang:diff decode:true ">            var message = {
                phoneNumber: phoneNumberField.value,
                body: bodyField.value
              };
-           idb.open('messages', 1, function(upgradeDb) {
-             upgradeDb.createObjectStore('outbox', { autoIncrement : true, keyPath: 'id' });
-           }).then(function(db) {
-             var transaction = db.transaction('outbox', 'readwrite');
-             return transaction.objectStore('outbox').put(message);
+           store.outbox('readwrite').then(function(outbox) {
+             return outbox.put(message);
            }).then(function() {</pre>
Import our store to the Service Worker file.
<pre class="lang:javascript decode:true mark:2">importScripts('/js/idb.js');
importScripts('/js/store.js');

self.addEventListener('sync', function(event) {
  event.waitUntil(
    // do asynchronous tasks here
  );
});</pre>

Now we can use our store to get all the messages without repeating ourselves.

<pre class="lang:javascript decode:true mark:3-8">self.addEventListener('sync', function(event) {
  event.waitUntil(
    store.outbox('readonly').then(function(outbox) {
      return outbox.getAll();
    }).then(function(messages) {
      // send the messages
    }).catch(function(err) { console.error(err); });
  );
});</pre>

We have an array of messages to send. For each message we need to make a POST request to the original path, "/messages", with the details of the message. Then we'll read the response and if it is a success we'll remove the message from the database.

As all those actions are asynchronous, we need to use `Promise.all` to wait for them all to successfully complete. That should look like this in sw.js:

<pre class="lang:javascript decode:true mark:4-21">    store.outbox('readonly').then(function(outbox) {
      return outbox.getAll();
    }).then(function(messages) {
      return Promise.all(messages.map(function(message) {
        return fetch('/messages', {
          method: 'POST',
          body: JSON.stringify(message),
          headers: {
            'Accept': 'application/json',
            'X-Requested-With': 'XMLHttpRequest',
            'Content-Type': 'application/json'
          }
        }).then(function(response) {  
          return response.json();
        }).then(function(data) {
          if (data.result === 'success') {
            return store.outbox('readwrite').then(function(outbox) {
              return outbox.delete(message.id);
            });
          }
        })
    }).catch(function(err) { console.error(err); });</pre>

With all that in place we are ready to test. Reload the page and send a message. You should receive it straight away. You should also receive the first message you tried to send as that was waiting in IndexedDB too.

Now disconnect from the Internet and try to send a message. Wait a while, turn the network back on and whoosh, there go the messages!

<img class="aligncenter size-full" src="https://twilioinc.wpengine.com/wp-content/uploads/2017/02/success.gif" alt="Now when you send a message while offline it will be sent when you go back online." width="643" height="540" />

## Network resilient messages

Using the Background Sync API for the Service Worker we now have network resilient message sending. No longer will users have to keep a web site open waiting to see if what they just did worked. We are getting closer to the experience that a native application can provide a user, like the built in SMS app I mentioned at the start.

This is just an initial implementation, the <a href="https://github.com/philnash/sms-messages-app/tree/background-sync">full code of which is on GitHub</a>.

There are more places we can try to improve the user experience in this application too, like:

* Improving the UI to show messages that haven't sent yet
* Including a timeout or method for cancelling sent messages
* <a href="https://www.twilio.com/blog/2016/02/web-powered-sms-inbox-with-service-worker-push-notifications.html">Push notifications</a> to notify about message failure

Sending messages is not the only use case for background sync either. I can see this being useful for things like:

* Adding items to a shopping cart
* <a href="https://www.twilio.com/docs/tutorials/walkthrough/click-to-call/node/express">Requesting a callback from a web form</a>
* Downloading an article in the background (see Jake Archibald's <a href="https://github.com/jakearchibald/offline-wikipedia">Offline Wikipedia</a> project for an example)
* Creating channels, sending messages, inviting users or any other action within a <a href="https://www.twilio.com/docs/api/chat">Programmable Chat</a> application

I hope you're excited about the potential for the Background Sync API, let me know what you think or if you have any ideas for what you might use it for. I can be reached in the comments here, on Twitter at <a href="https://twitter.com/philnash">@philnash</a> or by email at philnash@twilio.com.

___

_[Send messages when you're back online with Service Workers and Background Sync](https://www.twilio.com/blog/2017/02/send-messages-when-youre-back-online-with-service-workers-and-background-sync.html) was originally published on the [Twilio Blog](https://www.twilio.com/blog/) on February 17, 2017._