Change Streams, Coming Soon in MongoDB 3.6
Tags: iot, mongodb, changestreams, esp8266
Published: 2017-11-09 17:40:51

---
title: "Change Streams, Coming Soon in MongoDB 3.6"
published: true    
tags: IoT,MongoDB,Change Streams,ESP8266
canonical_url: https://www.kenwalger.com/blog/nosql/mongodb/change-streams-coming-soon-mongodb-3-6/
cover_image: https://thepracticaldev.s3.amazonaws.com/i/eqv28brenvwegebcivwe.png
---

Gone are the days when expectations of information were not instantaneous access to the data. In today’s world, people expect real-time results and information. What’s the moisture content of the soil around grapevine X right&nbsp; **now**? Not yesterday, not five minutes ago, but now. Being notified about constantly changing data points is critical in many of today’s applications. In the upcoming 3.6 release of [MongoDB](http://www.mongodb.com) a new feature is being introduced to assist with these real-time expectations: Change Streams.

### Change Streams Defined

Change streams are being implemented in the driver with a new [aggregation](https://docs.mongodb.com/master/aggregation/) operator, `$changeStream` and `watch` method in the API. We can now specify a&nbsp;`$changeStream` stage at the beginning of our pipeline and request that notifications are sent for specific changes to a particular collection.

### Example Use Case

Let’s take the soil moisture example I stated earlier. Imagine that you have a farm with multiple&nbsp;[moisture sensors](https://www.amazon.com/gp/product/B06ZYVBRND/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B06ZYVBRND&linkCode=as2&tag=kenwalgersite-20&linkId=2e3515c3da625962cfafe8e820620418) ![](https://www.kenwalger.com/blog//ir-na.amazon-adsystem.com/e/ir?t=kenwalgersite-20&l=am2&o=1&a=B06ZYVBRND)&nbsp;or a&nbsp;[TMP36 Analog temperature](https://www.amazon.com/gp/product/B011NEQ7J4/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B011NEQ7J4&linkCode=as2&tag=kenwalgersite-20&linkId=c8ee5dcc279985c2897c08b0ef5a504f)&nbsp;sensor ![](https://www.kenwalger.com/blog//ir-na.amazon-adsystem.com/e/ir?t=kenwalgersite-20&l=am2&o=1&a=B011NEQ7J4) connected to something like a [NodeMCU ESP8266](https://www.amazon.com/gp/product/B010O1G1ES/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B010O1G1ES&linkCode=as2&tag=kenwalgersite-20&linkId=85c3b431dbcb2773387af28ff09a6e4f) ![](https://www.kenwalger.com/blog//ir-na.amazon-adsystem.com/e/ir?t=kenwalgersite-20&l=am2&o=1&a=B010O1G1ES)&nbsp;or [LoPy](https://www.amazon.com/gp/product/B06W9GBL2W/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B06W9GBL2W&linkCode=as2&tag=kenwalgersite-20&linkId=11f52ad3f601e4e50c7a7799699da948) ![](https://www.kenwalger.com/blog//ir-na.amazon-adsystem.com/e/ir?t=kenwalgersite-20&l=am2&o=1&a=B06W9GBL2W) device. Without going into detail about how to set the IoT network up, for the sake of our use case we can assume that they are sending readings back to a MongoDB server. Perhaps the data is being stored on [MongoDB Atlas](https://www.kenwalger.com/blog/nosql/mongodb/mongodb-atlas/) even. I have, however, written about the [LoPy microcontroller](https://www.kenwalger.com/blog/iot/lopy/) and [ESP8266 with temperature](https://www.kenwalger.com/blog/iot/micropython/micropython-temperature-sensor/) sensors in the past if you are interested in learning more.

Anyway, for the sake of argument, let’s say that we need to know when the moisture content of the soil drops below a certain level. Moisture values on similar devices are measured as a raw number between 0-1024. The higher the number, the more moisture in the soil. For our use case, we need to know when the moisture level drops below 350 (too dry).

###### Change Stream in action

Since these recordings are frequently changing, we can set up a change stream to get notified. It’s fairly straightforward to establish a notification with the `watch` method. We could set something like this up:

```
change_stream = client.winery.collection.watch([
    {'$match': {
        'operationType': {'$in': ['insert']}
    }},
    {'$match': {
        'moisture_level': {'$lt': 350}
    }}
]);

for change in change_stream:
    turn_on_water();
    # Something else magical
```

Now when we have an insert into the database where the `moisture_level` is below 350, we’ll turn our watering system on along with whatever else needs to be done. We could set up a similar change stream for when the moisture level gets high enough to warrant&nbsp;shutting the system off at 450.

This is but one example of where change streams would be a useful tool. Updating reports dynamically on data changes, publish/subscribe types of functionality, brokers, and many other use cases are now open to native MongoDB actions.

### Change Steam Details

You’ll notice that there is an `operationType` being looked at there. Change streams look at five different events, or Operation Types

- Insert
- Delete
- Replace (everything but the unique id)
- Update
- Invalidate (in cases of an invalid cursor being returned)

Change streams also utilize access controls and can be set up based on [user roles](https://docs.mongodb.com/manual/reference/built-in-roles/). They present a defined API and enable scaling across primaries and secondaries. MongoDB is supporting up to 1,000 concurrent change streams per data bearing node in 3.6.

They are durable, meaning they rely on the data being majority committed in a [replica set](https://docs.mongodb.com/manual/replication/index.html). The order of the changes in data is important too, right? 3.6 also introduces a global logical clock which, among other uses, will ensure that clients always will get changes in the order they occurred on the server. This is very important for `replace` and `update` operations.

### Wrap Up

There are a lot more technical details about change streams forthcoming as 3.6 moves closer to production and documentation is further flushed out. Change streams are inherent in the 3.6 Server and are available in both the community and enterprise versions of the product. MongoDB Version 3.6 is available to [download](https://www.mongodb.com/download-center) today as a release candidate. Which means that while the high-level concepts of change streams discussed here likely will remain in tack in the final release, there may be some subtle changes. That being said, change streams are a great feature being implemented in the new release. If your applications require&nbsp;or could use real-time information, I would encourage you to have a look.

* * *

Follow me on Twitter [@kenwalger](https://www.twitter.com/kenwalger) to get the latest updates on my postings.

There are a few MongoDB specific terms in this post. I created a [MongoDB Dictionary](https://www.echoskillstore.com/MongoDB-Dictionary/45103) skill for the [Amazon Echo](https://www.amazon.com/gp/product/B01DFKC2SO/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B01DFKC2SO&linkCode=as2&tag=kenwalgersite-20&linkId=f9e513223de2525a72b95cf9561db55b)&nbsp;line of products. Check it out and you can say “Alexa, ask MongoDB for the definition of a&nbsp;document? and get a helpful response.

[![Facebook](https://i1.wp.com/www.kenwalger.com/blog/wp-content/plugins/social-media-feather/synved-social/image/social/regular/96x96/facebook.png?resize=48%2C48&ssl=1 "Share on Facebook")](http://www.facebook.com/sharer.php?u=https%3A%2F%2Fwww.kenwalger.com%2Fblog%2Fnosql%2Fmongodb%2Fchange-streams-coming-soon-mongodb-3-6%2F&t=Change%20Streams%2C%20Coming%20Soon%20in%20MongoDB%203.6&s=100&p%5Burl%5D=https%3A%2F%2Fwww.kenwalger.com%2Fblog%2Fnosql%2Fmongodb%2Fchange-streams-coming-soon-mongodb-3-6%2F&p%5Bimages%5D%5B0%5D=https%3A%2F%2Fi2.wp.com%2Fwww.kenwalger.com%2Fblog%2Fwp-content%2Fuploads%2F2017%2F10%2Fchange_streams_feature.png%3Ffit%3D125%252C125%26ssl%3D1&p%5Btitle%5D=Change%20Streams%2C%20Coming%20Soon%20in%20MongoDB%203.6 "Share on Facebook")[![twitter](https://i0.wp.com/www.kenwalger.com/blog/wp-content/plugins/social-media-feather/synved-social/image/social/regular/96x96/twitter.png?resize=48%2C48&ssl=1 "Share on Twitter")](https://twitter.com/intent/tweet?url=https%3A%2F%2Fwww.kenwalger.com%2Fblog%2Fnosql%2Fmongodb%2Fchange-streams-coming-soon-mongodb-3-6%2F&text=Hey%20check%20this%20out "Share on Twitter")[![google_plus](https://i1.wp.com/www.kenwalger.com/blog/wp-content/plugins/social-media-feather/synved-social/image/social/regular/96x96/google_plus.png?resize=48%2C48&ssl=1 "Share on Google+")](https://plus.google.com/share?url=https%3A%2F%2Fwww.kenwalger.com%2Fblog%2Fnosql%2Fmongodb%2Fchange-streams-coming-soon-mongodb-3-6%2F "Share on Google+")[![reddit](https://i1.wp.com/www.kenwalger.com/blog/wp-content/plugins/social-media-feather/synved-social/image/social/regular/96x96/reddit.png?resize=48%2C48&ssl=1 "Share on Reddit")](http://www.reddit.com/submit?url=https%3A%2F%2Fwww.kenwalger.com%2Fblog%2Fnosql%2Fmongodb%2Fchange-streams-coming-soon-mongodb-3-6%2F&title=Change%20Streams%2C%20Coming%20Soon%20in%20MongoDB%203.6 "Share on Reddit")[![linkedin](https://i2.wp.com/www.kenwalger.com/blog/wp-content/plugins/social-media-feather/synved-social/image/social/regular/96x96/linkedin.png?resize=48%2C48&ssl=1 "Share on Linkedin")](http://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fwww.kenwalger.com%2Fblog%2Fnosql%2Fmongodb%2Fchange-streams-coming-soon-mongodb-3-6%2F&title=Change%20Streams%2C%20Coming%20Soon%20in%20MongoDB%203.6 "Share on Linkedin")[![mail](https://i2.wp.com/www.kenwalger.com/blog/wp-content/plugins/social-media-feather/synved-social/image/social/regular/96x96/mail.png?resize=48%2C48&ssl=1 "Share by email")](mailto:?subject=Change%20Streams%2C%20Coming%20Soon%20in%20MongoDB%203.6&body=Hey%20check%20this%20out:%20https%3A%2F%2Fwww.kenwalger.com%2Fblog%2Fnosql%2Fmongodb%2Fchange-streams-coming-soon-mongodb-3-6%2F "Share by email")

The post [Change Streams, Coming Soon in MongoDB 3.6](https://www.kenwalger.com/blog/nosql/mongodb/change-streams-coming-soon-mongodb-3-6/) appeared first on [Blog of Ken W. Alger](https://www.kenwalger.com/blog).