Make Your Own Air Quality Logger
By: Andrew Doss
Tags: Data Engineering, Makers, IoT, Postgres, Raspberry Pi

Log IoT data to a cloud database using Python on a Raspberry Pi

For about $30 in parts, you can build your own home air quality logger. We’ll walk through how and also provide a general pattern that you can adapt for other types of sensor data that you may want to log to a cloud database.

Measure what matters to you

Years ago, I got back into C++ because I couldn’t sleep at night. I lived in a century-old apartment building in Seattle where my unit had the thermostat that controlled the boiler for the entire building. My landlord asked me to not open my windows in the winter because the thermostat would sense the cold and counter by roasting every other unit in the building.

One winter, my apartment started feeling really hot at around 3am every night. Being an engineer, I hacked together a DIY temperature logger using a thermistor and an Arduino to get some data. I plotted the overnight temperatures and confirmed there actually was a sharp spike early each morning.

What would you like to measure? With an ever-expanding array of low-cost sensors on the market, $10 wifi-enabled computers, and instant cloud databases, it’s never been easier to measure what matters to you. Individuals have even started crowd-sourcing air quality monitoring databases that can supplement government data and meaningfully contribute to tracking the impact of issues like climate change.

In this guide, we’ll walk you through the steps to build your own air quality monitor. Air quality is an important public health issue, particularly with ever-escalating wildfires in the West. However, the general pattern we cover here can also be adapted to other types of sensors and measurements like temperature, humidity, noise, and more.

Three easy pieces

We’ll walk through three easy pieces that can be put together to log sensor data to a cloud database:

Reading sensor data Processing sensor data into records Inserting records into a cloud database

Step 1 will vary with different sensor and I/O types, but 2 and 3 will be fairly similar in many cases. We list the components we used for our example implementation — you can use any computer running Python in place of the Pi Zero W.

Step 1. Reading sensor data

Sensors use various means to detect or measure a physical property and output a signal. For our air quality measurement example, we use a Nova Fitness SDS011 that can be found online for as little as $20.

The SDS011 draws air through a small chamber and uses the variation in properties of a laser shining onto a diode to infer the ambient particulate matter concentration. For this reason, the air inlet and outlets should be shaded from direct light.

The SDS011 is well-reviewed within its low-cost class, but all low-cost air quality sensors have limitations relative to several-thousand-dollar EPA-certified monitors. Air quality sensors are useful tools but should not be relied on for critical applications without learning more about their usage.

The Nova Fitness SDS011 sensor contains a small fan, laser, and diode for measuring ambient particulate matter concentrations.

The SDS011 sensor comes with its own serial USB adapter, enabling data to be read with three lines of Python from any computer with a USB port.

In the code snippet below, we import the pyserial package and construct a Serial object. The Serial object interfaces with the OS to read serial data from the USB serial port, named /dev/ttyUSB0 in a Linux system. You can look up the equivalent port name for non-Linux systems.

The SDS011 sends one ten-byte message every second. We use the Serial object to capture one message by reading ten bytes of data into a Python bytes object.

Just three lines of code to read the sensor data.

If you print the ten byte message, you will see a string of ten bytes in a hexadecimal representation that looks something like this:

b'\xaa\xc0\x13\x00\x2f\x00\x12\x92\xe6\xab'

Don’t worry if you’re not familiar with bytes objects or hexadecimal, we show how to parse the data in the next step.

Step 2. Processing sensor data into records

Now that we’ve read raw bytes data from the sensor, it’s time to process the sensor data into a record for our database.

To start, we consulted the sensor’s specification sheet. The contents of the ten bytes is summarized in the illustration below:

The first and last bytes are used for the communication protocol and the inner six bytes contain the data payload.

The first two and last two bytes are used by the protocol between the sensor and the OS and enable pyserial to read the correct ten-byte message sequences. You can ignore those bytes for our purposes.

Between the protocol bytes, two bytes are allocated to each of the PM₂.₅ measurement, the PM₁₀ measurement, and the sensor’s identification number.

We can parse each of those two-byte values using the code snippet below:

We can parse the bytes we need for different values to construct Python ints and floats.

We simply slice the bytes we need from the message, cast the bytes to an integer, then scale the result, if needed. The byte_order parameter is used to specify whether the bytes for each value are ordered from left to right, or right to left.

The scaler is used in cases where an integer is representing a value with digits to the right of the decimal. With the SDS011, we need to scale by 0.1 to capture the single decimal point precision of the sensor.

Using parse_value , we can create records for insertion to a database using the code snippet below:

How we create a record using the parse_value function

Starting at line 16, we define the measurements that we need to parse with config mapping each measurement to its start byte, number of bytes, and scaling factor.

In lines 21–23, we specify the number of messages to aggregate per database record, 300, and collect the corresponding messages in a list. With one message per second, 300 messages corresponds to five minutes of data. Aggregating sampled sensor data on an edge device, such as a Raspberry Pi, is a common IoT practice to capture information at the granularity that’s actually needed while reducing network and storage costs.

At line 23, we create a record using the create_record function defined on line 4. The create_record function compiles a record dictionary containing the location, sensor ID number, UTC timestamp, and mean particulate matter measurements from the sample period.

At this point, we’ve created a dictionary with a record that’s ready for insertion into a database:

{

‘datetime’: ‘2021–09–02 03:44:57.968544’,

‘location’: ‘outside’,

‘sensor_id’: 37394,

‘pm_2_5’: 2.64,

‘pm_10’: 4.71

}

Step 3. Inserting records into a cloud database

We’ve arrived at the final step of inserting records into a cloud database. For this step, we need to create a cloud database and then connect with Python to insert our record.

bit.io is the easiest way to instantly create a standards-compliant Postgres database and load your data into one place (and it’s free for most hobby-scale use cases). You simply sign up (no credit card required), follow the prompts to “create a repo” (your own private database), then follow the “connecting to bit.io” docs to get a Postgres connection string for your new database.