Building a Real-Time IoT Dashboard with HarperDB and Node.js
Tags: iot, dashboard, charts, node
Published: 2024-06-28 11:33:46

---
title: Building a Real-Time IoT Dashboard with HarperDB and Node.js
published: true
date: 2024-06-28 10:36:03 UTC
tags: iot,dashboard,charts,Nodejs
cover_image: https://cdn.hashnode.com/res/hashnode/image/upload/v1719570472697/c9f104f6-c696-43e9-856b-c3bda9e9eb42.jpeg?w=1600&h=840&fit=crop&crop=entropy&auto=compress,format&format=webp
canonical_url: https://www.thatamazingprogrammer.com/building-a-real-time-iot-dashboard-with-harperdb-and-nodejs
---

Welcome to an exciting journey into the world of real-time data with HarperDB and Node.js! Imagine having an IoT dashboard that updates in real-time, giving instant insights into your data. Sounds intriguing, right? That's precisely what you'll build in this tutorial.

HarperDB is a powerful tool that combines a distributed database, cache, application, and streaming system into a single process. This unique combination allows for high performance and simplicity at scale, making it an excellent choice for real-time applications. In this project, you'll leverage HarperDB's capabilities to create a real-time IoT dashboard.

By the end of this tutorial, you'll have a fully functional dashboard that can visualize data from IoT devices in real-time.

You'll learn how to:

Set up and configure HarperDB
Simulate an IoT device sending data
Build a Node.js server to handle real-time data
Create a dynamic frontend to display data

Whether you're a seasoned developer or just getting started, this tutorial is designed to be hands-on and straightforward. Each step will guide you through the process, ensuring you understand how everything fits together. Ready to dive in and start building? Let's get started!

## Project Setup

Start by setting up HarperDB and then create the necessary project structure. Follow these steps to get everything in place.

**1. Installing and Configuring HarperDB**

First, you'll need to install HarperDB. If you haven't already, head to the HarperDB website for the installation instructions tailored to your operating system.
After installing HarperDB, you need to start the HarperDB server. Open your terminal and run the following command:

```bash
harperdb run
```

You should see a message indicating that HarperDB is up and running. By default, HarperDB runs on port 9925. You can access the HarperDB Studio at http://localhost:9925 to manage your database.

![Terminal output of starting HarperDB](https://cdn.hashnode.com/res/hashnode/image/upload/v1719569968577/6aacb9d6-3fb0-4f27-9622-6b61ed214eca.png?auto=compress,format&format=webp)

**2. Setting Up the Project Directory and Necessary Files**

Now, set up the project directory. Open your terminal and navigate to your preferred workspace. Create a new directory for our project and navigate into it:

```bash
mkdir real-time-iot-dashboard
cd real-time-iot-dashboard
```

Next, initialize a new Node.js project:

```bash
npm init -y
```

This command creates a package.json file with the default settings. Now, install the required dependencies. You’ll need Express for our server, Axios for making HTTP requests, and WebSocket for real-time communication:

```bash
npm install express axios ws
```

With the dependencies installed, you'll need to create the necessary files and directories. Run the following commands:

```bash
mkdir public
touch server.js public/index.html
```

Here’s a quick overview of what each file will do:
server.js: This will be the main server file, handling HTTP and WebSocket connections.
public/index.html: This will be the frontend file, displaying the real-time data.

**3. Configuring HarperDB**

Before moving on, you must set up a table in HarperDB to store the IoT data.

Create a file called `schema.graphql` in your project directory, then paste the following code:

```json
type SensorData @table @export {
  id: ID @primaryKey
  timestamp: String
  temperature: String
  humidity: String
}
```

In a separate terminal window, run the following command to have HarperDB monitor the application directory:

```bash
harperdb dev .
```

You can learn more about creating tables in the Applications section.

## Simulating an IoT Device

With the project setup complete, it's time to simulate an IoT device sending data to HarperDB. Create a Node.js script that generates random sensor data and sends it to the HarperDB instance. Follow these steps to get started.

**1. Creating the Node.js Script**

First, create a new file in the project directory called simulate_device.js:

```bash
touch simulate_device.js
```

Open this file in your preferred code editor and start by requiring the necessary modules. Use Axios to send HTTP requests to HarperDB.

```javascript
const axios = require('axios');
const { v4: uuidv4 } = require('uuid');
```

The uuid library will generate unique IDs for the sensor data. Install this library by running:

```javascript
npm install uuid
```

**2. Generating Random Sensor Data**

Next, create a function to generate random sensor data. Assume the IoT device measures temperature and humidity.

```javascript
function generateSensorData() {
    return {
        id: uuidv4(),
        timestamp: new Date().toISOString(),
        temperature: (Math.random() * 40).toFixed(2), // Random temperature between 0 and 40
        humidity: (Math.random() * 100).toFixed(2) // Random humidity between 0 and 100
    };
}
```

**3. Sending Data to HarperDB**

Now, create a function to send the generated sensor data to HarperDB using the REST API.

```javascript
async function sendDataToHarperDB(data) {
const config = {
    		method: 'post',
    		url: 'http://localhost:9926/SensorData',
    		headers: {
        		'Content-Type': 'application/json'
    		},
    		data: JSON.stringify(data)
	};

	try {
    		const response = await axios(config);
    		console.log('Data sent successfully:', response.data);
	} catch (error) {
    		console.error('Error sending data:', error);
	}
}
```

**4. Automating Data Generation and Sending**

Finally, create a loop to generate and send data at regular intervals. For instance, send data every 5 seconds.

```javascript
setInterval(() => {
const data = generateSensorData();
	sendDataToHarperDB(data);
}, 5000);
```

The complete `simulate_device.js` script should look like this:

```javascript
const axios = require('axios');
const { v4: uuidv4 } = require('uuid');

function generateSensorData() {
	return {
    	id: uuidv4(),
    	timestamp: new Date().toISOString(),
    	temperature: (Math.random() * 40).toFixed(2),
    	humidity: (Math.random() * 100).toFixed(2)
	};
}

async function sendDataToHarperDB(data) {
	const config = {
    	method: 'post',
    	url: 'http://localhost:9926/SensorData',
    	headers: {
        	'Content-Type': 'application/json'
    	},
    	data: JSON.stringify(data)
	};

	try {
    	const response = await axios(config);
    	console.log('Data sent successfully:', response.data);
	} catch (error) {
    	console.error('Error sending data:', error);
	}
}

setInterval(() => {
	const data = generateSensorData();
	sendDataToHarperDB(data);
}, 5000);


```

Run the script using the following command:


```bash
node simulate_device.js
```

The script will start generating random sensor data and sending it to HarperDB every 5 seconds. This setup simulates an IoT device and provides data for the real-time dashboard.
In the next section, you'll build the real-time data ingestion setup, ensuring the IoT data flows smoothly into the dashboard.

## Real-Time Data Ingestion

Now that your IoT device simulation sends data to HarperDB, it's time to set up real-time data ingestion. This setup will allow the frontend to receive and display data as soon as it's available. Follow these steps to create a seamless flow of data from HarperDB to the dashboard.

**1. Setting Up the Node.js Server**

First, you'll create a Node.js server to handle real-time data communication. This server will use WebSocket to push updates to the frontend.

Open the `server.js` file and start by requiring the necessary modules:

```javascript
const express = require('express');
const http = require('http');
const WebSocket = require('ws');
const axios = require('axios');
```

Next, set up the basic Express server and HTTP server:

```javascript
const app = express();
const server = http.createServer(app);
const wss = new WebSocket.Server({ server });

// Serve static files from the 'public' directory
app.use(express.static('public'));

// Serve index.html at the root endpoint
app.get('/', (req, res) => {
	res.sendFile(__dirname + '/public/index.html');
});

// Start the server on port 3000
server.listen(3000, () => {
	console.log('Server is running on http://localhost:3000');
});
```

**2. Handling WebSocket Connections**

Set up the WebSocket server to handle connections and send real-time data to connected clients:

```javascript
wss.on('connection', (ws) => {
    console.log('Client connected');
    ws.on('close', () => {
        console.log('Client disconnected');
    });
});
```

**3. Fetching Data from HarperDB**

To fetch data from HarperDB, create a function that retrieves the latest sensor data. Use this function to periodically check HarperDB for new data and send updates to the frontend:

```javascript
async function fetchDataAndUpdateClients() {
	const config = {
    	method: 'post',
    	url: 'http://localhost:9925',  // HarperDB endpoint
    	headers: {
        	'Content-Type': 'application/json'
    	},
    	data: JSON.stringify({
        	operation: 'sql',
        	sql: 'SELECT * FROM data.SensorData ORDER BY timestamp DESC LIMIT 1'
    	})
	};
    
	try {
    	const response = await axios(config);
    	const latestData = response.data;
   	 
    	// Check if the response data is an array (expected format)
    	if (Array.isArray(latestData)) {
        	// Broadcast latest data to all connected clients
        	wss.clients.forEach((client) => {
            	if (client.readyState === WebSocket.OPEN) {
                	client.send(JSON.stringify(latestData));
            	}
        	});
    	} else {
        	console.error('Unexpected data format:', latestData);
    	}
	} catch (error) {
    	console.error('Error fetching data:', error);
	}
}

// Periodically fetch data and update clients every 5 seconds
setInterval(fetchDataAndUpdateClients, 5000);
```

This function queries HarperDB for the latest sensor data and sends it to all connected WebSocket clients every 5 seconds.

If you go to http://localhost:9925/SensorData you should see something like this:

![Output of the basic dashboard showing the id, timestamp, temperature, and humitity](https://cdn.hashnode.com/res/hashnode/image/upload/v1719570227418/a52a4d65-3a33-459a-8696-a09e348eff87.png?auto=compress,format&format=webp)

**4. Setting Up the Frontend to Receive Data**

Finally, configure the frontend to connect to the WebSocket server and display the received data. Open the `public/index.html` file and add the following code:

```html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Real-Time IoT Dashboard</title>
</head>
<body>
    <h1>Real-Time IoT Dashboard</h1>
    <div id="dataContainer">
        <p>Waiting for data...</p>
    </div>
    <script>
        const dataContainer = document.getElementById('dataContainer');
        const ws = new WebSocket('ws://localhost:3000');

        ws.onmessage = (event) => {
            const data = JSON.parse(event.data);
            if (Array.isArray(data)) {
                dataContainer.innerHTML = ''; // Clear previous data
                data.forEach(sensorData => {
                    const { id, timestamp, temperature, humidity } = sensorData;
                    const dataDiv = document.createElement('div');
                    dataDiv.classList.add('data-item');
                    dataDiv.innerHTML = `
                        <p>ID: ${id}</p>
                        <p>Timestamp: ${timestamp}</p>
                        <p>Temperature: ${temperature} °C</p>
                        <p>Humidity: ${humidity} %</p>
                        <hr>
                    `;
                    dataContainer.appendChild(dataDiv);
                });
            } else {
                console.error('Received unexpected data format:', data);
                // Handle single data object case, if needed
                const { id } = data;
                const dataDiv = document.createElement('div');
                dataDiv.classList.add('data-item');
                dataDiv.innerHTML = `
                    <p>ID: ${id}</p>
                    <p>Unexpected data format, check console for details.</p>
                `;
                dataContainer.appendChild(dataDiv);
            }
        };

        ws.onopen = () => {
            console.log('WebSocket connection established.');
        };

        ws.onclose = () => {
            console.log('WebSocket connection closed.');
        };
    </script>
</body>
</html>
```

This code establishes a WebSocket connection to the server and updates the page with the latest sensor data.

At this point the application should look like this:



## Building the Dashboard

Now that you've set up real-time data ingestion, it's time to build a polished and interactive dashboard to visualize the IoT data. Follow these steps to enhance the user interface and create dynamic charts.

**1. Setting Up Chart.js**

To create beautiful charts, use Chart.js. First, add the Chart.js library to your project. Include the following script tag in the <head> section of your `public/index.html` file:

```html
<head>
    <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/chartjs-adapter-date-fns@3.0.0"></script>
</head>
```
**2. Creating the HTML Structure**

Modify the HTML structure to include a canvas element for the chart. Update the `public/index.html` file:

```html
<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<title>Real-Time IoT Dashboard</title>
	<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
	<script src="https://cdn.jsdelivr.net/npm/chartjs-adapter-date-fns@3.0.0"></script>
	<link rel="stylesheet" href="styles.css">
	<script>
    	console.log("Chart.js loaded:", Chart);  // Check if Chart.js is loaded
	</script>
</head>
<body>
	<h1>Real-Time IoT Dashboard</h1>
	<div id="dataContainer">
    	<p>Waiting for data...</p>
	</div>
	<canvas id="myChart" width="400" height="200"></canvas>
	<script>
    	const dataContainer = document.getElementById('dataContainer');
    	const ctx = document.getElementById('myChart').getContext('2d');
    	const ws = new WebSocket('ws://localhost:3000');
    	let chart;
   	 
    	ws.onmessage = (event) => {
        	const data = JSON.parse(event.data);
        	if (Array.isArray(data)) {
            	dataContainer.innerHTML = ''; // Clear previous data
            	data.forEach(sensorData => {
                	const { id, timestamp, temperature, humidity } = sensorData;
                	const dataDiv = document.createElement('div');
                	dataDiv.classList.add('data-item');
                	dataDiv.innerHTML = `
                    	<p>ID: ${id}</p>
                    	<p>Timestamp: ${timestamp}</p>
                    	<p>Temperature: ${temperature} °C</p>
                    	<p>Humidity: ${humidity} %</p>
                    	<hr>
                	`;
                	dataContainer.appendChild(dataDiv);
                	updateChart(sensorData);
            	});
        	} else {
            	console.error('Received unexpected data format:', data);
            	// Handle single data object case, if needed
            	const { id } = data;
            	const dataDiv = document.createElement('div');
            	dataDiv.classList.add('data-item');
            	dataDiv.innerHTML = `
                	<p>ID: ${id}</p>
                	<p>Unexpected data format, check console for details.</p>
            	`;
            	dataContainer.appendChild(dataDiv);
        	}
    	};

    	ws.onopen = () => {
        	console.log('WebSocket connection established.');
    	};

    	ws.onclose = () => {
        	console.log('WebSocket connection closed.');
    	};

    	function createChart(data) {
        	chart = new Chart(ctx, {
            	type: 'line',
            	data: {
                	labels: [data.timestamp],
                	datasets: [
                    	{
                        	label: 'Temperature (°C)',
                        	data: [data.temperature],
                        	borderColor: 'rgba(255, 99, 132, 1)',
                        	borderWidth: 1,
                        	fill: false
                    	},
                    	{
                        	label: 'Humidity (%)',
                        	data: [data.humidity],
                        	borderColor: 'rgba(54, 162, 235, 1)',
                        	borderWidth: 1,
                        	fill: false
                    	}
                	]
            	},
            	options: {
                	scales: {
                    	x: {
                        	type: 'time',
                        	time: {
                            	unit: 'minute'
                        	}
                    	},
                    	y: {
                        	beginAtZero: true
                    	}
                	}
            	}
        	});
    	}

    	function updateChart(data) {
        	if (chart) {
            	chart.destroy(); // Destroy the existing chart instance
        	}
        	createChart(data);
    	}
	</script>
</body>
</html>
```

**3. Initializing the Chart**

The `createChart` function initializes the chart with the first data point received. It sets up two datasets, one for temperature and one for humidity. The `updateChart` function updates the chart with new data points as they arrive.

**4. Updating the Chart**

Each time new data is received through the WebSocket, the `updateChart` function adds the latest data to the chart and updates the display.

**5. Styling the Dashboard**

Add some basic CSS to improve the dashboard's appearance. Create a `public/styles.css` file and include it in the HTML file:

```css
body {
    font-family: Arial, sans-serif;
    display: flex;
    flex-direction: column;
    align-items: center;
    justify-content: center;
    height: 100vh;
    margin: 0;
}
h1 {
    margin-bottom: 20px;
}
#data {
    margin-bottom: 20px;
    text-align: center;
}
canvas {
    max-width: 100%;
    height: auto;
}
```

Ensure your dashboard is optimized for mobile devices by implementing responsive design principles. Adjust layout and styling to provide a seamless user experience across different screen sizes.

```css
/* Example CSS for responsive design */
@media (max-width: 768px) {
	body {
    	font-size: 14px;
	}
	/* Adjust other styles as needed */
}
```

Look at this thing of beauty!

![The finished dashboard showing a graph of the temperature and humidity](https://cdn.hashnode.com/res/hashnode/image/upload/v1719570257170/8ab54698-5c3c-47d2-ae58-322f02bcc424.png?auto=compress,format&format=webp)

Your interactive dashboard is now complete! It receives real-time data from the simulated IoT device and visualizes it using Chart.js. The dynamic chart updates automatically as new data arrives, providing a clear and engaging way to monitor sensor readings.

Consider adding data analysis and insights features, such as trend analysis, statistical summaries, or predictive analytics based on historical data trends. These enhancements can provide valuable insights for users.

By enhancing data visualization on the frontend, you're creating a more engaging and informative IoT dashboard. Experiment with these customization options to tailor the dashboard to your project's requirements and improve user interaction with real-time sensor data.


## Wrapping it up!

Congratulations on building your own real-time IoT dashboard using HarperDB and Chart.js! Throughout this tutorial, we've covered essential steps to set up a robust data pipeline, visualize sensor data dynamically, and enhance user interaction with real-time alerts and responsive design.

By customizing Chart.js configurations and integrating additional widgets, you've learned how to tailor data visualization to specific project needs. Implementing real-time alerts and notifications ensures timely updates on critical sensor readings, enhancing the dashboard's utility.
Remember, optimizing for mobile and responsive design ensures a seamless user experience across devices, making your dashboard accessible anytime, anywhere. Exploring further enhancements like data analysis and insights can provide deeper insights into sensor trends and performance metrics.

As you refine and expand your IoT dashboard, don't hesitate to experiment with different visualizations and features. Embrace a hands-on approach, iterate based on user feedback, and stay curious about new technologies and possibilities.

Keep building, learning, and pushing the boundaries of what's possible with IoT and data visualization. Share your creations with the developer community and showcase your skills in crafting meaningful and impactful technical solutions.

**Happy coding!**

Here are some other resources you might enjoy!

How to Build a Real-Time Crypto Dashboard with HarperDB
In this tutorial, Michael King demonstrates creating a straightforward dynamic dashboard to display cryptocurrency prices and news articles using HarperDB. The article covers setting up the database, using Angular for the frontend, and integrating RSS feed parsing.

Real-Time Communication with HarperDB
Learn about using WebSocket servers and HarperDB's leaf stream for real-time communication. This article explores setting up custom functions and creating a real-time chat application.

Build a Dynamic REST API with Custom Functions
Terra Roush's tutorial guides you through building a REST API with custom functions in HarperDB. It covers creating endpoints, handling requests, and integrating with your application.