Realtime Data Visualization with Peak Detection using Socket.IO, D3 and React 📈📈
Tags: dataviz, javascript, react, d3js
Published: 2019-10-30 11:06:02

---
published: true
title: Realtime Data Visualization with Peak Detection using Socket.IO, D3 and React 📈📈
tags: DataViz, JavaScript, React, D3js
cover_image: https://iammowgoud.com/blog/assets/imgs/peak-detection/thumb.png
canonical_url: https://iammowgoud.com/blog/Realtime-Data-Visualization-with-Peak-Detection-using-SocketIO-D3-and-React/
---


*Originally posted on my [personal blog](https://iammowgoud.com/blog/Realtime-Data-Visualization-with-Peak-Detection-using-SocketIO-D3-and-React/) (better formatting)*

> **TLDR:** This post is **UX research** + **fullstack programming tutorial** for building a web application that streams time series data to be visualized in realtime using multiple Javascript frameworks.
>
> [Github](https://github.com/iammowgoud/Peak-Detection-Visualization) | [Demo](https://peak-detection-visualization.herokuapp.com/)

# The Why?

IoT is growing rapidly these days and one aspect that's key to the success of this kind of projects is data visualization. Design and UX is actually one of the most critical parts in any IoT project.

Any SME may have a very good infrastructure and complex integrations, but what would that do if their systems lack the tools to transform this gathered into actionable insights that can deliver real business value.

One thing is to collect some data from those new cool devices, another is to visualize it and make it accessible for your business or even your customer.

## Use case: what is happening here?

In this project I'm trying to build a React app that can consume data using Socket.io from a live source of data (we'll call it a **Sensor**) and visualize the time series data (we'll call it **Readings**) in realtime using D3.js

To simulate the **Sensor**, I created a very simple server using Express.js that rotate over a JSON file and keeps on continuously sending the **Readings** one by one *every 1.5 seconds*.

TBH I didn't spend much time on the server development and didn't implement proper error handling/reporting because that's not the scope of the project.

It's worth noting that while this project is built to work with a ***stateful*** WebSockets API assuming a realtime streaming use case. We could also make it support batch streaming REST API with a periodic server pinging on a longer interval. This really depends on the nature of the data and what level of granularity you want to have. I'm assuming the customer wants to do deep analysis and monitoring down to the interval of 1.5 seconds.

## Peak detection

In the world of Real-Time Signal Analysis, **peak detection** is a hot topic for many engineering fields including chemistry, biology, biomedical, optics, astrophysics and energy systems.

To add some spice to this sample project, I included in the sample data the ***Moving Z-score*** of each data-point beside the actual sensor value.

The Moving Z-score is a mathematical model for measuring the anomalousness of each point in sequential time series. One of its main parameters is the window size *(w)*. Given that the moving Z-score is the number of standard deviations each data-point is away from the mean, *(w)* limits the window in which we calculate the such statistical parameters. In this specific use case, the mean and standard deviation are computed only over the previous *(w)* observations.

In this scenario the Z-score would be a binary (0/1) value that tells whether there's a **"peak"** at this point or not.

 ![Z-Score](https://iammowgoud.com/blog/assets/imgs/peak-detection/z-score.svg)

In this project I got hold of a dataset with manually pre-calculated z-score to detect if the sensor readings increase  suddenly (aka ***peaks***) over a certain period of time. I included the scores in the mock data on **server side**.

Later, I'm planning to revisit this project to do this calculation on **client-side**, but given we need a (w) of data-points to get the first value so there would be a little bit of lag in initializing the visualization.

# UX Research

So now we know that we have two pieces of information at any certain point of time:

1. Actual sensor reading
2. A binary value for whether there's a peak or not

Let's start by asking ourselves a few questions:

- How to show those 2 values **effectively** and **clearly**?
- How to create **readable** and meaningful time series chart?
- How to design unique and usable experience for the user interacting with the visualization?

### Data Visualization should be "beautiful"

This is a must. Whichever use case you have or application you are building, if it's an IoT monitoring tool or a fancy marketing dashboard, an ugly looking visualization will discourage your end users from looking at it and may actually **prevent** them from understanding the meaning behind it and what insights the data entails.

But what is an "ugly" visualization? What makes it "beautiful" and visually appealing instead?

Well, this is subjective. There is no silver bullet. But with some conventional wisdom and lots of experience you realize that you should consider following:

- **Use the right type of diagram:**

Each type of a diagram was designed to represent data in a different way and focus on one aspect of the data. Obviously, graphing population of distant cities on world map wouldn't be the best option and using a pie chart to display more than 2-3 variables is a big no, and so on.

  Although there's some interesting visualizations for time series like Stream and Gantt charts and there's always room of creativity and artistic improvisation in data visualization, customers tend to like what they are familiar with. And we only have two variables in this project.

  *Well...this is a time series. It's gonna be a line graph.*

 ![Line Chart](https://iammowgoud.com/blog/assets/imgs/peak-detection/steps/line.svg)

- **Avoid overcrowded areas:**

  Too much elements in a tiny space can only be justified in very limited cases. Points in a scatter plot for example can be tolerated but also, it's called a **scatter** plot for a reason. Only when the crowded (or *unscattered*) data points have the same meaning you can allow crowding them together to show **density**, but if they have different meaning and you can't them apart because of the crowd jam ,you are doing something wrong.

  *This is why I started my visualization by drawing it in its simplest form then adding on top of it. I removed all of the [chart junk](https://en.wikipedia.org/wiki/Chartjunk) for now and will add whatever I need along the way.*

- **Avoid mismatching or vague colors:**

  Colors are very subjective too, and they are associated with different feelings. Some feelings are obvious like hot red and warm blue, but if your data doesn't represent temperature? Also, some feelings or ideas associated with certain colors are cultural and differ from one target group to another.

  There's lots of science behind [color theory](https://www.smashingmagazine.com/2010/01/color-theory-for-designers-part-1-the-meaning-of-color/) and why we perceive colors the way we do.
  So, for this challenge what I do is that I stick with some of the famous palettes that have proved to work over time. You can use this [cool color wheel by Adobe](https://color.adobe.com/explore) to find some preset palettes or [create](https://color.adobe.com/create) your own based on color harmonies like Analogous, Triad, or Complementary colors, then they have an ***Amazing*** feature where you can Copy CSS or Less.

  *For this project, I went with this simple palette that has 2 shades of green and 2 shades of red.*

  ![Colors](https://iammowgoud.com/blog/assets/imgs/peak-detection/steps/colors.svg)

### Compare & contrast

> Comparisons and contrasts lead to conclusions

A visualization has to reflect the meaning of data and be built as simple as possible to make comparisons easier so the user can draw conclusions.

First thing we need to contrast here is the Readings series to the Z-scores series, *so instead of showing the two series in different graphs we can overlay the peaks over the original signal (Readings) and decreased **opacity** to 10%*.

![Z-score Alignment](https://iammowgoud.com/blog/assets/imgs/peak-detection/steps/z-score-overlay.svg)

We face a problem here with **scales** or the unit of each value. You can't place a binary value in a line chart along a numerical value like the sensor readings.

In this case we need to improvise. To show the Z-score as a pink region over the line series in D3, I converted it to an **area series** to span the whole height of the graph. I normalized the 0-1 values to be 0-X where X is the highest value of readings displayed currently in view.

We also need to provide the user with a way to ***compare*** the sensors data to each other. Why? So, the customer can see if the peak pattern is happening in one sensor or in all of them, and most importantly if the pattern is happening across all sensors at the ***exact time*** or if there's a ***shift***.

Since I'm assuming there's only 3 sensors we are visualizing, we can't really use  a [small multiple](https://en.wikipedia.org/wiki/Small_multiple). What we can do is stack the 3 graphs we have on top of each other, making sure that all graphs are **horizontally aligned** to each other.

![Vertical Alignment](https://iammowgoud.com/blog/assets/imgs/peak-detection/steps/vertical-alignment.svg)

### Usability & Interaction

***Usability*** is the ease of access of an interface. It’s a sub-discipline of UX. Although UX design and usability are sometimes used interchangeably, usability has grown to have to be more than the ease of access. Usability now is [measurable](https://www.testingtime.com/en/blog/measuring-your-ux-and-usability/). Measuring usability is out of the scope of this blog post, so will will take a holistic approach towards increasing usability in general.

Since we are here, we need to introduce a new terms: **Dashboards** and  **Widgets**. A dashboard shows various semi-related visualizations that deliver a shared business value but not necessary from the same data source. Widgets are the building blocks of a dashboard.

The cards you've been seeing throughout the previous sections are all widgets. What do we need to consider now to make each single card/widget user friendly and most importantly relate them to each other ?

**Labels & Controls**

We need to show several labels to guide users where to look and help them understand what are they looking at, for this project we need to include the following:

- **Titles:**
  dashboard title and sensor title.
- Connectivity indicator:
  Here I'm assuming that the dashboard may get disconnect from a sensor for any reason. This happens a lot in IoT applications. We need to inform the user if one chart is ***outdated***.
- **Time series legend:**
  This will have ***2 functions***, it will tell the user which is the actual reading and which is the peak area, and it will act as a **toggle** so the user can *show/hide one of the two variables*.
- **Axes:**
  Beside the fact that we need to show the units and values of each dimension, we need to make it clear in which direction time is moving.
- **(Extra element) Last reading Timestamp:**
  Since for the x-axis, I'm truncating the time stamp to show only the seconds (:20, :30,...). I added the Full timestamp of the last reading in the bottom right corner of the widget.

**States**

Any frontend component goes through a cycle of several [states](https://en.wikipedia.org/wiki/Finite-state_machine). These states are driven by the business logic, in our case we have the following states for *each* Widget:

- **Connecting:**
  This is the initial state when the page is loading and we don't have enough information to show to the user.

- **Disconnected:**
  This is when a widget is disconnected for a server or client error. We also show the HTTP error message for debugging and [helping users report their issues](https://ux.stackexchange.com/questions/84838/is-it-important-to-display-http-error-codes-to-users).

  In this scenario we can't only rely on the Connectivity indicator, we need to **explain to the user that the current data in view is not active**. *So we set the whole line graph opacity to 50%.*

- **Connected:** Everything is perfect 👌🏽

- **(Extra UI state) Mouseout:**
  This is mainly to make the visualization **less cluttered** and more visually appealing.

  Although this is debatable and some designers don't favor it, I removed the x-axis and last Reading timestamp if the user is not hovering on a Widget.

  My rationale behind this is that the customer is not really concerned with the exact time of each point but rather the main focus points of this visualization are the pink shaded Peak areas.
  If users really want to know when that happened, they can hover on the graph.

***Final design**:*
![UI States](https://iammowgoud.com/blog/assets/imgs/peak-detection/steps/controls.svg)

[Live Demo](https://peak-detection-visualization.herokuapp.com/)

# The How

## Engineering the solution

The frontend app contains two main classes:

- Chart: [src/components/Chart.js](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/src/components/Chart.js)
  - This is the main React component that connects to the relevant **sensor** to stream  **readings**, store it then does some data manipulation logic and finally initialize and update the D3 chart.
  - The React component has 1 **required prop** `sensorID` and optional `x-ticks` which has a default value of 20 and a max value of 50.
- D3TsChart: [src/d3-helpers/d3-ts-chart.js](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/src/d3-helpers/d3-ts-chart.js)
  - This is the custom class that handles the Time Series Chart graphics, and everything related to the chart SVG.
  - **Readings** are passed to this class to be rendered in DOM using D3 but never stored in the class itself. Data lives in the `Chart` component state.

![Component](https://iammowgoud.com/blog/assets/imgs/peak-detection/component.svg)

**File organization:**

```md
:root             // React component
  > api           // Express JS App
  > src
    > components
    > d3-helpers
    > styles
  > public
```

## Backend

The backend server is very simple, it's just a single Express.js file along with the the [data.json](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/api/data.json) file.

The data file contains mock data for 3 sensors. You can connect to the socket by pinging `https://localhost:4001?socket={sensorId}`. **sensorId** can only be 1 or 2 or 3 *for now*.

You can begin by creating the `api` folder and installing the 2 needed packages:

```bash
npm install -s socket.io express
```

First thing we need to import the server requirements and initialize the Express.js server (`app`) and wrap it by the socket.IO server (`io`). We will also import the JSON data and set a const `INTERVAL` of 1.5 seconds. This is how frequent we will emit data to each client connected to the server

```javascript
const http = require('http');
const express = require('express');
const socketIO = require('socket.io');

const app = express();
const server = http.createServer(app);
const io = socketIO(server);

const port = process.env.PORT || 4001;

const INTERVAL = 1500;
const sensorData = require('./data.json');
```

To keep track of each client connected to the server we will create a custom object that will keep track of 1. which sensor data was requested by the client, 2. index of the next data point to serve, and 3. the `setInterval` reference that will emit data each 1.5 seconds (`INTERVAL`). Then we will store one object like this in a custom dictionary attached to the Socket.IO `io` object.

```javascript
// Connection object interface
// {
//     sensorId,
//     index,
//     interval
// }

io.connections = {};
```

The idea behind storing the `setInterval` is that we need to emit the data periodically and we will also need to stop (`clearInterval`) this interval when a client disconnects from the server.

Now we need to listen to and handle the clients `connect` and `disconnect` in the Socket.IO server, and then emit data accordingly using the `emitData` function:

```javascript
io.on('connection', (socket) => {
  const connectionId = socket.id;
  const sensorId = Number(socket.handshake.query['sensor']); //parse the sensorID
  console.log(`New client connected with id:${connectionId}`);

  // Add a client connection to the custom dictionary
  io.connections[connectionId] = {
    sensorId,
    index: 0,
    interval: setInterval(() => emitData(connectionId, socket), INTERVAL)
  };

  // Remove connection
  socket.on('disconnect', () => {
    clearInterval(io.connections[connectionId].interval)
    io.connections[connectionId] = undefined;

    console.log(`Client ${connectionId} disconnected`)
  });
});
```

After that we need to implement the `emitData()` function which basically:

1. Selects the relevant sensor data from the data file
2. calls getReading() which gets **One** Reading from the data file
3. Store the next Reading index in the connection object we created in `io.connections` dictionary last snippet.
4. Emit the reading with the event name 'reading'. We will listen to this in the client app next section.

```javascript
const emitData = (connectionId, socket) => {
  let conn = io.connections[connectionId]
  const { newIndex, response } = getNextReading(sensorData[conn.sensorId - 1], conn.index);

  console.log(`Emitted to client: ${connectionId}, sensor id:${conn.sensorId}, index:  ${conn.index}`);

  socket.emit("reading", JSON.stringify(response));

  conn.index = newIndex;
}

// Get the next reading for the selected socket
const getNextReading = (data, index) => {
  response = {
    timestamp: Date.now(),
    value: data.readings[index],
    zscore: data.zScores[index]
  };

  return { newIndex: (index + 1) % data.readings.length, response };
}
```

***Now if you add `"start": "node index.js"` to the scripts property in `package.json` file and then run `npm start` in the [api](https://github.com/iammowgoud/Peak-Detection-Visualization/tree/master/api) folder, the server will be up and running and read to serve clients.***

We can test the server using this [awesome Electron app](https://electronjs.org/apps/socket-io-tester) ...or go to the next section and start implementing the React app 👏🏽👏🏽👏🏽

## Frontend

As mentioned before the client app will basically contain the main React component `Chart` that renders one chart and is responsible of **controlling** and **passing data** to the D3.js chart that lives inside the a separate custom [`d3-ts-chart`](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/src/d3-helpers/d3-ts-chart.js) class.

### React App and `Chart` component

To initialize the React app we will use `create-react-app`. You can install that globally by running `npm i -g create-react-app`.

Then to initialize the actual code template we run `create-react-app realtime-client`. This will create a folder with the name "realtime-client" and npm install the needed packages inside it.

If you cd into the folder and run a simple `react-scripts start` you should have a simple react app built and served in your browser on `http://localhost:3000/`.

*Note that this will be the root folder of the project and the backend server will live in a subdirectory inside it with the name `api`.*

Now we need to install the extra packages we will use in the project. cd into the folder and run `npm i -s socket.io node-sass d3`.

I'm using `node-sass` to write the app styles which means you need to rename all the `.css` files into `.scss` and change the reference in the `index.js` file.

***Let's build a component***

The final Chart component is a big one. I will focus on the important parts here.

We will need to define some basics stuff:

1. The series list: which is a list of information about the series/lines that will be graphed. This is we will pass to the `D3TsChart` later to initialize the chart.
2. `tsChart` is the `D3TsChart` object that we will code later and it's the one responsible of all D3 related operations.
3. `socket` is the socketId object that we will use to connect to the server and listen to the data.
4. State: The React component state in which we will store the data and some info and flags about the chart.

**So the initial Component should start as the following:**

```jsx
import React from 'react';
import ReactDOM from 'react-dom';

import socketIOClient from 'socket.io-client';
import D3TsChart from '../d3-helpers/d3-ts-chart';

export class Chart extends React.Component {

  seriesList = [
    {
      name: 'sensor-data',
      type: 'LINE',
      stroke: '#038C7E',
      strokeWidth: 5,
      label: 'Readings',
      labelClass: 'readings',
    },
    {
      name: 'z-score',
      type: 'AREA',
      fill: 'rgba(216, 13, 49, 0.2)',
      stroke: 'transparent',
      strokeWidth: 0,
      label: 'Peaks',
      labelClass: 'z-score',
    }
  ]

  tsChart = new D3TsChart();

  socket;

  state = {
    data: [],
    lastTimestamp: null,
    connected: false,
    error: ''
  }

  componentDidMount() { }

  render = () => (
    <div className="card">
      <div className='chart-container'></div>
    </div>
  )
}
export default Chart;
```

Now we need to connect to the socket.IO server and fetch data for **one sensor** by its id. We will pass `sensorId` to the component as a **prop**. This should be done in the `componentDidMount()` function. After passing the component HTML element reference to `tsChart` and initializing the 2 lines to be drawn by D3, it will call the `connect()` function and will `disconnect()` in `componentWillUnmount()`.

Also notice that we listen to the "reading" event coming from the server and attach the `storeReading` handler to it.

**componentDidMount():**

```javascript
componentDidMount() {
    if (this.props['sensorId'] === undefined) throw new Error('You have to pass "sensorId" prop to Chart component');

    // Component enclosing DIV HTML reference.
    const parentRef = ReactDOM.findDOMNode(this);

    this.tsChart.init({
      // Let D3 draw the chart SVG inside .chart-container div
      elRef: parentRef.getElementsByClassName('chart-container')[0],
      classList: {
        svg: 'z-chart'
      }
    });

    this.tsChart.addSeries(this.seriesList[0]); // readings
    this.tsChart.addSeries(this.seriesList[1]); //z-score

    this.connect();
  }

  connect = () => {
    this.socket = socketIOClient(`/?sensor=${this.props.sensorId}`);
    this.socket.on('reading', this.storeReading);

    // Various Errors handling
    SOCKETIO_ERRORS.forEach(errType => {
      this.socket.on(errType, (error) => this.setError(errType, error));
    });
  }

  componentWillUnmount() {
    this.socket.disconnect();
  }
```

The Socket.IO several errors strings and other constants are to be found in the top of the file:

```javascript
const SOCKETIO_ERRORS = ['reconnect_error', 'connect_error', 'connect_timeout', 'connect_failed', 'error'];
const MAX_POINTS_TO_STORE = 20;
```

Now we need to implement the `storeReading` function that will store the readings into the component `state` and pass the new data to the `tsChart` object.

We first push the new reading to the current data then we update the `state.data` with the last `MAX_POINTS_TO_STORE` items. We also store some **meta data** like the `connected` indicator and the `lastTimestamp` to be displayed in UI. LAstly, we call the `updateChart()` method.

**storeReading():**

```javascript
  storeReading = (response) => {
    const reading = JSON.parse(response);
    this.setState((prevState) => {
      const data = prevState.data;
      const pointsToStore = Math.max(data.length - MAX_POINTS_TO_STORE, 0);

      data.push(reading);

      return {
        data: data.slice(pointsToStore),
        connected: true,
        error: false,
        lastTimestamp: new Date(data[data.length - 1].timestamp).toLocaleTimeString()
      };
    });

    this.updateChart();
  }
```

`updateChart()` is implemented as a separate function because this is where we calculate `highestValueInView` from the Readings series. This is done so we can ***normalize*** the zscores 0/1 and replaces the 1s with the highest value. This will essentially make the Peaks Area series take the whole height of the current data in view.

**updateChart():**

```javascript
  updateChart() {
    const data = this.state.data;
    const highestValueInView = Math.max(...data.map(p => p.value));
    const zLine = data.map(p => ({
        timestamp: p.timestamp,
        value: p.zscore ? highestValueInView : 0
        })
      );

    this.tsChart.adjustAxes(data);
    this.tsChart.setSeriesData('sensor-data', data, false);
    this.tsChart.setSeriesData('z-score', zLine, false);
  }
```

This all the basic logic needed to pass the data to the `D2TsChart` class object.

Now we need to update the `render()` function to display the meta data we store in `state`:

**final render():**

```jsx
 render = () => (
    <div className="card">

      <h2>{!this.state.lastTimestamp ? 'Connecting...' : `Sensor ${this.props.sensorId}`}</h2>

      <span className={'status ' + (this.state.connected ? 'success' : 'danger')}>
        {this.state.error}
        <i className="pulse"></i>
        {this.state.connected ? 'Connected' : 'Disconnected'}
      </span>

      <div className={'chart-container ' + (this.state.error ? 'faded' : '')}></div>

      <span className={'timestamp ' + (this.state.connected ? 'success' : 'danger')}>
        {this.state.connected ? '' : 'Last reading was at '}
        {this.state.lastTimestamp}
      </span>

    </div>
  )
```

Finally we need to update the React [`index.js`](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/src/index.js) to include the charts for the **3 Sensors** we can fetch from API.

**index.js:**

```jsx
import React from 'react';
import ReactDOM from 'react-dom';
import './styles/main.scss';
import Chart from './components/Chart';

ReactDOM.render(
  <div>
    <h1>Peak Detection Dashboard</h1>

    <Chart sensorId="1" />
    <Chart sensorId="2" />
    <Chart sensorId="3" />
  </div>
  , document.getElementById('root'));
```

You can find all the needed `scss` styles in the [`styles`](https://github.com/iammowgoud/Peak-Detection-Visualization/tree/master/src/styles) directory.

### D3 Time Series line graph

Here's where all the actual **"graphing"** happens. This is the class where we import [D3.js library](https://square.github.io/intro-to-d3/) and use it to append the different SVG elements to the HTML element stored in`elRef`.

We need to set some constants like `TRANSITION_DURATION` and `MAX_Y_TICKS`, and *for now* we only support two `SERIES_TYPES` in graphing: `LINE` and `AREA`.

**So this is how we start with the basic class:**

```javascript
import * as d3 from 'd3';

const SERIES_TYPES = ['LINE', 'AREA'];
const TRANSITION_DURATION = 100;
const MAX_Y_TICKS = 6;

export default class D3TsChart {
  margin = { top: 10, right: 30, bottom: 30, left: 30 };

  outerWidth; outerHeight;

   init({ elRef, width, height, classList }) {
    this.elRef = elRef;

    // If no width/height specified, SVG will inherit container element dimensions
    if (width === undefined) this.responsiveWidth = true;
    if (height === undefined) this.responsiveHeight = true;

    this.outerWidth = width || this.elRef.offsetWidth;
    this.outerHeight = height || this.elRef.offsetHeight;

    this.classList = classList || {};

    this.draw();
  }
}
```

You will notice that we pass some initial config to the chart in the `init` function including a width and height which are used to set up the graph layout according to the **[Margin Convention](https://bl.ocks.org/mbostock/3019563)**.

**draw():**

```javascript
  draw() {
    // Main SVG
    this.svg = d3.select(this.elRef)
      .append('svg')
      .attr('width', this.outerWidth)
      .attr('height', this.outerHeight)
      .classed(this.classList.svg || null, true);

    //Inner box group  (deducting margins)
    this.group = this.svg.append('g')
      .attr('width', this.outerWidth - this.margin.left - this.margin.right)
      .attr('height', this.outerHeight - this.margin.top - this.margin.bottom)
      .attr('transform', `translate(${this.margin.left} , ${this.margin.top})`)
      .classed(this.classList.group || null, true);

    // X Axis init
    this.xScale
      .range([0, this.outerWidth - this.margin.left - this.margin.right]);
    this.xAxisRef = this.group.append('g')
      .attr('transform', `translate(0,${this.outerHeight - this.margin.bottom})`)
      .classed('x-axis', true);

    // Y Axis init
    this.yScale
      .range([this.outerHeight - this.margin.bottom, 0]);
    this.yAxisRef = this.group.append('g')
      .attr('transform', 'translate(0, 0)')
      .classed('y-axis', true);
  }
```

Here we set the main SVG (with margins) and the inner group then we set the [scales](https://www.d3indepth.com/scales/) for X-axis and Y-axis.

Now we need to add the functions that will draw the series (line and area) inside the SVG.

**Add Series:**

```javascript
  addSeries({ name, type, fill, stroke, strokeWidth, id }) {
    if (this.seriesDict[name]) throw new Error('Series name must be unique!');
    if (!SERIES_TYPES.includes(type)) throw new Error('Series type not supported!');

    this.seriesDict[name] = {
      type,
      ref: this.group.append('path')
        .attr('fill', fill || 'none')
        .attr('stroke', stroke || 'black')
        .attr('stroke-width', strokeWidth || 2)
        .classed('series', true)
        .classed('hidden', false)
    };
  }

  setSeriesData(name, data, adjustAxes = true) {
    const series = this.seriesDict[name];

    switch (series.type) {
      case 'AREA':
        this.updateAreaSeries(series, data);
        break;
      case 'LINE':
      default:
        this.updateLineSeries(series, data);
        break;
    }
  }
```

**Updating data of a single series:**

```javascript

  updateLineSeries(series, data) {
    series.ref
      .datum(data)
      .transition().duration(TRANSITION_DURATION).ease(d3.easeQuadIn)
      .attr('d', d3.line()
        .x((d) => { return this.xScale(d.timestamp); })
        .y((d) => { return this.yScale(d.value); })
      );
  }

  updateAreaSeries(series, data) {
    series.ref
      .datum(data)
      .transition().duration(TRANSITION_DURATION).ease(d3.easeQuadIn)
      .attr('d', d3.area()
        .x((d) => { return this.xScale(d.timestamp); })
        .y0(this.yScale(0))
        .y1((d) => {
          return this.yScale(d.value);
        })
      );
  }
```

Then finally we will have a function to adjust the axes to the current data in view.

**adjustAxes():**

```javascript
 adjustAxes(data) {
    const maxValue = d3.max(data, (d) => d.value);

    this.xScale.domain(d3.extent(data, (d) => d.timestamp));
    this.xAxisRef
      .transition().duration(TRANSITION_DURATION).ease(d3.easeLinear)
      .call(d3.axisBottom(this.xScale));

    this.yScale.domain([0, maxValue]);
    this.yAxisRef
      .transition().duration(TRANSITION_DURATION).ease(d3.easeLinear)
      .call(
        d3.axisLeft(this.yScale)
          .ticks(maxValue < MAX_Y_TICKS ? maxValue : MAX_Y_TICKS)
          .tickFormat(d3.format('d'))
      );
  }
```

You can have a look a deeper a look at this class in the  D3TsChart definition file [src/d3-helpers/d3-ts-chart.js](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/src/d3-helpers/d3-ts-chart.js).

## Deploying to Heroku

To deploy to Heroku this app to [heroku](https://www.heroku.com/) we need to setup the app so it:

1. Builds the react app and move the static webapp to `api/public`
2. Run the Express.js server

We can do so by adding the proper commands in the root [package.json](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/package.json) file.

**Package.json:**

```json
...
  "scripts": {
    "start": "node api/index.js",
    "prebuild": "rm -r api/public",
    "build": "react-scripts build",
    "postbuild": "mv build api/public"
  }
...
```

Heroku will [automatically detect](https://devcenter.heroku.com/articles/deploying-nodejs) that this is a Node.js app and will run the npm commands correctly.

The **final step** here is to set up the ExpressJS app so it serves the static app in `api/public` directory.

**ExpressJS [index.js](https://github.com/iammowgoud/Peak-Detection-Visualization/blob/master/api/index.js):**

```javascript
  app.use(express.static(__dirname + '/public'));
```

<br>

***That's all folks.***

***Related Links:***

- [Moving Z-score Algorithm](https://turi.com/learn/userguide/anomaly_detection/moving_zscore.html)
- [Chart junk](https://en.wikipedia.org/wiki/Chartjunk)
- [Adobe Color](https://color.adobe.com/explore)
- [Beautiful Evidence Book by Edward R. Tufte](https://www.amazon.com/Beautiful-Evidence-Edward-R-Tufte/dp/1930824165)
- [Intro to Socket.IO](https://medium.com/@chathuranga94/introduction-to-socket-io-600025322cd2)
- [Intro to React](https://reactjs.org/tutorial/tutorial.html)
- [Intro to D3.js](https://square.github.io/intro-to-d3/)
- [D3 Margin Convention](https://bl.ocks.org/mbostock/3019563)
- [D3 Scale functions in depth](https://www.d3indepth.com/scales/)
- [Deploying Node.js Apps on Heroku](https://devcenter.heroku.com/articles/deploying-nodejs)