Build Your Own Uptime Monitor with MeteorJS + Fetch + Plotly.js ☄️🔭
Tags: javascript, tutorial, meteor, beginners
Published: 2024-03-08 13:44:52

In this tutorial I'll going to show you how to create your own uptime monitor in six easy steps. The app will use of the following stack, database to frontend:

- [MongoDB](https://www.mongodb.com/) to store our data as documents, close to JS objects
- [MeteorJS](https://www.meteor.com/) to build and run our app and handle the data layer out-of-the-box
- [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) at the heart of the monitoring requests to the external server
- [React](https://react.dev/) as our frontend library
- [Plotly](https://plotly.com/javascript/) as our chart renderer

## Table of Contents

<!-- START doctoc generated TOC please keep comment here to allow auto update -->

- [Concept and Architecture](#concept-and-architecture)
- [Create a New React Project with MeteorJS](#create-a-new-react-project-with-meteorjs)
- [Implementation](#implementation)
  - [Step 1 - Configuration and Configurability](#step-1-configuration-and-configurability)
  - [Step 2 - Create Collections](#step-2-create-collections)
  - [Step 3 - Create a Handler for the Collections](#step-3-create-a-handler-for-the-collections)
  - [Step 4 - Create an Uptime Request](#step-4-create-an-uptime-request)
  - [Step 5 - Server Startup](#step-5-server-startup)
  - [Step 6 - The Monitoring Frontend](#step-6-the-monitoring-frontend)
- [Where to go from here](#where-to-go-from-here)
- [About me](#about-me)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

## Concept and Architecture

1. There is an HTTP endpoint on your external server that is constantly requested by your Meteor app's server using `fetch`.

2. Each response and it's status, including timeouts and latency measures, is stored as a "document" in the Database, which is MongoDB by default. 

3. Meteor "watches" the collection of documents and "publishes" updates of the collection to clients, that "subscribe" to it. This allows live-monitoring to be set up for custom periods of time. 

4. The React client application subscribes to the live changes and uses [Plotly](https://plotly.com/javascript/react/) to display them as a line graph showing uptime and response time for a given period.

![Uptime Monitor App Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4lm5591fz42ywnfylmfj.png)

> Please note that you should only use this for websites that you own or have the permission to monitor. Make sure that you always act within the law.

## Create a New React Project with MeteorJS

If you do not have MeteorJS installed on your system, you may want to perform this quick install of the latest stable version:

```sh
$ npm install -g meteor
# on Linux||MacOS, you can also use the shell-installer script via
$ curl https://install.meteor.com/ | sh
```

Then, it's time to create a new project, once the `meteor` tool is installed and available:

```sh
$ meteor create uptime
```

This creates a new React-based Meteor project in the folder `uptime`. 

> Note: If you don't want to use React, then you can also see what other frontends are available by running `meteor create --help`. Note, however, that this tutorial focuses on Meteor+React
 
Go to the `uptime` folder and install some dependencies, then run the project for the first time:

```js
$ cd uptime
$ meteor npm install react-plotly.js plotly.js node-abort-controller
$ echo "{}" >> settings.json
$ meteor --settings=settings.json
```
Your project is now running on http://localhost:3000

## Implementation

Most of the following steps are on the `server`. However, shared (isomorphic) code between client and server is placed in a separate folder called `/imports`. A new React project should have this folder by default, though.

### Step 1 - Configuration and Configurability

For our uptime measurements we use a simple [HTTP HEAD request](https://developer.mozilla.org/en-US/docs/web/http/methods/head). We use HEAD, because there is no response body sent over the wire and being parsed. 

However, the request can be configured using [Meteor's `settings.json` file](https://docs.meteor.com/api/core#Meteor-settings), similar to `.env`, allowing for easy changes in the future without the need to modify any code.

Let's create a new monitor in `settings.json`:

```json
{
  "monitors": {
    "myDomain": {
      "name": "MyDomainXYZ",
      "method": "HEAD",
      "url": "https://mycooldomain.xyz/reachability",
      "timeout": 10000,
      "interval": 5000,
      "response": 204
    }
  }
}
```

This configuration involves sending a HEAD request to the specified URL every 5 seconds and measuring the response time. The expected response is a 204 status code, but if there is no response within 10 seconds, the server will be considered 'down' and the request will be aborted.

It is important to note that using other request methods besides HEAD may take longer to process and could potentially consume more server resources.


### Step 2 - Create Collections

To send data to the client and persist it between server restarts, two MongoDB collections are required: one to preserve the overall monitor status and another to store the summary of each request as a document.

Create both collections in `/imports/collections/MonitorCollection.js` and `/imports/collections/StatusCollection.js`:

```js
import { Mongo } from 'meteor/mongo'
export const MonitorCollection = new Mongo.Collection('monitor')
```

```js
import { Mongo } from 'meteor/mongo'
export const StatusCollection = new Mongo.Collection('status')
```

They will be used later to automatically sync the data from the server to the client using "publish/subscribe".

### Step 3 - Create a Handler for the Collections

Instead of directly accessing the collections we actually want to control inflow and outflow of data. For this we create a `Monitor` object in `server/Monitor.js`, that will manage all this for us.

```js
import { Meteor } from 'meteor/meteor'
import { MonitorCollection } from '../imports/collections/MonitorCollection'
import { StatusCollection } from '../imports/collections/StatusCollection'

export const Monitor = {}
```

First, we want to be able to initially create a status for a configured domain, so that it's status/info is available, even if we haven't fired a request yet:

```js
// ... continue in server/Monitor.js

Monitor.init = async ({ name, url, method, interval }) => {
  return StatusCollection.upsertAsync({ name }, {
    $set: { name, url, method, interval } 
  })
}
```

This will also update the status, in case we changed our configuration in `settings.js`.

Okay, now let's think about what we actually want to measure. We need a `name` as key, a timestamp of the request, a duration, a `hasTImedOut` flag and an error, in case we want to read error details.

Furthermore, we want to update status only once it has changed and not every time a new document arrives. A simple first implementation could look like this:

```js
// ... continue in server/Monitor.js

Monitor.add = async document => {
  const { name, hasTimedOut, duration, error } = document
  const status = await StatusCollection.findOneAsync({ name })
  let statusChanged = false

  // we just went down :-/
  if (status.up && hasTimedOut) {
    status.up = false
    statusChanged = true
  }

  // we're back in business :-)
  if (!status.up && !hasTimedOut) {
    status.up = true
    statusChanged = true
  }

  if (statusChanged) {
    console.debug('[Monitor]: status changed:', status)
    await StatusCollection.updateAsync(status._id, { $set: status })
    // here is a good spot to send an Email to
    // inform about the status change!
  }

  await MonitorCollection.insertAsync(document)
}
```

Finally, we want to define the amount of data, send to the client. We do this by defining Meteor Publications. On the one hand, we want to publish the status of the monitor as a single document:

```js
Meteor.publish('status', function ({ name }) {
  return StatusCollection.find({ name },  { limit: 1 })
})
```

On the other hand we want a good bunch of request summaries, that will help us to build a nice graph of the uptime and response duration:

```js
// ... continue in server/Monitor.js

Meteor.publish('monitor', function ({ name }) {
  return MonitorCollection.find({ name }, { limit: 250, hint: { $natural: -1 }})
})
```

Note the `hint: { $natural: -1 }` which tells the collection to so a natural backwards search (start with the newest entry), omitting the need for `sort`.


### Step 4 - Create an Uptime Request

Now, that we have set up a collection management, we still need a handler that fires and evaluates our requests by our given configuration.

For this we create a `class` this time. Create a new file `server/UptimeRequest.js` and start with the following:

```js
import { fetch } from 'meteor/fetch'
import { AbortController } from "node-abort-controller"
import { PerformanceObserver, performance } from 'node:perf_hooks'

export class UptimeRequest {
  constructor ({ name, method, url, timeout, response }) {
    // config
    this.name = name
    this.method = method
    this.url = url
    this.timeout = timeout
    this.expectedResponse = response
    this.actualResponse = null

    // results / measurements
    this.response = null
    this.timestamp = null
    this.performance = null
    this.error = null
    this.hasTimedOut = null
    this.response = null
  }

  async fire () {
    // here we implement the actual request
    // and measure the outcome
  }

  toDocument () {
    // here we will return results that matter
  }
}
```

As you can see there is no `interval` as this class will be instantiated each time for a new request, thus one instance represents one request.

Let's implement the `fire` method:

```js
//... in the UptimeRequest class

async fire () {
    this.timestamp = new Date()

    // prepare performance measures
    // see https://nodejs.org/api/perf_hooks.html#performanceentryduration
    const observer = new PerformanceObserver((items) => {
      items.getEntries().forEach((entry) => {
        this.performance = entry
      })
    })
    observer.observe({ entryTypes: ["measure"], buffer: true })

    const start = `${this.name} - start`
    const ended = `${this.name} - ended`
    const controller = new AbortController()
    const options = {
      method: this.method,
      priority: 'high',
      redirect: 'error',
      signal: controller.signal
    }

    const timeout = setTimeout(() => {
      this.hasTimedOut = true
      controller.abort()
    }, this.timeout)

    try {
      performance.mark(start)
      this.response = await fetch(this.url, options)
    } catch (error) {
      this.error = error
    } finally {
      performance.mark(ended)
      clearTimeout(timeout)
      performance.measure(this.url, start, ended)
      observer.disconnect()
    }

    this.actualResponse = this.response?.status

    // post checkup
    if (!this.error && this.response.status !== this.expectedResponse) {
      this.error = new Error('Response code mismatch')
    }

    // normalize error
    if (this.error) {
      const error = {}
      Object.getOwnPropertyNames(this.error).forEach(prop => {
        error[prop] = this.error[prop]
      })
      delete error.stack
      this.error = error
    }
  }

// ...
```

Without going into too much details, this method creates a fetch request with a so called `AbortSignal` that aborts the request, once our given timeout has been exceeded. On top it's surrounded by Node's native performance measurement to measure the overall request-response duration.

You can read up everything that happens in this method using the following resources:

- https://nodejs.org/api/perf_hooks.html#performanceentryduration
- https://developer.mozilla.org/en-US/docs/Web/API/AbortController
- https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch
- https://stackoverflow.com/a/26199752/3098783

Finally we want to provide a summary as document to the outside consumers by implementing a simple `toDocument` method:

```js
//... in the UptimeRequest class

  toDocument () {
    return {
      name: this.name,
      response: {
        expected: this.expectedResponse,
        actual: this.actualResponse,
      },
      fired: this.timestamp,
      duration: this.performance?.duration ?? 0,
      hasTimedOut: !!this.hasTimedOut,
      error: this.error
    }
  }

// ...
```

### Step 5 - Server Startup

We can finally run our server-side code that will request the external server and saves the results and status in the database. For this, remove all code in `server/main.js` and replace it with this one:

```js
import { Meteor } from 'meteor/meteor'
import { UptimeRequest } from './UptimeRequest'
import { Monitor } from './Monitor'

Meteor.startup(async () => {
  const { myDomain } = Meteor.settings.monitors

  await Monitor.init(myDomain)

  const monitor = async () => {
    const request = new UptimeRequest(myDomain)
    await request.fire()
    return Monitor.add(request.toDocument())
  }

  Meteor.setInterval(monitor, myDomain.interval)
})
```
On startup we first register or update our domain (see the `Monitor.init` method in step 1.3) and then run a request for each interval, adding the result to the collections and updating the status.



### Step 6 - The Monitoring Frontend

The client part is much shorter and will actually fit in one step. This is, because here it combines the magic of Meteor's reactivity together with the magic of Plotly's auto-generation of charts, just to save you hours and hours of coding! 🧙‍♂️

For this you only need to change the file `imports/ui/App.jsx` to the following:

```jsx
import React from 'react'
import { MonitorCollection } from '../collections/MonitorCollection'
import { StatusCollection } from '../collections/StatusCollection'
import { useTracker, useSubscribe, useFind } from 'meteor/react-meteor-data'
import Plot from 'react-plotly.js'

const name = 'MyDomainXYZ'

export const App = () => {
  // subscribe to status, call
  // statusLoading() to know if it's loading or not
  const statusLoading = useSubscribe('status', { name })

  // subscribe to monitor docs,
  // use monitorLoading() to know if it's loading or not
  const monitorLoading = useSubscribe('monitor', { name })

  // the actual status doc, which will be undefined,
  // as long as the subscription is still loading
  const [status] = useFind(() => StatusCollection.find())
  const [uptime, duration] = useTracker(() => {
    const docs = MonitorCollection.find({}, {
      // make sure the client sorts the docs,
      // in case they arrive in a different order
      // than the server queried them
      sort: { fired: 1 }
    })
    const updatime = {
      x: [], // doc index as Date
      y: [], // up or down
      type: 'scatter'
    }

    const duration = {
      x: [],
      y: [],
      type: 'scatter'
    }
    docs.forEach((doc, index) => {
      updatime.x.push(doc.fired)
      duration.x.push(doc.fired)

      updatime.y.push(doc.hasTimedOut ? 'down' : 'up')
      duration.y.push(doc.hasTimedOut ? 0 : doc.duration)
    })
    return [updatime, duration]
  }, [])

  return (
    <div>
      <h1>Uptime Monitor for "{name}"</h1>
      <StatusInfo status={status} loading={statusLoading() || monitorLoading()}/>
      <Plot data={[uptime]} layout={{ width: 800, height: 400, title: 'Uptime' }} config={{ responsive: true }}/>
      <Plot data={[duration]} layout={{ width: 800, height: 400, title: 'Duration' }} config={{ responsive: true }}/>
    </div>
  )
}

// pure component, could be used with memo
const StatusInfo = ({ status, loading }) => {
  if (loading || !status) return (<div>Status loading...</div>)

  return (
    <div>
      <span>Status: {status.up ? 'up' : 'down'}</span><span> | </span>
      <span>URL: {status.url}</span><span> | </span>
      <span>Method: {status.method}</span><span> | </span>
      <span>Interval: {status.interval / 1000}s</span>
    </div>
  )
}
```

The result are two line-graphs that update every 5 seconds, as defined in our config:

![Final plot result](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tlijiwop6yf9lv3qmvzh.png)

The magic behind this very short code is actually the hooks, provided by [the `react-meteor-data` package](https://packosphere.com/meteor/react-meteor-data), which is already included in every new Meteor-React project. Here are some excerpts:

#### `useTracker`

> You can use the useTracker hook to get the value of a Tracker reactive function in your React "function components." The reactive function will get re-run whenever its reactive inputs change, and the component will re-render with the new value.

To provide some more context: This hook uses Meteor's [Tracker](https://docs.meteor.com/api/tracker.html) to listen on changes in a Data-set, such as our two collections. Under the hood there is an observer that runs, every time a document is added, removed or updated on this collection, invoking a Tracker computation, which in turn invokes a new React computation, similar to what you already know from other React hooks.

#### `useSubscribe`

> At its core, it is a very simple wrapper around useTracker (with no deps) to create the subscription in a safe way, and allows you to avoid some of the ceremony around defining a factory and defining deps. Just pass the name of your subscription, and your arguments.

#### `useFind`

> The useFind hook can substantially speed up the rendering (and rerendering) of lists coming from mongo queries (subscriptions). It does this by controlling document object references. By providing a highly tailored cursor management within the hook, using the Cursor.observe API, useFind carefully updates only the object references changed during a DDP update.

## Where to go from here

The app in this tutorial is far from complete. However, I compiled a few ideas for your next steps with this one:

- update to Meteor 3.0 and deploy to an ARM device like a Raspberry PI (we may cover this in the future if demand is high)
- [deploy your app to Meteor Cloud (managed hsoting)](https://www.meteor.com/cloud?utm_source=jkuester&utm_medium=online&utm_campaign=Q2-2022-Ambassadors) or your own infrastructure using [Meteor-Up](https://meteor-up.com/)
- extend the app to monitor multiple clients
- use a job / cron-job library, instead of `setInterval`
- create a dashboard for your monitors
- add [Meteor Accounts](https://docs.meteor.com/api/accounts.html) to manage users, authentication etc.
- improve the Plot layouts and make them visually more appealing
- use caching and capped collections or redis-oplog for scaling

---
## About me 👋

I regularly publish articles about **Meteor.js** and **JavaScript** here on dev.to. I also recently co-hosted the [weekly Meteor.js Community Podcast](https://www.youtube.com/@meteorjscommunity), which covers the latest in Meteor.js and the community.

You can also find me (and contact me) on [GitHub](https://github.com/jankapunkt/), [Twitter/X](https://twitter.com/kuester_jan) and [LinkedIn](https://www.linkedin.com/in/jan-kuester/).

If you like what you read and want to support me, you can [sponsor me on GitHub](https://github.com/jankapunkt), [send me a tip via PayPal](https://paypal.me/kuesterjan) or sponsor a book from [my Amazon wishlist](https://www.amazon.de/-/en/hz/wishlist/ls/12YMIY0QNH9TK?ref_=list_d_wl_lfu_nav_1).