Monitor Your Flask Web Application Automatically With Flask-Monitoring-Dashboard
By: Johan Settlin
Tags: Monitoring, Flask, Python, Web Development, DevOps

Part 2: Flask-Monitoring-Dashboard

How to set up Flask-Monitoring-Dashboard

Installation

pip install flask_monitoringdashboard

Hopefully, pip will install the correct dependencies automatically.

If you get dependency errors in the form:

flask-monitoringdashboard <version> has requirement <correct_package>, but you’ll have <wrong_package> which is incompatible.

You will have to install <correct_package> manually.

To automate this process in the future, list all the correct packages in a requirements.txt file in the root directory of your flask app. Here’s the requirements.txt used in this tutorial:

APScheduler==3.6.3

click==7.1.1

colorhash==1.0.2

configparser==5.0.0

Flask==1.1.2

Flask-MonitoringDashboard==3.0.9

itsdangerous==1.1.0

Jinja2==2.11.1

MarkupSafe==1.1.1

numpy==1.18.3

psutil==5.7.0

pytz==2019.3

six==1.14.0

SQLAlchemy==1.3.16

tzlocal==2.0.0

Werkzeug==1.0.1

Use this command to install all the packages listed in requirements.txt:

pip install -r requirements.txt

After successfully installing Flask-Monitoring-Dashboard in the virtual environment, you can add it to your app with these two lines:

import flask_monitoringdashboard as dashboard

dashboard.bind(app)

app.py should now look something like this:

from flask import Flask import flask_monitoringdashboard as dashboard app = Flask(__name__)

dashboard.bind(app) @app.route('/')

def hello_world():

return 'Flask-Monitoring-Dashboard tutorial' @app.route('/endpoint1')

def endpoint1():

time.sleep(0.20)

return 'Endpoint1', 400 @app.route('/endpoint2')

def endpoint2():

time.sleep(5)

return 'Endpoint2'

Restart the server and localhost/dashboard should display the Flask-Monitoring-Dashboard login screen.

Configuration

The next step is to create a configuration file for the dashboard. The path to the configuration file is set by adding this line to your app:

dashboard.config.init_from(file='/<path to file>/<config file>')

Alternatively, an environment variable can be used by adding this line instead:

dashboard.config.init_from(envvar='FLASK_MONITORING_DASHBOARD_CONFIG')

which must be set before running the flask app:

Unix:

export FLASK_MONITORING_DASHBOARD_CONFIG=/<path to file>/<config file> Windows:

set FLASK_MONITORING_DASHBOARD_CONFIG=/<path to file>/<config file>

Let’s name our config file ‘config.cfg’ and put it in the app’s root directory:

app = Flask(__name__)

dashboard.config.init_from(file='/config.cfg') # Make sure to call config.init_from() before bind()

dashboard.bind(app)

The config file we use contains all settings and looks like this:

[dashboard]

APP_VERSION=1.0

CUSTOM_LINK=dashboard

MONITOR_LEVEL=3

OUTLIER_DETECTION_CONSTANT=2. [authentication]

USERNAME=admin

PASSWORD=admin

GUEST_USERNAME=guest

GUEST_PASSWORD=['dashboardguest!', 'second_pw!']

SECURITY_TOKEN=cc83733cb0af8b884ff6577086b87909 [database]

DATABASE=sqlite:///flask_monitoringdashboard.db [visualization]

TIMEZONE=Europe/Stockholm

COLORS={'main':'[0,97,255]', 'static':'[255,153,0]'}

As you can see there is 4 different sections in the configuration file, dashboard, authentication, database and visualization.

The dashboard section:

APP_VERSION specify which version of the application (your own Flask application) that is running. This can be used to compare the performance of different versions later.

specify which version of the application (your own Flask application) that is running. This can be used to compare the performance of different versions later. CUSTOM_LINK specify the urls to the monitoring service

specify the urls to the monitoring service MONITOR_LEVEL sets the level of how much you want to monitor your endpoints (there is four levels 0,1,2,3. See below for a detailed explanation of each level)

sets the level of how much you want to monitor your endpoints (there is four levels 0,1,2,3. See below for a detailed explanation of each level) OUTLIER_DETECTION_CONSTANT sets a constant that is used to calculate when information about outliers should be stored in the database (average execution time * constant).

The authentication section:

USERNAME & PASSWORD is used for logging in to the dashboard

is used for logging in to the dashboard GUEST_USERNAME & _PASSWORD is to get access to the dashboard with only read rights.

is to get access to the dashboard with only read rights. SECURITY_TOKEN token that is used for exporting the data.

The database section:

DATABASE the database that results from the monitoring are stored in.

The visualization section:

TIMEZONE the timezone for converting a UTC timestamp to a local timestamp

the timezone for converting a UTC timestamp to a local timestamp COLORS here you can color your endpoints syntax {‘endpoint-name’:[color]}

Note: If you want your monitoring to automatically update the version of your application and you are using git there is a property called GIT where you can specify the path to your project and the version will automatically update based on the commit hash in the dashboard section.

How to use Flask-Monitoring-Dashboard

Dashboard Overview

Logging in to Flask-Monitoring-Dashboard will bring up the Dashboard overview. The overview has a table with general statistics for each endpoint in your app. There is a Number of hits tab and a Median request duration tab. The Monitoring-level column is used to set monitoring level for each endpoint:

The monitoring level of an endpoint decides what type of data should be collected for it. Flask-Monitoring-Dashboard can collect number of hits (utilization), request durations (performance), extra information about slow requests (outliers), and performance per line of code (profiler).

Monitoring levels

Level 0: Disabled, only collects time since last request.

Level 1: Level 0 + Utilization and Performance

Level 2: Level 1 + Outliers

Level 3: Level 2 + Profiler

Note: see the documentation for information about how outliers are identified and how the profiler works.

Application graphs

Built-in graphs with data for all endpoints (with monitoring level => 1) are available in the Dashboard section of the main menu.

Hourly API Utilization: This graph shows total number of hits (all endpoints) each hour for all days in the selected interval. Interval is selected with the Start date and End date fields in graph options.

This graph shows total number of hits (all endpoints) each hour for all days in the selected interval. Interval is selected with the Start date and End date fields in graph options. Multi Version API Utilization: This graph shows number of hits for selected endpoints in the selected application versions. Which endpoints and versions to display in the graph can be selected in graph options. The APP_VERSION field in the config determines current version.

This graph shows number of hits for selected endpoints in the selected application versions. Which endpoints and versions to display in the graph can be selected in graph options. The APP_VERSION field in the config determines current version. Daily API Utilization: This graph shows number of hits each day for each endpoint in the selected interval. Interval is selected with the Start date and End date fields in graph options.

This graph shows number of hits each day for each endpoint in the selected interval. Interval is selected with the Start date and End date fields in graph options. API Performance: This graph shows duration of all requests, including median duration, for selected endpoints. Endpoints are selected in graph options.

This graph shows duration of all requests, including median duration, for selected endpoints. Endpoints are selected in graph options. Reporting: On this page, graphs that compare application performance between two time periods can be generated. Performance is compared in terms of average latency and status code distribution. Use the tabs to choose which intervals to compare. Custom intervals can be chosen in the Custom tab.

Endpoint graphs

Clicking on an endpoint in the Overview table will bring up built-in graphs for that endpoint. All built-in graphs for the endpoint will listed in the endpoint’s section of the main menu.

Hourly API Utilization: Requires monitoring level 1 or above. This graph shows number of hits each hour for all days in the selected interval for the endpoint. Interval is selected with the Start date and End date fields in graph options. Requires monitoring level 1 or above.

Requires monitoring level 1 or above. This graph shows number of hits each hour for all days in the selected interval for the endpoint. Interval is selected with the Start date and End date fields in graph options. Requires monitoring level 1 or above. User-Focused Multi Version API Utilization: This graph shows number of hits per user and version for the endpoint. Which users and versions to display in the graph can be selected in graph options. Requires monitoring level 1 or above.

This graph shows number of hits per user and version for the endpoint. Which users and versions to display in the graph can be selected in graph options. Requires monitoring level 1 or above. IP-Focused Multi Version API Utilization: This graph shows number of hits per IP and version for the endpoint. Which IPs and versions to display in the graph can be selected in graph options. Requires monitoring level 1 or above.

This graph shows number of hits per IP and version for the endpoint. Which IPs and versions to display in the graph can be selected in graph options. Requires monitoring level 1 or above. Per-Version performance: This graph shows all requests for the endpoint per version. Which versions to display in the graph can be selected in graph options. Requires monitoring level 1 or above.

This graph shows all requests for the endpoint per version. Which versions to display in the graph can be selected in graph options. Requires monitoring level 1 or above. Per-User performance: This graph shows all requests for the endpoint per user. Which users to display in the graph can be selected in graph options. Requires monitoring level 1 or above.

This graph shows all requests for the endpoint per user. Which users to display in the graph can be selected in graph options. Requires monitoring level 1 or above. Profiler: This page shows execution paths as trees for each request in the endpoint, with execution time and percentage for each step in execution. Requires monitoring level 3.

This page shows execution paths as trees for each request in the endpoint, with execution time and percentage for each step in execution. Requires monitoring level 3. Grouped Profiler: This page shows the combined execution path as a tree of all requests in the endpoint, execution time and percentage for each step in execution. Requires monitoring level 3.

This page shows the combined execution path as a tree of all requests in the endpoint, execution time and percentage for each step in execution. Requires monitoring level 3. Outliers: This page shows extra information collected on outliers. The graph shows cpu load (%) for the outliers. Requires monitoring level 2 or above.

This page shows extra information collected on outliers. The graph shows cpu load (%) for the outliers. Requires monitoring level 2 or above. Status Code Distribution: Shows status code distribution for the endpoint. Requires monitoring level 1 or above.

Create your own custom graph

You have the possibility to create tailored graphs that collect data that suits your needs. This could be the number of registered users, number of purchases of your product today or anything that comes up to your mind.

To create your own graph for monitoring you use the dashboard.add_graph() function. The function takes four arguments title, function, schedule type, schedule dictionary.

The title is a string containing the desired name of the graph, the function is the function that will be executed according to the scheduler and return the wanted data to monitor, scheduler type can be either “cron” or “interval” depending on which type you prefer and the scheduler dictionary is a dict specifies when to run the job.

We will create our own graph by first creating a function that returns the data we want to show in the graph.

In our app.py file add the following code after dashboard.config.init_from(file=’config.cfg’):

def numberOfNewCustomers():

return float(randint(1,5))

then we should create a new scheduler dictionary (we will schedule for every 10 second to get fast results)

numberOfNewCustomers_schedule = {'seconds': 10}

If you would like to create a scheduling for every day at midnight you could use (use the ‘cron’ arguments for this one):

midnight_schedule = {'month':"*",

'day': "*",

'hour': 23,

'minute': 59,

'second': 00}

The next step is to create a new dashboard by adding the following code:

dashboard.add_graph("Every 10 Seconds", numberOfNewCustomers, "interval", **numberOfNewCustomers_schedule) dashboard.bind(app)

Notice that dashboard.bind(app) is after the add_graph call

In this example we use the interval scheduling type you can also use cron as argument.

This will generate a new graph under custom graphs in the UI called “Every 10 Seconds”

Note: if the data is not shown in the graph there is a bug that the end date is set to today. To fix this you can scroll down under your graph and set the end date to tomorrow, this is a bug that is reported and might be fixed by now.

The End

That concludes our tutorial. We hope that you have found this tutorial on how to use the Flask-Monitoring-Dashboard extension in your flask projects helpful. The content of this tutorial is sourced from Flask-Monitoring-Dashboard’s great official documentation, which we encourage you to visit if you want to learn more.