Monitor your Python application with full stack open source APM tool - SigNoz
Tags: python, devops, opensource, monitoring
Published: 2021-06-14 12:44:17

If you want to check our Github repo before diving in 👇

<a href="https://bit.ly/3prv3WN">![SigNoz github repo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ogp68gbgfrp3vtigh3ki.png)</a>

__The cost of a millisecond.__

TABB Group, a financial services industry research firm, [estimates](https://research.tabbgroup.com/report/v06-007-value-millisecond-finding-optimal-speed-trading-infrastructure) that if a broker's electronic trading platform is 5 milliseconds behind the competition, it could cost $4 million in revenue per millisecond.

The cost of latency is too high in the financial services industry, and the same is true for almost any software-based business today. For Google, half a second delay in search results caused a 20% drop in traffic. Half a second is enough to kill user satisfaction to a point where they abandon an app's service.

While a user sees a screen, there are thousands of services in the background taking care of a user's request. In a microservices architecture, the challenge for engineering teams is to constantly figure out areas of optimization in a complex distributed network. And the solution starts with setting up a robust monitoring infrastructure for the application's production environment.

Capturing and analyzing data about your production environment is critical. You need to proactively solve stability and performance issues in your web application to avoid system failures and ensure a smooth user experience.

And to do that, you need insights into how your infrastructure handles user requests. With SigNoz, you can start monitoring your app in a few simple steps, and with an easy-to-use dashboard, you can quickly identify bottlenecks in your services.

## Introducing SigNoz
[SigNoz](https://github.com/SigNoz/signoz) is a full-stack open-source application monitoring and observability platform which can be installed within your infra. You can track metrics like p99 latency, error rates for your services, external API calls, and individual endpoints. With service maps, you can quickly assess the health of your services.

<img width="100%" style="width:100%" src="https://media.giphy.com/media/c8t8MnT0WDVBlM9sod/giphy.gif">

And once you know the affected service, trace data can help you identify the exact code causing the issue. Using SigNoz dashboard, you can visualize your traces easily with flamegraphs.

![Flamegraphs on SigNoz dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bxnezrbcvik2l0qd0iuv.png)<figcaption>Distributed tracing visualized with flamegraphs on SigNoz dashboard</figcaption>

Now let's get down to some action and see everything for yourself.

We have set up a [sample ToDo Python app](https://github.com/SigNoz/sample-flask-app) based on Flask web framework, which uses MongoDB as a database to demonstrate how SigNoz works. We will divide the tutorial into two parts:

1. Installing SigNoz
2. Instrumenting sample app to start monitoring

## Part 1 - Installing SigNoz

__Install Docker__
You can install Docker by following the steps listed on their website [here](https://www.docker.com/get-started). For this tutorial, you can choose the Docker Desktop option based on the system you have.

![Docker installation landing page](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hj2h0mewmcwyf8j72ypw.png)

__Clone SigNoz GitHub repository__
From your terminal use the following command to clone SigNoz's GitHub repository.
```
git clone https://github.com/SigNoz/signoz.git
```

__Update path to signoz/deploy and install SigNoz__
The deploy folder contains the files necessary for deploying SigNoz through Docker.
```
cd signoz/deploy/
./install.sh
```
You will be asked to select one of the 2 ways to proceed:

1. Clickhouse as database (default)
2. Kafka + Druid setup to handle scale (recommended for production use)

Trying out SigNoz with clickhouse database takes less than 1GB of memory and for this tutorial, we will use that option.

![SigNoz installation process on mac terminal](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uenodd3f8igg2tghijmv.png)

You will get the following message once the installation is complete.

![Success message once the installation is complete](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/phjjj8ns2l0wmi1pghsp.png)

Once the installation runs successfully, the UI should be accessible at port 3000. Wait for 2-3 mins for the data to be available to frontend.

![SigNoz dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qtmmoslacztt1j5cxqk9.png)

The applications shown in the dashboard are from a sample app called Hot R.O.D that comes with the installation bundle. It has 4 microservices being monitored: Frontend, Customer, Driver and Route. You can access the Hot R.O.D application UI at: http://localhost:9000/

Now that you have SigNoz up and running, let's see how instrumentation works. Instrumentation is the process of implementing code instructions to monitor your application's performance. Instrumentation is key to see how your application handles the real world.

SigNoz supports [OpenTelemetry](https://opentelemetry.io/) as the primary way for users to instrument their application. OpenTelemetry is a single, vendor-agnostic instrumentation library per language with support for both automatic and manual instrumentation. You don't need to worry about instrumentation in this tutorial. OpenTelemetry comes with all currently available [instrumentation](https://github.com/open-telemetry/opentelemetry-python).

## Part 2 - Instrumenting sample app to start monitoring

### Prerequisites

__Python 3.4 or newer__
If you do not have Python installed on your system, you can download it from the link [here](https://www.python.org/downloads/). Check the version of Python using <code>python3 --version</code> on your terminal to see if Python is properly installed or not.

__MongoDB__
If you already have MongoDB services running on your system, you can skip this step.

For macOS:
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-os-x/

For Linux:
https://docs.mongodb.com/manual/administration/install-on-linux/

For Windows:
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-windows/

On MacOS the installation is done using Homebrew's brew package manager. Once the installation is done, don't forget to start MongoDB services using <code>brew services start mongodb/brew/mongodb-community@4.4</code> on your macOS terminal.

![Message on mac terminal on successfully starting mongodb services](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4vyq8a96v3trzyutdpci.png)

### Steps

__Clone sample Flask app repository__
From your terminal use the following command to clone sample Flask app GitHub repository.
```
git clone https://github.com/SigNoz/sample-flask-app.git
```

--Update path to sample-flask-app & check if the app is running__
Check if the app is working or not using the following command:
```
cd sample-flask-app
python3 app.py
```

![Message on mac terminal on running python app](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qpln6xiapqvuq27qmd71.png)<figcaption>On my mac terminal</figcaption>

You can now access the UI of the app on your local host: http://localhost:5000/

![UI of python app at port 5000](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gwf63ad5n6ck8pde4uip.png)<figcaption>TODO reminder app with Flask and MongoDB
</figcaption>

Press 'Ctrl + C' to exit the app once you have made sure it is running properly.

__Set up OpenTelemetry Python instrumentation library__
Your app folder contains a file called requirements.txt. This file contains all the necessary commands to set up OpenTelemetry python instrumentation library. All the mandatory packages required to start the instrumentation are installed with the help of this file. Make sure your path is updated to the root directory of your sample app and run the following command:
```
pip3 install -r requirements.txt
```
If it hangs while installing <code>grpcio</code> during __pip3 install opentelemetry-exporter-otlp__ then follow below steps as suggested in this stackoverflow [link](https://stackoverflow.com/questions/56357794/unable-to-install-grpcio-using-pip-install-grpcio/62500932#62500932)

* pip3 install --upgrade pip
* python3 -m pip install --upgrade setuptools
* pip3 install --no-cache-dir --force-reinstall -Iv grpcio

__Install application specific packages__
This step is required to install packages specific to the application. Make sure to run this command in the root directory of your installed application. This command figures out which instrumentation packages the user might want to install and installs it for them:

```
opentelemetry-bootstrap --action=install
```

__Configure a span exporter and run your application__
You're almost done. In the last step, you just need to configure a few environment variables for your OTLP exporters. Environment variables that need to be configured:

* SERVICE_NAME - application service name (you can name it as you like)
* ENDPOINT_ADDRESS - OTLP gRPC collector endpoint address (IP of SigNoz)

After taking care of these environment variables, you only need to run your instrumented application. Accomplish all these by using the following command at your terminal.

```
OTEL_RESOURCE_ATTRIBUTES=service.name=pythonApp OTEL_EXPORTER_OTLP_ENDPOINT="http://<IP of SigNoz>:4317" opentelemetry-instrument python3 app.py
```
<Ip of SigNoz> can be replaced with localhost in this case. Hence, the final command becomes:

```
OTEL_RESOURCE_ATTRIBUTES=service.name=pythonApp OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" opentelemetry-instrument python3 app.py
```

And, congratulations! You have instrumented your sample Python app. Hit the endpoint of your app at port 5000, and wait for 1-2 mins. You can now access the SigNoz dashboard at http://localhost:3000 to monitor your app for performance metrics.

![Sample app listed in applications on SigNoz dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2k4mrfzha0on568yax07.png)

Now that you can monitor your app performance, let's have a brief look at how to identify issues causing high latency in the app.

In just 5 easy steps, our dashboard lets you drill down to events causing a delay in your deployed apps 👇

1. __Choose the service you want to inspect__
   ![List of application on SigNoz dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2oxu5piq4ro9iwqxvhzt.png)

2. __Choose the timestamp where latency is high and click on view traces__
   ![Inspect latency graph and click on view traces for high latency timestamps](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2mbmhfdu0vmczzm8nuj5.png)

3. __Choose the trace ID with the highest latency__
   ![Sort and choose the trace ID with highest latency](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/40jgfgum0b1x42620jqi.png)

4. __Inspect distributed traces with flamegraph__
   ![Flamegraphs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/58fvwt6eaet6nb07iwwh.png)

5. __Zero in on the highest latency event and take action__
   ![Identify specific event causing the delay](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/efon6ci18irrwwu0f0m1.png)


With these steps, you can resolve latency issues in your app quickly. For distributed microservices architecture, distributed tracing enables teams to resolve issues quickly by providing a central dashboard where they can inspect how user requests perform across services.

If you need any help with trying out SigNoz, feel free to mail me at ankit.anand@signoz.io.

At SigNoz, we are committed to making the best open-source, self-hosted tool for application performance monitoring. Feel free to check out our GitHub repo here:

{% github signoz/signoz %}