Docker Logs, set limits!
Tags: docker, devops
Published: 2023-02-06 18:34:28

---
title: Docker Logs, set limits!
published: true
description: 
tags: #docker #devops
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-02-03 15:06 +0000
---
#The problem...
I have a docker-compose.yaml which contains a simple python script. The job of this script is to subscribe to MQTT broker and then write these messages to a database. 
##The mistake...
Inside this script I have a simple innocent print statement, which prints the captured message.
We talk about 30 messages per second, and this service already runs for 3 plus months. So..., a lot of messages.

## I need more space!!!
First of all, this script runs in Ubuntu server. At some point the hard disk doesn't have enough space. My first though is that there is much data inside my database.
But the problem here is that I get messages every second (~3600x30 messages!) which eventually I replace with their average value per hour. Again I think the problem is the database.
##VACUUM IT!!!
I use **Postgresql**, so, I read the [documentation](https://www.postgresql.org/docs/current/sql-vacuum.html). 
Here we are! Let's release some space. At this point I don't want to use the **VACUUM FULL** because it causes downtimes...
After that, nothing changed dramatically in my server's hard drive..
## Docker df
I think my first move should be this one. Check the space which is taken by my docker container,
```bash
docker system df
```
Hmmm, here I am missing ~6.5Gb of space which at that point I assumed was being taken up by docker... somehow...

## Logs, Logs everywhere..
Next move is to see the logs of the container
```bash
docker logs <container id>
```
I see all these print statements which start from the beginning of container's existence.
So, I thought where these logs are stored?
```bash
 docker inspect --format='{{.LogPath}}' <my-app or container-id>
```
The result will be something like this,

```bash
/var/lib/docker/containers/46d66d306b4a7f95051dbf2aa7b27864d78d84bfac7dc685c4b94985a1ee30c5/46d66d306b4a7f95051dbf2aa7b27864d78d84bfac7dc685c4b94985a1ee30c5-json.log
```
Here you are!!!
Ok, then I cd to this directory and run,
```bash 
ls -l -h
```
To see 6.5Gb of logs!!!
Here was my precious Gb!!!
<img width="100%" style="width:100%" src="https://media.giphy.com/media/3oFyCVxsQn6RBa0r5u/giphy.gif">

## How clear this logs?
The first fast and easy way, without need to re-build your container is,
```bash
sudo sh -c 'echo "" > $(docker inspect --format="{{.LogPath}}" my-app)'
```
Be aware, this is a temporary solution! You will come up with the same problem in the near future!

## Set limits in docker-compose
You can set limits of logs for your container in your docker-compose.yaml
```yaml
logging:
      driver: "json-file"
      options:
        max-size: 50m
```
Here we set limit of 50mb of log file.

## Set limits in docker run
```bash
docker run \
      --log-driver local --log-opt max-size=10m \
      alpine echo hello world
```
Another example, set 10mb limit of logs with docker command
[To be continued...](https://dev.to/spartakos87/docker-logs-continue-ld0)
#Sources
* [https://www.misterpki.com/docker-log-file-size/](https://www.misterpki.com/docker-log-file-size/)
* [https://www.howtogeek.com/devops/how-to-clear-logs-of-running-docker-containers/](https://www.howtogeek.com/devops/how-to-clear-logs-of-running-docker-containers/)
* [https://docs.docker.com/config/containers/logging/local/](https://docs.docker.com/config/containers/logging/local/)