How to keep your docker installation clean?
By: Maciek Opała
Tags: Programming, Docker, DevOps, Monitoring

Docker is a powerful tool that significantly simplifies deployment, testing and prototyping of various applications and/or services. When heavily used it may turn out that considerable amount of the machine’s storage memory is occupied with docker files — sometimes up to tens of gigabytes or even more! How to save disk space? What can be safely removed, what cannot? This is what this post is about!

Photo by James Pond on Unsplash

Should I really care?

Well.. It depends, as usual ;) Actually, if you’re rich enough and whenever your machines are short of resources you can afford to buy additional disk space, CPU, etc. you don’t to have to worry at all. But keeping your system running clean, tidy and up-to-date is about being a good professional — so when you look at it from this perspective, then — yes, you should look after the system to make it running smoothly. Good news is that docker doesn’t use system resources besides disk space for non-running containers, images or unmounted volumes.

What is the resources usage?

Before any cleanup is started you need to know if it is even necessary — how can you actually know how much disk space is currently used by docker?

First of all you need to know where to check the disk occupancy. The following command:

docker info

will give you comprehensive information about the current docker installation. In particular there’s one key Docker Root Dir which typically points to /var/lib/docker and is a place where docker files are kept. To verify how much space is used by this path you can run: du -h --max-depth=1 /var/lib/docker which gives output similar too:

372K /var/lib/docker/containers

4.0K /var/lib/docker/tmp

20M /var/lib/docker/image

72K /var/lib/docker/buildkit

20K /var/lib/docker/builder

96K /var/lib/docker/network

4.0K /var/lib/docker/trust

20K /var/lib/docker/plugins

199M /var/lib/docker/volumes

312K /var/lib/docker/containerd

4.0K /var/lib/docker/swarm

4.0K /var/lib/docker/runtimes

2.0G /var/lib/docker/aufs

2.2G /var/lib/docker/

and clearly shows how much space is used by a given component.

Another useful command in docker monitoring is:

docker system df

Which gives the following output:

TYPE TOTAL ACTIVE SIZE RECLAIMABLE

Images 9 2 1.48GB 700.1MB (47%) Containers 2 1 54.48kB 21.64kB (39%)

Local Volumes 1 1 206.8MB 0B (0%)

Build Cache 0 0 0B 0B

and shows the current storage usage — what’s interesting is the value in RECLAIMABLE column. It represents the storage that will be freed when all non-running, but existing containers (images) have been removed.

Now, when you know how much space is used by all the components in general, it might be useful to know what is the occupancy of a particular element e.g. container, image or volume.

Containers

To know how much space is taken by a particular container you can run:

docker container ls -s

which shows all running containers, along with the SIZE column (which was added with -s or --size option)

CONTAINER ID IMAGE (...) SIZE

c55ec9a85ddd sonatype/nexus3 (...) 32.8kB (virtual 539MB)

where 32.8kB represents the writable layer of the container while virtual 539MB is the size of a read-only image on top of which the given container is run. Mind that it’s a bit misleading. SIZE value does not contain the size of the volumes used by containers and other metadata — for more info, please have a look at this post — so this is not the whole space that is taken by a given container.

Images

When it comes to the images it’s straightforward. Size is included into the output by default, so running:

docker image ls

gives:

REPOSITORY TAG IMAGE ID CREATED SIZE sonatype/nexus3 latest 5d3600fd5cf2 5 weeks ago 539MB

where SIZE represents the whole space taken by the image and all its parent images.

Volumes

Finally, the volumes. There’s no command that shows the size of a given volume by default so you need to combine docker volume and du . Firstly, run:

docker volume ls

to get the list of the volumes:

DRIVER VOLUME NAME

local 69915af8ddf99

then, run:

docker volume inspect 69915af8ddf99

to get the volume’s metadata ( Mountpoint is what interests you):

[{

"CreatedAt": "2019-03-21T14:56:27Z",

"Driver": "local",

"Labels": null,

"Mountpoint": "/var/lib/docker/volumes/69915af8ddf99/_data",

"Name": "69915af8ddf99",

"Options": null,

"Scope": "local"

}]

and finally use du -h /var/lib/docker/volumes/69915af8ddf99/_data to know the space being used by given volume.

Now when you know how to monitor the disk space used by docker, it’s time to learn how to free the reclaimable memory.

For all types of docker components described here:

containers

images

volumes

networks

you need to remember that a component that is being used by docker cannot be removed. Such “used” components are for instance:

a running container

an image or volume that is tied to a running container

a network that is used by a running container

On attempt to remove a e.g. running container you will get the following error:

docker container rm c55ec9a85ddd Error response from daemon: You cannot remove a running container c55ec9a85dddd8e62100369ab2624d2463aa7ce22e3f2e732d88a4d7dcf79b46. Stop the container before attempting removal or force remove

How to cleanup docker containers?

To list the IDs of stopped containers you can use (see here for more filtering options which enable you to filter by e.g. exit code):

docker container ls --filter "status=exited" -aq

and then combine this command with docker container rm in the following way:

docker container rm $(docker container ls --filter "status=exited" -aq)

(Please mind that well implemented linux commands have an upper limit for the number of arguments passed — in this case it works as expected but you can consider using xargs )

docker container rm has two useful options:

-v or --volumes which will remove volumes associated with the container being removed

or which will remove volumes associated with the container being removed -f or --force which will force the removal of the container with SIGKILL

There’s also an useful option for docker container run command, namely --rm . It will remove the container automatically when it exits.

How to cleanup docker images?

To remove docker images run:

docker rmi $(docker image ls -aq)

In case of an image that is still used by docker an error will occur and such image will not be removed. Typically you remove dangling (untagged) images — which are useless, they just occupy the free space — and it can be done in the following way:

docker rmi $(docker image ls -aq --filter "dangling=true")

As with containers, there’s also -f or --force option available. What’s important here is that it’s quite safe to use, since images that are tied to running containers will not be removed even when -f option is present.

There’s also a similar option to docker container run --rm , namely docker image build --rm (which is used to remove intermediate containers after successful build) but the latter defaults to true, so it’s not needed to be passed explicitly.

How to cleanup docker volumes?

To remove docker volumes run:

docker volume rm $(docker volume ls -q)

Similarly to images there’s an -f (force) option available for docker volume rm and an -f (filter) option for docker volume ls . The latter can be used to filter only dangling (not tied to any running container volumes).

How to cleanup docker networks?

Docker networks are the least memory consuming components (these are just configuration files) so there’s not much gain from removing the docker networks. It also happens very rarely that you need to remove an existing docker network. However, as I wrote at the beginning, it’s not about space only, but about having it all clean and tidy.

To remove docker networks run:

docker network rm <NETWORK_ID>

Contrary to the previous components, there’s no -f (force) option provided. Similarly to them you can use docker network ls to filter the networks you’d like to remove.

Docker prune

The commands described in the previous sections are used to remove components of a particular type. But there’s a command that allows you to remove all redundant objects at once, namely docker system prune (available since version 1.13). It will remove all unused containers, networks and images (dangling). Optionally — if --volumes options is passed — volumes and (with -a option) also dangling images. Volumes are not removed by default, since it may contain some relevant/sensitive data. That’s why you need to state explicitly that you want the volumes to be removed too. After running the command you will see a warning:

WARNING! This will remove: - all stopped containers

- all networks not used by at least one container

- all volumes not used by at least one container

- all dangling images

- all dangling build cache Are you sure you want to continue? [y/N]

and will be prompted for a confirmation unless -f (force) option is present.

What’s more prune command is available for the components described in this post: containers, images, volumes and networks.

Remember to use it with caution!

Periodical cleanup

An idea to keep your docker ecosystem running clean and not overwhelmed by needless data is to run a periodical cleanup. But instead of setting it up on your own you can use a dedicated docker image that was already checked by other users. Here it goes!