How tf does Docker Networking even work?!😵‍💫
Tags: webdev, tutorial, devops, docker
Published: 2024-01-02 13:00:00

Ever wondered what happens after running `docker-compose up`? IP tables, DNS, and virtual networks are just vague buzzwords for you? Well, it's your lucky day! Follow along and learn some new stuff that will help you to better understand Docker magic ✨🧑‍🎓

![lol](https://media0.giphy.com/media/IzjhI7ggjDlEnMxZMu/giphy.gif?cid=ecf05e47m9vvzbky53in6y7fxhzai4vovi0e4sg65hw1xv6d&ep=v1_gifs_search&rid=giphy.gif&ct=g)

Warning: This is a beginners post. If you already have a good grasp of networking aspects this might not be for you:)

## The Setup

If you have done literally *anything* with Docker Compose, this will look very similar to you:

```yaml
services:
  web:
    build: .
    ports:
      - "8000:8000"
  db:
    image: postgres
```

This is basically the hello-world of Docker. One locally defined web service that is exposed on port `8000` and one internal database. You will probably also know that by writing `8000:8000`, the _internal_ port `8000` of the container is also exposed at your host port `8000`. This will allow you to access the web service by opening your browser and navigating to `http://localhost:8000`. While the postgres database is _somehow_ not reachable for you, it is reachable from the web container. So let's start with the first of many **WHAT?!** and dig a bit deeper. What actually happens when we run `docker-compose up`?

PS: You can also follow a long by cloning this Github repository and executing the commands as we go. This should work for MacOS and Linux, no guarantee for Windows 😬

## Step 1. `docker-compose up`

When you run `docker-compose up`, 3 things happen:
1. A virtual network gets created called `dockernetworking_default` (or whatever your working directory is called + "_default")
2. Both `web` and `db` container start and get added to the network
3. `web` also makes the port `8000` available on your host

This sounds easy enough, but I am asking the same thing again: **WHAT?!**

What does it mean to create a network? How do we add container to a network? And why do we even need to make ports available to the host, its all on the same computer right?!

## Step 2. Creating a "network"

When you run `docker-compose up` a so called "bridge network" that are isolated on only your local machine, meaning that no other hosts can access them (the alternative would be an overlay network).

(`docker-compose up` does the equivalent to `docker network create -d bridge my-bridge-network`)

Each container in the network gets a unique IP address and name that they can be reached on.

Let's take a closer look at what is going on. After running `docker-compose up`, execute `docker network ls`. This will list all of your docker networks, one line should look like this: 


![docker network ls](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p0d9s1oe8k9nwu00zct3.png)

You can then "inspect" it (print the configuration) using 
`docker inspect docker-networking_default | jq` (jq is just a nice add-on to add json syntax highlighting)

This will then print this monster of a json string:

```json

[
  {
    "Name": "docker-networking_default",
    "Id": "d968443acaf789d7d2e9ae18a82371f663baa3019288b18a13c5f38c432aa5df",
    "Created": "2023-12-15T17:25:27.011248Z",
    "Scope": "local",
    "Driver": "bridge",
    "EnableIPv6": false,
    "IPAM": {
      "Driver": "default",
      "Options": null,
      "Config": [
        {
          "Subnet": "172.21.0.0/16",
          "Gateway": "172.21.0.1"
        }
      ]
    },
    "Internal": false,
    "Attachable": false,
    "Ingress": false,
    "ConfigFrom": {
      "Network": ""
    },
    "ConfigOnly": false,
    "Containers": {
      "7dc4807614c7a4660b9bd34f23dc91610afcdffad8b39af288b9160c3ce34e6e": {
        "Name": "docker-networking-web-1",
        "EndpointID": "25250386497a2d3093ac133ab0892595128c5734eedb8e039744deccfa1c5c06",
        "MacAddress": "02:42:ac:15:00:03",
        "IPv4Address": "172.21.0.3/16",
        "IPv6Address": ""
      },
      "ac105fd2a292cda4136aedd0d00c88635f2a140ba3e2359c7e0267514ed9c8b5": {
        "Name": "docker-networking-db-1",
        "EndpointID": "e26c41754ad5016b7fd9ea0777cbbaa27546209cc8c610db7648e5ed33189538",
        "MacAddress": "02:42:ac:15:00:02",
        "IPv4Address": "172.21.0.2/16",
        "IPv6Address": ""
      }
    },
    "Options": {},
    "Labels": {
      "com.docker.compose.network": "default",
      "com.docker.compose.project": "docker-networking",
      "com.docker.compose.version": "2.23.0"
    }
  }
]
```


There is a lot of information here, but what I want to focus on are two important lines:

```json
{
 "Subnet": "172.21.0.0/16",
 "Gateway": "172.21.0.1"
}
```
This means that all your container IP addresses will be in the `172.21.0.0/16` subnet, or in other words between `172.21.0.1 - 172.21.255.254`. Although the first one at `172.21.0.1` will be reserved for your gateway. You can confirm this by opening your web containers shell with `docker exec -ti docker-networking-web-1 sh` and executing `hostname -I`. This will print `172.21.0.3` or `172.21.0.2`! Btw, the gateway is (as the name suggests) the piece that is connecting the virtual docker network with your hosts network.

For the same reason why we use domain names in the internet instead of IP adresses, each container also gets a domain name that can be used instead of the IP adress (which can also change!)

For example, if you go into your web container with 
`docker exec -ti docker-networking-web-1 sh` you can run `ping db`:


![ping db](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wmvufep15q0pqul5a16s.png)

You should see that the domain name `db` gets translated to `172.21.0.2`! Awesome right, but **WHAT?!** how does your container know that?!

## 3. DNS in Docker

Something that I didn't tell you before is that not just the gateway IP is reserved, `172.21.0.1` is also reserved for dockers internal DNS server.

If you once again go into your web container with
`docker exec -ti docker-networking-web-1 sh`
you can run 
`dig db`
to see how the hostname `db` gets resolved by your DNS.
```
# dig db  

; <<>> DiG 9.18.18-0ubuntu0.22.04.1-Ubuntu <<>> db
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37963
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;db.                            IN      A

;; ANSWER SECTION:
db.                     600     IN      A       172.21.0.2

;; Query time: 1 msec
;; SERVER: 127.0.0.11#53(127.0.0.11) (UDP)
;; WHEN: Fri Dec 15 18:32:12 UTC 2023
;; MSG SIZE  rcvd: 38
```

To be fair, this is fairly unreadable. But the only thing we really care about are two lines. First, we see that there is an A record for db pointing at `172.21.0.2`, which (who would have thought!) is also the IP address of our database container.
```
db.                     600     IN      A       172.21.0.2
```

Below that, we see that the DNS server we used is at `127.0.0.11#53`, which is the default local DNS server.
```
;; SERVER: 127.0.0.11#53(127.0.0.11) (UDP)
```
## 4. Exposing Ports

Ever wondered why you need to map your container port to your host port? The abstract and easy answer is that the network of your containers is isolated from the one of your host. This is done for security reasons but also removes side-effects from weird setups you might have. So in order to connect from your host to a docker container, there needs to be some sort of translation layer in-between. Luckily, this is something that the docker daemon handles for you! If you start your containers with `docker-compose up` and you mapped some ports, you can use this command:
`lsof -i -P -n | grep LISTEN` to find what applications are listening on what ports. Somewhere you will find the host port that you declared. This process will re-route the traffic to the "hidden" docker network. Pretty awesome right? If you want to know more, I suggest reading more into how virtual networks and network isolation in linux work!

## 5. Exposing Containers to the Internet

Docker networking usually stops on your machine. If you want to expose your apps to the internet you can either use a reverse-proxy like [Caddy](https://caddyserver.com/) or a reverse-tunneling tool like ngrok or [Livecycle](https://livecycle.io/?utm_source=devto&utm_medium=article&utm_campaign=jonas). Tools like caddy are great for production-ready and deployed apps, while something like [Livecycle](https://livecycle.io/?utm_source=devto&utm_medium=article&utm_campaign=jonas) is more for a prototyping/collaboration use-case. While this is of course not necessary, at some point you will have to share your app with the world:)

I previously wrote [about how I use Livecycle for Hackathons](https://dev.to/code42cate/how-to-win-any-hackathon-3i99), but the TL;DR goes like this:
1. Install the [Livecycle](https://livecycle.io/?utm_source=devto&utm_medium=article&utm_campaign=jonas) Docker Desktop extension
2. Click on "share" in the Docker Desktop [Livecycle](https://livecycle.io/?utm_source=devto&utm_medium=article&utm_campaign=jonas) tab
3. Your locally hosted docker-compose apps will now be tunneled to a publicly available web server with a random domain.
4. If you shutdown your compose app, the tunnel will go down as well!


![Livecycle extension](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/63gnrk6yz5ctpkg1voef.png)




## Conclusion

There are so many things that I would love to cover and also go way deeper into technical OS-level details. But to keep this brief and somewhat entertaining I am stopping here. If you have any questions or suggestions on what to cover next, please let me know in the comments! As always, I hope you learned something new:)

Cheers, Jonas