Repository: traefik/traefik (language: go)

Issue:
Use IPv6 address from Docker API for URL when available
### Welcome!

- [X] Yes, I've searched similar issues on [GitHub](https://github.com/traefik/traefik/issues) and didn't find any.
- [X] Yes, I've searched similar issues on the [Traefik community forum](https://community.containo.us) and didn't find any.

### What did you do?

1. Configure Docker to support IPv6 addresses.
2. Create docker-compose with Traefik + some service.
3. Configure labels on said service to enable Traefik
4. Expect Traefik to use IPv6 address for the service URL

### What did you see instead?

Traefik picks the containers IPv4 address for the backend server URL (even when IPv6 address is available).

I'd expect the URL to be `http://[fd10:255:3::2]:80`.

![image](https://user-images.githubusercontent.com/359918/154824608-f236864f-8ec3-445a-9a58-a9813fc85db0.png)

```
root@docker1:/srv/config/docker/test# docker inspect test-whoami
[
    {
        […]
            "Networks": {
                "test": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "test-whoami",
                        "a7c6ec8c1357"
                    ],
                    "NetworkID": "c2ccfda57d35f43d8e4dbaaeb9beb97c118f47c9454410c2132914af4cf2c640",
                    "EndpointID": "8f6c92c67c1ef4ab5faf861aadec883d91103cc1a6ab0133d8fb62aea30a7b71",
                    "Gateway": "10.128.3.1",
                    "IPAddress": "10.128.3.2",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "fd10:255:3::1",
                    "GlobalIPv6Address": "fd10:255:3::2",
                    "GlobalIPv6PrefixLen": 64,
                    "MacAddress": "02:42:0a:80:03:02",
                    "DriverOpts": null
                }
            }
        }
    }
]
```

### What version of Traefik are you using?

```
Version:      2.6.1
Codename:     rocamadour
Go version:   go1.17.7
Built:        2022-02-14T16:50:25Z
OS/Arch:      linux/amd64
```

### What is your environment & configuration?

Host OS: Debian Bullseye
docker: 20.10.5+dfsg1-1+deb11u1
docker-compose: 1.25.0-1


docker-compose.yml:
```yaml
version: '2.1'

services:
  test-traefik:
    container_name: test-traefik
    hostname: test-traefik
    image: "traefik:latest"
    restart: "${RESTART}"  
    environment:
      - TZ=${TZ}
    ports:
      - "80:80/tcp"
      - "443:443/tcp"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ${CONFIG_BASE}/traefik:/etc/traefik
      - /srv/letsencrypt/webssl:/certs
    labels: 
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.rule=Host(`traefik.${TRAEFIK_DOMAIN}`)"
      - "traefik.http.routers.traefik.tls=true"
      - "traefik.http.routers.traefik.service=api@internal"
      - "traefik.http.services.traefik.loadbalancer.server.port=8080"

  test-whoami:
    container_name: test-whoami
    hostname: test-whoami
    image: traefik/whoami:latest
    restart: "${RESTART}"
    environment:
      - TZ=${TZ}
    labels: 
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.rule=Host(`whoami.${TRAEFIK_DOMAIN}`)"
      - "traefik.http.routers.whoami.tls=true"
      - "traefik.http.routers.whoami.service=test-whoami-test"

networks:
  default:
    name: test
    enable_ipv6: true
    driver_opts:
      com.docker.network.enable_ipv6: "true"
    ipam:
      driver: default
      config:
       - subnet: fd10:255:3::/64
       - subnet: 10.128.3.0/24
```


/etc/traefik/traefik.yml
```yaml
log:
  level: DEBUG

api:
  dashboard: true
  debug: true
  insecure: false

entryPoints:
  web:
    address: :80
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https
  websecure:
    address: :443
  traefik:
    address: :8080

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    watch: true
  file:
    directory: /etc/traefik/dynamic
    watch: true
 ```


### If applicable, please paste the log output in DEBUG level

_No response_


Discussion / hints:
One alternative would be to use the hostname of the container (`test-whoami`), rather than the IPv6-address directly, but the container doesn't seem to prioritize IPv6 above IPv4 currently (which probably would have to be solved in Docker). Not sure if Traefik could prioritize IPv6 above IPv4 if both are present in a hostname/FQDN, or if that is left to some other part of the stack.

```
/ # hostname
test-traefik
/ # nslookup test-whoami
Server:		127.0.0.11
Address:	127.0.0.11:53

Non-authoritative answer:
Name:	test-whoami
Address: fd10:255:3::2

Non-authoritative answer:
Name:	test-whoami
Address: 10.128.3.2

/ # dig test-whoami

; <<>> DiG 9.16.25 <<>> test-whoami
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23282
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;test-whoami.			IN	A

;; ANSWER SECTION:
test-whoami.		600	IN	A	10.128.3.2

;; Query time: 0 msec
;; SERVER: 127.0.0.11#53(127.0.0.11)
;; WHEN: Sun Feb 20 02:26:33 CET 2022
;; MSG SIZE  rcvd: 56

/ # dig test-whoami aaaa

; <<>> DiG 9.16.25 <<>> test-whoami aaaa
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41712
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;test-whoami.			IN	AAAA

;; ANSWER SECTION:
test-whoami.		600	IN	AAAA	fd10:255:3::2

;; Query time: 0 msec
;; SERVER: 127.0.0.11#53(127.0.0.11)
;; WHEN: Sun Feb 20 02:26:36 CET 2022
;; MSG SIZE  rcvd: 68

/ # curl -o- http://test-whoami
Hostname: test-whoami
IP: 127.0.0.1
IP: ::1
IP: 10.128.3.2
IP: fd10:255:3::2
IP: fe80::42:aff:fe80:302
RemoteAddr: 10.128.3.3:49310
GET / HTTP/1.1
Host: test-whoami
User-Agent: curl/7.79.1
Accept: */*

/ # curl -6 -o- http://test-whoami
Hostname: test-whoami
IP: 127.0.0.1
IP: ::1
IP: 10.128.3.2
IP: fd10:255:3::2
IP: fe80::42:aff:fe80:302
RemoteAddr: [fd10:255:3::3]:53314
GET / HTTP/1.1
Host: test-whoami
User-Agent: curl/7.79.1
Accept: */*

```

Adding manual dynamic config via `/etc/traefik/dynamic/test.yml`;

```yaml
http:
  services:
    whoami-host:
      loadBalancer:
        servers:
        - url: "http://whoami-test:80"
    whoami-ipv6:
      loadBalancer:
        servers:
        - url: "http://[fd10:255:3::2]:80"
  routers:
    whoami-host:
      entryPoints:
        - "websecure"
      rule: "Host(`whoami-host.foo.bar`)"
      service: "whoami-host"
      tls: true
    whoami-ipv6:
      entryPoints:
        - "websecure"
      rule: "Host(`whoami-ipv6.foo.bar`)"
      service: "whoami-ipv6"
      tls: true
```

Trying `https://whoami-host.foo.bar`, it seems like it cannot resolve (even if it works with both dig and nslookup on the same container);

```
test-traefik    | time="2022-02-20T02:12:49+01:00" level=debug msg="'502 Bad Gateway' caused by: dial tcp: lookup whoami-test on 127.0.0.11:53: no such host"
```

Trying `https://whoami-ipv6.foo.bar` works as expected:

```
jocke@keklolwtf: ~ $ curl -k -o- https://whoami-ipv6.foo.bar
Hostname: test-whoami
IP: 127.0.0.1
IP: ::1
IP: 10.128.3.2
IP: fd10:255:3::2
IP: fe80::42:aff:fe80:302
RemoteAddr: [fd10:255:3::3]:53318
GET / HTTP/1.1
Host: whoami-ipv6.foo.bar
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 2001::x
X-Forwarded-Host: whoami-ipv6.foo.bar
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: test-traefik
X-Real-Ip: 2001::x
```

edit: Re-trying this setup, it seems as the host variant works, but Traefik seems to prefer the IPv4 address:

```
jocke@keklolwtf: ~ $ curl -k -o- https://whoami-host.foo.bar
Hostname: test-whoami
IP: 127.0.0.1
IP: ::1
IP: 10.128.3.2
IP: fd10:255:3::2
IP: fe80::42:aff:fe80:302
RemoteAddr: 10.128.3.3:53318
GET / HTTP/1.1
Host: whoami-host.foo.bar
User-Agent: curl/7.64.1
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 2001::x
X-Forwarded-Host: whoami-host.foo.bar
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Forwarded-Server: test-traefik
X-Real-Ip: 2001::x
```
The issue with unresolved hostname seems to be caused by unstable internal DNS in Docker, which seems to be a somewhat known problem. Using external DNS, it seems to work fine.

That leaves us with the question wether or not to use hostname or IPv6 address. A default priority between the three (IPv4, IPv6, hostname) should probably be decided, and then maybe a configuration parameter to override/choose between them, would be the most flexible? Preferable globally and per router/label, so that you have the option to override the global value (either the default or specifically set).
we have probably the same issue (on how docker chooses IPAMConfig).
We would like to use dual-stack with our services. So we did configure docker to use ipv6 and ip6tables experimental. 

What we experience is that when the backend node has ipv6 assigned, we receive a bad gateway even service inside docker is exposed on ipv6 (not sure why that happens)

Ie here is inspect section of continer using traefik targeted overlay docker network:

```json
           "net--proxy": {
                    "IPAMConfig": {
                        "IPv6Address": "fd00:3::14"
                    },
                    "Links": null,
                    "Aliases": [
                        "809b3cba85d9"
                    ],
                    "NetworkID": "f07y65n4xa2p4gd7i0zgexk7x",
                    "EndpointID": "794ee9c4d20296bc4712f2e44a4dd0c77fcd5ce4c1bb7da1e93fcb11c3cc053d",
                    "Gateway": "",
                    "IPAddress": "10.100.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "fd00:3::14",
                    "GlobalIPv6PrefixLen": 80,
                    "MacAddress": "obsolete",
                    "DriverOpts": null
                },
```

Results with above IPAMConfig and following traffic backend:

![image](https://user-images.githubusercontent.com/9593365/157055424-9b516142-bd44-4a1f-a7e4-52683eca5469.png)

And if stop container when autostarting sometimes it get ipv4 and all works fine.

@joachimtingvold Hello. We're not discarding your experience, and the validity of the issue, but we're kindof wondering about whether it is an actual problem corresponding to a real use-case.

In other words, since the containers are all within their own subnetwork, we suspect that in that space there's always going to be enough IPv4 addresses for the containers, right? If yes, then what would be the actual need for IPv6 in this context? Even further, Is there a use-case which would require their subnetwork to be IPv6 only?

> In other words, since the containers are all within their own subnetwork, we suspect that in that space there's always going to be enough IPv4 addresses for the containers, right?

For most it would be sufficient, yes, but there are already multiple organizations that had to move away from IPv4 due to lack of address space (Facebook, Microsoft, et al).

> If yes, then what would be the actual need for IPv6 in this context?

Due to lack of IPv4 assignments from RIRs, there are now multiple organizations deploying single-stacked networks (IPv6 only). There are also organizations that tries to use IPv6-only where they can (ourselves included), simply for ease of deployment (why spend time deploying dual-stack, when all resources are accessed from IPv6-capable clients, or through dual-stacked loadbalancer?).

We have come far enough in IPv6 transition that I think it's fair to ask "why use IPv4 if you don't need to?".

> Even further, Is there a use-case which would require their subnetwork to be IPv6 only?

I have not tested v6-only environment, but that would be a fair usecase. The way Traefik works now, such a setup would completely break, I think? (i.e. it would not produce a suitable service, since it only seem to look for the `IPAddress` field in the Docker API?).

For dual-stacked setups, there's always the discussion on what to prefer; IPv4 versus IPv6. Since Docker currently seems to default to IPv4 (when resolving hostnames that has both A and AAAA records), I don't mind Traefik prefering IPv4 over IPv6 if both are present. However, I'd like it to at least be able to pick the IPv6 address if it's the only one available (IPv6-only/single-stacked), and preferably also an option/flag to prefer IPv6 over IPv4 if both are present.

Since at some time IPv6 will be generally preferred over IPv4, one could also argue if the "prefer IPv6"-flag should be future-proofed by flipping the logic; in stead of the flag preferring IPv6 over IPv4, it would prefer IPv4 over IPv6. The flag would (for now) default to true (i.e. prefer IPv4 over IPv6). You could then override it if-need-be, and whenever IPv6 becomes generally preferred, we could just switch the flag default value to false.
It feels there is some confusion between the public IP ranges and private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, 100.64.0.0/10).

RIRs manage the public IP ranges.
The problem with IPv4 is about public IP ranges and IPv6 is the right solution.

The IPs provided by Docker for the containers are in the private IP ranges.
I think you will never have enough containers to overflow the private IPv4 IP ranges (about 20 million addresses)


The need to get a full IPv6 stack is understandable, but the arguments about the number of IPs for a private network with IPv4 are a bit ambiguous.

Maybe I missed something.
There is no confusion on my part, at least. I'm perfectly aware of the differences between public- and private IPv4-ranges. They are, however, related in the way that organizations unable to get (sufficient) public IPv4 addresses might have to resort building parts of their network, or even the whole network, only using IPv6 (there are even some IPv6-only ISPs out there). They might have some public IPv4 at their demarc, use IPv6-only in their transport networks, and use IPv4-over-IPv6 transition mechanisms (4rd et al) to transport some of the IPv4 traffic.

Point being; just because you *can* use IPv4, does not mean you should, and thus, Traefik should imho fully support IPv6, even in Docker. Our business is small, but we still do IPv6-only where we can, and just dual-stack the few services that are public-facing (loadbalancers et al). Saves us a lot of time/effort trying to dual-stack everything.

Fix patch:
diff --git a/docs/content/providers/docker.md b/docs/content/providers/docker.md
index 95603a7b4b..bb6feecef5 100644
--- a/docs/content/providers/docker.md
+++ b/docs/content/providers/docker.md
@@ -140,6 +140,13 @@ On Linux, for versions of Docker older than 20.10.0, for `host.docker.internal`
 as an `extra_host` to the Traefik container, using the `--add-host` flag. For example, to set it to the IP address of
 the bridge interface (`docker0` by default): `--add-host=host.docker.internal:172.17.0.1`
 
+### IPv4 && IPv6
+
+When using a docker stack that uses IPv6,
+Traefik will use the IPv4 container IP before its IPv6 counterpart.
+Therefore, on an IPv6 Docker stack,
+Traefik will use the IPv6 container IP.
+
 ### Docker API Access
 
 Traefik requires access to the docker socket to get its dynamic configuration.
diff --git a/pkg/provider/docker/docker.go b/pkg/provider/docker/docker.go
index 192fc49992..ab094b16bc 100644
--- a/pkg/provider/docker/docker.go
+++ b/pkg/provider/docker/docker.go
@@ -411,10 +411,15 @@ func parseContainer(container dockertypes.ContainerJSON) dockerData {
 		if container.NetworkSettings.Networks != nil {
 			dData.NetworkSettings.Networks = make(map[string]*networkData)
 			for name, containerNetwork := range container.NetworkSettings.Networks {
+				addr := containerNetwork.IPAddress
+				if addr == "" {
+					addr = containerNetwork.GlobalIPv6Address
+				}
+
 				dData.NetworkSettings.Networks[name] = &networkData{
 					ID:   containerNetwork.NetworkID,
 					Name: name,
-					Addr: containerNetwork.IPAddress,
+					Addr: addr,
 				}
 			}
 		}


Test patch:
diff --git a/pkg/provider/docker/builder_test.go b/pkg/provider/docker/builder_test.go
index 28b8d225c3..424dc7a3e0 100644
--- a/pkg/provider/docker/builder_test.go
+++ b/pkg/provider/docker/builder_test.go
@@ -77,6 +77,12 @@ func ipv4(ip string) func(*network.EndpointSettings) {
 	}
 }
 
+func ipv6(ip string) func(*network.EndpointSettings) {
+	return func(s *network.EndpointSettings) {
+		s.GlobalIPv6Address = ip
+	}
+}
+
 func swarmTask(id string, ops ...func(*swarm.Task)) swarm.Task {
 	task := &swarm.Task{
 		ID: id,
diff --git a/pkg/provider/docker/config_test.go b/pkg/provider/docker/config_test.go
index f58999459d..ed8b58b7c7 100644
--- a/pkg/provider/docker/config_test.go
+++ b/pkg/provider/docker/config_test.go
@@ -3511,6 +3511,22 @@ func TestDockerGetIPAddress(t *testing.T) {
 			network:  "testnet",
 			expected: "10.11.12.13",
 		},
+		{
+			desc: "one ipv6 network, network label",
+			container: containerJSON(
+				withNetwork("testnet", ipv6("fd00:1:2:3:4::")),
+			),
+			network:  "testnet",
+			expected: "fd00:1:2:3:4::",
+		},
+		{
+			desc: "two network ipv4 + ipv6, network label",
+			container: containerJSON(
+				withNetwork("testnet", ipv4("10.11.12.13"), ipv6("fd00:1:2:3:4::")),
+			),
+			network:  "testnet",
+			expected: "10.11.12.13",
+		},
 		{
 			desc: "two networks, network label",
 			container: containerJSON(