Repository: moby/moby (language: go)

Issue:
Windows containers does not restart after Windows Server 2019 restart.
<!--
If you are reporting a new issue, make sure that we do not have any duplicates
already open. You can ensure this by searching the issue list for this
repository. If there is a duplicate, please close your issue and add a comment
to the existing issue instead.

If you suspect your issue is a bug, please edit your issue description to
include the BUG REPORT INFORMATION shown below. If you fail to provide this
information within 7 days, we cannot debug your issue and will close it. We
will, however, reopen it if you later provide the information.

For more information about reporting issues, see
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#reporting-other-issues

---------------------------------------------------
GENERAL SUPPORT INFORMATION
---------------------------------------------------

The GitHub issue tracker is for bug reports and feature requests.
General support for **docker** can be found at the following locations:

- Docker Support Forums - https://forums.docker.com
- Slack - community.docker.com #general channel
- Post a question on StackOverflow, using the Docker tag

General support for **moby** can be found at the following locations:

- Moby Project Forums - https://forums.mobyproject.org
- Slack - community.docker.com #moby-project channel
- Post a question on StackOverflow, using the Moby tag

---------------------------------------------------
BUG REPORT INFORMATION
---------------------------------------------------
Use the commands below to provide key information from your environment:
You do NOT have to include this information if this is a FEATURE REQUEST
-->

Windows Server 2019 does not start containers automatically on reboot, even when all docker-compose are labelled as restart: always.

<!--
Briefly describe the problem you are having in a few paragraphs.
-->

**Steps to reproduce the issue:**
1. Open PowerShell with admin privileges.
2. Run any docker-compose file: `docker-compose up`
```
version: "3"

services:

  postgres_db:
    image: stellirin/postgres-windows:12-1809
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password
    ports:
      - "5432:5432"
```
3. Execute Restart-Computer.
When the machine starts up, execute docker ps in PowerShell with admin privileges. 

**Describe the results you received:**
docker ps shows no containers running.
Running `docker start $(docker ps -a -q -f status=exited)` returns:
```
Error response from daemon: network 325cbdb7ce3b29e0c744d76783d94bc7637293b6efce7d912669581d5af5e4eb not found
Error response from daemon: network f555844ad59f8e1e9d95a01a8fce580ac1f2a0582acb2475c218fb8aab5bd1fb not found
Error: failed to start containers: 3cf6c5c3207a, 1df07dd99139
```


**Describe the results you expected:**
docker ps should should show containers running.

**Additional information you deem important (e.g. issue happens only occasionally):**
Happens every time on Windows Server 2019.

**Output of `docker version`:**

```
Client: Mirantis Container Runtime
 Version:           20.10.9
 API version:       1.41
 Go version:        go1.16.12m2
 Git commit:        591094d
 Built:             12/21/2021 21:34:30
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Mirantis Container Runtime
 Engine:
  Version:          20.10.9
  API version:      1.41 (minimum version 1.24)
  Go version:       go1.16.12m2
  Git commit:       9b96ce992b
  Built:            12/21/2021 21:33:06
  OS/Arch:          windows/amd64
  Experimental:     false
```

**Output of `docker info`:**

```
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  cluster: Manage Mirantis Container Cloud clusters (Mirantis Inc., v1.9.0)
  registry: Manage Docker registries (Docker Inc., 0.1.0)

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 2
 Server Version: 20.10.9
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: process
 Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
 Operating System: Windows Server 2019 Standard Version 1809 (OS Build 17763.1817)
 OSType: windows
 Architecture: x86_64
 CPUs: 8
 Total Memory: 16GiB
 Name: WIN-T721FCB2TP1
 ID: NDM2:5C34:GAV5:QDFF:TF52:327Y:HQUW:EZU5:ROH3:4T5B:QPML:KZ7H
 Docker Root Dir: C:\ProgramData\docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
```

**Additional environment details (AWS, VirtualBox, physical, etc.):**
Windows Server 2019 (Version 1809)

Server on is a VM on a physical machine located inside my office building.  

Very very similar issue to https://github.com/moby/moby/issues/39623.  However, it seems like my network is disappearing on restart.



Discussion / hints:
I think that support for those non-default NAT networks in Windows are not ever fully implemented which why there is known bugs. Workaround is use default one by adding this to compose file:
```yaml
networks:
  default:
    external:
      name: nat
```

However as you looks to be running PostgreSQL I highly recommend to use Linux version of it instead. Linux implementation of Docker is much more well tested.
Hello ! I faced the same issue recently and I applied this same workaround, but I definitely don't understand waht is the root cause.
If you create a container and a network with the non default network it works perfectly until the machine reboot.

I traced a lot of things, and the issue seems around Hyper-V VM Swithches because it deletes and recreates network interfaces after each reboot. That cause "network xxxx not found" when the container tries to start. But if you look at `docker network ls` you see you "new" network with the same name but a different sha256.

I also tried to stick at maximum to the default network config so I tried something like :
```yaml
networks:
  elastic:
    driver: nat
    driver_opts:
      com.docker.network.windowsshim.networkname: elastic
```
To have a named network interface, but in this case, at reboot the interface s not even recreated !
Saying that, what I didn't tried is to name the interface "nat"... Maybe it could work by some magic...

Anyway, what I don't really understand is **how** it works for the default nat network !? because the behaviour seems the same (adapter deleted and recreated).

Which component is not doing its job as expected ?
* moby 
* hcsshim
* hyper-v
@gillg as far I know the issue is that libnetwork is not aware that HNS does remove those networks and it should re-create them. Default "nat" network gets created because after same way like it does after when dockerd is started first time on new server.

You can see that behavior by checking with PowerShell command `Get-HnsNetwork` before and after service reboot / server reboot.

Fix patch:
diff --git a/daemon/daemon_windows.go b/daemon/daemon_windows.go
index 7acc06c7f7f07..3dc8e09aeb8d4 100644
--- a/daemon/daemon_windows.go
+++ b/daemon/daemon_windows.go
@@ -264,6 +264,35 @@ func (daemon *Daemon) initNetworkController(activeSandboxes map[string]interface
 		}
 
 		if !found {
+			// non-default nat networks should be re-created if missing from HNS
+			if v.Type() == "nat" && v.Name() != "nat" {
+				_, _, v4Conf, v6Conf := v.Info().IpamConfig()
+				netOption := map[string]string{}
+				for k, v := range v.Info().DriverOptions() {
+					if k != winlibnetwork.NetworkName && k != winlibnetwork.HNSID {
+						netOption[k] = v
+					}
+				}
+				name := v.Name()
+				id := v.ID()
+
+				err = v.Delete()
+				if err != nil {
+					logrus.Errorf("Error occurred when removing network %v", err)
+				}
+
+				_, err := daemon.netController.NewNetwork("nat", name, id,
+					libnetwork.NetworkOptionGeneric(options.Generic{
+						netlabel.GenericData: netOption,
+					}),
+					libnetwork.NetworkOptionIpam("default", "", v4Conf, v6Conf, nil),
+				)
+				if err != nil {
+					logrus.Errorf("Error occurred when creating network %v", err)
+				}
+				continue
+			}
+
 			// global networks should not be deleted by local HNS
 			if v.Info().Scope() != datastore.GlobalScope {
 				err = v.Delete()


Test patch: