Repository: moby/moby (language: go)

Issue:
panic: interface {} is nil in libnetwork/drivers/ipvlan
### Description

After updating to docker-ce 23 from 20.10.23 the daemon would no longer start, checking with `dockerd --debug` I see the shortened error output attached under additional info

I assume it may be a compatability issue with the new docker version, but from what I'm getting I have no way of knowing what causes the issue exactly

### Reproduce

Not very sure, but my assumption is it may have to do with having an ipvlan setup on the previous version and then upgrading, I'd have to dig into it some more

1. Create ipvlan in docker-ce 20.10.23
2. Upgrade docker-ce to latest
3. I assume it would fail

### Expected behavior

For docker to startup or provide a clearer message about what might be wrong here

### docker version

```bash
Client: Docker Engine - Community
 Version:           23.0.0
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        e92dd87
 Built:             Wed Feb  1 17:44:38 2023
 OS/Arch:           linux/arm64
 Context:           default
```


### docker info

```bash
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.5.0
    Path:     /home/elp/.docker/cli-plugins/docker-compose

Server:
```


### Additional Info

```
DEBU[2023-02-05T00:33:18.957555317+01:00] Network (7640c03) restored
DEBU[2023-02-05T00:33:18.958571817+01:00] Endpoint (08537ee) restored to network (445d1e5)
INFO[2023-02-05T00:33:18.959434188+01:00] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=plugins.moby
INFO[2023-02-05T00:33:18.959697891+01:00] stopping event stream following graceful shutdown  error="context canceled" module=libcontainerd namespace=moby
panic: interface conversion: interface {} is nil, not string

goroutine 1 [running]:
github.com/docker/docker/libnetwork/drivers/ipvlan.(*configuration).UnmarshalJSON(0x40011533b0, {0x400069c2d0, 0xef, 0xef})
        /go/src/github.com/docker/docker/libnetwork/drivers/ipvlan/ipvlan_store.go:196 +0x414
encoding/json.(*decodeState).object(0x4001153440, {0x5597157640?, 0x40011533b0?, 0x559524115c?})
        /usr/local/go/src/encoding/json/decode.go:613 +0x650
encoding/json.(*decodeState).value(0x4001153440, {0x5597157640?, 0x40011533b0?, 0x559524005c?})
        /usr/local/go/src/encoding/json/decode.go:374 +0x40
encoding/json.(*decodeState).unmarshal(0x4001153440, {0x5597157640?, 0x40011533b0?})
        /usr/local/go/src/encoding/json/decode.go:181 +0x204
encoding/json.Unmarshal({0x400069c2d0, 0xef, 0xef}, {0x5597157640, 0x40011533b0})
        /usr/local/go/src/encoding/json/decode.go:108 +0xf4
github.com/docker/docker/libnetwork/drivers/ipvlan.(*configuration).SetValue(0x4000d18050?, {0x400069c2d0?, 0x23?, 0x23?})
        /go/src/github.com/docker/docker/libnetwork/drivers/ipvlan/ipvlan_store.go:230 +0x38
github.com/docker/docker/libnetwork/datastore.(*cache).kmap(0x400121ef18, {0x5597253860, 0x4001153320})
        /go/src/github.com/docker/docker/libnetwork/datastore/cache.go:61 +0x550
github.com/docker/docker/libnetwork/datastore.(*cache).list(0x400121ef18, {0x5597253860?, 0x4001153320?})
        /go/src/github.com/docker/docker/libnetwork/datastore/cache.go:164 +0x44
github.com/docker/docker/libnetwork/datastore.(*datastore).List(0x4000cb1140?, {0x4000d1a240?, 0x6?}, {0x5597253860?, 0x4001153320?})
        /go/src/github.com/docker/docker/libnetwork/datastore/datastore.go:517 +0x11c
github.com/docker/docker/libnetwork/drivers/ipvlan.(*driver).populateNetworks(0x4000ccfc50)
        /go/src/github.com/docker/docker/libnetwork/drivers/ipvlan/ipvlan_store.go:77 +0x14c
github.com/docker/docker/libnetwork/drivers/ipvlan.(*driver).initStore(0x4000ccfc50, 0x5595eb5314?)
        /go/src/github.com/docker/docker/libnetwork/drivers/ipvlan/ipvlan_store.go:62 +0x170
github.com/docker/docker/libnetwork/drivers/ipvlan.Init({0x5597230fc0, 0x4001106d80}, 0x6?)
        /go/src/github.com/docker/docker/libnetwork/drivers/ipvlan/ipvlan.go:74 +0x6c
github.com/docker/docker/libnetwork/drvregistry.(*DrvRegistry).AddDriver(...)
        /go/src/github.com/docker/docker/libnetwork/drvregistry/drvregistry.go:72
github.com/docker/docker/libnetwork.New({0x40006e2620, 0x8, 0xe})
        /go/src/github.com/docker/docker/libnetwork/controller.go:217 +0x514
github.com/docker/docker/daemon.(*Daemon).initNetworkController(0x40009c6240, 0x4001089ec0)
        /go/src/github.com/docker/docker/daemon/daemon_unix.go:851 +0x3c
github.com/docker/docker/daemon.(*Daemon).restore(0x40009c6240)
        /go/src/github.com/docker/docker/daemon/daemon.go:478 +0x414
github.com/docker/docker/daemon.NewDaemon({0x5597245018?, 0x400077e3c0}, 0x4000142c00, 0x400098b770)
        /go/src/github.com/docker/docker/daemon/daemon.go:1085 +0x23a4
main.(*DaemonCli).start(0x400098a450, 0x4000980a80)
        /go/src/github.com/docker/docker/cmd/dockerd/daemon.go:200 +0x7e8
main.runDaemon(...)
        /go/src/github.com/docker/docker/cmd/dockerd/docker_unix.go:14
main.newDaemonCommand.func1(0x4000736c00?, {0x40009637b0?, 0x1?, 0x1?})
        /go/src/github.com/docker/docker/cmd/dockerd/docker.go:38 +0x68
github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).execute(0x4000736c00, {0x400004e070, 0x1, 0x1})
        /go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:916 +0x5c8
github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x4000736c00)
        /go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:1044 +0x35c
github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).Execute(...)
        /go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:968
main.main()
        /go/src/github.com/docker/docker/cmd/dockerd/docker.go:102 +0x170
```


Discussion / hints:
This is indeed in the restore of an ipvlan network -- you should be able to delete the state in `/var/lib/docker/network` to get up-and-running in the short term.

/cc @corhere to take a look at the failure.
I can reproduce this using the instructions in the issue body:
* create ipvlan network on 20.10.23
* upgrade to 23.0.0
* observe panic

I can also confirm the mitigation of removing the network state store will unstick the daemon. The exact same network can then be recreated and used as usual.

My test network is from [docs.docker.com](https://docs.docker.com/network/ipvlan/):
```
docker network create -d ipvlan \
    --subnet=192.168.1.0/24 \
    --gateway=192.168.1.1 \
    -o ipvlan_mode=l2 \
    -o parent=eth0 db_net
```
Can confirm, when I remove the ipvlan network before upgrading the upgrade runs fine and the docker daemon starts up, adding the ipvlan back in afterwards works without issues as well.
The panicking line is https://github.com/moby/moby/blob/d7573ab8672555762688f4c7ab8cc69ae8ec1a47/libnetwork/drivers/ipvlan/ipvlan_store.go#L196
which blames to https://github.com/moby/moby/pull/44027.
> it was a clean cherry-pick, and should be low risk

Famous last words @thaJeztah 😁 
🤦 😞 😭 

Fix patch:
diff --git a/libnetwork/drivers/ipvlan/ipvlan_store.go b/libnetwork/drivers/ipvlan/ipvlan_store.go
index c5ace3f4da52b..b55ef427ef4f9 100644
--- a/libnetwork/drivers/ipvlan/ipvlan_store.go
+++ b/libnetwork/drivers/ipvlan/ipvlan_store.go
@@ -193,7 +193,12 @@ func (config *configuration) UnmarshalJSON(b []byte) error {
 	config.Mtu = int(nMap["Mtu"].(float64))
 	config.Parent = nMap["Parent"].(string)
 	config.IpvlanMode = nMap["IpvlanMode"].(string)
-	config.IpvlanFlag = nMap["IpvlanFlag"].(string)
+	if v, ok := nMap["IpvlanFlag"]; ok {
+		config.IpvlanFlag = v.(string)
+	} else {
+		// Migrate config from an older daemon which did not have the flag configurable.
+		config.IpvlanFlag = flagBridge
+	}
 	config.Internal = nMap["Internal"].(bool)
 	config.CreatedSlaveLink = nMap["CreatedSubIface"].(bool)
 	if v, ok := nMap["Ipv4Subnets"]; ok {


Test patch: