Repository: containrrr/watchtower (go)

Issue:
Healtcheck not updated
### Describe the bug

An image specifies the health check with various arguments, e. g. the following health check:
```dockerfile
HEALTHCHECK --interval=5s --timeout=2s --start-period=10s --retries=3 CMD [ "/bin/bash", "/healthcheck.sh" ]
```

container: (`$> docker inspect kjn2kj3498`)
```jsonc
// ...
"Healthcheck": {
  "Test": [
    "CMD",
    "/bin/bash",
    "/healthcheck.sh"
  ],
  "Interval": 5000000000,
  "Timeout": 2000000000,
  "StartPeriod": 10000000000,
  "Retries": 3
},
```

The image author updates the health check parameters, e. g. timeout will be increased to 15s, and retries to 4:
```dockerfile
HEALTHCHECK --interval=5s --timeout=15s --start-period=10s --retries=4 CMD [ "/bin/bash", "/healthcheck.sh" ]
```

Watchtower updates the container by pulling the new image and recreate a new container of the new image. But the new container runs with the **old** healthcheck arguments instead of the updated ones:

`$> docker inspect ui43j3na`
```jsonc
// ...
"Healthcheck": {
  "Test": [
    "CMD",
    "/bin/bash",
    "/healthcheck.sh"
  ],
  "Interval": 5000000000,
  "Timeout": 2000000000,
  "StartPeriod": 10000000000,
  "Retries": 3
},
```

If you manually re-create the container, e. g. using `docker-compose pull` and `docker-compose up -d`, the updated health check parameters will be applied to the new container:

`$> docker inspect asdn2904ds`
```jsonc
// ...
  "Healthcheck": {
    "Test": [
        "CMD",
        "/bin/bash",
        "/healthcheck.sh"
    ],
    "Interval": 60000000000,
    "Timeout": 15000000000,
    "StartPeriod": 600000000000,
    "Retries": 4
  },
```

### Steps to reproduce

1. Create a simplified dockerfile:
```dockerfile
FROM ubuntu

HEALTHCHECK --interval=5s --timeout=2s --start-period=10s --retries=3 CMD [ "/usr/bin/sleep", "1s"  ]

ENTRYPOINT [ "/usr/bin/sleep", "10m" ]
```
2. build the image: `docker build --no-cache -t healthcheck-image .`
3. publish the image: `docker push healthcheck-image`
4. Create a container + watchtower with the following compose file:
```yaml
version: '3'
services:
  healthcheck-test:
    image: healtcheck-image
    labels:
      - com.centurylinklabs.watchtower.enable=true

  watchtower:
    image: containrrr/watchtower
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    command: --interval 10
    environment:
      - WATCHTOWER_LABEL_ENABLE=true
```
5. Inspect the container to check the healthcheck configuration
```sh
$> docker inspect --format "{{ .Config.Healthcheck }}" container-id
{[CMD /usr/bin/sleep 1s] 5s 2s 10s 3}
```
6. Update the healthcheck `timeout` and `retries` Parameter
```dockerfile
FROM ubuntu

HEALTHCHECK --interval=5s --timeout=15s --start-period=10s --retries=4 CMD [ "/usr/bin/sleep", "1s"  ]

ENTRYPOINT [ "/usr/bin/sleep", "10m" ]
```
7. rebuild and republish the image
8. Wait for watchtower to update the container
9. re-check the container healthcheck configuration
```sh
$> docker inspect --format "{{ .Config.Healthcheck }}" container-id
{[CMD /usr/bin/sleep 1s] 5s 2s 10s 3}
```

Expected:
```sh
$> docker inspect --format "{{ .Config.Healthcheck }}" container-id
{[CMD /usr/bin/sleep 1s] 5s 15s 10s 4}
```

Do the same with `docker-compose up -d --force-recreate`. The expected healthcheck configuration will be applied.

### Expected behavior

The container should be recreated with the updated healtcheck configuration

### Screenshots

_No response_

<details>

<summary>

### Environment

</summary>

```
docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.16.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 9
  Running: 2
  Paused: 0
  Stopped: 7
 Images: 347
 Server Version: 23.0.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.10.102.1-microsoft-standard-WSL2
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 12.27GiB
 Name: Wehrstedt
 ID: DDGC:2DRA:UQYD:5LEP:PAUW:AA4Y:XA6G:MJD5:SCWX:U6MM:3D5Z:Z2IV
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 20
   Base: 192.168.0.0/16, Size: 24

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
```
</details>

### Your logs

```text
Attaching to watchtower-test_watchtower_1, watchtower-test_healthcheck-test_1
watchtower_1        | time="2023-03-21T11:43:49Z" level=debug msg="Sleeping for a second to ensure the docker api client has been properly initialized."
watchtower_1        | time="2023-03-21T11:43:50Z" level=debug msg="Making sure everything is sane before starting"
watchtower_1        | time="2023-03-21T11:43:50Z" level=debug msg="Retrieving running containers"
watchtower_1        | time="2023-03-21T11:43:50Z" level=debug msg="There are no additional watchtower containers"
watchtower_1        | time="2023-03-21T11:43:50Z" level=debug msg="Watchtower HTTP API skipped."
watchtower_1        | time="2023-03-21T11:43:50Z" level=info msg="Watchtower 1.5.3"
watchtower_1        | time="2023-03-21T11:43:50Z" level=info msg="Using no notifications"
watchtower_1        | time="2023-03-21T11:43:50Z" level=info msg="Only checking containers using enable label"
watchtower_1        | time="2023-03-21T11:43:50Z" level=info msg="Scheduling first run: 2023-03-21 11:44:00 +0000 UTC"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Checking containers for updated images"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Retrieving running containers"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Trying to load authentication credentials." container=/watchtower-test_healthcheck-test_1 image="<registry-url>/healtcheck-image:latest"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Loaded auth credentials for user <user>, on registry <registry-url>/healtcheck-image:latest, from file /config.json"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Got image name: <registry-url>/healtcheck-image:latest"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Credentials loaded"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Checking if pull is needed" container=/watchtower-test_healthcheck-test_1 image="<registry-url>/healtcheck-image:latest"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Building challenge URL" URL="https://<registry-url>/v2/"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Got response to challenge request" header="Bearer realm=\"https://<registry-url>/jwt/auth\",service=\"container_registry\"" status="401 Unauthorized"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Checking challenge header content" realm="https://<registry-url>/jwt/auth" service=container_registry
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Setting scope for auth token" image=<registry-url>/healtcheck-image scope="repository:<registry-url>/healtcheck-image:pull"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Credentials found."
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Parsing image ref" host=<registry-url> image=healtcheck-image normalized="<registry-url>/healtcheck-image:latest" tag=latest
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Doing a HEAD request to fetch a digest" url="https://<registry-url>/v2/healtcheck-image/manifests/latest"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Could not do a head request for \"<registry-url>/healtcheck-image:latest\", falling back to regular pull." container=/watchtower-test_healthcheck-test_1 image="<registry-url>/healtcheck-image:latest"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Reason: registry responded to head request with \"401 Unauthorized\", auth: \"Bearer realm=\\\"https://<registry-url>/jwt/auth\\\",service=\\\"container_registry\\\",scope=\\\"repository:healtcheck-image:pull\\\",error=\\\"insufficient_scope\\\"\"" container=/watchtower-test_healthcheck-test_1 image="<registry-url>/healtcheck-image:latest"
watchtower_1        | time="2023-03-21T11:44:30Z" level=debug msg="Pulling image" container=/watchtower-test_healthcheck-test_1 image="<registry-url>/healtcheck-image:latest"
watchtower_1        | time="2023-03-21T11:44:30Z" level=info msg="Found new <registry-url>/healtcheck-image:latest image (feb17fe278fe)"
watchtower_1        | time="2023-03-21T11:44:30Z" level=info msg="Stopping /watchtower-test_healthcheck-test_1 (f29ee537d965) with SIGTERM"
watchtower_1        | time="2023-03-21T11:44:40Z" level=debug msg="Skipped another update already running."
watchtower_1        | time="2023-03-21T11:44:40Z" level=debug msg="Scheduled next run: 2023-03-21 11:44:50 +0000 UTC"
watchtower_1        | time="2023-03-21T11:44:40Z" level=debug msg="Removing container f29ee537d965"
watchtower-test_healthcheck-test_1 exited with code 137
watchtower_1        | time="2023-03-21T11:44:41Z" level=info msg="Creating /watchtower-test_healthcheck-test_1"
watchtower_1        | time="2023-03-21T11:44:41Z" level=debug msg="Starting container /watchtower-test_healthcheck-test_1 (de887b1bc2fc)"
watchtower_1        | time="2023-03-21T11:44:41Z" level=info msg="Session done" Failed=0 Scanned=1 Updated=1 notify=no
```


### Additional context

_No response_

Pull request:
fix: use new healthcheck config if not overridden
This PR will clear out the create container healthcheck configuration if it's equal to the original image's configuration. Doing this allows any new configuration from the updated image to be used instead.

Fixes #1601 

Patch:
diff --git a/pkg/container/container.go b/pkg/container/container.go
index b18e53c..0433c1e 100644
--- a/pkg/container/container.go
+++ b/pkg/container/container.go
@@ -304,6 +304,27 @@ func (c Container) GetCreateConfig() *dockercontainer.Config {
 		}
 	}
 
+	// Clear HEALTHCHECK configuration (if default)
+	if util.SliceEqual(config.Healthcheck.Test, imageConfig.Healthcheck.Test) {
+		config.Healthcheck.Test = nil
+	}
+
+	if config.Healthcheck.Retries == imageConfig.Healthcheck.Retries {
+		config.Healthcheck.Retries = 0
+	}
+
+	if config.Healthcheck.Interval == imageConfig.Healthcheck.Interval {
+		config.Healthcheck.Interval = 0
+	}
+
+	if config.Healthcheck.Timeout == imageConfig.Healthcheck.Timeout {
+		config.Healthcheck.Timeout = 0
+	}
+
+	if config.Healthcheck.StartPeriod == imageConfig.Healthcheck.StartPeriod {
+		config.Healthcheck.StartPeriod = 0
+	}
+
 	config.Env = util.SliceSubtract(config.Env, imageConfig.Env)
 
 	config.Labels = util.StringMapSubtract(config.Labels, imageConfig.Labels)


Test patch:
diff --git a/pkg/container/container_mock_test.go b/pkg/container/container_mock_test.go
index 96a7834..8aa1470 100644
--- a/pkg/container/container_mock_test.go
+++ b/pkg/container/container_mock_test.go
@@ -21,7 +21,8 @@ func MockContainer(updates ...MockContainerUpdate) *Container {
 		},
 	}
 	image := types.ImageInspect{
-		ID: "image_id",
+		ID:     "image_id",
+		Config: &dockerContainer.Config{},
 	}
 
 	for _, update := range updates {
@@ -64,3 +65,15 @@ func WithContainerState(state types.ContainerState) MockContainerUpdate {
 		cnt.State = &state
 	}
 }
+
+func WithHealthcheck(healthConfig dockerContainer.HealthConfig) MockContainerUpdate {
+	return func(cnt *types.ContainerJSON, img *types.ImageInspect) {
+		cnt.Config.Healthcheck = &healthConfig
+	}
+}
+
+func WithImageHealthcheck(healthConfig dockerContainer.HealthConfig) MockContainerUpdate {
+	return func(cnt *types.ContainerJSON, img *types.ImageInspect) {
+		img.Config.Healthcheck = &healthConfig
+	}
+}
diff --git a/pkg/container/container_test.go b/pkg/container/container_test.go
index afc2bc1..8b9ed58 100644
--- a/pkg/container/container_test.go
+++ b/pkg/container/container_test.go
@@ -2,6 +2,7 @@ package container
 
 import (
 	"github.com/containrrr/watchtower/pkg/types"
+	dc "github.com/docker/docker/api/types/container"
 	"github.com/docker/go-connections/nat"
 	. "github.com/onsi/ginkgo"
 	. "github.com/onsi/gomega"
@@ -67,6 +68,63 @@ var _ = Describe("the container", func() {
 			})
 		})
 	})
+	Describe("GetCreateConfig", func() {
+		When("container healthcheck config is equal to image config", func() {
+			It("should return empty healthcheck values", func() {
+				c := MockContainer(WithHealthcheck(dc.HealthConfig{
+					Test: []string{"/usr/bin/sleep", "1s"},
+				}), WithImageHealthcheck(dc.HealthConfig{
+					Test: []string{"/usr/bin/sleep", "1s"},
+				}))
+				Expect(c.GetCreateConfig().Healthcheck).To(Equal(&dc.HealthConfig{}))
+
+				c = MockContainer(WithHealthcheck(dc.HealthConfig{
+					Timeout: 30,
+				}), WithImageHealthcheck(dc.HealthConfig{
+					Timeout: 30,
+				}))
+				Expect(c.GetCreateConfig().Healthcheck).To(Equal(&dc.HealthConfig{}))
+
+				c = MockContainer(WithHealthcheck(dc.HealthConfig{
+					StartPeriod: 30,
+				}), WithImageHealthcheck(dc.HealthConfig{
+					StartPeriod: 30,
+				}))
+				Expect(c.GetCreateConfig().Healthcheck).To(Equal(&dc.HealthConfig{}))
+
+				c = MockContainer(WithHealthcheck(dc.HealthConfig{
+					Retries: 30,
+				}), WithImageHealthcheck(dc.HealthConfig{
+					Retries: 30,
+				}))
+				Expect(c.GetCreateConfig().Healthcheck).To(Equal(&dc.HealthConfig{}))
+			})
+		})
+		When("container healthcheck config is different to image config", func() {
+			It("should return the container healthcheck values", func() {
+				c := MockContainer(WithHealthcheck(dc.HealthConfig{
+					Test:        []string{"/usr/bin/sleep", "1s"},
+					Interval:    30,
+					Timeout:     30,
+					StartPeriod: 10,
+					Retries:     2,
+				}), WithImageHealthcheck(dc.HealthConfig{
+					Test:        []string{"/usr/bin/sleep", "10s"},
+					Interval:    10,
+					Timeout:     60,
+					StartPeriod: 30,
+					Retries:     10,
+				}))
+				Expect(c.GetCreateConfig().Healthcheck).To(Equal(&dc.HealthConfig{
+					Test:        []string{"/usr/bin/sleep", "1s"},
+					Interval:    30,
+					Timeout:     30,
+					StartPeriod: 10,
+					Retries:     2,
+				}))
+			})
+		})
+	})
 	When("asked for metadata", func() {
 		var c *Container
 		BeforeEach(func() {