Change:
// HashContainer returns the hash of the container. It is used to compare // the running container with its desired spec. <ins> // Note: remember to update hashValues in container_hash_test.go as well. </ins> func HashContainer(container *v1.Container) uint64 { hash := fnv.New32a() <del> hashutil.DeepHashObject(hash, *container) </del> <ins> // Omit nil or empty field when calculating hash value // Please see https://github.com/kubernetes/kubernetes/issues/53644 containerJson, _ := json.Marshal(container) hashutil.DeepHashObject(hash, containerJson) </ins> return uint64(hash.Sum32()) }

Discussion:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: During the update from v1.7.1 to v1.8.0 all docker containers were restarted, because container spec hash changed. What you expected to happen: No restart of the docker containers, because they did not change at all. How to reproduce it (as minimally and precisely as possible): get yourself a k8s cluster in v1.7.x update (at least the kubelet) to v1.8.0 Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration**: AWS OS (e.g. from /etc/os-release):Kernel (e.g. ):Install tools: Others:
/area kubelet
I ran into this earlier today. I was able to update the control plane & kube-proxy to 1.8.1 without any problems. Once I upgraded the kubelet though it caused container restarts.
/sig apps /sig api-machinery
the log message one receives is at: The function, that calculates the hashes is at: The hash function seems to not have changed. I suspect changes in the object itself, which is at: One can see: but:
But the new field does not initiated, so the restart seems unreasonable.
the result i tested was that the spew pkg also dumps the new add field(with nil value) to the hasher. is the new field. so, if the container use , then new MountPropagationMode will be included into the hash input, the hash result changed. other new fields are in the same situation.
Do you have any idea about preventing this?
We've hit this as well. Caused a cluster-wide restart of every container.
i also hit this issue as well, when I am upgrading cluster from v1.7.4 to v1.8.5. especially, the kubelet version from v1.7.4 to v1.8.5
Still seems to happen in 1.9.0
I can confirm this. v1.9.0 has introduced a new field attribute for , which is causing the hash value get changed.
It will exist when we use docker. I remember had described it.

There are details in this issues below:
thanks for digging out the old issue! Yes, any new field to the container spec would cause the hash to change. This is not a new issue but new fields are to the container spec rarely (compared with pod spec) so it doesn't happen so often. As discussed in , a new way to generate the hash (e.g., using only updatable fields) or other method for kubelet to examine and compare the state of the container against the spec is needed. Calling this (i.e., container restart) out in the 1.9 release note would be nice.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
/close
Just wondering, why don't we calculate the container hash by since container image is the only mutable field? I know we have to do some refactoring to , but wouldn't it be more reliable?