Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --my mutating webhook calculates a hash of the , allows reinvocation; no side effects my validating webhook validates the hash of the and fails because the has changed webhook auditing indicates that my webhooks are the only ones processing the upon serializing the spec to JSON for both the mutator and validator, the only change is related to the "kube-api-access-xyz" projected volume, managed by a built-in k8s admission plugin the aforementioned plugin does not appear to mutate the reinvocation context, even through it may mutate a pod (robbing other mutators of the chance to observe the change): it appears that both my hooks see the injected volume, but the validation hook sees the injected SA volume with a "defaultMode" field set, and my mutation hook does NOT see this field set my validation hook should calculate the same hash mutating, validating webhooks are implemented atop controller-runtime v0.8.3, pod objects are being decoded into targets Kubernetes version (use ): 1.21.3 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
cc
interesting... defaults are applied when going from versioned → internal server decodes v1 Pod request → applies defaults → converts to internal Pod -in admission control operates on the internal Pod service account admission injects the kube-api-access volume admission is called internal Pod → converts to v1 Pod → serializes → sends to mutating webhook mutating webhook returns patch patch is applied to v1 Pod JSON resulting v1 Pod JSON is decoded → defaults applied (defaultMode is set on the api-token volume here) → converted to internal Pod webhook admission is called internal Pod → converts to v1 Pod → serializes → sends to validating webhook That means if any built-in admission plugin sets up data that would be modified by applying defaulting to the pod, the data sent to mutating webhooks will be missing those applied defaults until the first mutating webhook that returns a patch that changes the object
Possible solutions: defaults are applied to the internal → versioned object before invoking webhooks built-in admission always populates data matching default values for unset fields
As an aside, I'm not sure what you're using the hashing for, but be aware that hashing is fragile in the presence of defaults and new fields over time. A hash you add to a pod spec in one release might not match a hash computed in a different release, or with different admission in place.
/triage accepted /assign /priority backlog

Discussion:
"time" corev1 "k8s.io/api/core/v1" <ins> v1 "k8s.io/api/core/v1" </ins> "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/util/sets"

podutil "k8s.io/kubernetes/pkg/api/pod" api "k8s.io/kubernetes/pkg/apis/core" "k8s.io/kubernetes/pkg/serviceaccount" <ins> "k8s.io/utils/pointer" </ins> ) const (

// TokenVolumeSource returns the projected volume source for service account token. func TokenVolumeSource() *api.ProjectedVolumeSource { return &api.ProjectedVolumeSource{ <ins> // explicitly set default value, see #104464 DefaultMode: pointer.Int32(v1.ProjectedVolumeSourceDefaultMode), </ins> Sources: []api.VolumeProjection{ { ServiceAccountToken: &api.ServiceAccountTokenProjection{

"strings" "testing" <ins> "github.com/google/go-cmp/cmp" </ins> "github.com/stretchr/testify/assert" corev1 "k8s.io/api/core/v1" <ins> v1 "k8s.io/api/core/v1" </ins> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "k8s.io/apimachinery/pkg/util/diff"

"k8s.io/client-go/informers" "k8s.io/client-go/kubernetes/fake" api "k8s.io/kubernetes/pkg/apis/core" <ins> v1defaults "k8s.io/kubernetes/pkg/apis/core/v1" </ins> "k8s.io/kubernetes/pkg/controller" kubelet "k8s.io/kubernetes/pkg/kubelet/types" <ins> utilpointer "k8s.io/utils/pointer" </ins> ) func TestIgnoresNonCreate(t *testing.T) {

}, }) <del> pod := &api.Pod{ Spec: api.PodSpec{ Containers: []api.Container{{}}, </del> <ins> v1PodIn := &v1.Pod{ Spec: v1.PodSpec{ Containers: []v1.Container{{}}, </ins> }, } <ins> v1defaults.SetObjectDefaults_Pod(v1PodIn) pod := &api.Pod{} if err := v1defaults.Convert_v1_Pod_To_core_Pod(v1PodIn, pod, nil); err != nil { t.Fatal(err) } </ins> attrs := admission.NewAttributesRecord(pod, nil, api.Kind("Pod").WithVersion("version"), ns, "myname", api.Resource("pods").WithVersion("version"), "", admission.Create, &metav1.CreateOptions{}, false, nil) err := admissiontesting.WithReinvocationTesting(t, admit).Admit(context.TODO(), attrs, nil) if err != nil {

{ConfigMap: &api.ConfigMapProjection{LocalObjectReference: api.LocalObjectReference{Name: "kube-root-ca.crt"}, Items: []api.KeyToPath{{Key: "ca.crt", Path: "ca.crt"}}}}, {DownwardAPI: &api.DownwardAPIProjection{Items: []api.DownwardAPIVolumeFile{{Path: "namespace", FieldRef: &api.ObjectFieldSelector{APIVersion: "v1", FieldPath: "metadata.namespace"}}}}}, }, <ins> DefaultMode: utilpointer.Int32(0644), </ins> }, }, }}

if !reflect.DeepEqual(expectedVolumeMounts, pod.Spec.Containers[0].VolumeMounts) { t.Errorf("unexpected volumes: %s", diff.ObjectReflectDiff(expectedVolumeMounts, pod.Spec.Containers[0].VolumeMounts)) } <ins> // ensure result converted to v1 matches defaulted object v1PodOut := &v1.Pod{} if err := v1defaults.Convert_core_Pod_To_v1_Pod(pod, v1PodOut, nil); err != nil { t.Fatal(err) } v1PodOutDefaulted := v1PodOut.DeepCopy() v1defaults.SetObjectDefaults_Pod(v1PodOutDefaulted) if !reflect.DeepEqual(v1PodOut, v1PodOutDefaulted) { t.Error(cmp.Diff(v1PodOut, v1PodOutDefaulted)) } </ins> } func TestFetchesUncachedServiceAccount(t *testing.T) {