Repository: kubernetes/kops (go)

Issue:
kops 1.29.0 can't deploy aws-iam-authenticator pods - image tag not found in registry
/kind bug

**1. What `kops` version are you running? The command `kops version`, will display
 this information.**
kops 1.29.0

**2. What Kubernetes version are you running? `kubectl version` will print the
 version if a cluster is running or provide the Kubernetes version specified as
 a `kops` flag.**
v1.29.5

**3. What cloud provider are you using?**
AWS

**4. What commands did you run?  What is the simplest way to reproduce this issue?**
Enabled in cluster spec
```
  authentication:
    aws: {}
```

**5. What happened after the commands executed?**
aws-iam-authenticator pods can't pull their specified image because it's missing in the registry

```
  Warning  Failed       3m8s (x3 over 3m53s)   kubelet            Failed to pull image "602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27": rpc error: code = NotFound desc = failed to pull and unpack image "602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27": failed to resolve reference "602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27": 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27: not found
  Warning  Failed       3m8s (x3 over 3m53s)   kubelet            Error: ErrImagePull
  Normal   BackOff      2m41s (x4 over 3m52s)  kubelet            Back-off pulling image "602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27"
  Warning  Failed       2m41s (x4 over 3m52s)  kubelet            Error: ImagePullBackOff
```


**6. What did you expect to happen?**
aws-iam-authenticator pods running

**7. Please provide your cluster manifest. Execute
  `kops get --name my.example.com -o yaml` to display your cluster manifest.
  You may want to remove your cluster name and other sensitive information.**

```yaml

apiVersion: kops.k8s.io/v1alpha2
kind: Cluster
metadata:
  creationTimestamp: "2024-06-05T12:22:06Z"
  generation: 3
  name: XXXXXXXXXXXXXXXXXX
spec:
  additionalPolicies:
    master: |
      [
        {
          "Effect": "Allow",
          "Action": ["ecr:BatchImportUpstreamImage","ecr:CreateRepository","ecr:TagResource"],
          "Resource": ["*"]
        }
      ]
    node: |
      [
        {
          "Effect": "Allow",
          "Action": ["ecr:BatchImportUpstreamImage","ecr:CreateRepository","ecr:TagResource"],
          "Resource": ["*"]
        }
      ]
  api:
    dns: {}
  authentication:
    aws: {}
  authorization:
    rbac: {}
  certManager:
    enabled: true
    managed: false
  channel: stable
  cloudLabels:
    Environment: development
  cloudProvider: aws
  configBase: s3://XXXXXXXXXXXXXXXXXXXX/kops/XXXXXXXXXXXXXXXXXXX
  dnsZone: XXXXXXXXXXXXXXXXXXXX
  etcdClusters:
  - cpuRequest: 200m
    etcdMembers:
    - encryptedVolume: true
      instanceGroup: control-plane-us-east-2a
      name: a
    - encryptedVolume: true
      instanceGroup: control-plane-us-east-2b
      name: b
    - encryptedVolume: true
      instanceGroup: control-plane-us-east-2c
      name: c
    manager:
      backupRetentionDays: 90
    memoryRequest: 100Mi
    name: main
  - cpuRequest: 100m
    etcdMembers:
    - encryptedVolume: true
      instanceGroup: control-plane-us-east-2a
      name: a
    - encryptedVolume: true
      instanceGroup: control-plane-us-east-2b
      name: b
    - encryptedVolume: true
      instanceGroup: control-plane-us-east-2c
      name: c
    manager:
      backupRetentionDays: 90
    memoryRequest: 100Mi
    name: events
  iam:
    allowContainerRegistry: true
    legacy: false
    useServiceAccountExternalPermissions: true
  kubeDNS:
    nodeLocalDNS:
      enabled: true
    provider: CoreDNS
  kubeProxy:
    enabled: false
  kubelet:
    anonymousAuth: false
  kubernetesApiAccess:
  - 0.0.0.0/0
  - ::/0
  kubernetesVersion: 1.29.5
  masterPublicName: api.XXXXXXX
  metricsServer:
    enabled: true
    insecure: false
  networkCIDR: 10.105.0.0/16
  networkID: vpc-05323df512cb1db11
  networking:
    cilium:
      enableNodePort: true
      hubble:
        enabled: true
  nonMasqueradeCIDR: 100.64.0.0/10
  podIdentityWebhook:
    enabled: true
  serviceAccountIssuerDiscovery:
    discoveryStore: s3://XXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXX
    enableAWSOIDCProvider: true
  sshAccess:
  - 0.0.0.0/0
  - ::/0
  subnets:
  - cidr: 10.105.64.0/19
    ipv6CIDR: /64#40
    name: us-east-2a
    type: Public
    zone: us-east-2a
  - cidr: 10.105.96.0/19
    ipv6CIDR: /64#41
    name: us-east-2b
    type: Public
    zone: us-east-2b
  - cidr: 10.105.128.0/19
    ipv6CIDR: /64#42
    name: us-east-2c
    type: Public
    zone: us-east-2c
  sysctlParameters:
  - net.ipv4.neigh.default.gc_thresh2=28672
  - net.ipv4.neigh.default.gc_thresh3=32768
  topology:
    dns:
      type: Public

```

**8. Please run the commands with most verbose logging by adding the `-v 10` flag.
  Paste the logs into this report, or in a gist and provide the gist link here.**

**9. Anything else do we need to know?**


Pull request:
Update aws-iam-authenticator image
Temporary workaround for https://github.com/kubernetes-sigs/aws-iam-authenticator/issues/723

Closes #16602

This should unblock the 1.30 beta release

Patch:
diff --git a/upup/models/cloudup/resources/addons/authentication.aws/k8s-1.12.yaml.template b/upup/models/cloudup/resources/addons/authentication.aws/k8s-1.12.yaml.template
index b297a6897f..c5ca31e6c9 100644
--- a/upup/models/cloudup/resources/addons/authentication.aws/k8s-1.12.yaml.template
+++ b/upup/models/cloudup/resources/addons/authentication.aws/k8s-1.12.yaml.template
@@ -164,7 +164,7 @@ spec:
       # - output (output kubeconfig to plug into your apiserver configuration, mounted from the host)
       containers:
       - name: aws-iam-authenticator
-        image: {{ or .Authentication.AWS.Image "602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27" }}
+        image: {{ or .Authentication.AWS.Image "public.ecr.aws/eks-distro/kubernetes-sigs/aws-iam-authenticator:v0.6.20-eks-1-30-7" }}
         args:
         - server
         {{- if or (not .Authentication.AWS.BackendMode) (contains "MountedFile" .Authentication.AWS.BackendMode) }}


Test patch:
diff --git a/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-authentication.aws-k8s-1.12_content b/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-authentication.aws-k8s-1.12_content
index 32bd24845b..26293b1294 100644
--- a/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-authentication.aws-k8s-1.12_content
+++ b/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-authentication.aws-k8s-1.12_content
@@ -164,7 +164,7 @@ spec:
         - --config=/etc/aws-iam-authenticator/config.yaml
         - --state-dir=/var/aws-iam-authenticator
         - --kubeconfig-pregenerated=true
-        image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27
+        image: public.ecr.aws/eks-distro/kubernetes-sigs/aws-iam-authenticator:v0.6.20-eks-1-30-7
         livenessProbe:
           httpGet:
             host: 127.0.0.1
diff --git a/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-bootstrap_content b/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-bootstrap_content
index 947d6a249c..a1a911a588 100644
--- a/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-bootstrap_content
+++ b/tests/integration/update_cluster/complex/data/aws_s3_object_complex.example.com-addons-bootstrap_content
@@ -99,7 +99,7 @@ spec:
     version: 9.99.0
   - id: k8s-1.12
     manifest: authentication.aws/k8s-1.12.yaml
-    manifestHash: 904efad45a2ff29d5abc3357121317223e3d1b9256be90ba38178954fee50687
+    manifestHash: 4a72854108b5bf11c547c832153a13abc6b26952723ae7a9a9f8dd7126b8116c
     name: authentication.aws
     selector:
       role.kubernetes.io/authentication: "1"
diff --git a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/authentication.aws-k8s-1.12.yaml b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/authentication.aws-k8s-1.12.yaml
index 5ffeefe756..a9f631528b 100644
--- a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/authentication.aws-k8s-1.12.yaml
+++ b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/authentication.aws-k8s-1.12.yaml
@@ -166,7 +166,7 @@ spec:
         - --state-dir=/var/aws-iam-authenticator
         - --kubeconfig-pregenerated=true
         - --backend-mode=CRD,MountedFile
-        image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27
+        image: public.ecr.aws/eks-distro/kubernetes-sigs/aws-iam-authenticator:v0.6.20-eks-1-30-7
         livenessProbe:
           httpGet:
             host: 127.0.0.1
diff --git a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/manifest.yaml b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/manifest.yaml
index ad6f0bdd56..edb51f4037 100644
--- a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/manifest.yaml
+++ b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/crd/manifest.yaml
@@ -99,7 +99,7 @@ spec:
     version: 9.99.0
   - id: k8s-1.12
     manifest: authentication.aws/k8s-1.12.yaml
-    manifestHash: e38c1aa72a3d40a7548fdd055f7baa1c24d9a63f0d336fe82aeffbcb6592610e
+    manifestHash: 4debe9956ffc92cd7c8da44e6fcc4c9705d51faa6640be9a59e8d86e0854c7ad
     name: authentication.aws
     selector:
       role.kubernetes.io/authentication: "1"
diff --git a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/authentication.aws-k8s-1.12.yaml b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/authentication.aws-k8s-1.12.yaml
index 93ec710011..176a58e181 100644
--- a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/authentication.aws-k8s-1.12.yaml
+++ b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/authentication.aws-k8s-1.12.yaml
@@ -165,7 +165,7 @@ spec:
         - --state-dir=/var/aws-iam-authenticator
         - --kubeconfig-pregenerated=true
         - --backend-mode=CRD
-        image: 602401143452.dkr.ecr.us-west-2.amazonaws.com/amazon/aws-iam-authenticator:v0.5.27
+        image: public.ecr.aws/eks-distro/kubernetes-sigs/aws-iam-authenticator:v0.6.20-eks-1-30-7
         livenessProbe:
           httpGet:
             host: 127.0.0.1
diff --git a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/manifest.yaml b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/manifest.yaml
index ffa0804a80..3e437fcb65 100644
--- a/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/manifest.yaml
+++ b/upup/pkg/fi/cloudup/tests/bootstrapchannelbuilder/awsiamauthenticator/mappings/manifest.yaml
@@ -99,7 +99,7 @@ spec:
     version: 9.99.0
   - id: k8s-1.12
     manifest: authentication.aws/k8s-1.12.yaml
-    manifestHash: 76df887736bf7023f791eb8921ad0b1df8e55dcd1af7f24d138fe1176687d4f5
+    manifestHash: c8a71f31e741c1938991952cf16728801dc2e2f1b81f92aef39b0d9c250a3f94
     name: authentication.aws
     selector:
       role.kubernetes.io/authentication: "1"