Change:
Part of umbrella to migrate kubernetes e2e test images/registries to community-owned infrastructure: The following images are used by 1.20 e2e tests We should migrate them to community owned infrastructure (ref: for deciding how to implement a mirror registry) We should consider doing this for release branches as well, which may use different images/tags. So these wouldn't necessarily be cherry-picks Two possible approaches: change the registry in source (I would prefer eliminating the extra "docker library" registry, and consolidate to a single e2e test image registry) start running tests with a config that swaps out for some other registry /sig testing /sig release /area release-eng /wg k8s-infra /milestone v1.21
/kind cleanup
/triage accepted /help cc: Give a shout if any of you are interested in working on this!
I'm in! /assign
/remove-help
cc
/cc
I've had 2 PRs opened for this, before it was decited that the test jobs would use the registry mirror configuration. Has there been any consensus on the approach? If so, I can un-abandon these PRs if necessary:
ooh thanks for sharing I don't think there has been any consensus, though I personally think your work on kubernetes/kubernetes is relevant, especially that you've covered everything except gluster and redis.
We already have the redis image here [1], we'll just have to trigger a job for it afterwards. Indeed, I was missing the gluster image, I only considered the image for it.
so I've been thinking about this for a bit and I'm a bit concerned with the visibility of this setup to test authors -- I'm not sure if this is straightforward / intuitive enough for them to follow. alternatively, are there other lighter (in terms of maintenance) solutions that we can pursue?
Hm, what do you mean by the visibility of this setup of test authors? If I'm not mistaken, the current goal is to not rely so heavily on dockerhub for testing anymore. For this purpose we are mirroring these images into our own registries, and then we are going to update to use our own registry instead of the dockerhub one. Test authors shouldn't be impacted by this, IMO. From their point of view, it's the same image, from a different source. And in terms of maintenance... hm, techically, if someone would want to use an newer image, say , so it would be consistent.
yes, I think we're on the same page on this yeah what I'm wondering is: are there ways we make it even less significant work on the first step of modifying and files? if that step could disappear altogether, especially in scenario of adding new images, that would be great.
I don't really see how. Other images are doing something similar too. For example, the pause image also has its BASEIMAGEs and VERSION in its Makefile. Not all images are built exactly the same, or have the same base image, or are built for the same arch types. Some of them are from scratch, others are from an alpine image, others are from a different alpine image, others are from a debian image... and then we also have windows base images. So, having those BASEIMAGE files addresses this issue. Plus, it also allows us to control what platforms we build for and what we test for. For example, the s390x support wasn't there from the start, it was afterwards. Another example, we want to add the base images at some point. Anyways, modifying the BASEIMAGE files is not very common, but when it is, there's a good reason behind it, so it can't really be optional. Plus, adding / modifying the BASEIMAGE and VERSION files is not significant work. For example, mirroring the 7 images in this PR [1] probably took me ~10 minutes or so. From what I can tell, you want some fast way to just say: "mirror me this image entirely onto our registry". I don't know if we have something like that, and if we do, it'll probably also mirror images with arch types we don't need, and most likely won't have images for platforms we actually need (for example, the ` images on dockerhub does not have any Windows images, but we need them). The test images building process for the test images was not designed for mirroring images, but it can easily do it and we can use it to do so. In addition to that, we can even extend those manifest lists with Windows support [2]. [1] [2]
oh right, apologies for the confusion. I was referring to having a Dockerfile as a whole, but your explanation is very helpful for me to gain more context about image usage in our infrastructure -- thank you for elaborating I agree the what isn't significant, but the where worries me a bit. I guess in some ways I kind of wish that there is way for this work to simply exist in without touching k/k proper, though now I'm not too sure if that would be overall less complex than this approach. yeah, I was trying to find more solutions towards this direction though I haven't found anything significant which aligns with this topic.
/priority important-soon
In order to complete this, we are going to need these 2 trivial PRs: Retrigger image builder jobs for some images: Small fix for Windows httpd images:
seems like much of the work has been done by Claudiu or I do not have access to finish -- I'll unassign myself to keep this aligned with the state of things /unassign
/assign who has done a lot of the work for this

Discussion:
to pull the images from. Sample file: ```yaml <del> dockerLibraryRegistry: your-awesome-registry </del> e2eRegistry: your-awesome-registry promoterE2eRegistry: your-awesome-registry gcRegistry: your-awesome-registry

// RegistryList holds public and private image registries type RegistryList struct { GcAuthenticatedRegistry string `yaml:"gcAuthenticatedRegistry"` <del> DockerLibraryRegistry   string `yaml:"dockerLibraryRegistry"` DockerGluster           string `yaml:"dockerGluster"` </del> E2eRegistry             string `yaml:"e2eRegistry"` E2eVolumeRegistry       string `yaml:"e2eVolumeRegistry"` PromoterE2eRegistry     string `yaml:"promoterE2eRegistry"`

func initReg() RegistryList { registry := RegistryList{ GcAuthenticatedRegistry: "gcr.io/authenticated-image-pulling", <del> DockerLibraryRegistry:   "docker.io/library", DockerGluster:           "docker.io/gluster", </del> E2eRegistry:             "gcr.io/kubernetes-e2e-test-images", E2eVolumeRegistry:       "gcr.io/kubernetes-e2e-test-images/volume", PromoterE2eRegistry:     "k8s.gcr.io/e2e-test-images",

PrivateRegistry = registry.PrivateRegistry // Preconfigured image configs <del> dockerLibraryRegistry   = registry.DockerLibraryRegistry dockerGluster           = registry.DockerGluster </del> <ins> dockerLibraryRegistry   = "docker.io/library" </ins> e2eRegistry             = registry.E2eRegistry e2eVolumeRegistry       = registry.E2eVolumeRegistry promoterE2eRegistry     = registry.PromoterE2eRegistry

configs[AuthenticatedWindowsNanoServer] = Config{gcAuthenticatedRegistry, "windows-nanoserver", "v1"} configs[APIServer] = Config{promoterE2eRegistry, "sample-apiserver", "1.17.4"} configs[AppArmorLoader] = Config{promoterE2eRegistry, "apparmor-loader", "1.3"} <del> configs[BusyBox] = Config{dockerLibraryRegistry, "busybox", "1.29"} </del> <ins> configs[BusyBox] = Config{promoterE2eRegistry, "busybox", "1.29"} </ins> configs[CheckMetadataConcealment] = Config{promoterE2eRegistry, "metadata-concealment", "1.6"} configs[CudaVectorAdd] = Config{e2eRegistry, "cuda-vector-add", "1.0"} configs[CudaVectorAdd2] = Config{promoterE2eRegistry, "cuda-vector-add", "2.2"} configs[DebianIptables] = Config{buildImageRegistry, "debian-iptables", "buster-v1.5.0"} configs[EchoServer] = Config{promoterE2eRegistry, "echoserver", "2.3"} configs[Etcd] = Config{gcEtcdRegistry, "etcd", "3.4.13-0"} <del> configs[GlusterDynamicProvisioner] = Config{dockerGluster, "glusterdynamic-provisioner", "v1.0"} configs[Httpd] = Config{dockerLibraryRegistry, "httpd", "2.4.38-alpine"} configs[HttpdNew] = Config{dockerLibraryRegistry, "httpd", "2.4.39-alpine"} </del> <ins> configs[GlusterDynamicProvisioner] = Config{promoterE2eRegistry, "glusterdynamic-provisioner", "v1.0"} configs[Httpd] = Config{promoterE2eRegistry, "httpd", "2.4.38-alpine"} configs[HttpdNew] = Config{promoterE2eRegistry, "httpd", "2.4.39-alpine"} </ins> configs[InvalidRegistryImage] = Config{invalidRegistry, "alpine", "3.1"} configs[IpcUtils] = Config{promoterE2eRegistry, "ipc-utils", "1.2"} configs[JessieDnsutils] = Config{promoterE2eRegistry, "jessie-dnsutils", "1.4"} configs[Kitten] = Config{promoterE2eRegistry, "kitten", "1.4"} configs[Nautilus] = Config{promoterE2eRegistry, "nautilus", "1.4"} configs[NFSProvisioner] = Config{sigStorageRegistry, "nfs-provisioner", "v2.2.2"} <del> configs[Nginx] = Config{dockerLibraryRegistry, "nginx", "1.14-alpine"} configs[NginxNew] = Config{dockerLibraryRegistry, "nginx", "1.15-alpine"} </del> <ins> configs[Nginx] = Config{promoterE2eRegistry, "nginx", "1.14-alpine"} configs[NginxNew] = Config{promoterE2eRegistry, "nginx", "1.15-alpine"} </ins> configs[NodePerfNpbEp] = Config{promoterE2eRegistry, "node-perf/npb-ep", "1.1"} configs[NodePerfNpbIs] = Config{promoterE2eRegistry, "node-perf/npb-is", "1.1"} configs[NodePerfTfWideDeep] = Config{promoterE2eRegistry, "node-perf/tf-wide-deep", "1.1"}

configs[NonRoot] = Config{promoterE2eRegistry, "nonroot", "1.1"} // Pause - when these values are updated, also update cmd/kubelet/app/options/container_runtime.go configs[Pause] = Config{gcRegistry, "pause", "3.4.1"} <del> configs[Perl] = Config{dockerLibraryRegistry, "perl", "5.26"} </del> <ins> configs[Perl] = Config{promoterE2eRegistry, "perl", "5.26"} </ins> configs[PrometheusDummyExporter] = Config{gcRegistry, "prometheus-dummy-exporter", "v0.1.0"} configs[PrometheusToSd] = Config{gcRegistry, "prometheus-to-sd", "v0.5.0"} configs[Redis] = Config{promoterE2eRegistry, "redis", "5.0.5-alpine"}