Change:
# NOTE(claudiub): A non-default builder instance is needed in order to build Windows images. all-push: all-container-registry push-manifest <ins> push-manifest: SHELL:=/bin/bash </ins> push-manifest: docker manifest create --amend $(IMAGE):$(TAG) $(shell echo $(ALL_OS_ARCH) | sed -e "s~[^ ]*~$(IMAGE):$(TAG)-&~g") set -x; for arch in $(ALL_ARCH.linux); do docker manifest annotate --os linux --arch $${arch} ${IMAGE}:${TAG} ${IMAGE}:${TAG}-linux-$${arch}; done # For Windows images, we also need to include the "os.version" in the manifest list, so the Windows node can pull the proper image it needs. # we use awk to also trim the quotes around the OS version string. set -x;  <ins> # tagToKernelVersionMap maps the container images tags for different Windows Server releases (ex: ltsc2022 for Windows Server 2022) # to the kernel version for that OS release (ex: 20348 for Windows Server 2022). This is needed to fetch the servicing revision from the # pause base image manifest (which containers an entry for each Windows Server version) so we can add the approrite 'os.version' # field to the pause image manifest. declare -A tagToKernelVersionMap=( ['1809']='17763' ['20H2']='19042' ['ltsc2022']='20348' ); </ins> for arch in $(ALL_ARCH.windows);  do  for osversion in ${ALL_OSVERSIONS.windows}; do  <del> BASEIMAGE=${BASE.windows}:$${osversion};  full_version=`docker manifest inspect ${BASE.windows}:$${osversion} | grep "os.version" | head -n 1 | awk -F" '{print $$4}'` || true;  </del> <ins> full_version=`docker manifest inspect ${BASE.windows} | grep "10.0.$${tagToKernelVersionMap[$$osversion]}" | head -n 1 | awk -F" '{print $$4}'` || true;  </ins> docker manifest annotate --os windows --arch $${arch} --os-version $${full_version} ${IMAGE}:${TAG} ${IMAGE}:${TAG}-windows-$${arch}-$${osversion};  done;  done

Discussion:
The Windows pause images is built using buildkit on Linux hosts and this configuration does not allow for the Windows registry to me modified during the container build process. When Windows containers are built on Windows host machines part of the build process creates 'delta' registry files when get stored in the layer files. These 'delta' registry hive files are applied to the containers active registry when the containers are created. Chaning the registry files in the container image layers without having the 'delta' files present will result in the changes NOT being present in the container image. We have an increasing demand for the ability to modify registry in the pause image on Windows such as Disable some services like DiagTrack that negatively impact performance Change behavior of RPCSS so it doesn't hijack RPC port (135) for all containers in the pod Things are working as expected given the current build process. There are linux utilities like that can modify Windows registry files that I have used to test this out. After running the container produced from these files you can see the registry changes made by persisted in the container image but these changes are not present in the container's active registry. vs One possible solution is to create an intermediate image based on (the current base image for the Windows pause image) that performs all the desired registry operations that is built on a Windows machine and published someplace (MCR maybe?) and the use that as the container base image for the Windows pause image. <detailsN/A </details<detailsN/A </details<detailsN/A </details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc
cc
my first concern was. will a community member be able to build this image themselves (will the sources be available and up-to-date and the image built from it). We could also do a github hosted windows runner to build / publish this new image as well. let's please revisit this once we wrap 1.24.

/milestone v1.25
We created which has the source/docker files used to build . AKS has made a pause image based on image available to some customers out of necessity to unblock some scenarios. We'd like to upstream the changes - and do so with as little destabilization to Kubernetes project as possible.
Seems like the intermediate base image step would be reasonable. I don't think community members can build the existing base windows images either for the same reason of depending on windows?
The windows pause image is currently based on 'nanoserver' which is one of several container base images provided by Microsoft (and only Microsoft can produce container base images).
yep. i'd expect community downstream teams (for example) would like to start with this nanoserver, have a set of dockerfiles to build this new image ... then build their own pause images based on this new image
or in other words if MSFT wants to roll all the changes into nanoserver itself (since we all already know that only MSFT can produce this), then there is no debate after that
We can't roll these changes into nanoserver because they may break other scenarios. Because of that - having an intermediate layer built somewhere was the best solution we came up with without forking the build process for the pause image. Microsoft is willing to host build/host the intermediate layers/images from open-sourced docker files. If we want to build the intermediate layers somewhere else too (like github hosted Windows runners) that would be fine too.
/sig windows
If this intermediate image used only by k8s then it is a good idea to build the image via GH actions in the kubernetes org so that k8s maintainers have better control on this or if this image is going to be generic for anyone to use then MSFT can build one and host it somewhere.
Picking this back up since 1.24 release is behind everyone. - Do you think a separate github repo for the dockerfile and github actions staging/prod container repos would be appropriate here? If so I can get started on creating these. Also - Is it possible to push to the staging repos from a github action job instead of PROW jobs? We'd need to use github actions to build the Windows image on a Windows machine.

I supposed we can store the .tar files as release artifacts if needed and then use a PROW to job push them to the staging repo if needed but being able to push for the action would be simpler.
/cc as FYI
I see two possible paths forward Continue to host the source for the intermediate layer at , publish the image to , and then update the Windows pause build docker files in k/k to consume this image instead of the or Create a new kubernetes-sigs repo to host the source and build steps for the image, publish the image to , and then update the Windows pause build docker files in k/k to consume this image. I'm fine with either approach but am not sure who the right folks to discuss this with are.
This looks more promising because this image you are creating is not a specific change for the pause container, IMO if we can change that image name to something generic and use the same wherever required.
I think those both sound reasonable personally. I think it's technically feasible to push from GitHub actions to staging GCR with workload identity but nobody has done this yet in Kubernetes.