Change:
There is now a test tag from I remember a tag being approved in SIG Architecture and SIG Testing, but I don't remember a tag. If this was approved I think we need to broadcast it more widely as many existing test setups including current conformance testing appear to now need to skip / exclude this tag. cc /area conformance /sig architecture /sig testing
/assign
bug may not be the appropriate type, but seemed like the closest issue template to start from initially.
TL;DR nope, that was not something that I'm aware of.
cc
The email thread is in
the in that thread says: That's not the case here.
we should file a PR marked 1.14 and to yank that tag, just so we don't forget. Of course if the decision changes in the email thread, we can revisit the PR. ok?
makes sense to me
Catching up. Quick comment before I read the threads: Documented kinds of tests are currently here: WindowsOnly had been hypothesized, but not yet to that list.
Also, are there already tests tagged as SIG Windows?
On conformance tests: Windows isn't part of conformance yet, so no conformance test should have this tag. And no conformance test should contain any Skip directives. They should die in that case:
I looked at and don't understand why the Windows image test wasn't written as a separate test. To avoid either refactoring or copy/pasting? It looks like a pretty obscure approach -- auto-skipping individual test cases that contain a particular tag?
Pretty much. But it is no longer the case, since all those "should pull image from X" test cases have been refactored. (look at to see how it looks now)
Yes, that's better, thanks. To answer my own question regarding , here's an example: which also invokes inside the test. So we're agreed that we don't need yet?
now removes all instances of the [WindowsOnly] tag.

Discussion:
// Images used for ConformanceContainer are not added into NodeImageWhiteList, because this test is // testing image pulling, these images don't need to be prepulled. The ImagePullPolicy // is v1.PullAlways, so it won't be blocked by framework image white list check. <del> imagePullTest := func(image string, hasSecret bool, expectedPhase v1.PodPhase, expectedPullStatus bool) { </del> <ins> imagePullTest := func(image string, hasSecret bool, expectedPhase v1.PodPhase, expectedPullStatus bool, windowsImage bool) { command := []string{"/bin/sh", "-c", "while true; do sleep 1; done"} if windowsImage { // -t: Ping the specified host until stopped. command = []string{"ping", "-t", "localhost"} } </ins> container := ConformanceContainer{ PodClient: f.PodClient(), Container: v1.Container{ Name:            "image-pull-test", Image:           image, <del> Command:         []string{"/bin/sh", "-c", "while true; do sleep 1; done"}, </del> <ins> Command:         command, </ins> ImagePullPolicy: v1.PullAlways, }, RestartPolicy: v1.RestartPolicyNever,

It("should not be able to pull image from invalid registry [NodeConformance]", func() { image := "invalid.com/invalid/alpine:3.1" <del> imagePullTest(image, false, v1.PodPending, true) </del> <ins> imagePullTest(image, false, v1.PodPending, true, false) </ins> }) It("should not be able to pull non-existing image from gcr.io [NodeConformance]", func() { image := "k8s.gcr.io/invalid-image:invalid-tag" <del> imagePullTest(image, false, v1.PodPending, true) </del> <ins> imagePullTest(image, false, v1.PodPending, true, false) </ins> }) <del> // TODO(claudiub): Add a Windows equivalent test. </del> It("should be able to pull image from gcr.io [LinuxOnly] [NodeConformance]", func() { image := "gcr.io/google-containers/debian-base:0.4.1" <del> imagePullTest(image, false, v1.PodRunning, false) </del> <ins> imagePullTest(image, false, v1.PodRunning, false, false) }) It("should be able to pull image from gcr.io [NodeConformance]", func() { framework.SkipUnlessNodeOSDistroIs("windows") image := "gcr.io/kubernetes-e2e-test-images/windows-nanoserver:v1" imagePullTest(image, false, v1.PodRunning, false, true) </ins> }) It("should be able to pull image from docker hub [LinuxOnly] [NodeConformance]", func() { image := "alpine:3.7" <del> imagePullTest(image, false, v1.PodRunning, false) </del> <ins> imagePullTest(image, false, v1.PodRunning, false, false) </ins> }) <del> It("should be able to pull image from docker hub [WindowsOnly] [NodeConformance]", func() { </del> <ins> It("should be able to pull image from docker hub [NodeConformance]", func() { </ins> framework.SkipUnlessNodeOSDistroIs("windows") <ins> // TODO(claudiub): Switch to nanoserver image manifest list. </ins> image := "e2eteam/busybox:1.29" <del> imagePullTest(image, false, v1.PodRunning, false) </del> <ins> imagePullTest(image, false, v1.PodRunning, false, true) </ins> }) It("should not be able to pull from private registry without secret [NodeConformance]", func() { image := "gcr.io/authenticated-image-pulling/alpine:3.7" <del> imagePullTest(image, false, v1.PodPending, true) </del> <ins> imagePullTest(image, false, v1.PodPending, true, false) </ins> }) It("should be able to pull from private registry with secret [LinuxOnly] [NodeConformance]", func() { image := "gcr.io/authenticated-image-pulling/alpine:3.7" <del> imagePullTest(image, true, v1.PodRunning, false) </del> <ins> imagePullTest(image, true, v1.PodRunning, false, false) }) It("should be able to pull from private registry with secret [NodeConformance]", func() { framework.SkipUnlessNodeOSDistroIs("windows") image := "gcr.io/authenticated-image-pulling/windows-nanoserver:v1" imagePullTest(image, true, v1.PodRunning, false, true) </ins> }) }) })