Change:
return setupCRDAndVerifySchema(f, schema, expect, groupSuffix, versions...) } <del> func setupCRDWithShortName(f *framework.Framework, schema []byte, groupSuffix, shortName string, versions ...string) (*crd.TestCrd, error) { expect := schema if schema == nil { // to be backwards compatible, we expect CRD controller to treat // CRD with nil schema specially and publish an empty schema expect = []byte(`type: object`) } setShortName := func(crd *apiextensionsv1.CustomResourceDefinition) { crd.Spec.Names.ShortNames = []string{shortName} } return setupCRDAndVerifySchemaWithOptions(f, schema, expect, groupSuffix, versions, setShortName) } </del> func setupCRDAndVerifySchema(f *framework.Framework, schema, expect []byte, groupSuffix string, versions ...string) (*crd.TestCrd, error) { return setupCRDAndVerifySchemaWithOptions(f, schema, expect, groupSuffix, versions) }

Discussion:
Which jobs are failing: gce-master-scale-correctness (ci-kubernetes-e2e-gce-scale-correctness) Which test(s) are failing: [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD with validation schema [Conformance] Since when has it been failing: 04-26 05:02 PDT Testgrid link: Reason for failure: Apr 26 12:28:01.219: FAIL: failed to explain error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://35.185.126.243 -- explain Command stdout: stderr: the server doesn't have a resource type "ksvc" error: exit status 1 Full Stack Trace () +0x1545 (0xc002ca6100) +0x324 (0xc002ca6100) +0x2b testing.tRunner(0xc002ca6100, 0x4b8d060) +0xc9 created by testing.(T).Run +0x350 [AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] ï¿½[1mSTEPï¿½[0m: Collecting events from namespace "crd-publish-openapi-4377". ï¿½[1mSTEPï¿½[0m: Found 0 events. Apr 26 12:28:01.314: INFO: POD NODE PHASE GRACE CONDITIONS Apr 26 12:28:01.314: INFO: Apr 26 12:28:02.899: INFO: skipping dumping cluster info - cluster too large Apr 26 12:28:02.899: INFO: Waiting up to 3m0s for all (but 50) nodes to be ready ï¿½[1mSTEPï¿½[0m: Destroying namespace "crd-publish-openapi-4377" for this suite. Anything else we need to know: /cc /cc /milestone v1.19 /priority critical-urgent
You must be a member of the GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your and have them propose you as an additional delegate for this responsibility.

<detailsIn response to : Which jobs are failing: gce-master-scale-correctness (ci-kubernetes-e2e-gce-scale-correctness) [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] works for CRD with validation schema [Conformance] 04-26 05:02 PDT Apr 26 12:28:01.219: FAIL: failed to explain error running /workspace/kubernetes/platforms/linux/amd64/kubectl --server=https://35.185.126.243 -- explain Command stdout: the server doesn't have a resource type "ksvc" exit status 1 () (0xc002ca6100) (0xc002ca6100) testing.tRunner(0xc002ca6100, 0x4b8d060) created by testing.(T).Run [AfterEach] [sig-api-machinery] CustomResourcePublishOpenAPI [Privileged:ClusterAdmin] ï¿½[1mSTEPï¿½[0m: Collecting events from namespace "crd-publish-openapi-4377". ï¿½[1mSTEPï¿½[0m: Found 0 events. Apr 26 12:28:01.314: INFO: POD NODE PHASE GRACE CONDITIONS Apr 26 12:28:01.314: INFO: Apr 26 12:28:02.899: INFO: skipping dumping cluster info - cluster too large Apr 26 12:28:02.899: INFO: Waiting up to 3m0s for all (but 50) nodes to be ready ï¿½[1mSTEPï¿½[0m: Destroying namespace "crd-publish-openapi-4377" for this suite. /cc /milestone v1.19 </details
/milestone v1.19 Thanks for opening this! I'll get you on milestone maintainers soon :) Also, was taking a look at this as well and noticed this test was recently :
As noted in , this modified an existing conformance test and made it flaky. Please move the new test out of the conformance test and mark it as flaky until this issue has been resolved.

Cc for flake introduced in
It is a bit weird that I cannot reproduce this problem on my local kind cluster by running even after 20 times I suspect this is a cache issue since I once created a crd without short name and forgot to delete it in the test.
/retitle [Flaky Test]: gce-master-scale-correctness (ci-kubernetes-e2e-gce-scale-correctness) /kind flake /remove-kind failing-test
/assign
I can reproduce this problem by the following steps now. It looks like that this problem has something to do with discovery cache. <pre$ cat <summary<detailsapiVersion: kind: CustomResourceDefinition metadata: name: spec: conversion: strategy: None group: crd-publish-openapi-test- names: kind: Service listKind: ServiceList plural: services singular: service shortNames: ["ksvc"] scope: Namespaced versions: name: v1alpha1 schema: openAPIV3Schema: properties: spec: description: Specification of CustomService properties: dummy: description: Dummy property. type: string type: object type: object served: true storage: true </details</summary$ kubectl get crd No resources found in default namespace. $ rm -r $ kubectl explain the server doesn't have a resource type "ksvc" $ kubectl create -f created $ kubectl explain the server doesn't have a resource type "ksvc" $ kubectl explain <summary<detailsKIND: Service VERSION: crd-publish-openapi-test- RESOURCE: spec <ObjectDESCRIPTION: Specification of CustomService FIELDS: dummy <stringDummy property. </details</summary</pre
looks like
/assign