Change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If this may be security issue, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /sig testing What happened: Tests checking DNS responses can have false positives, such as "should provide DNS for services". The tests rely on dig-ing certain domain names [1]: test -n "$(dig +notcp +noall +answer +search kubernetes.default A)" && echo OK /results/ The problems could be that, if a host / container doesn't have any entries in their : ubuntu dig +notcp +noall +answer +search kubernetes.default A ;; connection timed out; no servers could be reached , causing the test to pass and to write "OK" into the results file. [1] What you expected to happen: The tests shouldn't be able to pass if dig failed. How to reproduce it (as minimally and precisely as possible): Comment all nameservers or configure a bogus DNS server in, and run: test -n "$(dig +notcp +noall +answer +search kubernetes.default A)" && echo OK Anything else we need to know?: Environment: Kubernetes version (use ): 1.9.6 Cloud provider or hardware configuration: N/A OS (e.g. from /etc/os-release): "Ubuntu 16.04.3 LTS" Kernel (e.g. ): Linux ubuntu 4.4.0-87-generic
/assign
: But one of the purpose of the test is to ensure that the DNS of the cluster is replying. In other words, if it does not reply then we EXPECT this test to fail. If you are not using DNS then isn't it better to SKIP the test ?
I believe means to say "false negatives" ... meaning that in some situations, the test will errantly pass, when it should fail.
oops . Looks like I misread the description, and agree that the dig should work.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale

Discussion:
} fileName := fmt.Sprintf("%s_udp@%s", fileNamePrefix, name) fileNames = append(fileNames, fileName) <del> probeCmd += fmt.Sprintf(`test -n "$$(dig +notcp +noall +answer +search %s %s)" && echo OK > /results/%s;`, name, lookup, fileName) </del> <ins> probeCmd += fmt.Sprintf(`check="$$(dig +notcp +noall +answer +search %s %s)" && test -n "$$check" && echo OK > /results/%s;`, name, lookup, fileName) </ins> fileName = fmt.Sprintf("%s_tcp@%s", fileNamePrefix, name) fileNames = append(fileNames, fileName) <del> probeCmd += fmt.Sprintf(`test -n "$$(dig +tcp +noall +answer +search %s %s)" && echo OK > /results/%s;`, name, lookup, fileName) </del> <ins> probeCmd += fmt.Sprintf(`check="$$(dig +tcp +noall +answer +search %s %s)" && test -n "$$check" && echo OK > /results/%s;`, name, lookup, fileName) </ins> } for _, name := range hostEntries {

podARecByUDPFileName := fmt.Sprintf("%s_udp@PodARecord", fileNamePrefix) podARecByTCPFileName := fmt.Sprintf("%s_tcp@PodARecord", fileNamePrefix) probeCmd += fmt.Sprintf(`podARec=$$(hostname -i| awk -F. '{print $$1"-"$$2"-"$$3"-"$$4".%s.pod.cluster.local"}');`, namespace) <del> probeCmd += fmt.Sprintf(`test -n "$$(dig +notcp +noall +answer +search $${podARec} A)" && echo OK > /results/%s;`, podARecByUDPFileName) probeCmd += fmt.Sprintf(`test -n "$$(dig +tcp +noall +answer +search $${podARec} A)" && echo OK > /results/%s;`, podARecByTCPFileName) </del> <ins> probeCmd += fmt.Sprintf(`check="$$(dig +notcp +noall +answer +search $${podARec} A)" && test -n "$$check" && echo OK > /results/%s;`, podARecByUDPFileName) probeCmd += fmt.Sprintf(`check="$$(dig +tcp +noall +answer +search $${podARec} A)" && test -n "$$check" && echo OK > /results/%s;`, podARecByTCPFileName) </ins> fileNames = append(fileNames, podARecByUDPFileName) fileNames = append(fileNames, podARecByTCPFileName)

ptrLookup := fmt.Sprintf("%s.in-addr.arpa.", strings.Join(reverseArray(strings.Split(ptrLookupIP, ".")), ".")) ptrRecByUDPFileName := fmt.Sprintf("%s_udp@PTR", ptrLookupIP) ptrRecByTCPFileName := fmt.Sprintf("%s_tcp@PTR", ptrLookupIP) <del> probeCmd += fmt.Sprintf(`test -n "$$(dig +notcp +noall +answer +search %s PTR)" && echo OK > /results/%s;`, ptrLookup, ptrRecByUDPFileName) probeCmd += fmt.Sprintf(`test -n "$$(dig +tcp +noall +answer +search %s PTR)" && echo OK > /results/%s;`, ptrLookup, ptrRecByTCPFileName) </del> <ins> probeCmd += fmt.Sprintf(`check="$$(dig +notcp +noall +answer +search %s PTR)" && test -n "$$check" && echo OK > /results/%s;`, ptrLookup, ptrRecByUDPFileName) probeCmd += fmt.Sprintf(`check="$$(dig +tcp +noall +answer +search %s PTR)" && test -n "$$check" && echo OK > /results/%s;`, ptrLookup, ptrRecByTCPFileName) </ins> fileNames = append(fileNames, ptrRecByUDPFileName) fileNames = append(fileNames, ptrRecByTCPFileName) }