Code change:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: CRI-O e2e Which test(s) are failing: e2e suite [sig-network] Services should be able to change the type from ExternalName to NodePort e2e suite [sig-network] Services should be able to create a functioning NodePort service 19s See link: https://openshift-gce- Since when has it been failing: Anything else we need to know:
/sig network
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry. ‍
/milestone v1.16 /assign FYI Looks like two other release-blocking tests too: - The changes link testgrid gives () suggests two other PR's, but I agree that seems the likely culprit
Based on the , Service is reachable on and but failing to reach on node's IP over nodePort i.e. . From the log, it is not clear why its not reachable on node's internal IP. Could you provide from the CRI-O based node when trying to reach nodePort service. Secondly, need to make sure, node is listening on allocated nodePort. As this issue seems much on system-level, I need to setup k8s cluster/node with CRI-O runtime and troubleshoot to find the root cause.
[sig-network] ESIPP [Slow] [DisabledForLargeClusters] should work from pods [sig-network] Services should only allow access from service loadbalancer source ranges [Slow] seems flaky to me. Most of time execPod starts running and might be terminating immediately after some time which causes this issue. [1.] e2e is failing as might have terminated and so no PodIP leading to invalid . [2.] empty . The same issue had been observed in for E2E and to fix the issue, we introduced execPod deployment to make sure execPod would remain in desired running state. It's better to address this change in another PR and emphasize using pausePod deployment.
Thanks, for helping me setting up CRI-O based cluster. Kubernetes e2e suite [sig-network] Services should be able to change the type from ExternalName to NodePort e2e suite [sig-network] Services should be able to create a functioning NodePort service 19s From log, it is clear that, these services are failing to reach over allocated nodePort. But on my CRIO based testing cluster, they are reachable. or Could you please check if they failing at your end too or it's testgrid Job-specific issue?
Hey, I did some local tests and was able to reproduce the failure. I started the cluster via (check that the variable is correct and does not point to ): Executed the tests by (again, watch out for the variable to be correct): Error output: Since I'm now able to reproduce the issue I can go further in debugging and find out how to fix this. Is anyone furhter, or If not, then I will target for fixing the issue this week. :)
Thanks for reproducing this. FYI we've seem similar failure on , but I haven't reproduced the problem locally yet. We are looking to at least turn on verbose on the test command for debugging:
The issue is that we try to automatically discover the node IP by obtaining its status here: In our setup the node has IP which for sure does not work if we try to access it within the pod. I'm thinking about a proper solution and will propose it as a PR.
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry. ‍
/assign

Related discussion:
text: testEndpointReachability(clusterIP, sp.Port, sp.Protocol, execPod) } } func testReachabilityOverNodePorts(nodes *v1.NodeList, sp v1.ServicePort, pod *v1.Pod) { internalAddrs := e2enode.CollectAddresses(nodes, v1.NodeInternalIP) externalAddrs := e2enode.CollectAddresses(nodes, v1.NodeExternalIP) for _, internalAddr := range internalAddrs { <ins> // If the node's internal address points to localhost, then we are not // able to test the service reachability via that address if isInvalidOrLocalhostAddress(internalAddr) { e2elog.Logf("skipping testEndpointReachability() for internal adddress %s", internalAddr) continue } </ins> testEndpointReachability(internalAddr, sp.NodePort, sp.Protocol, pod) } for _, externalAddr := range externalAddrs {