Change:
introduced somethng that really should be e2e tested.
Would be nice to get this in in the 2 weeks we have after 1.2 branch cut, if no more flakes crop up (but I'm not too optimistic). Bumping to next-release.
/sig network I'm assuming sig-network owns kube-proxy
I thought I noted a change related to this recently, but I may be mistaken.
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten /remove-lifecycle stale
/lifecycle frozen /remove-lifecycle stale
/remove-lifecycle rotten
Going to try to take this on.
thanks!
I've had some trouble constructing a test for this. should I determine the timeout? There doesn't appear to be an existing mechanism for waiting for service creation. seem to time out rather than be refused - I'm confused by this, as it seems to be the opposite of the intended behavior.
can look at the endpoints + wait for longer than kube-proxy sync time. refused means packets got to their destination but there was nothing listening there. If traffic is blocked, then you will just get a timeout.
I created a service with no matching pods to test manually outside of the e2e suite. # curl test-api curl: (7) Failed to connect to athena-api port 80: Connection timed out # curl --connect-timeout 8 test-api curl: (28) Connection timed out after 8000 milliseconds Looking back at original example in I see that I'm getting the "before" behaviour. An iptables REJECT should send a denial, not a timeout. Am I looking at the wrong thing here? To me this looks like a regression, but I don't think that's likely.
I have a cluster from HEAD as of 2 weeks ago: So indeed it seems to not work. If I hit that same service IP from the VM itself I immediately get curl: (7) Failed to connect to 10.0.199.54 port 80: Connection refused. If I insert an iptables jump to KUBE-SERVICES at the front of FORWARD, it correctly rejects. This seems in accordance with the iptables flowchart. I'll do some spelunking to see if there's a reason NOT to reject in this case. Vallery Lancey <:
Can you check IPVS mode?
This is why we need tests. Sigh.
Proposed fix is - still needs your e2e :)
Thanks Tim, if I have time I’ll get a PR up tonight.
Sure will check IPVS mode.
Base on my test, IPVS has no such issue.

Discussion:
import ( "bytes" <ins> "errors" </ins> "fmt" "math/rand" "net"

By("verifying service-disabled is still not up") framework.ExpectNoError(framework.VerifyServeHostnameServiceDown(cs, host, svcDisabledIP, servicePort)) }) <ins> It("should be rejected when no endpoints exist", func() { namespace := f.Namespace.Name serviceName := "no-pods" jig := framework.NewServiceTestJig(cs, serviceName) nodes := jig.GetNodes(framework.MaxNodesForEndpointsTests) labels := map[string]string{ "nopods": "nopods", } port := 80 ports := []v1.ServicePort{{ Port:       int32(port), TargetPort: intstr.FromInt(80), }} By("creating a service with no endpoints") _, err := jig.CreateServiceWithServicePort(labels, namespace, ports) if err != nil { framework.Failf("Failed to create service: %v", err) } nodeName := nodes.Items[0].Name podName := "execpod-noendpoints" By(fmt.Sprintf("creating %v on node %v", podName, nodeName)) execPodName := framework.CreateExecPodOrFail(f.ClientSet, namespace, podName, func(pod *v1.Pod) { pod.Spec.NodeName = nodeName }) execPod, err := f.ClientSet.CoreV1().Pods(namespace).Get(execPodName, metav1.GetOptions{}) framework.ExpectNoError(err) serviceAddress := net.JoinHostPort(serviceName, strconv.Itoa(port)) framework.Logf("waiting up to %v wget %v", framework.KubeProxyEndpointLagTimeout, serviceAddress) cmd := fmt.Sprintf(`wget -T 3 -qO- %v`, serviceAddress) By(fmt.Sprintf("hitting service %v from pod %v on node %v", serviceAddress, podName, nodeName)) expectedErr := "connection refused" if pollErr := wait.PollImmediate(framework.Poll, framework.KubeProxyEndpointLagTimeout, func() (bool, error) { _, err := framework.RunHostCmd(execPod.Namespace, execPod.Name, cmd) if err != nil { if strings.Contains(strings.ToLower(err.Error()), expectedErr) { framework.Logf("error contained '%s', as expected: %s", expectedErr, err.Error()) return true, nil } else { framework.Logf("error didn't contain '%s', keep trying: %s", expectedErr, err.Error()) return false, nil } } else { return true, errors.New("expected wget call to fail") } }); pollErr != nil { Expect(pollErr).NotTo(HaveOccurred()) } }) </ins> }) // TODO: Get rid of [DisabledForLargeClusters] tag when issue #56138 is fixed.