Change:
We need a test case that will verify that NodePort works on all nodes in the cluster for a Windows application. Here's an example manual test: Run with this yaml Once the pod is running, get the nodeport and make sure it's accessible on each node IP for that port. /good-first-issue /kind test /sig windows
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : We need a test case that will verify that NodePort works on all nodes in the cluster for a Windows application. /good-first-issue /kind test /sig windows </details
/assign adelina-t
Something is wierd here: What I did: Fresh environment, 2 linux nodes agents, 2 windows agents, deployed with aks-engine v.29.1. Using k8s 13.2. I used the exact spec you indicated: As you can see, it is assigned to the node: 2923k8s011. This is the first ever pod ( other than kube-system, that don't land on windows ) created on the windows nodes. CURL on the first windows node: Curl on the other cluster nodes works: kubeproxy logs on the first node only contain this: After I create a bunch of other pods, to make sure one gets scheduled on the windows node, the initial pod will be accessible via nod port from the first node as well. What I figure is that kubeproxy can't start because there is no HNS-Network called azure on the node until the first pod lands on the node ( c:kkubeletstart.ps1 deletes it ).
Cc
/milestone v1.14
per - Azure-CNI only creates the network when a pod is scheduled. Kube-proxy waits for the network to be created. Therefore if no pod has been scheduled, kube-proxy won't be active on that node. This won't affect Flannel or OVN deployments because they have a daemon already running that creates the network.
is going to take a look cc
NodePort service doesn't work on my mixed-OS clusters (1 k8s linux master + 1 k8s linux worker + 1 windows worker) as well. I depends on as network solution for windows node, other linux node use calico for networking. all works well except accept access NodePort service on windows node, it seems the Port is not get stared on windows node, anyone who want to get more info for this issue, please let me know, thanks
this message show in the kube-proxy logs periodically.
please send us the debug logs.
thanks, below is my log, please take a look,
i looked at the logs i dont see any LB Policy programmed in HNS for the IP aspnetappsvc NodePort 10.0.15.35 <none80:/TCP 8m14s. Could you please confirm that this is the nodeport we are looking to debug? In the log files you collected please look @ and see if the policy you were trying to program is there or not. if its there and it doesnt work then its an HNS issue. if the policy is not there in that file then its a KubeProxy issue where the policy itself is not programmed in HNS. Then look at Kubeproxy to see why its failing to plumb the policy.
thanks for your suggestion, i will retry it and append the result here.
Below is my retest full steps and output: it seems the ip of aspnetapp and aspnetappsvc not in , but I don't see any error in kube-proxy logs except Any comments ? I need more debug on kube-proxy according to your suggestion
I just tried it with and it works for me. I get 200. azureuser curl -s -o /dev/null -w "%{httpcode}" http://10.255.255.5: 200azureuser -s -o /dev/null -w "%{httpcode}" http://10.255.255.5: azureuser curl -s -o /dev/null -w "%{httpcode}" http://10.240.0.4: 200azureuser kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME 1116k8s000 Ready agent 5d15h v1.13.3 10.240.0.35 <noneWindows Server Datacenter 10.0..316 docker://18.9.0 1116k8s001 Ready agent 5d15h v1.13.3 10.240.0.4 <noneWindows Server Datacenter 10.0..316 docker://18.9.0 k8s-master--0 Ready master 5d16h v1.13.3 10.255.255.5 <noneUbuntu 16.04.5 LTS 4.15.0-1037-azure docker://3.0.4 azureuser curl -s -o /dev/null -w "%{httpcode}" http://10.240.0.35: 200azureuser get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE iis NodePort 10.0.179.188 <none80:/TCP 5d kubernetes ClusterIP 10.0.0.1 <none443/TCP 5d16h azureuser exit
/assign immuzz
GitHub didn't allow me to assign the following users: immuzz. Note that only and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time. For more information please see <detailsIn response to : /assign immuzz Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
CC I'm able to create a test for it which is passing successfully. I will be doing a PR shortly
it looks like this needs a rebase. There were a lot of merges yesterday and unfortunately this one was the victim.
/milestone clear
Yeah .. I just merged. Let's hope this time works.

Discussion:
"hybrid_network.go", "memory_limits.go", "networking.go", <ins> "service.go", </ins> "volumes.go", ], importpath = "k8s.io/kubernetes/test/e2e/windows",

"//staging/src/k8s.io/apimachinery/pkg/util/sets:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/util/uuid:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/watch:go_default_library", <ins> "//staging/src/k8s.io/client-go/kubernetes:go_default_library", </ins> "//staging/src/k8s.io/client-go/kubernetes/scheme:go_default_library", "//staging/src/k8s.io/client-go/tools/cache:go_default_library", "//staging/src/k8s.io/kubelet/config/v1beta1:go_default_library",

<ins> /* Copyright 2019 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ package windows import ( "fmt" v1 "k8s.io/api/core/v1" clientset "k8s.io/client-go/kubernetes" "k8s.io/kubernetes/test/e2e/framework" "github.com/onsi/ginkgo" ) var _ = SIGDescribe("Services", func() { f := framework.NewDefaultFramework("services") var cs clientset.Interface ginkgo.BeforeEach(func() { //Only for Windows containers framework.SkipUnlessNodeOSDistroIs("windows") cs = f.ClientSet }) ginkgo.It("should be able to create a functioning NodePort service for Windows", func() { serviceName := "nodeport-test" ns := f.Namespace.Name jig := framework.NewServiceTestJig(cs, serviceName) nodeIP := framework.PickNodeIP(jig.Client) ginkgo.By("creating service " + serviceName + " with type=NodePort in namespace " + ns) service := jig.CreateTCPServiceOrFail(ns, func(svc *v1.Service) { svc.Spec.Type = v1.ServiceTypeNodePort }) jig.SanityCheckService(service, v1.ServiceTypeNodePort) nodePort := int(service.Spec.Ports[0].NodePort) ginkgo.By("creating Pod to be part of service " + serviceName) jig.RunOrFail(ns, nil) //using hybrid_network methods ginkgo.By("creating Windows testing Pod") windowsPod := createTestPod(f, windowsBusyBoximage, windowsOS) ginkgo.By(fmt.Sprintf("checking connectivity Pod to curl http://%s:%d", nodeIP, nodePort)) assertConsistentConnectivity(f, windowsPod.ObjectMeta.Name, windowsOS, windowsCheck(fmt.Sprintf("http://%s:%d", nodeIP, nodePort))) }) }) </ins>