Code change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If this may be security issue, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: On Windows Server with sandbox support the CNI gets called for all the containers within the pod including the sandbox container itself. What you expected to happen: The CNI should be called only for the sandbox container since they share the same network details. How to reproduce it (as minimally and precisely as possible): kubelet Pod at the parameters that get passed when the CNI is called Anything else we need to know?: This applies to Windows Server with sandbox support. The CNI should be called only for the infra container. Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/sig windows /sig network /kind bug /assign /cc
Reopen this because of
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
has been merged now. /close
Closing this issue. <detailsIn response to : has been merged now. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
has this issue been fixed? Is the CNI getting called only for the infra container? From what I see, the CNI still gets called for every single container from within a pod:
See comments It is required for windows containers.
The way I understand it, it's that the DNS issue will be fixed in Windows RS5 and there will not be required additional calls. The workaround at the moment is to call the CNI multiple times until the DNS issue gets fixed in Windows. As far as I know, that is the only reason why the additional calls to the CNI are required.
Is this true? Do we still need to invoke cni for each container in RS5? And even with new cni dns capabilities ()?
the PR that you linked it's unrelated to this issue. The CNI should be called for the sandbox pod which is the infra container. All the other containers created within the same pod are sharing all network details, excepting DNS. This is an issue in 1709 and 1803 where not everything gets replicated properly to the other containers. As a workaround for this, you need to call AttachEndpoint from HCS on the newly created container network interface only to replicate the DNS as well. You can also see more details about this issue here:
Can you re-open the issue in the meanwhile since this issue still exists?
I still think this isn't a issue, but a requirement for windows containers. Even after RS5, we should probably support old windows versions. WDYH
Why would the spec of the CNI should be different on Windows? On Linux you know that the sandbox ID that will get passed to the CNI is always the sandbox container. On Windows the sandbox ID won't be only that, it will also be an ID of a container from within the same pod. Does this sound like normal behavior to you? For now I agree, the workaround is to call the CNI for every single container because of the Windows issue with DNS. But this doesn't mean we have to stick with this for future Windows releases which will not require calling the CNI ADD multiple times. Also, this is a hack for now () which needs to be fixed. Please re-open the issue until we get feedback from others as well, thanks.
OK. Reopened the issue. Let's wait a while for feedbacks from other folks.
Apologize for the delay. Windows doenst have the namespace support native on the platform. we this support as part of RS5 Platform release. We are working on making runtime spec changes and CRI/ContainerD changes to use this model. here is a runtime spec PR that got merged. This is orthogonal to the DNS issue. Once we are done with the CRI/ContainerD changes then if u use CRI/containerD as runtime and if your OS version is RS5 and above then you don't need this second CNI add call. But as Pengfei said, if you are on a downlevel version or using dockershim as Runtime then you will have to make the second CNI call. So in my opinion this code will stay as workaround forever until we remove the support for downlevel OS. Hope this helps. I would recommend closing this issue.
thanks for the confirmation. In my opinion this should stay open until it gets fixed at least for future Windows releases, because the way it is at the moment, the CNI will still be called multiple times regardless of the Windows version. Why would you recommend closing this issue? Shouldn't this be fixed for futures Windows releases? The other calls will be useless when Windows gets proper namespace support. Also, I recommend watching this presentation about the CNI which had place at the KubeCon from May I don't see why we shouldn't have this open until the issue gets fixed for future releases and we stick with the official SPEC of the CNI.
and I had talked before about end of life of downlevel OS like 1803 and 1709. I think we should keep this open or open a new Issue to track updating the code so that the CNI is not called multiple times
This code would exists as far as we support 1709, 1803. The new workflow to use the windows namespace is getting implemented in containerd/cri. We can move kubernetes to use this new flow only for RS5 and above, if we plan to do it here as well. There isn't a good way to switch between 2 workflows of cni callflow wrt the Windows version hence we are not planning to change here till 1709 & 1803 exists, without adding some ugly code.
thanks for the explanation
I totally agree that this has to be fixed for future release and we are actively working on it. I was recommending to close this issue and open a feature to track this request. We should be able to raise a PR in another 2 weeks to support native network namespace support. This code will stay as long as we support down-level OS'es. I am fine either way. I am trying to close any open issues that is not a bug on Windows Networking(Platform).
/kind feature Although from a kubernetes standpoint, this isn't a new feature. It doesn't need a new KEP and won't change any API. It will just be a future Windows network code cleanup.
/kind cleanup
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
- can you clarify a few things so we know what's required to fix this? 's the minimum OS version needed? Windows Server 2019, or a future release? this be possible to fix for dockershim, or just CRI-containerD? Thanks!
What's the minimum OS version needed? Windows Server 2019, or a future release? [Dinesh]: Minimum OS version needed is 2019 Will this be possible to fix for dockershim, or just CRI-containerD? [Dinesh]: yes its possible to fix for Dockershim as well. but we are starting it with ContainerD. ContainerD is already done and available for testing.
excellent. thank you Dinesh!
ping - you have logs coming in email for this. Can you help us determine if it's needed for 1.14 vs 1.15?
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry. ‍
Ping
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry. ‍
If this issue has been triaged, please comment . If you aren't able to handle this issue, consider unassigning yourself and/or adding the label. I am a bot run by vllry. ‍
Status on this?

Related discussion:
text: continue } <del> // Versions and feature support // ============================ // Windows version == Windows Server, Version 1709,, Supports both sandbox and non-sandbox case // Windows version == Windows Server 2016   Support only non-sandbox case // Windows version < Windows Server 2016 is Not Supported // Sandbox support in Windows mandates CNI Plugin. // Presence of CONTAINER_NETWORK flag is considered as non-Sandbox cases here // Todo: Add a kernel version check for more validation if networkMode := os.Getenv("CONTAINER_NETWORK"); networkMode == "" { // On Windows, every container that is created in a Sandbox, needs to invoke CNI plugin again for adding the Network, // with the shared container name as NetNS info, // This is passed down to the platform to replicate some necessary information to the new container // // This place is chosen as a hack for now, since ds.getIP would end up calling CNI's addToNetwork // That is why addToNetwork is required to be idempotent // Instead of relying on this call, an explicit call to addToNetwork should be // done immediately after ContainerCreation, in case of Windows only. TBD Issue # to handle this if r.HostConfig.Isolation == kubeletapis.HypervIsolationValue { // Hyper-V only supports one container per Pod yet and the container will have a different // IP address from sandbox. Return the first non-sandbox container IP as POD IP. // TODO(feiskyer): remove this workaround after Hyper-V supports multiple containers per Pod. if containerIP := ds.getIP(c.ID, r); containerIP != "" { return containerIP } } else { // Do not return any IP, so that we would continue and get the IP of the Sandbox ds.getIP(sandboxID, r) } } else { // ds.getIP will call the CNI plugin to fetch the IP if containerIP := ds.getIP(c.ID, r); containerIP != "" { return containerIP } </del> <ins> if containerIP := ds.getIP(c.ID, r); containerIP != "" { return containerIP </ins> } }