Chapter 2. Windows Container Support for Red Hat OpenShift release notes 2.1. About Windows Container Support for Red Hat OpenShift Red Hat OpenShift support for Windows Containers enables running Windows compute nodes in an OpenShift Container Platform cluster. Running Windows workloads is possible by using the Red Hat Windows Machine Config Operator (WMCO) to install and manage Windows nodes. With Windows nodes available, you can run Windows container workloads in OpenShift Container Platform. The release notes for Red Hat OpenShift support for Windows Containers tracks the development of the WMCO, which provides all Windows container workload capabilities in OpenShift Container Platform. 2.2. Getting support Red Hat OpenShift support for Windows Containers is provided and available as an optional, installable component. Red Hat OpenShift support for Windows Containers is not part of the OpenShift Container Platform subscription. It requires an additional Red Hat subscription and is supported according to the Scope of coverage and Service level agreements . You must have this separate subscription to receive support for Red Hat OpenShift support for Windows Containers. Without this additional Red Hat subscription, deploying Windows container workloads in production clusters is not supported. You can request support through the Red Hat Customer Portal . For more information, see the Red Hat OpenShift Container Platform Life Cycle Policy document for Red Hat OpenShift support for Windows Containers . If you do not have this additional Red Hat subscription, you can use the Community Windows Machine Config Operator, a distribution that lacks official support. 2.3. Release notes for Red Hat Windows Machine Config Operator 4.0.2 Issued: 2023-3-28 The WMCO 4.0.2 is now available with bug fixes. The components of the WMCO were released in RHBA-2023:1488 . 2.3.1. Bug fixes Previously, containers on Windows nodes were assigned a hard-coded DNS server IP address (such as 172.30.0.10) based on the default subnet. This caused DNS resolution to fail in certain situations. This fix removes the hard-coded DNS address, and Windows containers are now assigned a valid DNS server IP address that is based on the cluster DNS configuration. As a result, DNS resolution works as expected for Windows containers. ( BZ#2020350 ) 2.4. Release notes for Red Hat Windows Machine Config Operator 4.0.1 Issued: 2021-12-13 The WMCO 4.0.1 is now available with bug fixes. The components of the WMCO were released in RHBA-2021:4757 . 2.4.1. Bug fixes Previously, the windows-exporter metrics endpoint object contained a reference to a deleted machine. This incorrect reference caused the WMCO to ignore deleted events for machines with invalid IP addresses. This bug fix removes the validation of the machine object from the event filtering, allowing the windows-exporter metrics endpoint object to correctly update when the machine is still in the Deleting phase. ( BZ#2008992 ) Previously, deleting the node associated with a Windows Machine object returned a reconciliation error upon restart of the Operator. This bug fix opts not to react or reconcile when the node referenced by a Windows machine in the Running state is not found within the cluster, preventing any error loop and standardizing functionality with Linux machine objects. ( BZ#2009474 ) Previously, certain commands being run by the WMCO in Windows VMs were not parsed correctly by PowerShell. This caused Windows VMs with PowerShell as its default SSH shell to be unable to join to a cluster as a node. The WMCO now identifies the default SSH shell of a Windows VM and runs the associated commands accordingly. This new capability allows Windows VMs with PowerShell as the default SSH shell to be configured as nodes in a cluster. ( BZ#2014707 ) Previously, encrypted usernames were being generated with extra tags, which caused them not to display correctly. This bug fix removes the extra tags, allowing the encrypted username to display correctly. ( BZ#2016712 ) Previously, the WMCO did not properly associate BYOH Windows VMs with their Node object when the VM was specified with a DNS object. This caused the WMCO to attempt to configure VMs that were already fully configured. The WMCO now resolves VMs specified by a DNS address when looking for an associated node. ( BZ#2020648 ) 2.5. Release notes for Red Hat Windows Machine Config Operator 4.0.0 This release of the WMCO provides bug fixes for running Windows compute nodes in an OpenShift Container Platform cluster. The components of the WMCO 4.0.0 were released in RHBA-2021:3702 . 2.5.1. Bug fixes Previously, the WMCO used the raw user-provided instance address when creating Bring-Your-Own-Host (BYOH) Windows nodes. This caused BYOH Windows instances to not join an OpenShift Container Platform cluster. This bug fix ensures user-provided DNS names resolve to valid IPv4 addresses, and that the resolved value is used when creating BYOH Windows instances. Now BYOH instances with differing hostnames and DNS addresses can be configured as Windows Nodes. ( BZ#1995684 ) Previously, the WMCO performed direct comparisons using unresolved instance addresses when identifying instance-to-node associations. This caused BYOH Windows instances configured to join an OpenShift Container Platform cluster to be removed. This bug fix validates DNS addresses by performing DNS lookups of entries that are added to the windows-instances config map. Now the WMCO can properly identify configured instance-to-node relationships, preventing any premature removals of BYOH nodes. ( BZ#2005126 ) 2.5.2. Known issues The file system graphs available in the web console do not display for Windows nodes. This issue is caused by changes in the file system queries, which will be fixed in a future release of WMCO. ( BZ#1930347 ) For clusters installed on VMware vSphere, the WMCO ignored the Deleting phase notification event, leaving incorrect node information in the windows-exporter metrics endpoint. This resulted in an invalid mapping for the Prometheus metrics endpoint. This bug has been fixed; the WMCO now recognizes the Deleting phase notification event and maps the Prometheus metrics endpoint appropriately. ( BZ#1995341 ) When the RunAsUser permission is set in the security context of a Linux-based pod, the projected files have the correct permissions set, including container user ownership. However, when the Windows equivalent RunAsUsername permission is set in a Windows pod, the kubelet is prevented from setting correct ownership on the files in the projected volume. This problem can get exacerbated when used in conjunction with a hostPath volume where best practices are not followed. For example, giving a pod access to the C:\var\lib\kubelet\pods\ folder results in that pod being able to access service account tokens from other pods. By default, the projected files will have the following ownership, as shown in this example Windows projected volume file: Path : Microsoft.PowerShell.Core\FileSystem::C:\var\run\secrets\kubernetes.io\serviceaccount\..2021_08_31_22_22_18.318230061\ca.crt Owner : BUILTIN\Administrators Group : NT AUTHORITY\SYSTEM Access : NT AUTHORITY\SYSTEM Allow FullControl BUILTIN\Administrators Allow FullControl BUILTIN\Users Allow ReadAndExecute, Synchronize Audit : Sddl : O:BAG:SYD:AI(A;ID;FA;;;SY)(A;ID;FA;;;BA)(A;ID;0x1200a9;;;BU) This indicates all administrator users, like someone with the ContainerAdministrator role, have read, write, and execute access, while non-administrator users have read and execute access. Important OpenShift Container Platform applies the RunAsUser security context to all pods irrespective of its operating system. This means Windows pods automatically have the RunAsUser permission applied to its security context. In addition, if a Windows pod is created with a projected volume with the default RunAsUser permission set, the pod gets stuck in the ContainerCreating phase. To handle these issues, OpenShift Container Platform forces the file permission handling in projected service account volumes set in the security context of the pod to not be honored for projected volumes on Windows ( BZ#1971745 ). Note that this behavior for Windows pods is how file permission handling used to work for all pod types prior to OpenShift Container Platform 4.7. 2.6. Windows Machine Config Operator prerequisites The following information details the supported platform versions, Windows Server versions, and networking configurations for the Windows Machine Config Operator. See the vSphere documentation for any information that is relevant to only that platform. 2.6.1. Supported platforms based on OpenShift Container Platform and WMCO versions Platform Supported OpenShift Container Platform version Supported WMCO version Installer-provisioned infrastructure installation support User-provisioned infrastructure installation support Amazon Web Services (AWS) 4.6+ WMCO 1.0+ GA Microsoft Azure 4.6+ WMCO 1.0+ GA VMware vSphere 4.7+ WMCO 2.0+ GA 2.6.2. Supported platforms for Bring-Your-Own-Host (BYOH) instances based on OpenShift Container Platform and WMCO versions Platform Supported OpenShift Container Platform version Supported WMCO version BYOH for installer-provisioned infrastructure installation support BYOH for user-provisioned infrastructure installation support Amazon Web Services (AWS) 4.8+ WMCO 3.1+ GA Microsoft Azure 4.8+ WMCO 3.1+ GA VMware vSphere 4.8+ WMCO 3.1+ GA Provider agnostic (Platform: none) 4.8+ WMCO 3.1+ GA 2.6.3. Supported Windows Server versions The following table lists the supported Windows Server version based on the applicable platform. Any unlisted Windows Server version is not supported and will cause errors. To prevent these errors, only use the appropriate version according to the platform in use. Platform Supported Windows Server version Amazon Web Services (AWS) Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019 Microsoft Azure Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019 VMware vSphere Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2022 (OS Build 20348.681 or later) bare metal Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019 2.6.4. Supported networking Hybrid networking with OVN-Kubernetes is the only supported networking configuration. See the additional resources below for more information on this functionality. The following tables outline the type of networking configuration and Windows Server versions to use based on your platform. You must specify the network configuration when you install the cluster. Be aware that OpenShift SDN networking is the default network for OpenShift Container Platform clusters. However, OpenShift SDN is not supported by WMCO. Table 2.1. Platform networking support Platform Supported networking Amazon Web Services (AWS) Hybrid networking with OVN-Kubernetes Microsoft Azure Hybrid networking with OVN-Kubernetes VMware vSphere Hybrid networking with OVN-Kubernetes with a custom VXLAN port bare metal Hybrid networking with OVN-Kubernetes Table 2.2. Hybrid OVN-Kubernetes Windows Server support Hybrid networking with OVN-Kubernetes Supported Windows Server version Default VXLAN port Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019 Custom VXLAN port Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2022 (OS Build 20348.681 or later) Important Running Windows container workloads is not supported for clusters in a restricted network or disconnected environment. Version 4.x of the WMCO is only compatible with OpenShift Container Platform 4.9. 2.7. Known limitations Note the following limitations when working with Windows nodes managed by the WMCO (Windows nodes): The following OpenShift Container Platform features are not supported on Windows nodes: Red Hat OpenShift Developer CLI (odo) Image builds OpenShift Pipelines OpenShift Service Mesh OpenShift monitoring of user-defined projects OpenShift Serverless Horizontal Pod Autoscaling Vertical Pod Autoscaling The following Red Hat features are not supported on Windows nodes: Red Hat cost management Red Hat OpenShift Local Windows nodes do not support pulling container images from private registries. You can use images from public registries or pre-pull the images. Windows nodes do not support workloads created by using deployment configs. You can use a deployment or other method to deploy workloads. Windows nodes are not supported in clusters that use a cluster-wide proxy. This is because the WMCO is not able to route traffic through the proxy connection for the workloads. Windows nodes are not supported in clusters that are in a disconnected environment. Red Hat OpenShift support for Windows Containers does not support adding Windows nodes to a cluster through a trunk port. The only supported networking configuration for adding Windows nodes is through an access port that carries traffic for the VLAN. Red Hat OpenShift support for Windows Containers supports only in-tree storage drivers for all cloud providers. Kubernetes has identified the following node feature limitations : Huge pages are not supported for Windows containers. Privileged containers are not supported for Windows containers. Pod termination grace periods require the containerd container runtime to be installed on the Windows node. Kubernetes has identified several API compatibility issues .

Commands:
Path : Microsoft.PowerShell.Core\FileSystem::C:\var\run\secrets\kubernetes.io\serviceaccount\..2021_08_31_22_22_18.318230061\ca.crt Owner : BUILTIN\Administrators Group : NT AUTHORITY\SYSTEM Access : NT AUTHORITY\SYSTEM Allow FullControl BUILTIN\Administrators Allow FullControl BUILTIN\Users Allow ReadAndExecute, Synchronize Audit : Sddl : O:BAG:SYD:AI(A;ID;FA;;;SY)(A;ID;FA;;;BA)(A;ID;0x1200a9;;;BU)