Chapter 13. Managing machines with the Cluster API 13.1. About the Cluster API Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . The Cluster API is an upstream project that is integrated into OpenShift Container Platform as a Technology Preview for Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, Red Hat OpenStack Platform (RHOSP), and VMware vSphere. 13.1.1. Cluster API overview You can use the Cluster API to create and manage compute machine sets and compute machines in your OpenShift Container Platform cluster. This capability is in addition or an alternative to managing machines with the Machine API. For OpenShift Container Platform 4.18 clusters, you can use the Cluster API to perform node host provisioning management actions after the cluster installation finishes. This system enables an elastic, dynamic provisioning method on top of public or private cloud infrastructure. With the Cluster API Technology Preview, you can create compute machines and compute machine sets on OpenShift Container Platform clusters for supported providers. You can also explore the features that are enabled by this implementation that might not be available with the Machine API. 13.1.1.1. Cluster API benefits By using the Cluster API, OpenShift Container Platform users and developers gain the following advantages: The option to use upstream community Cluster API infrastructure providers that might not be supported by the Machine API. The opportunity to collaborate with third parties who maintain machine controllers for infrastructure providers. The ability to use the same set of Kubernetes tools for infrastructure management in OpenShift Container Platform. The ability to create compute machine sets by using the Cluster API that support features that are not available with the Machine API. 13.1.1.2. Cluster API limitations Using the Cluster API to manage machines is a Technology Preview feature and has the following limitations: To use this feature, you must enable the TechPreviewNoUpgrade feature set. Important Enabling this feature set cannot be undone and prevents minor version updates. Only Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, Red Hat OpenStack Platform (RHOSP), and VMware vSphere clusters can use the Cluster API. You must manually create some of the primary resources that the Cluster API requires. For more information, see "Getting started with the Cluster API". You cannot use the Cluster API to manage control plane machines. Migration of existing compute machine sets created by the Machine API to Cluster API compute machine sets is not supported. Full feature parity with the Machine API is not available. For clusters that use the Cluster API, OpenShift CLI ( oc ) commands prioritize Cluster API objects over Machine API objects. This behavior impacts any oc command that acts upon any object that is represented in both the Cluster API and the Machine API. For more information and a workaround for this issue, see "Referencing the intended objects when using the CLI" in the troubleshooting content. Additional resources Enabling features using feature gates Getting started with the Cluster API Referencing the intended objects when using the CLI 13.1.2. Cluster API architecture The OpenShift Container Platform integration of the upstream Cluster API is implemented and managed by the Cluster CAPI Operator. The Cluster CAPI Operator and its operands are provisioned in the openshift-cluster-api namespace, in contrast to the Machine API, which uses the openshift-machine-api namespace. 13.1.2.1. The Cluster CAPI Operator The Cluster CAPI Operator is an OpenShift Container Platform Operator that maintains the lifecycle of Cluster API resources. This Operator is responsible for all administrative tasks related to deploying the Cluster API project within an OpenShift Container Platform cluster. If a cluster is configured correctly to allow the use of the Cluster API, the Cluster CAPI Operator installs the Cluster API components on the cluster. For more information, see the "Cluster CAPI Operator" entry in the Cluster Operators reference content. Additional resources Cluster CAPI Operator 13.1.2.2. Cluster API primary resources The Cluster API consists of the following primary resources. For the Technology Preview of this feature, you must create some of these resources manually in the openshift-cluster-api namespace. Cluster A fundamental unit that represents a cluster that is managed by the Cluster API. Infrastructure cluster A provider-specific resource that defines properties that all of the compute machine sets in the cluster share, such as the region and subnets. Machine template A provider-specific template that defines the properties of the machines that a compute machine set creates. Machine set A group of machines. Compute machine sets are to machines as replica sets are to pods. To add machines or scale them down, change the replicas field on the compute machine set custom resource to meet your compute needs. With the Cluster API, a compute machine set references a Cluster object and a provider-specific machine template. Machine A fundamental unit that describes the host for a node. The Cluster API creates machines based on the configuration in the machine template. 13.2. Getting started with the Cluster API Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . For the Cluster API Technology Preview, you must manually create some of the primary resources that the Cluster API requires. 13.2.1. Creating the Cluster API primary resources To create the Cluster API primary resources, you must obtain the cluster ID value, which you use for the <cluster_name> parameter in the cluster resource manifest. 13.2.1.1. Obtaining the cluster ID value You can find the cluster ID value by using the OpenShift CLI ( oc ). Prerequisites You have deployed an OpenShift Container Platform cluster. You have access to the cluster using an account with cluster-admin permissions. You have installed the OpenShift CLI ( oc ). Procedure Obtain the value of the cluster ID by running the following command: USD oc get infrastructure cluster \ -o jsonpath='{.status.infrastructureName}' You can create the Cluster API primary resources manually by creating YAML manifest files and applying them with the OpenShift CLI ( oc ). 13.2.1.2. Creating the Cluster API cluster resource You can create the cluster resource by creating a YAML manifest file and applying it with the OpenShift CLI ( oc ). Prerequisites You have deployed an OpenShift Container Platform cluster. You have enabled the use of the Cluster API. You have access to the cluster using an account with cluster-admin permissions. You have installed the OpenShift CLI ( oc ). You have the cluster ID value. Procedure Create a YAML file similar to the following. This procedure uses <cluster_resource_file>.yaml as an example file name. apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: name: <cluster_name> 1 namespace: openshift-cluster-api spec: infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: <infrastructure_kind> 2 name: <cluster_name> namespace: openshift-cluster-api 1 Specify the cluster ID as the name of the cluster. 2 Specify the infrastructure kind for the cluster. The following values are valid: Cluster cloud provider Value Amazon Web Services (AWS) AWSCluster Google Cloud Platform (GCP) GCPCluster Microsoft Azure AzureCluster Red Hat OpenStack Platform (RHOSP) OpenStackCluster VMware vSphere VSphereCluster Create the cluster CR by running the following command: USD oc create -f <cluster_resource_file>.yaml Verification Confirm that the cluster CR exists by running the following command: USD oc get cluster Example output NAME PHASE AGE VERSION <cluster_name> Provisioning 4h6m The cluster resource is ready when the value of PHASE is Provisioned . Additional resources Cluster API configuration 13.2.1.3. Creating a Cluster API machine template You can create a provider-specific machine template resource by creating a YAML manifest file and applying it with the OpenShift CLI ( oc ). Prerequisites You have deployed an OpenShift Container Platform cluster. You have enabled the use of the Cluster API. You have access to the cluster using an account with cluster-admin permissions. You have installed the OpenShift CLI ( oc ). You have created and applied the cluster resource. Procedure Create a YAML file similar to the following. This procedure uses <machine_template_resource_file>.yaml as an example file name. apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: <machine_template_kind> 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 1 Specify the machine template kind. This value must match the value for your platform. The following values are valid: Cluster cloud provider Value Amazon Web Services (AWS) AWSMachineTemplate Google Cloud Platform (GCP) GCPMachineTemplate Microsoft Azure AzureMachineTemplate Red Hat OpenStack Platform (RHOSP) OpenStackMachineTemplate VMware vSphere VSphereMachineTemplate 2 Specify a name for the machine template. 3 Specify the details for your environment. These parameters are provider specific. For more information, see the sample Cluster API machine template YAML for your provider. Create the machine template CR by running the following command: USD oc create -f <machine_template_resource_file>.yaml Verification Confirm that the machine template CR is created by running the following command: USD oc get <machine_template_kind> where <machine_template_kind> is the value that corresponds to your platform. Example output NAME AGE <template_name> 77m Additional resources Sample YAML for a Cluster API machine template resource on Amazon Web Services Sample YAML for a Cluster API machine template resource on Google Cloud Platform Sample YAML for a Cluster API machine template resource on Microsoft Azure Sample YAML for a Cluster API machine template resource on RHOSP Sample YAML for a Cluster API machine template resource on VMware vSphere 13.2.1.4. Creating a Cluster API compute machine set You can create compute machine sets that use the Cluster API to dynamically manage the machine compute resources for specific workloads of your choice. Prerequisites You have deployed an OpenShift Container Platform cluster. You have enabled the use of the Cluster API. You have access to the cluster using an account with cluster-admin permissions. You have installed the OpenShift CLI ( oc ). You have created the cluster and machine template resources. Procedure Create a YAML file similar to the following. This procedure uses <machine_set_resource_file>.yaml as an example file name. apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api spec: clusterName: <cluster_name> 2 replicas: 1 selector: matchLabels: test: example template: metadata: labels: test: example spec: 3 # ... 1 Specify a name for the compute machine set. The cluster ID, machine role, and region form a typical pattern for this value in the following format: <cluster_name>-<role>-<region> . 2 Specify the name of the cluster. 3 Specify the details for your environment. These parameters are provider specific. For more information, see the sample Cluster API compute machine set YAML for your provider. Create the compute machine set CR by running the following command: USD oc create -f <machine_set_resource_file>.yaml Confirm that the compute machine set CR is created by running the following command: USD oc get machineset -n openshift-cluster-api 1 1 Specify the openshift-cluster-api namespace. Example output NAME CLUSTER REPLICAS READY AVAILABLE AGE VERSION <machine_set_name> <cluster_name> 1 1 1 17m When the new compute machine set is available, the REPLICAS and AVAILABLE values match. If the compute machine set is not available, wait a few minutes and run the command again. Verification To verify that the compute machine set is creating machines according to your required configuration, review the lists of machines and nodes in the cluster by running the following commands: View the list of Cluster API machines: USD oc get machine -n openshift-cluster-api 1 1 Specify the openshift-cluster-api namespace. Example output NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_set_name>-<string_id> <cluster_name> <ip_address>.<region>.compute.internal <provider_id> Running 8m23s View the list of nodes: USD oc get node Example output NAME STATUS ROLES AGE VERSION <ip_address_1>.<region>.compute.internal Ready worker 5h14m v1.28.5 <ip_address_2>.<region>.compute.internal Ready master 5h19m v1.28.5 <ip_address_3>.<region>.compute.internal Ready worker 7m v1.28.5 Additional resources Sample YAML for a Cluster API compute machine set resource on Amazon Web Services Sample YAML for a Cluster API compute machine set resource on Google Cloud Platform Sample YAML for a Cluster API compute machine set resource on Microsoft Azure Sample YAML for a Cluster API compute machine set resource on RHOSP Sample YAML for a Cluster API compute machine set resource on VMware vSphere 13.3. Managing machines with the Cluster API Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . 13.3.1. Modifying a Cluster API machine template You can update the machine template resource for your cluster by modifying the YAML manifest file and applying it with the OpenShift CLI ( oc ). Prerequisites You have deployed an OpenShift Container Platform cluster that uses the Cluster API. You have access to the cluster using an account with cluster-admin permissions. You have installed the OpenShift CLI ( oc ). Procedure List the machine template resource for your cluster by running the following command: USD oc get <machine_template_kind> 1 1 Specify the value that corresponds to your platform. The following values are valid: Cluster cloud provider Value Amazon Web Services AWSMachineTemplate Google Cloud Platform GCPMachineTemplate Microsoft Azure AzureMachineTemplate RHOSP OpenStackMachineTemplate VMware vSphere VSphereMachineTemplate Example output NAME AGE <template_name> 77m Write the machine template resource for your cluster to a file that you can edit by running the following command: USD oc get <machine_template_kind> <template_name> -o yaml > <template_name>.yaml where <template_name> is the name of the machine template resource for your cluster. Make a copy of the <template_name>.yaml file with a different name. This procedure uses <modified_template_name>.yaml as an example file name. Use a text editor to make changes to the <modified_template_name>.yaml file that defines the updated machine template resource for your cluster. When editing the machine template resource, observe the following: The parameters in the spec stanza are provider specific. For more information, see the sample Cluster API machine template YAML for your provider. You must use a value for the metadata.name parameter that differs from any existing values. Important For any Cluster API compute machine sets that reference this template, you must update the spec.template.spec.infrastructureRef.name parameter to match the metadata.name value in the new machine template resource. Apply the machine template CR by running the following command: USD oc apply -f <modified_template_name>.yaml 1 1 Use the edited YAML file with a new name. steps For any Cluster API compute machine sets that reference this template, update the spec.template.spec.infrastructureRef.name parameter to match the metadata.name value in the new machine template resource. For more information, see "Modifying a compute machine set by using the CLI." Additional resources Sample YAML for a Cluster API machine template resource on Amazon Web Services Sample YAML for a Cluster API machine template resource on Google Cloud Platform Sample YAML for a Cluster API machine template resource on Microsoft Azure Sample YAML for a Cluster API machine template resource on RHOSP Sample YAML for a Cluster API machine template resource on VMware vSphere Modifying a compute machine set by using the CLI 13.3.2. Modifying a compute machine set by using the CLI You can modify the configuration of a compute machine set, and then propagate the changes to the machines in your cluster by using the CLI. By updating the compute machine set configuration, you can enable features or change the properties of the machines it creates. When you modify a compute machine set, your changes only apply to compute machines that are created after you save the updated MachineSet custom resource (CR). The changes do not affect existing machines. Note Changes made in the underlying cloud provider are not reflected in the Machine or MachineSet CRs. To adjust instance configuration in cluster-managed infrastructure, use the cluster-side resources. You can replace the existing machines with new ones that reflect the updated configuration by scaling the compute machine set to create twice the number of replicas and then scaling it down to the original number of replicas. If you need to scale a compute machine set without making other changes, you do not need to delete the machines. Note By default, the OpenShift Container Platform router pods are deployed on compute machines. Because the router is required to access some cluster resources, including the web console, do not scale the compute machine set to 0 unless you first relocate the router pods. The output examples in this procedure use the values for an AWS cluster. Prerequisites Your OpenShift Container Platform cluster uses the Cluster API. You are logged in to the cluster as an administrator by using the OpenShift CLI ( oc ). Procedure List the compute machine sets in your cluster by running the following command: USD oc get machinesets.cluster.x-k8s.io -n openshift-cluster-api Example output NAME CLUSTER REPLICAS READY AVAILABLE AGE VERSION <compute_machine_set_name_1> <cluster_name> 1 1 1 26m <compute_machine_set_name_2> <cluster_name> 1 1 1 26m Edit a compute machine set by running the following command: USD oc edit machinesets.cluster.x-k8s.io <machine_set_name> \ -n openshift-cluster-api Note the value of the spec.replicas field, because you need it when scaling the machine set to apply the changes. apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> namespace: openshift-cluster-api spec: replicas: 2 1 # ... 1 The examples in this procedure show a compute machine set that has a replicas value of 2 . Update the compute machine set CR with the configuration options that you want and save your changes. List the machines that are managed by the updated compute machine set by running the following command: USD oc get machines.cluster.x-k8s.io \ -n openshift-cluster-api \ -l cluster.x-k8s.io/set-name=<machine_set_name> Example output for an AWS cluster NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_original_1> <cluster_name> <original_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h <machine_name_original_2> <cluster_name> <original_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h For each machine that is managed by the updated compute machine set, set the delete annotation by running the following command: USD oc annotate machines.cluster.x-k8s.io/<machine_name_original_1> \ -n openshift-cluster-api \ cluster.x-k8s.io/delete-machine="true" To create replacement machines with the new configuration, scale the compute machine set to twice the number of replicas by running the following command: USD oc scale --replicas=4 \ 1 machinesets.cluster.x-k8s.io <machine_set_name> \ -n openshift-cluster-api 1 The original example value of 2 is doubled to 4 . List the machines that are managed by the updated compute machine set by running the following command: USD oc get machines.cluster.x-k8s.io \ -n openshift-cluster-api \ -l cluster.x-k8s.io/set-name=<machine_set_name> Example output for an AWS cluster NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_original_1> <cluster_name> <original_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h <machine_name_original_2> <cluster_name> <original_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h <machine_name_updated_1> <cluster_name> <updated_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Provisioned 55s <machine_name_updated_2> <cluster_name> <updated_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Provisioning 55s When the new machines are in the Running phase, you can scale the compute machine set to the original number of replicas. To remove the machines that were created with the old configuration, scale the compute machine set to the original number of replicas by running the following command: USD oc scale --replicas=2 \ 1 machinesets.cluster.x-k8s.io <machine_set_name> \ -n openshift-cluster-api 1 The original example value of 2 . Verification To verify that a machine created by the updated machine set has the correct configuration, examine the relevant fields in the CR for one of the new machines by running the following command: USD oc describe machines.cluster.x-k8s.io <machine_name_updated_1> \ -n openshift-cluster-api To verify that the compute machines without the updated configuration are deleted, list the machines that are managed by the updated compute machine set by running the following command: USD oc get machines.cluster.x-k8s.io \ -n openshift-cluster-api \ cluster.x-k8s.io/set-name=<machine_set_name> Example output while deletion is in progress for an AWS cluster NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_original_1> <cluster_name> <original_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_original_2> <cluster_name> <original_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_updated_1> <cluster_name> <updated_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_updated_2> <cluster_name> <updated_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m Example output when deletion is complete for an AWS cluster NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_updated_1> <cluster_name> <updated_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_updated_2> <cluster_name> <updated_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m Additional resources Sample YAML for a Cluster API compute machine set resource on Amazon Web Services Sample YAML for a Cluster API compute machine set resource on Google Cloud Platform Sample YAML for a Cluster API compute machine set resource on Microsoft Azure Sample YAML for a Cluster API compute machine set resource on RHOSP Sample YAML for a Cluster API compute machine set resource on VMware vSphere 13.4. Cluster API configuration Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . The following example YAML files show how to make the Cluster API primary resources work together and configure settings for the machines that they create that are appropriate for your environment. 13.4.1. Sample YAML for a Cluster API cluster resource The cluster resource defines the name and infrastructure provider for the cluster and is managed by the Cluster API. This resource has the same structure for all providers. apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: name: <cluster_name> 1 namespace: openshift-cluster-api spec: controlPlaneEndpoint: 2 host: <control_plane_endpoint_address> port: 6443 infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: <infrastructure_kind> 3 name: <cluster_name> namespace: openshift-cluster-api 1 Specify the name of the cluster. 2 Specify the IP address of the control plane endpoint and the port used to access it. 3 Specify the infrastructure kind for the cluster. The following values are valid: Cluster cloud provider Value Amazon Web Services AWSCluster GCP GCPCluster Azure AzureCluster RHOSP OpenStackCluster VMware vSphere VSphereCluster 13.4.2. Provider-specific configuration options The remaining Cluster API resources are provider-specific. For provider-specific configuration options for your cluster, see the following resources: Cluster API configuration options for Amazon Web Services Cluster API configuration options for Google Cloud Platform Cluster API configuration options for Microsoft Azure Cluster API configuration options for RHOSP Cluster API configuration options for VMware vSphere 13.5. Configuration options for Cluster API machines 13.5.1. Cluster API configuration options for Amazon Web Services Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . You can change the configuration of your Amazon Web Services (AWS) Cluster API machines by updating values in the Cluster API custom resource manifests. 13.5.1.1. Sample YAML for configuring Amazon Web Services clusters The following example YAML files show configurations for an Amazon Web Services cluster. 13.5.1.1.1. Sample YAML for a Cluster API machine template resource on Amazon Web Services The machine template resource is provider-specific and defines the basic properties of the machines that a compute machine set creates. The compute machine set references this template when creating machines. apiVersion: infrastructure.cluster.x-k8s.io/v1beta2 kind: AWSMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 uncompressedUserData: true iamInstanceProfile: # ... instanceType: m5.large ignition: storageType: UnencryptedUserData version: "3.2" ami: id: # ... subnet: filters: - name: tag:Name values: - # ... additionalSecurityGroups: - filters: - name: tag:Name values: - # ... 1 Specify the machine template kind. This value must match the value for your platform. 2 Specify a name for the machine template. 3 Specify the details for your environment. The values here are examples. 13.5.1.1.2. Sample YAML for a Cluster API compute machine set resource on Amazon Web Services The compute machine set resource defines additional properties of the machines that it creates. The compute machine set also references the cluster resource and machine template when creating machines. apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> 3 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AWSMachineTemplate 4 name: <template_name> 5 1 Specify a name for the compute machine set. The cluster ID, machine role, and region form a typical pattern for this value in the following format: <cluster_name>-<role>-<region> . 2 3 Specify the cluster ID as the name of the cluster. 4 Specify the machine template kind. This value must match the value for your platform. 5 Specify the machine template name. 13.5.2. Cluster API configuration options for Google Cloud Platform Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . You can change the configuration of your Google Cloud Platform (GCP) Cluster API machines by updating values in the Cluster API custom resource manifests. 13.5.2.1. Sample YAML for configuring Google Cloud Platform clusters The following example YAML files show configurations for a Google Cloud Platform cluster. 13.5.2.1.1. Sample YAML for a Cluster API machine template resource on Google Cloud Platform The machine template resource is provider-specific and defines the basic properties of the machines that a compute machine set creates. The compute machine set references this template when creating machines. apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: GCPMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 rootDeviceType: pd-ssd rootDeviceSize: 128 instanceType: n1-standard-4 image: projects/rhcos-cloud/global/images/rhcos-411-85-202203181601-0-gcp-x86-64 subnet: <cluster_name>-worker-subnet serviceAccounts: email: <service_account_email_address> scopes: - https://www.googleapis.com/auth/cloud-platform additionalLabels: kubernetes-io-cluster-<cluster_name>: owned additionalNetworkTags: - <cluster_name>-worker ipForwarding: Disabled 1 Specify the machine template kind. This value must match the value for your platform. 2 Specify a name for the machine template. 3 Specify the details for your environment. The values here are examples. 13.5.2.1.2. Sample YAML for a Cluster API compute machine set resource on Google Cloud Platform The compute machine set resource defines additional properties of the machines that it creates. The compute machine set also references the cluster resource and machine template when creating machines. apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> 3 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: GCPMachineTemplate 4 name: <template_name> 5 failureDomain: <failure_domain> 6 1 Specify a name for the compute machine set. The cluster ID, machine role, and region form a typical pattern for this value in the following format: <cluster_name>-<role>-<region> . 2 3 Specify the cluster ID as the name of the cluster. 4 Specify the machine template kind. This value must match the value for your platform. 5 Specify the machine template name. 6 Specify the failure domain within the GCP region. 13.5.3. Cluster API configuration options for Microsoft Azure Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . You can change the configuration of your Microsoft Azure Cluster API machines by updating values in the Cluster API custom resource manifests. 13.5.3.1. Sample YAML for configuring Microsoft Azure clusters The following example YAML files show configurations for an Azure cluster. 13.5.3.1.1. Sample YAML for a Cluster API machine template resource on Microsoft Azure The machine template resource is provider-specific and defines the basic properties of the machines that a compute machine set creates. The compute machine set references this template when creating machines. apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 disableExtensionOperations: true identity: UserAssigned image: id: /subscriptions/<subscription_id>/resourceGroups/<cluster_name>-rg/providers/Microsoft.Compute/galleries/gallery_<compliant_cluster_name>/images/<cluster_name>-gen2/versions/latest 4 networkInterfaces: - acceleratedNetworking: true privateIPConfigs: 1 subnetName: <cluster_name>-worker-subnet osDisk: diskSizeGB: 128 managedDisk: storageAccountType: Premium_LRS osType: Linux sshPublicKey: <ssh_key_value> userAssignedIdentities: - providerID: 'azure:///subscriptions/<subscription_id>/resourcegroups/<cluster_name>-rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<cluster_name>-identity' vmSize: Standard_D4s_v3 1 Specify the machine template kind. This value must match the value for your platform. 2 Specify a name for the machine template. 3 Specify the details for your environment. The values here are examples. 4 Specify an image that is compatible with your instance type. The Hyper-V generation V2 images created by the installation program have a -gen2 suffix, while V1 images have the same name without the suffix. Note Default OpenShift Container Platform cluster names contain hyphens ( - ), which are not compatible with Azure gallery name requirements. The value of <compliant_cluster_name> in this configuration must use underscores ( _ ) instead of hyphens to comply with these requirements. Other instances of <cluster_name> do not change. For example, a cluster name of jdoe-test-2m2np transforms to jdoe_test_2m2np . The full string for gallery_<compliant_cluster_name> in this example is gallery_jdoe_test_2m2np , not gallery_jdoe-test-2m2np . The complete value of spec.template.spec.image.id for this example value is /subscriptions/<subscription_id>/resourceGroups/jdoe-test-2m2np-rg/providers/Microsoft.Compute/galleries/gallery_jdoe_test_2m2np/images/jdoe-test-2m2np-gen2/versions/latest . 13.5.3.1.2. Sample YAML for a Cluster API compute machine set resource on Microsoft Azure The compute machine set resource defines additional properties of the machines that it creates. The compute machine set also references the cluster resource and machine template when creating machines. apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate 3 name: <template_name> 4 1 Specify a name for the compute machine set. The cluster ID, machine role, and region form a typical pattern for this value in the following format: <cluster_name>-<role>-<region> . 2 Specify the cluster ID as the name of the cluster. 3 Specify the machine template kind. This value must match the value for your platform. 4 Specify the machine template name. 13.5.4. Cluster API configuration options for Red Hat OpenStack Platform Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . You can change the configuration of your Red Hat OpenStack Platform (RHOSP) Cluster API machines by updating values in the Cluster API custom resource manifests. 13.5.4.1. Sample YAML for configuring RHOSP clusters The following example YAML files show configurations for a RHOSP cluster. 13.5.4.1.1. Sample YAML for a Cluster API machine template resource on RHOSP The machine template resource is provider-specific and defines the basic properties of the machines that a compute machine set creates. The compute machine set references this template when creating machines. apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: OpenStackMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 flavor: <openstack_node_machine_flavor> 4 image: filter: name: <openstack_image> 5 1 Specify the machine template kind. This value must match the value for your platform. 2 Specify a name for the machine template. 3 Specify the details for your environment. The values here are examples. 4 Specify the RHOSP flavor to use. For more information, see Creating flavors for launching instances . 5 Specify the image to use. 13.5.4.1.2. Sample YAML for a Cluster API compute machine set resource on RHOSP The compute machine set resource defines additional properties of the machines that it creates. The compute machine set also references the infrastructure resource and machine template when creating machines. apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api spec: clusterName: <cluster_name> 2 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data 3 clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: OpenStackMachineTemplate 4 name: <template_name> 5 failureDomain: <nova_availability_zone> 6 1 Specify a name for the compute machine set. 2 Specify the cluster ID as the name of the cluster. 3 For the Cluster API Technology Preview, the Operator can use the worker user data secret from the openshift-machine-api namespace. 4 Specify the machine template kind. This value must match the value for your platform. 5 Specify the machine template name. 6 Optional: Specify the name of the Nova availability zone for the machine set to create machines in. If you do not specify a value, machines are not restricted to a specific availability zone. 13.5.5. Cluster API configuration options for VMware vSphere Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . You can change the configuration of your VMware vSphere Cluster API machines by updating values in the Cluster API custom resource manifests. 13.5.5.1. Sample YAML for configuring VMware vSphere clusters The following example YAML files show configurations for a VMware vSphere cluster. 13.5.5.1.1. Sample YAML for a Cluster API machine template resource on VMware vSphere The machine template resource is provider-specific and defines the basic properties of the machines that a compute machine set creates. The compute machine set references this template when creating machines. apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: VSphereMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 template: <vm_template_name> 4 server: <vcenter_server_ip> 5 diskGiB: 128 cloneMode: linkedClone 6 datacenter: <vcenter_data_center_name> 7 datastore: <vcenter_datastore_name> 8 folder: <vcenter_vm_folder_path> 9 resourcePool: <vsphere_resource_pool> 10 numCPUs: 4 memoryMiB: 16384 network: devices: - dhcp4: true networkName: "<vm_network_name>" 11 1 Specify the machine template kind. This value must match the value for your platform. 2 Specify a name for the machine template. 3 Specify the details for your environment. The values here are examples. 4 Specify the vSphere VM template to use, such as user-5ddjd-rhcos . 5 Specify the vCenter server IP or fully qualified domain name. 6 Specify the type of VM clone to use. The following values are valid: fullClone linkedClone When using the linkedClone type, the disk size matches the clone source instead of using the diskGiB value. For more information, see the vSphere documentation about VM clone types. 7 Specify the vCenter data center to deploy the compute machine set on. 8 Specify the vCenter datastore to deploy the compute machine set on. 9 Specify the path to the vSphere VM folder in vCenter, such as /dc1/vm/user-inst-5ddjd . 10 Specify the vSphere resource pool for your VMs. 11 Specify the vSphere VM network to deploy the compute machine set to. This VM network must be where other compute machines reside in the cluster. 13.5.5.1.2. Sample YAML for a Cluster API compute machine set resource on VMware vSphere The compute machine set resource defines additional properties of the machines that it creates. The compute machine set also references the cluster resource and machine template when creating machines. apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> 3 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: VSphereMachineTemplate 4 name: <template_name> 5 failureDomain: 6 - name: <failure_domain_name> region: <region_a> zone: <zone_a> server: <vcenter_server_name> topology: datacenter: <region_a_data_center> computeCluster: "</region_a_data_center/host/zone_a_cluster>" resourcePool: "</region_a_data_center/host/zone_a_cluster/Resources/resource_pool>" datastore: "</region_a_data_center/datastore/datastore_a>" networks: - port-group 1 Specify a name for the compute machine set. The cluster ID, machine role, and region form a typical pattern for this value in the following format: <cluster_name>-<role>-<region> . 2 3 Specify the cluster ID as the name of the cluster. 4 Specify the machine template kind. This value must match the value for your platform. 5 Specify the machine template name. 6 Specify the failure domain configuration details. Note Using multiple regions and zones on a vSphere cluster that uses the Cluster API is not a validated configuration. 13.6. Troubleshooting clusters that use the Cluster API Important Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . Use the information in this section to understand and recover from issues you might encounter. Generally, troubleshooting steps for problems with the Cluster API are similar to those steps for problems with the Machine API. The Cluster CAPI Operator and its operands are provisioned in the openshift-cluster-api namespace, whereas the Machine API uses the openshift-machine-api namespace. When using oc commands that reference a namespace, be sure to reference the correct one. 13.6.1. Referencing the intended objects when using the CLI For clusters that use the Cluster API, OpenShift CLI ( oc ) commands prioritize Cluster API objects over Machine API objects. This behavior impacts any oc command that acts upon any object that is represented in both the Cluster API and the Machine API. This explanation uses the oc delete machine command, which deletes a machine, as an example. Cause When you run an oc command, oc communicates with the Kube API server to determine which objects to act upon. The Kube API server uses the first installed custom resource definition (CRD) it encounters alphabetically when an oc command is run. CRDs for Cluster API objects are in the cluster.x-k8s.io group, while CRDs for Machine API objects are in the machine.openshift.io group. Because the letter c precedes the letter m alphabetically, the Kube API server matches on the Cluster API object CRD. As a result, the oc command acts upon Cluster API objects. Consequences Due to this behavior, the following unintended outcomes can occur on a cluster that uses the Cluster API: For namespaces that contain both types of objects, commands such as oc get machine return only Cluster API objects. For namespaces that contain only Machine API objects, commands such as oc get machine return no results. Workaround You can ensure that oc commands act on the type of objects you intend by using the corresponding fully qualified name. Prerequisites You have access to the cluster using an account with cluster-admin permissions. You have installed the OpenShift CLI ( oc ). Procedure To delete a Machine API machine, use the fully qualified name machine.machine.openshift.io when running the oc delete machine command: USD oc delete machine.machine.openshift.io <machine_name> To delete a Cluster API machine, use the fully qualified name machine.cluster.x-k8s.io when running the oc delete machine command: USD oc delete machine.cluster.x-k8s.io <machine_name>

Commands:
oc get infrastructure cluster -o jsonpath='{.status.infrastructureName}'
apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: name: <cluster_name> 1 namespace: openshift-cluster-api spec: infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: <infrastructure_kind> 2 name: <cluster_name> namespace: openshift-cluster-api
oc create -f <cluster_resource_file>.yaml
oc get cluster
NAME PHASE AGE VERSION <cluster_name> Provisioning 4h6m
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: <machine_template_kind> 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3
oc create -f <machine_template_resource_file>.yaml
oc get <machine_template_kind>
NAME AGE <template_name> 77m
apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api spec: clusterName: <cluster_name> 2 replicas: 1 selector: matchLabels: test: example template: metadata: labels: test: example spec: 3
oc create -f <machine_set_resource_file>.yaml
oc get machineset -n openshift-cluster-api 1
NAME CLUSTER REPLICAS READY AVAILABLE AGE VERSION <machine_set_name> <cluster_name> 1 1 1 17m
oc get machine -n openshift-cluster-api 1
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_set_name>-<string_id> <cluster_name> <ip_address>.<region>.compute.internal <provider_id> Running 8m23s
oc get node
NAME STATUS ROLES AGE VERSION <ip_address_1>.<region>.compute.internal Ready worker 5h14m v1.28.5 <ip_address_2>.<region>.compute.internal Ready master 5h19m v1.28.5 <ip_address_3>.<region>.compute.internal Ready worker 7m v1.28.5
oc get <machine_template_kind> 1
NAME AGE <template_name> 77m
oc get <machine_template_kind> <template_name> -o yaml > <template_name>.yaml
oc apply -f <modified_template_name>.yaml 1
oc get machinesets.cluster.x-k8s.io -n openshift-cluster-api
NAME CLUSTER REPLICAS READY AVAILABLE AGE VERSION <compute_machine_set_name_1> <cluster_name> 1 1 1 26m <compute_machine_set_name_2> <cluster_name> 1 1 1 26m
oc edit machinesets.cluster.x-k8s.io <machine_set_name> -n openshift-cluster-api
apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> namespace: openshift-cluster-api spec: replicas: 2 1
oc get machines.cluster.x-k8s.io -n openshift-cluster-api -l cluster.x-k8s.io/set-name=<machine_set_name>
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_original_1> <cluster_name> <original_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h <machine_name_original_2> <cluster_name> <original_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h
oc annotate machines.cluster.x-k8s.io/<machine_name_original_1> -n openshift-cluster-api cluster.x-k8s.io/delete-machine="true"
oc scale --replicas=4 \ 1 machinesets.cluster.x-k8s.io <machine_set_name> -n openshift-cluster-api
oc get machines.cluster.x-k8s.io -n openshift-cluster-api -l cluster.x-k8s.io/set-name=<machine_set_name>
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_original_1> <cluster_name> <original_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h <machine_name_original_2> <cluster_name> <original_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 4h <machine_name_updated_1> <cluster_name> <updated_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Provisioned 55s <machine_name_updated_2> <cluster_name> <updated_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Provisioning 55s
oc scale --replicas=2 \ 1 machinesets.cluster.x-k8s.io <machine_set_name> -n openshift-cluster-api
oc describe machines.cluster.x-k8s.io <machine_name_updated_1> -n openshift-cluster-api
oc get machines.cluster.x-k8s.io -n openshift-cluster-api cluster.x-k8s.io/set-name=<machine_set_name>
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_original_1> <cluster_name> <original_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_original_2> <cluster_name> <original_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_updated_1> <cluster_name> <updated_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_updated_2> <cluster_name> <updated_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION <machine_name_updated_1> <cluster_name> <updated_1_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m <machine_name_updated_2> <cluster_name> <updated_2_ip>.<region>.compute.internal aws:///us-east-2a/i-04e7b2cbd61fd2075 Running 18m
apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: name: <cluster_name> 1 namespace: openshift-cluster-api spec: controlPlaneEndpoint: 2 host: <control_plane_endpoint_address> port: 6443 infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: <infrastructure_kind> 3 name: <cluster_name> namespace: openshift-cluster-api
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2 kind: AWSMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 uncompressedUserData: true iamInstanceProfile: # instanceType: m5.large ignition: storageType: UnencryptedUserData version: "3.2" ami: id: # subnet: filters: - name: tag:Name values: - # additionalSecurityGroups: - filters: - name: tag:Name values: - #
apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> 3 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AWSMachineTemplate 4 name: <template_name> 5
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: GCPMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 rootDeviceType: pd-ssd rootDeviceSize: 128 instanceType: n1-standard-4 image: projects/rhcos-cloud/global/images/rhcos-411-85-202203181601-0-gcp-x86-64 subnet: <cluster_name>-worker-subnet serviceAccounts: email: <service_account_email_address> scopes: - https://www.googleapis.com/auth/cloud-platform additionalLabels: kubernetes-io-cluster-<cluster_name>: owned additionalNetworkTags: - <cluster_name>-worker ipForwarding: Disabled
apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> 3 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: GCPMachineTemplate 4 name: <template_name> 5 failureDomain: <failure_domain> 6
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 disableExtensionOperations: true identity: UserAssigned image: id: /subscriptions/<subscription_id>/resourceGroups/<cluster_name>-rg/providers/Microsoft.Compute/galleries/gallery_<compliant_cluster_name>/images/<cluster_name>-gen2/versions/latest 4 networkInterfaces: - acceleratedNetworking: true privateIPConfigs: 1 subnetName: <cluster_name>-worker-subnet osDisk: diskSizeGB: 128 managedDisk: storageAccountType: Premium_LRS osType: Linux sshPublicKey: <ssh_key_value> userAssignedIdentities: - providerID: 'azure:///subscriptions/<subscription_id>/resourcegroups/<cluster_name>-rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<cluster_name>-identity' vmSize: Standard_D4s_v3
apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate 3 name: <template_name> 4
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: OpenStackMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 flavor: <openstack_node_machine_flavor> 4 image: filter: name: <openstack_image> 5
apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api spec: clusterName: <cluster_name> 2 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data 3 clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: OpenStackMachineTemplate 4 name: <template_name> 5 failureDomain: <nova_availability_zone> 6
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: VSphereMachineTemplate 1 metadata: name: <template_name> 2 namespace: openshift-cluster-api spec: template: spec: 3 template: <vm_template_name> 4 server: <vcenter_server_ip> 5 diskGiB: 128 cloneMode: linkedClone 6 datacenter: <vcenter_data_center_name> 7 datastore: <vcenter_datastore_name> 8 folder: <vcenter_vm_folder_path> 9 resourcePool: <vsphere_resource_pool> 10 numCPUs: 4 memoryMiB: 16384 network: devices: - dhcp4: true networkName: "<vm_network_name>" 11
apiVersion: cluster.x-k8s.io/v1beta1 kind: MachineSet metadata: name: <machine_set_name> 1 namespace: openshift-cluster-api labels: cluster.x-k8s.io/cluster-name: <cluster_name> 2 spec: clusterName: <cluster_name> 3 replicas: 1 selector: matchLabels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> template: metadata: labels: test: example cluster.x-k8s.io/cluster-name: <cluster_name> cluster.x-k8s.io/set-name: <machine_set_name> node-role.kubernetes.io/<role>: "" spec: bootstrap: dataSecretName: worker-user-data clusterName: <cluster_name> infrastructureRef: apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: VSphereMachineTemplate 4 name: <template_name> 5 failureDomain: 6 - name: <failure_domain_name> region: <region_a> zone: <zone_a> server: <vcenter_server_name> topology: datacenter: <region_a_data_center> computeCluster: "</region_a_data_center/host/zone_a_cluster>" resourcePool: "</region_a_data_center/host/zone_a_cluster/Resources/resource_pool>" datastore: "</region_a_data_center/datastore/datastore_a>" networks: - port-group
oc delete machine.machine.openshift.io <machine_name>
oc delete machine.cluster.x-k8s.io <machine_name>