Install ROSA with HCP clusters Red Hat OpenShift Service on AWS 4 Installing, accessing, and deleting Red Hat OpenShift Service on AWS (ROSA) clusters. Red Hat OpenShift Documentation Team

Commands:
git clone https://github.com/openshift-cs/terraform-vpc-example
cd terraform-vpc-example
terraform init
terraform plan -out rosa.tfplan -var region=<region>
terraform apply rosa.tfplan
export SUBNET_IDS=USD(terraform output -raw cluster-subnets-string)
echo USDSUBNET_IDS
subnet-0a6a57e0f784171aa,subnet-078e84e5b10ecf5b0
aws ec2 create-tags --resources <public-subnet-id> --region <aws_region> --tags Key=kubernetes.io/role/elb,Value=1
aws ec2 create-tags --resources <private-subnet-id> --region <aws_region> --tags Key=kubernetes.io/role/internal-elb,Value=1
aws ec2 describe-tags --filters "Name=resource-id,Values=<subnet_id>"
TAGS Name <subnet-id> subnet <prefix>-subnet-public1-us-east-1a TAGS kubernetes.io/role/elb <subnet-id> subnet 1
rosa create account-roles --hosted-cp
export ACCOUNT_ROLES_PREFIX=<account_role_prefix>
echo USDACCOUNT_ROLES_PREFIX
ManagedOpenShift
rosa create oidc-config --mode=auto --yes
? Would you like to create a Managed (Red Hat hosted) OIDC Configuration Yes I: Setting up managed OIDC configuration I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 13cdr6b If you are going to create a Hosted Control Plane cluster please include '--hosted-cp' I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName' ? Create the OIDC provider? Yes I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'
export OIDC_ID=<oidc_config_id> 1
echo USDOIDC_ID
13cdr6b
rosa list oidc-config
ID MANAGED ISSUER URL SECRET ARN 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 true https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2 233hvnrjoqu14jltk6lhbhf2tj11f8un false https://oidc-r7u1.s3.us-east-1.amazonaws.com aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN
export OPERATOR_ROLES_PREFIX=<prefix_name>
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX --oidc-config-id=USDOIDC_ID --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX 1 --oidc-config-id=USDOIDC_ID 2 --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role 3
? Role creation mode: auto ? Operator roles prefix: <pre-filled_prefix> 1 ? OIDC Configuration ID: 23soa2bgvpek9kmes9s7os0a39i13qm4 | https://dvbwgdztaeq9o.cloudfront.net/23soa2bgvpek9kmes9s7os0a39i13qm4 2 ? Create hosted control plane operator roles: Yes W: More than one Installer role found ? Installer role ARN: arn:aws:iam::4540112244:role/<prefix>-HCP-ROSA-Installer-Role ? Permissions boundary ARN (optional): I: Reusable OIDC Configuration detected. Validating trusted relationships to operator roles: I: Creating roles using 'arn:aws:iam::4540112244:user/<userName>' I: Created role '<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' I: Created role '<prefix>-openshift-cloud-network-config-controller-cloud-credenti' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti' I: Created role '<prefix>-kube-system-kube-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager' I: Created role '<prefix>-kube-system-capa-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager' I: Created role '<prefix>-kube-system-control-plane-operator' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator' I: Created role '<prefix>-kube-system-kms-provider' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider' I: Created role '<prefix>-openshift-image-registry-installer-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials' I: Created role '<prefix>-openshift-ingress-operator-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials' I: To create a cluster with these roles, run the following command: rosa create cluster --sts --oidc-config-id 23soa2bgvpek9kmes9s7os0a39i13qm4 --operator-roles-prefix <prefix> --hosted-cp
rosa list operator-roles
I: Fetching operator roles ROLE PREFIX AMOUNT IN BUNDLE <prefix> 8 ? Would you like to detail a specific prefix Yes 1 ? Operator Role Prefix: <prefix> ROLE NAME ROLE ARN VERSION MANAGED <prefix>-kube-system-capa-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager 4.13 No <prefix>-kube-system-control-plane-operator arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator 4.13 No <prefix>-kube-system-kms-provider arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider 4.13 No <prefix>-kube-system-kube-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager 4.13 No <prefix>-openshift-cloud-network-config-controller-cloud-credenti arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti 4.13 No <prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials 4.13 No <prefix>-openshift-image-registry-installer-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials 4.13 No <prefix>-openshift-ingress-operator-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials 4.13 No
rosa create cluster --cluster-name=<cluster_name> \ 1 --mode=auto --hosted-cp [--private] \ 2 --operator-roles-prefix <operator-role-prefix> \ 3 --external-id <external-id> \ 4 --oidc-config-id <id-of-oidc-configuration> --subnet-ids=<public-subnet-id>,<private-subnet-id>
rosa create cluster --private --cluster-name=<cluster_name> --mode=auto --hosted-cp --operator-roles-prefix=USDOPERATOR_ROLES_PREFIX --oidc-config-id=USDOIDC_ID --subnet-ids=USDSUBNET_IDS
rosa create cluster --cluster-name=<cluster_name> --mode=auto --hosted-cp --operator-roles-prefix=USDOPERATOR_ROLES_PREFIX --oidc-config-id=USDOIDC_ID --subnet-ids=USDSUBNET_IDS
rosa describe cluster --cluster=<cluster_name>
rosa logs install --cluster=<cluster_name> --watch \ <.>
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "iam:GetPolicyVersion", "iam:DeletePolicyVersion", "iam:CreatePolicyVersion", "iam:UpdateAssumeRolePolicy", "secretsmanager:DescribeSecret", "iam:ListRoleTags", "secretsmanager:PutSecretValue", "secretsmanager:CreateSecret", "iam:TagRole", "secretsmanager:DeleteSecret", "iam:UpdateOpenIDConnectProviderThumbprint", "iam:DeletePolicy", "iam:CreateRole", "iam:AttachRolePolicy", "iam:ListInstanceProfilesForRole", "secretsmanager:GetSecretValue", "iam:DetachRolePolicy", "iam:ListAttachedRolePolicies", "iam:ListPolicyTags", "iam:ListRolePolicies", "iam:DeleteOpenIDConnectProvider", "iam:DeleteInstanceProfile", "iam:GetRole", "iam:GetPolicy", "iam:ListEntitiesForPolicy", "iam:DeleteRole", "iam:TagPolicy", "iam:CreateOpenIDConnectProvider", "iam:CreatePolicy", "secretsmanager:GetResourcePolicy", "iam:ListPolicyVersions", "iam:UpdateRole", "iam:GetOpenIDConnectProvider", "iam:TagOpenIDConnectProvider", "secretsmanager:TagResource", "sts:AssumeRoleWithWebIdentity", "iam:ListRoles" ], "Resource": [ "arn:aws:secretsmanager:*:<ACCOUNT_ID>:secret:*", "arn:aws:iam::<ACCOUNT_ID>:instance-profile/*", "arn:aws:iam::<ACCOUNT_ID>:role/*", "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/*", "arn:aws:iam::<ACCOUNT_ID>:policy/*" ] }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "s3:*" ], "Resource": "*" } ] }
mkdir terraform-cluster && cd terraform-cluster
export RHCS_TOKEN=<your_offline_token>
echo USDRHCS_TOKEN
cat<<-EOF>main.tf # Copyright (c) 2023 Red Hat, Inc. # Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at # http://www.apache.org/licenses/LICENSE-2.0 # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 4.20.0" } rhcs = { version = ">= 1.6.3" source = "terraform-redhat/rhcs" } } } Export token using the RHCS_TOKEN environment variable provider "rhcs" {} provider "aws" { region = var.aws_region ignore_tags { key_prefixes = ["kubernetes.io/"] } default_tags { tags = var.default_aws_tags } } data "aws_availability_zones" "available" {} locals { # Extract availability zone names for the specified region, limit it to 3 if multi az or 1 if single region_azs = var.multi_az ? slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 3) : slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 1) } resource "random_string" "random_name" { length = 6 special = false upper = false } locals { worker_node_replicas = var.multi_az ? 3 : 2 # If cluster_name is not null, use that, otherwise generate a random cluster name cluster_name = coalesce(var.cluster_name, "rosa-\USD{random_string.random_name.result}") } The network validator requires an additional 60 seconds to validate Terraform clusters. resource "time_sleep" "wait_60_seconds" { count = var.create_vpc ? 1 : 0 depends_on = [module.vpc] create_duration = "60s" } module "rosa-hcp" { source = "terraform-redhat/rosa-hcp/rhcs" version = "1.6.3" cluster_name = local.cluster_name openshift_version = var.openshift_version account_role_prefix = local.cluster_name operator_role_prefix = local.cluster_name replicas = local.worker_node_replicas aws_availability_zones = local.region_azs create_oidc = true private = var.private_cluster aws_subnet_ids = var.create_vpc ? var.private_cluster ? module.vpc[0].private_subnets : concat(module.vpc[0].public_subnets, module.vpc[0].private_subnets) : var.aws_subnet_ids create_account_roles = true create_operator_roles = true Optional: Configure a cluster administrator user <.> # Option 1: Default cluster-admin user Create an administrator user (cluster-admin) and automatically generate a password by uncommenting the following parameter: create_admin_user = true Generated administrator credentials are displayed in terminal output. # Option 2: Specify administrator username and password Create an administrator user and define your own password by uncommenting and editing the values of the following parameters: admin_credentials_username = <username> admin_credentials_password = <password> depends_on = [time_sleep.wait_60_seconds] } EOF
cat<<-EOF>variables.tf # Copyright (c) 2023 Red Hat, Inc. # Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at # http://www.apache.org/licenses/LICENSE-2.0 # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # variable "openshift_version" { type = string default = "4.14.20" description = "Desired version of OpenShift for the cluster, for example '4.14.20'. If version is greater than the currently running version, an upgrade will be scheduled." } variable "create_vpc" { type = bool description = "If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'." } ROSA Cluster info variable "cluster_name" { default = null type = string description = "The name of the ROSA cluster to create" } variable "additional_tags" { default = { Terraform = "true" Environment = "dev" } description = "Additional AWS resource tags" type = map(string) } variable "multi_az" { type = bool description = "Multi AZ Cluster for High Availability" default = true } variable "worker_node_replicas" { default = 3 description = "Number of worker nodes to provision. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes" type = number } variable "aws_subnet_ids" { type = list(any) description = "A list of either the public or public + private subnet IDs to use for the cluster blocks to use for the cluster" default = ["subnet-01234567890abcdef", "subnet-01234567890abcdef", "subnet-01234567890abcdef"] } variable "private_cluster" { type = bool description = "If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'." } #VPC Info variable "vpc_name" { type = string description = "VPC Name" default = "tf-qs-vpc" } variable "vpc_cidr_block" { type = string description = "value of the CIDR block to use for the VPC" default = "10.0.0.0/16" } variable "private_subnet_cidrs" { type = list(any) description = "The CIDR blocks to use for the private subnets" default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] } variable "public_subnet_cidrs" { type = list(any) description = "The CIDR blocks to use for the public subnets" default = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] } variable "single_nat_gateway" { type = bool description = "Single NAT or per NAT for subnet" default = false } #AWS Info variable "aws_region" { type = string default = "us-east-2" } variable "default_aws_tags" { type = map(string) description = "Default tags for AWS" default = {} } EOF
cat<<-EOF>vpc.tf # Copyright (c) 2023 Red Hat, Inc. # Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at # http://www.apache.org/licenses/LICENSE-2.0 # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "5.1.2" count = var.create_vpc ? 1 : 0 name = var.vpc_name cidr = var.vpc_cidr_block azs = local.region_azs private_subnets = var.multi_az ? var.private_subnet_cidrs : [var.private_subnet_cidrs[0]] public_subnets = var.multi_az ? var.public_subnet_cidrs : [var.public_subnet_cidrs[0]] enable_nat_gateway = true single_nat_gateway = var.single_nat_gateway enable_dns_hostnames = true enable_dns_support = true tags = var.additional_tags } EOF
terraform init
terraform validate
Success! The configuration is valid.
terraform apply
var.create_vpc If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'. Enter a value: var.private_cluster If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'. Enter a value:
Plan: 63 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve.
rosa list clusters
ID NAME STATE TOPOLOGY 27c3snjsupa9obua74ba8se5kcj11269 rosa-tf-demo ready Classic (STS)
rosa list account-roles
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION AWS Managed ROSA-demo-Installer-Role Installer arn:aws:iam::<ID>:role/ROSA-demo-Installer-Role 4.14 No ROSA-demo-Support-Role Support arn:aws:iam::<ID>:role/ROSA-demo-Support-Role 4.14 No ROSA-demo-Worker-Role Worker arn:aws:iam::<ID>:role/ROSA-demo-Worker-Role 4.14 No
rosa list operator-roles
I: Fetching operator roles ROLE PREFIX AMOUNT IN BUNDLE rosa-demo 8
terraform destroy
var.create_vpc If you would like to create a new VPC, set this value to 'true.' If you do not want to create a new VPC, set this value to 'false.' Enter a value: var.private_cluster If you want to create a private cluster, set this value to 'true.' If you want a publicly available cluster, set this value to 'false.' Enter a value:
Plan: 0 to add, 0 to change, 63 to destroy. Do you really want to destroy all resources? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes
rosa list clusters
I: No clusters available
rosa list account-roles
I: Fetching account roles I: No account roles available
rosa list operator-roles
I: Fetching operator roles I: No operator roles available
git clone https://github.com/openshift-cs/terraform-vpc-example
cd terraform-vpc-example
terraform init
terraform plan -out rosa.tfplan -var region=<region>
terraform apply rosa.tfplan
export SUBNET_IDS=USD(terraform output -raw cluster-subnets-string)
echo USDSUBNET_IDS
subnet-0a6a57e0f784171aa,subnet-078e84e5b10ecf5b0
rosa create account-roles --hosted-cp
export ACCOUNT_ROLES_PREFIX=<account_role_prefix>
echo USDACCOUNT_ROLES_PREFIX
ManagedOpenShift
rosa create oidc-config --mode=auto --yes
? Would you like to create a Managed (Red Hat hosted) OIDC Configuration Yes I: Setting up managed OIDC configuration I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 13cdr6b If you are going to create a Hosted Control Plane cluster please include '--hosted-cp' I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName' ? Create the OIDC provider? Yes I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'
export OIDC_ID=<oidc_config_id> 1
echo USDOIDC_ID
13cdr6b
rosa list oidc-config
ID MANAGED ISSUER URL SECRET ARN 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 true https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2 233hvnrjoqu14jltk6lhbhf2tj11f8un false https://oidc-r7u1.s3.us-east-1.amazonaws.com aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN
export OPERATOR_ROLES_PREFIX=<prefix_name>
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX --oidc-config-id=USDOIDC_ID --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX 1 --oidc-config-id=USDOIDC_ID 2 --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role 3
? Role creation mode: auto ? Operator roles prefix: <pre-filled_prefix> 1 ? OIDC Configuration ID: 23soa2bgvpek9kmes9s7os0a39i13qm4 | https://dvbwgdztaeq9o.cloudfront.net/23soa2bgvpek9kmes9s7os0a39i13qm4 2 ? Create hosted control plane operator roles: Yes W: More than one Installer role found ? Installer role ARN: arn:aws:iam::4540112244:role/<prefix>-HCP-ROSA-Installer-Role ? Permissions boundary ARN (optional): I: Reusable OIDC Configuration detected. Validating trusted relationships to operator roles: I: Creating roles using 'arn:aws:iam::4540112244:user/<userName>' I: Created role '<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' I: Created role '<prefix>-openshift-cloud-network-config-controller-cloud-credenti' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti' I: Created role '<prefix>-kube-system-kube-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager' I: Created role '<prefix>-kube-system-capa-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager' I: Created role '<prefix>-kube-system-control-plane-operator' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator' I: Created role '<prefix>-kube-system-kms-provider' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider' I: Created role '<prefix>-openshift-image-registry-installer-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials' I: Created role '<prefix>-openshift-ingress-operator-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials' I: To create a cluster with these roles, run the following command: rosa create cluster --sts --oidc-config-id 23soa2bgvpek9kmes9s7os0a39i13qm4 --operator-roles-prefix <prefix> --hosted-cp
rosa list operator-roles
I: Fetching operator roles ROLE PREFIX AMOUNT IN BUNDLE <prefix> 8 ? Would you like to detail a specific prefix Yes 1 ? Operator Role Prefix: <prefix> ROLE NAME ROLE ARN VERSION MANAGED <prefix>-kube-system-capa-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager 4.13 No <prefix>-kube-system-control-plane-operator arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator 4.13 No <prefix>-kube-system-kms-provider arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider 4.13 No <prefix>-kube-system-kube-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager 4.13 No <prefix>-openshift-cloud-network-config-controller-cloud-credenti arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti 4.13 No <prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials 4.13 No <prefix>-openshift-image-registry-installer-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials 4.13 No <prefix>-openshift-ingress-operator-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials 4.13 No
KMS_ARN=USD(aws kms create-key --region USDAWS_REGION --description 'Custom ROSA Encryption Key' --tags TagKey=red-hat,TagValue=true --query KeyMetadata.Arn --output text)
echo USDKMS_ARN
AWS_ACCOUNT_ID=<aws_account_id>
{ "Version": "2012-10-17", "Id": "key-rosa-policy-1", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::USD{AWS_ACCOUNT_ID}:root" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "Installer Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/ManagedOpenShift-HCP-ROSA-Installer-Role" }, "Action": [ "kms:CreateGrant", "kms:DescribeKey", "kms:GenerateDataKeyWithoutPlaintext" ], "Resource": "*" }, { "Sid": "ROSA KubeControllerManager Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/<operator_role_prefix>-kube-system-kube-controller-manager" }, "Action": "kms:DescribeKey", "Resource": "*" }, { "Sid": "ROSA KMS Provider Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/<operator_role_prefix>-kube-system-kms-provider" }, "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:DescribeKey" ], "Resource": "*" }, { "Sid": "ROSA NodeManager Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/<operator_role_prefix>-kube-system-capa-controller-manager" }, "Action": [ "kms:DescribeKey", "kms:GenerateDataKeyWithoutPlaintext", "kms:CreateGrant" ], "Resource": "*" } ] }
cat rosa-key-policy.json
aws kms put-key-policy --key-id USDKMS_ARN --policy file://rosa-key-policy.json --policy-name default
rosa create cluster --cluster-name <cluster_name> --subnet-ids <private_subnet_id>,<public_subnet_id> --sts --mode auto --machine-cidr 10.0.0.0/16 --compute-machine-type m5.xlarge --hosted-cp --region <aws_region> --oidc-config-id USDOIDC_ID --kms-key-arn USDKMS_ARN \ 1 --etcd-encryption-kms-arn USDKMS_ARN \ 2 --operator-roles-prefix USDOPERATOR_ROLES_PREFIX
rosa create account-roles --hosted-cp
rosa create oidc-config --mode=auto --yes
I: Setting up managed OIDC configuration I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 28s4avcdt2l318r1jbk3ifmimkurk384 If you are going to create a Hosted Control Plane cluster please include '--hosted-cp' I: Creating OIDC provider using 'arn:aws:iam::46545644412:user/user' I: Created OIDC provider with ARN 'arn:aws:iam::46545644412:oidc-provider/oidc.op1.openshiftapps.com/28s4avcdt2l318r1jbk3ifmimkurk384'
rosa create operator-roles --hosted-cp --prefix <operator_roles_prefix> --oidc-config-id <oidc_config_id> --installer-role-arn arn:aws:iam::USD<account_roles_prefix>:role/USD<account_roles_prefix>-HCP-ROSA-Installer-Role
rosa create cluster --private --cluster-name=<cluster-name> --sts --mode=auto --hosted-cp --operator-roles-prefix <operator_role_prefix> --oidc-config-id <oidc_config_id> [--machine-cidr=<VPC CIDR>/16] --subnet-ids=<private-subnet-id1>[,<private-subnet-id2>,<private-subnet-id3>]
rosa describe cluster --cluster=<cluster_name>
rosa logs install --cluster=<cluster_name> --watch
export CLUSTER_NAME=<cluster_name>
echo USDCLUSTER_NAME
hcp-private
read -r VPCE_ID VPC_ID <<< USD(aws ec2 describe-vpc-endpoints --filters "Name=tag:api.openshift.com/id,Values=USD(rosa describe cluster -c USD{CLUSTER_NAME} -o yaml | grep '^id: ' | cut -d' ' -f2)" --query 'VpcEndpoints[].[VpcEndpointId,VpcId]' --output text)
export SG_ID=USD(aws ec2 create-security-group --description "Granting API access to USD{CLUSTER_NAME} from outside of VPC" --group-name "USD{CLUSTER_NAME}-api-sg" --vpc-id USDVPC_ID --output text)
aws ec2 authorize-security-group-ingress --group-id USDSG_ID --ip-permissions FromPort=443,ToPort=443,IpProtocol=tcp,IpRanges=[{CidrIp=<cidr-to-allow>}] <.>
aws ec2 modify-vpc-endpoint --vpc-endpoint-id USDVPCE_ID --add-security-group-ids USDSG_ID
rosa create cluster [...] --additional-allowed-principals <arn_string>
Name: mycluster Domain Prefix: mycluster Display Name: mycluster ID: <cluster-id> External ID: <cluster-id> Control Plane: ROSA Service Hosted OpenShift Version: 4.15.17 Channel Group: stable DNS: Not ready AWS Account: <aws_id> AWS Billing Account: <aws_id> API URL: Console URL: Region: us-east-2 Availability: - Control Plane: MultiAZ - Data Plane: SingleAZ Nodes: - Compute (desired): 2 - Compute (current): 0 Network: - Type: OVNKubernetes - Service CIDR: 172.30.0.0/16 - Machine CIDR: 10.0.0.0/16 - Pod CIDR: 10.128.0.0/14 - Host Prefix: /23 - Subnets: subnet-453e99d40, subnet-666847ce827 EC2 Metadata Http Tokens: optional Role (STS) ARN: arn:aws:iam::<aws_id>:role/mycluster-HCP-ROSA-Installer-Role Support Role ARN: arn:aws:iam::<aws_id>:role/mycluster-HCP-ROSA-Support-Role Instance IAM Roles: - Worker: arn:aws:iam::<aws_id>:role/mycluster-HCP-ROSA-Worker-Role Operator IAM Roles: - arn:aws:iam::<aws_id>:role/mycluster-kube-system-control-plane-operator - arn:aws:iam::<aws_id>:role/mycluster-openshift-cloud-network-config-controller-cloud-creden - arn:aws:iam::<aws_id>:role/mycluster-openshift-image-registry-installer-cloud-credentials - arn:aws:iam::<aws_id>:role/mycluster-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<aws_id>:role/mycluster-openshift-cluster-csi-drivers-ebs-cloud-credentials - arn:aws:iam::<aws_id>:role/mycluster-kube-system-kms-provider - arn:aws:iam::<aws_id>:role/mycluster-kube-system-kube-controller-manager - arn:aws:iam::<aws_id>:role/mycluster-kube-system-capa-controller-manager Managed Policies: Yes State: waiting (Waiting for user action) Private: No Delete Protection: Disabled Created: Jun 25 2024 13:36:37 UTC User Workload Monitoring: Enabled Details Page: https://console.redhat.com/openshift/details/s/Bvbok4O79q1Vg8 OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/vhufi5lap6vbl3jlq20e (Managed) Audit Log Forwarding: Disabled External Authentication: Disabled Additional Principals: arn:aws:iam::<aws_id>:role/additional-user-role
rosa edit cluster -c <cluster_name> --additional-allowed-principals <arn_string>
git clone https://github.com/openshift-cs/terraform-vpc-example
cd terraform-vpc-example/zero-egress
terraform init
terraform plan -out rosa-zero-egress.tfplan -var region=<aws_region> \ 1 -var 'availability_zones=["aws_region_1a","aws_region_1b","aws_region_1c"]'\ 2 -var vpc_cidr_block=10.0.0.0/16 \ 3 -var 'private_subnets=["10.0.0.0/24", "10.0.1.0/24", "10.0.2.0/24"]' 4
terraform apply rosa-zero-egress.tfplan
aws ec2 create-tags --resources <public-subnet-id> --region <aws_region> --tags Key=kubernetes.io/role/elb,Value=1
aws ec2 create-tags --resources <private-subnet-id> --region <aws_region> --tags Key=kubernetes.io/role/internal-elb,Value=1
aws ec2 describe-tags --filters "Name=resource-id,Values=<subnet_id>"
TAGS Name <subnet-id> subnet <prefix>-subnet-public1-us-east-1a TAGS kubernetes.io/role/elb <subnet-id> subnet 1
aws ec2 create-security-group --group-name allow-inbound-traffic --description "allow inbound traffic" --vpc-id <vpc_id> \ 1 --region <aws_region> \ 2
aws ec2 authorize-security-group-ingress --group-id <group_id> \ 1 --protocol -1 --port 0-0 --cidr <vpc_cidr> \ 2 --region <aws_region> \ 3
aws ec2 create-vpc-endpoint --vpc-id <vpc_id> \ 1 --service-name com.amazonaws.<aws_region>.sts \ 2 --vpc-endpoint-type Interface
aws ec2 create-vpc-endpoint --vpc-id <vpc_id> --service-name com.amazonaws.<aws_region>.ecr.dkr \ 1 --vpc-endpoint-type Interface
aws ec2 create-vpc-endpoint --vpc-id <vpc_id> --service-name com.amazonaws.<aws_region>.s3
rosa create account-roles --hosted-cp
aws iam attach-role-policy --role-name ManagedOpenShift-HCP-ROSA-Worker-Role \ 1 --policy-arn "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
export ACCOUNT_ROLES_PREFIX=<account_role_prefix>
echo USDACCOUNT_ROLES_PREFIX
ManagedOpenShift
rosa create oidc-config --mode=auto --yes
? Would you like to create a Managed (Red Hat hosted) OIDC Configuration Yes I: Setting up managed OIDC configuration I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 13cdr6b If you are going to create a Hosted Control Plane cluster please include '--hosted-cp' I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName' ? Create the OIDC provider? Yes I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'
export OIDC_ID=<oidc_config_id> 1
echo USDOIDC_ID
13cdr6b
rosa list oidc-config
ID MANAGED ISSUER URL SECRET ARN 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 true https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2 233hvnrjoqu14jltk6lhbhf2tj11f8un false https://oidc-r7u1.s3.us-east-1.amazonaws.com aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN
export OPERATOR_ROLES_PREFIX=<prefix_name>
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX --oidc-config-id=USDOIDC_ID --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX 1 --oidc-config-id=USDOIDC_ID 2 --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role 3
? Role creation mode: auto ? Operator roles prefix: <pre-filled_prefix> 1 ? OIDC Configuration ID: 23soa2bgvpek9kmes9s7os0a39i13qm4 | https://dvbwgdztaeq9o.cloudfront.net/23soa2bgvpek9kmes9s7os0a39i13qm4 2 ? Create hosted control plane operator roles: Yes W: More than one Installer role found ? Installer role ARN: arn:aws:iam::4540112244:role/<prefix>-HCP-ROSA-Installer-Role ? Permissions boundary ARN (optional): I: Reusable OIDC Configuration detected. Validating trusted relationships to operator roles: I: Creating roles using 'arn:aws:iam::4540112244:user/<userName>' I: Created role '<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' I: Created role '<prefix>-openshift-cloud-network-config-controller-cloud-credenti' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti' I: Created role '<prefix>-kube-system-kube-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager' I: Created role '<prefix>-kube-system-capa-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager' I: Created role '<prefix>-kube-system-control-plane-operator' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator' I: Created role '<prefix>-kube-system-kms-provider' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider' I: Created role '<prefix>-openshift-image-registry-installer-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials' I: Created role '<prefix>-openshift-ingress-operator-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials' I: To create a cluster with these roles, run the following command: rosa create cluster --sts --oidc-config-id 23soa2bgvpek9kmes9s7os0a39i13qm4 --operator-roles-prefix <prefix> --hosted-cp
rosa list operator-roles
I: Fetching operator roles ROLE PREFIX AMOUNT IN BUNDLE <prefix> 8 ? Would you like to detail a specific prefix Yes 1 ? Operator Role Prefix: <prefix> ROLE NAME ROLE ARN VERSION MANAGED <prefix>-kube-system-capa-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager 4.13 No <prefix>-kube-system-control-plane-operator arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator 4.13 No <prefix>-kube-system-kms-provider arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider 4.13 No <prefix>-kube-system-kube-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager 4.13 No <prefix>-openshift-cloud-network-config-controller-cloud-credenti arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti 4.13 No <prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials 4.13 No <prefix>-openshift-image-registry-installer-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials 4.13 No <prefix>-openshift-ingress-operator-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials 4.13 No
rosa create cluster --cluster-name=<cluster_name> \ <.> --mode=auto --hosted-cp [--private] \ <.> --operator-roles-prefix <operator-role-prefix> \ <.> --oidc-config-id <id-of-oidc-configuration> --subnet-ids=<private-subnet-id> --region <region> --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 --pod-cidr 10.128.0.0/14 --host-prefix 23 --billing-account <root-acct-id> \ <.> --properties zero_egress:true
rosa create cluster --private --cluster-name=<cluster_name> --mode=auto --hosted-cp --operator-roles-prefix=USDOPERATOR_ROLES_PREFIX --oidc-config-id=USDOIDC_ID --subnet-ids=USDSUBNET_IDS --region <region> --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 --pod-cidr 10.128.0.0/14 --host-prefix 23 --billing-account <root-acct-id> --private --properties zero_egress:true
rosa describe cluster --cluster=<cluster_name>
rosa logs install --cluster=<cluster_name> --watch \ <.>
rosa create cluster --hosted-cp --subnet-ids=USDSUBNET_IDS --oidc-config-id=USDOIDC_ID --cluster-name=<cluster_name> --operator-roles-prefix=USDOPERATOR_ROLES_PREFIX --external-auth-providers-enabled
rosa create cluster --cluster-name=<cluster_name> --sts --mode=auto --hosted-cp --operator-roles-prefix <operator-role-prefix> --oidc-config-id <ID-of-OIDC-configuration> --external-auth-providers-enabled --subnet-ids=<public-subnet-id>,<private-subnet-id>
rosa describe cluster --cluster=<cluster_name>
Name: rosa-ext-test Display Name: rosa-ext-test ID: <cluster_id> External ID: <cluster_ext_id> Control Plane: ROSA Service Hosted OpenShift Version: 4.18.0 Channel Group: stable DNS: <dns> AWS Account: <AWS_id> AWS Billing Account: <AWS_id> API URL: <ocm_api> Console URL: Region: us-east-1 Availability: - Control Plane: MultiAZ - Data Plane: SingleAZ Nodes: - Compute (desired): 2 - Compute (current): 0 Network: - Type: OVNKubernetes - Service CIDR: <service_cidr> - Machine CIDR: <machine_cidr> - Pod CIDR: <pod_cidr> - Host Prefix: /23 - Subnets: <subnet_ids> EC2 Metadata Http Tokens: optional Role (STS) ARN: arn:aws:iam::<AWS_id>:role/<account_roles_prefix>-HCP-ROSA-Installer-Role Support Role ARN: arn:aws:iam::<AWS_id>:role/<account_roles_prefix>-HCP-ROSA-Support-Role Instance IAM Roles: - Worker: arn:aws:iam::<AWS_id>:role/<account_roles_prefix>-HCP-ROSA-Worker-Role Operator IAM Roles: - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-openshift-cloud-network-config-controller-clo - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-kube-system-capa-controller-manager - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-kube-system-control-plane-operator - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-kube-system-kms-provider - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-kube-system-kube-controller-manager - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-openshift-image-registry-installer-cloud-cred - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<AWS_id>:role/<operator_roles_prefix>-openshift-cluster-csi-drivers-ebs-cloud-crede Managed Policies: Yes State: ready Private: No Created: Mar 29 2024 14:25:52 UTC User Workload Monitoring: Enabled Details Page: https://<url> OIDC Endpoint URL: https://<endpoint> (Managed) Audit Log Forwarding: Disabled External Authentication: Enabled 1
rosa create external-auth-provider -c <cluster_name>
I: Enabling interactive mode ? Name: 1 ? Issuer audiences: 2 ? The serving url of the token issuer: 3 ? CA file path (optional): 4 ? Claim mapping username: 5 ? Claim mapping groups: 6 ? Claim validation rule (optional): 7 ? Console client id (optional): 8
rosa create external-auth-provider --cluster=<cluster_id> --name=<provider_name> --issuer-url=<issuing_url> --issuer-audiences=<audience_id> --claim-mapping-username-claim=email --claim-mapping-groups-claim=groups --console-client-id=<client_id_for_app_registration> --console-client-secret=<client_secret>
I: Successfully created an external authentication provider for cluster '<cluster_id>'
rosa list external-auth-provider -c <cluster_name>
NAME ISSUER URL m-entra-id https://login.microsoftonline.com/<group_id>/v2.0
rosa describe external-auth-provider -c <cluster_name> --name <name_of_external_authentication>
ID: ms-entra-id Cluster ID: <cluster_id> Issuer audiences: - <audience_id> Issuer Url: https://login.microsoftonline.com/<group_id>/v2.0 Claim mappings group: groups Claim mappings username: email
rosa create break-glass-credential -c <cluster_name> -i 1
I: Enabling interactive mode ? Username (optional): 1 ? Expiration duration (optional): 2 I: Successfully created a break glass credential for cluster 'ac-hcp-test'.
rosa create break-glass-credential -c mycluster --username test-username --expiration 1h
rosa list break-glass-credential -c mycluster
ID USERNAME STATUS 2a7jli9n4phe6c02ul7ti91djtv2o51d test-user issued
rosa describe break-glass-credential <break_glass_credential_id> -c <cluster_name>
ID: 2a7jli9n4phe6c02ul7ti91djtv2o51d Username: test-user Expire at: Dec 28 2026 10:23:05 EDT Status: issued
mkdir ~/kubeconfigs
export CLUSTER_NAME=<cluster_name> && export KUBECONFIG=~/kubeconfigs/break-glass-USD{CLUSTER_NAME}.kubeconfig
rosa describe break-glass-credential <break_glass_credential_id> -c mycluster --kubeconfig
apiVersion: v1 clusters: - cluster: server: <server_url> name: cluster contexts: - context: cluster: cluster namespace: default user: test-username name: admin current-context: admin kind: Config preferences: {} users: - name: test-user user: client-certificate-data: <client-certificate-data> 1 client-key-data: <client-key-data> 2
rosa describe break-glass-credential <break_glass_credential_id> -c mycluster --kubeconfig > USDKUBECONFIG
rosa describe break-glass-credential <break_glass_credential_id> -c <cluster_name> --kubeconfig > USDKUBECONFIG
oc get nodes
NAME STATUS ROLES AGE VERSION ip-10-0-0-27.ec2.internal Ready worker 8m v1.28.7+f1b5f6c ip-10-0-0-67.ec2.internal Ready worker 9m v1.28.7+f1b5f6c
kubectl auth whoami
ATTRIBUTE VALUE Username system:customer-break-glass:test-user Groups [system:masters system:authenticated]
oc apply -f - <<EOF apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: rosa-hcp-admins roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - apiGroup: rbac.authorization.k8s.io kind: Group name: f715c264-ab90-45d5-8a29-2e91a609a895 EOF
clusterrolebinding.rbac.authorization.k8s.io/rosa-hcp-admins created
rosa revoke break-glass-credentials -c <cluster_name> 1
? Are you sure you want to revoke all the break glass credentials on cluster 'my-cluster'?: Yes I: Successfully requested revocation for all break glass credentials from cluster 'my-cluster'
rosa list break-glass-credential -c <cluster_name>
ID USERNAME STATUS 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 test-user awaiting_revocation
rosa describe break-glass-credential <break_glass_credential_id> -c <cluster_name>
ID: 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 Username: test-user Expire at: Dec 28 2026 10:23:05 EDT Status: issued Revoked at: Dec 27 2026 15:30:33 EDT
rosa list external-auth-provider -c <cluster_name>
NAME ISSUER URL entra-test https://login.microsoftonline.com/<group_id>/v2.0
rosa delete external-auth-provider <name_of_provider> -c <cluster_name>
? Are you sure you want to delete external authentication provider entra-test on cluster rosa-ext-test? Yes I: Successfully deleted external authentication provider 'entra-test' from cluster 'rosa-ext-test'
rosa list external-auth-provider -c <cluster_name>
E: there are no external authentication providers for this cluster
rosa create account-roles --hosted-cp
export ACCOUNT_ROLES_PREFIX=<account_role_prefix>
echo USDACCOUNT_ROLES_PREFIX
ManagedOpenShift
rosa create oidc-config --mode=auto --yes
? Would you like to create a Managed (Red Hat hosted) OIDC Configuration Yes I: Setting up managed OIDC configuration I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 13cdr6b If you are going to create a Hosted Control Plane cluster please include '--hosted-cp' I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName' ? Create the OIDC provider? Yes I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'
export OIDC_ID=<oidc_config_id> 1
echo USDOIDC_ID
13cdr6b
rosa list oidc-config
ID MANAGED ISSUER URL SECRET ARN 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 true https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2 233hvnrjoqu14jltk6lhbhf2tj11f8un false https://oidc-r7u1.s3.us-east-1.amazonaws.com aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN
export OPERATOR_ROLES_PREFIX=<prefix_name>
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX --oidc-config-id=USDOIDC_ID --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role
rosa create operator-roles --hosted-cp --prefix=USDOPERATOR_ROLES_PREFIX 1 --oidc-config-id=USDOIDC_ID 2 --installer-role-arn arn:aws:iam::USD{AWS_ACCOUNT_ID}:role/USD{ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role 3
? Role creation mode: auto ? Operator roles prefix: <pre-filled_prefix> 1 ? OIDC Configuration ID: 23soa2bgvpek9kmes9s7os0a39i13qm4 | https://dvbwgdztaeq9o.cloudfront.net/23soa2bgvpek9kmes9s7os0a39i13qm4 2 ? Create hosted control plane operator roles: Yes W: More than one Installer role found ? Installer role ARN: arn:aws:iam::4540112244:role/<prefix>-HCP-ROSA-Installer-Role ? Permissions boundary ARN (optional): I: Reusable OIDC Configuration detected. Validating trusted relationships to operator roles: I: Creating roles using 'arn:aws:iam::4540112244:user/<userName>' I: Created role '<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' I: Created role '<prefix>-openshift-cloud-network-config-controller-cloud-credenti' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti' I: Created role '<prefix>-kube-system-kube-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager' I: Created role '<prefix>-kube-system-capa-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager' I: Created role '<prefix>-kube-system-control-plane-operator' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator' I: Created role '<prefix>-kube-system-kms-provider' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider' I: Created role '<prefix>-openshift-image-registry-installer-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials' I: Created role '<prefix>-openshift-ingress-operator-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials' I: To create a cluster with these roles, run the following command: rosa create cluster --sts --oidc-config-id 23soa2bgvpek9kmes9s7os0a39i13qm4 --operator-roles-prefix <prefix> --hosted-cp
rosa list operator-roles
I: Fetching operator roles ROLE PREFIX AMOUNT IN BUNDLE <prefix> 8 ? Would you like to detail a specific prefix Yes 1 ? Operator Role Prefix: <prefix> ROLE NAME ROLE ARN VERSION MANAGED <prefix>-kube-system-capa-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager 4.13 No <prefix>-kube-system-control-plane-operator arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator 4.13 No <prefix>-kube-system-kms-provider arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider 4.13 No <prefix>-kube-system-kube-controller-manager arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager 4.13 No <prefix>-openshift-cloud-network-config-controller-cloud-credenti arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti 4.13 No <prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials 4.13 No <prefix>-openshift-image-registry-installer-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials 4.13 No <prefix>-openshift-ingress-operator-cloud-credentials arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials 4.13 No
rosa create cluster --cluster-name=<cluster_name> --sts --mode=auto --hosted-cp --operator-roles-prefix <operator-role-prefix> --oidc-config-id <ID-of-OIDC-configuration> --subnet-ids=<public-subnet-id>,<private-subnet-id> --no-cni
rosa create cluster --private --cluster-name=<cluster_name> --sts --mode=auto --hosted-cp --subnet-ids=<private-subnet-id> --no-cni
rosa create cluster --hosted-cp --subnet-ids=USDSUBNET_IDS --oidc-config-id=USDOIDC_ID --cluster-name=<cluster_name> --operator-roles-prefix=USDOPERATOR_ROLES_PREFIX --no-cni
rosa describe cluster --cluster=<cluster_name>
rosa logs install --cluster=<cluster_name> --watch 1
rosa describe cluster --cluster=<cluster_name>
Name: test_cluster Domain Prefix: test_cluster Display Name: test_cluster ID: <cluster_id> 1 External ID: <external_id> Control Plane: ROSA Service Hosted OpenShift Version: 4.18.0 Channel Group: stable DNS: test_cluster.l3cn.p3.openshiftapps.com AWS Account: <AWS_id> AWS Billing Account: <AWS_id> API URL: https://api.test_cluster.l3cn.p3.openshiftapps.com:443 Console URL: Region: us-east-1 Availability: - Control Plane: MultiAZ - Data Plane: SingleAZ Nodes: - Compute (desired): 2 - Compute (current): 0 Network: - Type: OVNKubernetes - Service CIDR: 172.30.0.0/16 - Machine CIDR: 10.0.0.0/16 - Pod CIDR: 10.128.0.0/14 - Host Prefix: /23 - Subnets: <subnet_ids> EC2 Metadata Http Tokens: optional Role (STS) ARN: arn:aws:iam::<AWS_id>:role/test_cluster-HCP-ROSA-Installer-Role Support Role ARN: arn:aws:iam::<AWS_id>:role/test_cluster-HCP-ROSA-Support-Role Instance IAM Roles: - Worker: arn:aws:iam::<AWS_id>:role/test_cluster-HCP-ROSA-Worker-Role Operator IAM Roles: 2 - arn:aws:iam::<AWS_id>:role/test_cluster-openshift-cloud-network-config-controller-cloud-crede - arn:aws:iam::<AWS_id>:role/test_cluster-openshift-image-registry-installer-cloud-credentials - arn:aws:iam::<AWS_id>:role/test_cluster-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<AWS_id>:role/test_cluster-kube-system-kube-controller-manager - arn:aws:iam::<AWS_id>:role/test_cluster-kube-system-capa-controller-manager - arn:aws:iam::<AWS_id>:role/test_cluster-kube-system-control-plane-operator - arn:aws:iam::<AWS_id>:role/hcpcluster-kube-system-kms-provider - arn:aws:iam::<AWS_id>:role/test_cluster-openshift-cluster-csi-drivers-ebs-cloud-credentials Managed Policies: Yes State: ready Private: No Created: Apr 16 2024 20:32:06 UTC User Workload Monitoring: Enabled Details Page: https://console.redhat.com/openshift/details/s/<cluster_id> OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/<cluster_id> (Managed) 3 Audit Log Forwarding: Disabled External Authentication: Disabled
rosa delete cluster --cluster=<cluster_name> --watch
rosa delete operator-roles --prefix <operator_role_prefix>
rosa delete oidc-provider --oidc-config-id <oidc_config_id>
rosa list account-roles
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION AWS Managed ManagedOpenShift-HCP-ROSA-Installer-Role Installer arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Installer-Role 4.18 Yes ManagedOpenShift-HCP-ROSA-Support-Role Support arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Support-Role 4.18 Yes ManagedOpenShift-HCP-ROSA-Worker-Role Worker arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Worker-Role 4.18 Yes
rosa delete account-roles --prefix <prefix> --mode auto 1
W: There are no classic account roles to be deleted I: Deleting hosted CP account roles ? Delete the account role 'delete-rosa-HCP-ROSA-Installer-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Installer-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Support-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Support-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Worker-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Worker-Role' I: Successfully deleted the hosted CP account roles
rosa list ocm-roles
I: Fetching ocm roles ROLE NAME ROLE ARN LINKED ADMIN AWS Managed ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> Yes Yes Yes
rosa unlink ocm-role --role-arn <arn> 1
I: Unlinking OCM role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' role from organization '<red_hat_organization_id>'? Yes I: Successfully unlinked role-arn 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' from organization account '<red_hat_organization_id>'
rosa delete ocm-role --role-arn <arn>
I: Deleting OCM role ? OCM Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> ? Delete 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' ocm role? Yes ? OCM role deletion mode: auto 1 I: Successfully deleted the OCM role
rosa list user-roles
I: Fetching user roles ROLE NAME ROLE ARN LINKED ManagedOpenShift-User-<ocm_user_name>-Role arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role Yes
rosa unlink user-role --role-arn <arn> 1
I: Unlinking user role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the current account '<ocm_user_account_id>'? Yes I: Successfully unlinked role ARN 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' from account '<ocm_user_account_id>'
rosa delete user-role --role-arn <arn>
I: Deleting user role ? User Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role ? Delete the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the AWS account? Yes ? User role deletion mode: auto 1 I: Successfully deleted the user role