Install ROSA Classic clusters Red Hat OpenShift Service on AWS 4 Installing, accessing, and deleting Red Hat OpenShift Service on AWS (ROSA) clusters. Red Hat OpenShift Documentation Team

Commands:
rosa create ocm-role
rosa create user-role
rosa list account-roles
rosa create account-roles
rosa create oidc-config --mode=auto --yes
? Would you like to create a Managed (Red Hat hosted) OIDC Configuration Yes I: Setting up managed OIDC configuration I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 13cdr6b If you are going to create a Hosted Control Plane cluster please include '--hosted-cp' I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName' ? Create the OIDC provider? Yes I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'
export OIDC_ID=<oidc_config_id> 1
echo USDOIDC_ID
13cdr6b
rosa list oidc-config
ID MANAGED ISSUER URL SECRET ARN 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 true https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2 233hvnrjoqu14jltk6lhbhf2tj11f8un false https://oidc-r7u1.s3.us-east-1.amazonaws.com aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN
rosa create account-roles --mode auto
rosa create cluster --cluster-name <cluster_name> \ 1 --sts --mode auto 2
rosa describe cluster --cluster <cluster_name|cluster_id>
rosa logs install --cluster <cluster_name|cluster_id> --watch 1
rosa create oidc-config --mode=auto --yes
? Would you like to create a Managed (Red Hat hosted) OIDC Configuration Yes I: Setting up managed OIDC configuration I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 13cdr6b If you are going to create a Hosted Control Plane cluster please include '--hosted-cp' I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName' ? Create the OIDC provider? Yes I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'
export OIDC_ID=<oidc_config_id> 1
echo USDOIDC_ID
13cdr6b
rosa list oidc-config
ID MANAGED ISSUER URL SECRET ARN 2330dbs0n8m3chkkr25gkkcd8pnj3lk2 true https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2 233hvnrjoqu14jltk6lhbhf2tj11f8un false https://oidc-r7u1.s3.us-east-1.amazonaws.com aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN
rosa login --token=<api_login_token> 1
I: Logged in as '<username>' on 'https://api.openshift.com'
rosa create ocm-role
I: Creating ocm role ? Role prefix: ManagedOpenShift 1 ? Enable admin capabilities for the OCM role (optional): No 2 ? Permissions boundary ARN (optional): 3 ? Role Path (optional): 4 ? Role creation mode: auto 5 I: Creating role using 'arn:aws:iam::<aws_account_id>:user/<aws_username>' ? Create the 'ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' role? Yes I: Created role 'ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' with ARN 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' I: Linking OCM role ? OCM Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> ? Link the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' role with organization '<red_hat_organization_id>'? Yes 6 I: Successfully linked role-arn 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' with organization account '<red_hat_organization_id>'
rosa link ocm-role <arn> 1
rosa create user-role
I: Creating User role ? Role prefix: ManagedOpenShift 1 ? Permissions boundary ARN (optional): 2 ? Role Path (optional): [? for help] 3 ? Role creation mode: auto 4 I: Creating ocm user role using 'arn:aws:iam::<aws_account_id>:user/<aws_username>' ? Create the 'ManagedOpenShift-User-<red_hat_username>-Role' role? Yes I: Created role 'ManagedOpenShift-User-<red_hat_username>-Role' with ARN 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<red_hat_username>-Role' I: Linking User role ? User Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<red_hat_username>-Role ? Link the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<red_hat_username>-Role' role with account '<red_hat_user_account_id>'? Yes 5 I: Successfully linked role ARN 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<red_hat_username>-Role' with account '<red_hat_user_account_id>'
rosa link user-role <arn> 1
rosa create account-roles
I: Logged in as '<red_hat_username>' on 'https://api.openshift.com' I: Validating AWS credentials I: AWS credentials are valid! I: Validating AWS quota I: AWS quota ok. If cluster installation fails, validate actual AWS resource usage against https://docs.openshift.com/rosa/rosa_getting_started/rosa-required-aws-service-quotas.html I: Verifying whether OpenShift command-line tool is available I: Current OpenShift Client Version: 4.0 I: Creating account roles ? Role prefix: ManagedOpenShift 1 ? Permissions boundary ARN (optional): 2 ? Path (optional): [? for help] 3 ? Role creation mode: auto 4 I: Creating roles using 'arn:aws:iam::<aws_account_number>:user/<aws_username>' ? Create the 'ManagedOpenShift-Installer-Role' role? Yes 5 I: Created role 'ManagedOpenShift-Installer-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-Installer-Role' ? Create the 'ManagedOpenShift-ControlPlane-Role' role? Yes 6 I: Created role 'ManagedOpenShift-ControlPlane-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-ControlPlane-Role' ? Create the 'ManagedOpenShift-Worker-Role' role? Yes 7 I: Created role 'ManagedOpenShift-Worker-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-Worker-Role' ? Create the 'ManagedOpenShift-Support-Role' role? Yes 8 I: Created role 'ManagedOpenShift-Support-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-Support-Role' I: To create a cluster with these roles, run the following command: rosa create cluster --sts
rosa create account-roles --interactive \ 1 --mode manual 2
I: Logged in as '<red_hat_username>' on 'https://api.openshift.com' I: Validating AWS credentials I: AWS credentials are valid! I: Validating AWS quota I: AWS quota ok. If cluster installation fails, validate actual AWS resource usage against https://docs.openshift.com/rosa/rosa_getting_started/rosa-required-aws-service-quotas.html I: Verifying whether OpenShift command-line tool is available I: Current OpenShift Client Version: 4.0 I: Creating account roles ? Role prefix: ManagedOpenShift 1 ? Permissions boundary ARN (optional): 2 ? Path (optional): [? for help] 3 ? Role creation mode: auto 4 I: Creating roles using 'arn:aws:iam::<aws_account_number>:user/<aws_username>' ? Create the 'ManagedOpenShift-Installer-Role' role? Yes 5 I: Created role 'ManagedOpenShift-Installer-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-Installer-Role' ? Create the 'ManagedOpenShift-ControlPlane-Role' role? Yes 6 I: Created role 'ManagedOpenShift-ControlPlane-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-ControlPlane-Role' ? Create the 'ManagedOpenShift-Worker-Role' role? Yes 7 I: Created role 'ManagedOpenShift-Worker-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-Worker-Role' ? Create the 'ManagedOpenShift-Support-Role' role? Yes 8 I: Created role 'ManagedOpenShift-Support-Role' with ARN 'arn:aws:iam::<aws_account_number>:role/ManagedOpenShift-Support-Role' I: To create a cluster with these roles, run the following command: rosa create cluster --sts
aws kms get-key-policy --key-id <key_id_or_arn> --policy-name default --output text > kms-key-policy.json 1
{ "Version": "2012-10-17", "Id": "key-rosa-policy-1", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<aws_account_id>:root" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "Allow ROSA use of the key", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role", 1 "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role", "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role", "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role", "arn:aws:iam::<aws_account_id>:role/<cluster_name>-xxxx-openshift-cluster-csi-drivers-ebs-cloud-credent" 2 ] }, "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ], "Resource": "*" }, { "Sid": "Allow attachment of persistent resources", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role", 3 "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role", "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role", "arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role", "arn:aws:iam::<aws_account_id>:role/<cluster_name>-xxxx-openshift-cluster-csi-drivers-ebs-cloud-credent" 4 ] }, "Action": [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ], "Resource": "*", "Condition": { "Bool": { "kms:GrantIsForAWSResource": "true" } } } ] }
aws kms put-key-policy --key-id <key_id_or_arn> \ 1 --policy file://kms-key-policy.json \ 2 --policy-name default
rosa create cluster --interactive --sts
I: Interactive mode enabled. Any optional fields can be left empty and a default will be selected. ? Cluster name: <cluster_name> ? Domain prefix: <domain_prefix> 1 ? Deploy cluster with Hosted Control Plane (optional): No ? Create cluster admin user: Yes 2 ? Create custom password for cluster admin: No 3 I: cluster admin user is cluster-admin I: cluster admin password is password ? OpenShift version: <openshift_version> 4 ? Configure the use of IMDSv2 for ec2 instances optional/required (optional): 5 I: Using arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role for the Installer role 6 I: Using arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role for the ControlPlane role I: Using arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role for the Worker role I: Using arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role for the Support role ? External ID (optional): 7 ? Operator roles prefix: <cluster_name>-<random_string> 8 ? Deploy cluster using pre registered OIDC Configuration ID: ? Tags (optional) 9 ? Multiple availability zones (optional): No 10 ? AWS region: us-east-1 ? PrivateLink cluster (optional): No ? Machine CIDR: 10.0.0.0/16 ? Service CIDR: 172.30.0.0/16 ? Pod CIDR: 10.128.0.0/14 ? Install into an existing VPC (optional): Yes 11 ? Subnet IDs (optional): ? Select availability zones (optional): No ? Enable Customer Managed key (optional): No 12 ? Compute nodes instance type (optional): ? Enable autoscaling (optional): No ? Compute nodes: 2 ? Worker machine pool labels (optional): ? Host prefix: 23 ? Additional Security Group IDs (optional): 13 ? > [*] sg-0e375ff0ec4a6cfa2 ('sg-1') ? > [ ] sg-0e525ef0ec4b2ada7 ('sg-2') ? Enable FIPS support: No 14 ? Encrypt etcd data: No 15 ? Disable Workload monitoring (optional): No I: Creating cluster '<cluster_name>' I: To create this cluster again in the future, you can run: rosa create cluster --cluster-name <cluster_name> --role-arn arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role --support-role-arn arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role --master-iam-role arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role --worker-iam-role arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role --operator-roles-prefix <cluster_name>-<random_string> --region us-east-1 --version 4.18.0 --additional-compute-security-group-ids sg-0e375ff0ec4a6cfa2 --additional-infra-security-group-ids sg-0e375ff0ec4a6cfa2 --additional-control-plane-security-group-ids sg-0e375ff0ec4a6cfa2 --replicas 2 --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 --pod-cidr 10.128.0.0/14 --host-prefix 23 16 I: To view a list of clusters and their status, run 'rosa list clusters' I: Cluster '<cluster_name>' has been created. I: Once the cluster is installed you will need to add an Identity Provider before you can login into the cluster. See 'rosa create idp --help' for more information.
rosa create operator-roles --mode manual --cluster <cluster_name|cluster_id> 1
rosa create oidc-provider --mode auto --cluster <cluster_name|cluster_id> 1
rosa describe cluster --cluster <cluster_name|cluster_id>
Name: <cluster_name> ID: <cluster_id> External ID: <external_id> OpenShift Version: <version> Channel Group: stable DNS: <cluster_name>.xxxx.p1.openshiftapps.com AWS Account: <aws_account_id> API URL: https://api.<cluster_name>.xxxx.p1.openshiftapps.com:6443 Console URL: https://console-openshift-console.apps.<cluster_name>.xxxx.p1.openshiftapps.com Region: <aws_region> Multi-AZ: false Nodes: - Master: 3 - Infra: 2 - Compute: 2 Network: - Service CIDR: 172.30.0.0/16 - Machine CIDR: 10.0.0.0/16 - Pod CIDR: 10.128.0.0/14 - Host Prefix: /23 STS Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role Support Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role Instance IAM Roles: - Master: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role - Worker: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role Operator IAM Roles: - arn:aws:iam::<aws_account_id>:role/<cluster_name>-xxxx-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<aws_account_id>:role/<cluster_name>-xxxx-openshift-cluster-csi-drivers-ebs-cloud-credent - arn:aws:iam::<aws_account_id>:role/<cluster_name>-xxxx-openshift-machine-api-aws-cloud-credentials - arn:aws:iam::<aws_account_id>:role/<cluster_name>-xxxx-openshift-cloud-credential-operator-cloud-crede - arn:aws:iam::<aws_account_id>:role/<cluster_name>-xxxx-openshift-image-registry-installer-cloud-creden Ec2 Metadata Http Tokens: optional State: ready Private: No Created: Oct 1 2021 08:12:25 UTC Details Page: https://console.redhat.com/openshift/details/s/<subscription_id> OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/<cluster_id>|<oidc_config_id> \ 1
rosa logs install --cluster <cluster_name|cluster_id> --watch 1
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "iam:GetPolicyVersion", "iam:DeletePolicyVersion", "iam:CreatePolicyVersion", "iam:UpdateAssumeRolePolicy", "secretsmanager:DescribeSecret", "iam:ListRoleTags", "secretsmanager:PutSecretValue", "secretsmanager:CreateSecret", "iam:TagRole", "secretsmanager:DeleteSecret", "iam:UpdateOpenIDConnectProviderThumbprint", "iam:DeletePolicy", "iam:CreateRole", "iam:AttachRolePolicy", "iam:ListInstanceProfilesForRole", "secretsmanager:GetSecretValue", "iam:DetachRolePolicy", "iam:ListAttachedRolePolicies", "iam:ListPolicyTags", "iam:ListRolePolicies", "iam:DeleteOpenIDConnectProvider", "iam:DeleteInstanceProfile", "iam:GetRole", "iam:GetPolicy", "iam:ListEntitiesForPolicy", "iam:DeleteRole", "iam:TagPolicy", "iam:CreateOpenIDConnectProvider", "iam:CreatePolicy", "secretsmanager:GetResourcePolicy", "iam:ListPolicyVersions", "iam:UpdateRole", "iam:GetOpenIDConnectProvider", "iam:TagOpenIDConnectProvider", "secretsmanager:TagResource", "sts:AssumeRoleWithWebIdentity", "iam:ListRoles" ], "Resource": [ "arn:aws:secretsmanager:*:<ACCOUNT_ID>:secret:*", "arn:aws:iam::<ACCOUNT_ID>:instance-profile/*", "arn:aws:iam::<ACCOUNT_ID>:role/*", "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/*", "arn:aws:iam::<ACCOUNT_ID>:policy/*" ] }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "s3:*" ], "Resource": "*" } ] }
mkdir terraform-cluster && cd terraform-cluster
export RHCS_TOKEN=<your_offline_token>
echo USDRHCS_TOKEN
cat<<-EOF>main.tf # Copyright (c) 2023 Red Hat, Inc. # Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at # http://www.apache.org/licenses/LICENSE-2.0 # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 4.20.0" } rhcs = { version = ">= 1.6.2" source = "terraform-redhat/rhcs" } } } Export token using the RHCS_TOKEN environment variable provider "rhcs" {} provider "aws" { region = var.aws_region ignore_tags { key_prefixes = ["kubernetes.io/"] } default_tags { tags = var.default_aws_tags } } data "aws_availability_zones" "available" {} locals { # The default setting creates 3 availability zones. Set to "false" to create a single availability zones. region_azs = var.multi_az ? slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 3) : slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 1) } resource "random_string" "random_name" { length = 6 special = false upper = false } locals { path = coalesce(var.path, "/") worker_node_replicas = try(var.worker_node_replicas, var.multi_az ? 3 : 2) # If cluster_name is not null, use that, otherwise generate a random cluster name cluster_name = coalesce(var.cluster_name, "rosa-\USD{random_string.random_name.result}") } The network validator requires an additional 60 seconds to validate Terraform clusters. resource "time_sleep" "wait_60_seconds" { count = var.create_vpc ? 1 : 0 depends_on = [module.vpc] create_duration = "60s" } module "rosa-classic" { source = "terraform-redhat/rosa-classic/rhcs" version = "1.5.0" cluster_name = local.cluster_name openshift_version = var.openshift_version account_role_prefix = local.cluster_name operator_role_prefix = local.cluster_name replicas = local.worker_node_replicas aws_availability_zones = local.region_azs create_oidc = true private = var.private_cluster aws_private_link = var.private_cluster aws_subnet_ids = var.create_vpc ? var.private_cluster ? module.vpc[0].private_subnets : concat(module.vpc[0].public_subnets, module.vpc[0].private_subnets) : var.aws_subnet_ids multi_az = var.multi_az create_account_roles = true create_operator_roles = true Optional: Configure a cluster administrator user <.> # Option 1: Default cluster-admin user Create an administrator user (cluster-admin) and automatically generate a password by uncommenting the following parameter: create_admin_user = true Generated administrator credentials are displayed in terminal output. # Option 2: Specify administrator username and password Create an administrator user and define your own password by uncommenting and editing the values of the following parameters: admin_credentials_username = <username> admin_credentials_password = <password> depends_on = [time_sleep.wait_60_seconds] } EOF
cat<<-EOF>variables.tf # Copyright (c) 2023 Red Hat, Inc. # Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at # http://www.apache.org/licenses/LICENSE-2.0 # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # variable "openshift_version" { type = string default = "4.14.20" description = "Desired version of OpenShift for the cluster, for example '4.14.20'. If version is greater than the currently running version, an upgrade will be scheduled." } variable "create_vpc" { type = bool description = "If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'." } ROSA Cluster info variable "cluster_name" { default = null type = string description = "The name of the ROSA cluster to create" } variable "additional_tags" { default = { Terraform = "true" Environment = "dev" } description = "Additional AWS resource tags" type = map(string) } variable "path" { description = "(Optional) The arn path for the account/operator roles as well as their policies." type = string default = null } variable "multi_az" { type = bool description = "Multi AZ Cluster for High Availability" default = true } variable "worker_node_replicas" { default = 3 description = "Number of worker nodes to provision. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes" type = number } variable "aws_subnet_ids" { type = list(any) description = "A list of either the public or public + private subnet IDs to use for the cluster blocks to use for the cluster" default = ["subnet-01234567890abcdef", "subnet-01234567890abcdef", "subnet-01234567890abcdef"] } variable "private_cluster" { type = bool description = "If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'." } #VPC Info variable "vpc_name" { type = string description = "VPC Name" default = "tf-qs-vpc" } variable "vpc_cidr_block" { type = string description = "value of the CIDR block to use for the VPC" default = "10.0.0.0/16" } variable "private_subnet_cidrs" { type = list(any) description = "The CIDR blocks to use for the private subnets" default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] } variable "public_subnet_cidrs" { type = list(any) description = "The CIDR blocks to use for the public subnets" default = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] } variable "single_nat_gateway" { type = bool description = "Single NAT or per NAT for subnet" default = false } #AWS Info variable "aws_region" { type = string default = "us-east-2" } variable "default_aws_tags" { type = map(string) description = "Default tags for AWS" default = {} } EOF
cat<<-EOF>vpc.tf # Copyright (c) 2023 Red Hat, Inc. # Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at # http://www.apache.org/licenses/LICENSE-2.0 # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. # module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "5.1.2" count = var.create_vpc ? 1 : 0 name = var.vpc_name cidr = var.vpc_cidr_block azs = local.region_azs private_subnets = var.private_subnet_cidrs public_subnets = var.public_subnet_cidrs enable_nat_gateway = true single_nat_gateway = var.single_nat_gateway enable_dns_hostnames = true enable_dns_support = true tags = var.additional_tags } EOF
terraform init
terraform validate
Success! The configuration is valid.
terraform apply
var.create_vpc If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'. Enter a value: var.private_cluster If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'. Enter a value:
Plan: 74 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes
rosa list clusters
ID NAME STATE TOPOLOGY 27c3snjsupa9obua74ba8se5kcj11269 rosa-tf-demo ready Classic (STS)
rosa list account-roles
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION AWS Managed ROSA-demo-ControlPlane-Role Control plane arn:aws:iam::<ID>:role/ROSA-demo-ControlPlane-Role 4.14 No ROSA-demo-Installer-Role Installer arn:aws:iam::<ID>:role/ROSA-demo-Installer-Role 4.14 No ROSA-demo-Support-Role Support arn:aws:iam::<ID>:role/ROSA-demo-Support-Role 4.14 No ROSA-demo-Worker-Role Worker arn:aws:iam::<ID>:role/ROSA-demo-Worker-Role 4.14 No
rosa list operator-roles
I: Fetching operator roles ROLE PREFIX AMOUNT IN BUNDLE rosa-demo 6
terraform destroy
var.create_vpc If you would like to create a new VPC, set this value to 'true.' If you do not want to create a new VPC, set this value to 'false.' Enter a value: var.private_cluster If you want to create a private cluster, set this value to 'true.' If you want a publicly available cluster, set this value to 'false.' Enter a value:
Plan: 0 to add, 0 to change, 74 to destroy. Do you really want to destroy all resources? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes
rosa list clusters
I: No clusters available
rosa list account-roles
I: Fetching account roles I: No account roles available
rosa list operator-roles
I: Fetching operator roles I: No operator roles available
rosa create cluster --private-link --cluster-name=<cluster-name> [--machine-cidr=<VPC CIDR>/16] --subnet-ids=<private-subnet-id>
rosa create cluster --private-link --multi-az --cluster-name=<cluster-name> [--machine-cidr=<VPC CIDR>/16] --subnet-ids=<private-subnet-id1>,<private-subnet-id2>,<private-subnet-id3>
rosa describe cluster --cluster=<cluster_name>
rosa logs install --cluster=<cluster_name> --watch
cat <<EOF > /tmp/shared-vpc-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ChangeResourceRecordSets", "route53:ListHostedZones", "route53:ListHostedZonesByName", "route53:ListResourceRecordSets", "route53:ChangeTagsForResource", "route53:GetAccountLimit", "route53:GetChange", "route53:GetHostedZone", "route53:ListTagsForResource", "route53:UpdateHostedZoneComment", "tag:GetResources", "tag:UntagResources" ], "Resource": "*" } ] } EOF
aws iam create-policy --policy-name SharedVPCPolicy --policy-document file:///tmp/shared-vpc-policy.json
cat <<EOF > /tmp/shared-vpc-role.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<Account-ID>:root" 1 }, "Action": "sts:AssumeRole" } ] } EOF
aws iam create-role --role-name <role_name> \ 1 --assume-role-policy-document file:///tmp/shared-vpc-role.json
aws iam attach-role-policy --role-name <role_name> --policy-arn \ 1 arn:aws:iam::<AWS_account_ID>:policy/SharedVPCPolicy 2
rosa create dns-domain
I: DNS domain '14eo.p1.openshiftapps.com' has been created. I: To view all DNS domains, run 'rosa list dns-domains'
rosa create oidc-config
I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 25tu67hq45rto1am3slpf5lq6jargg
rosa create operator-roles --oidc-config-id <oidc-config-ID> 1 --installer-role-arn <Installer_Role> 2 --shared-vpc-role-arn <Created_VPC_Role_Arn> 3 --prefix <operator-prefix> 4
{ "Version": "2012-10-17", "Statement": [ { "Sid": "Statement1", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-ingress-operator-cloud-credentials", "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-Installer-Role" ] }, "Action": "sts:AssumeRole" } ] }
rosa create cluster --cluster-name <cluster_name> --sts --operator-roles-prefix <prefix> --oidc-config-id <oidc_config_id> --region us-east-1 --subnet-ids <subnet_ids> --private-hosted-zone-id <hosted_zone_ID> --shared-vpc-role-arn <vpc-role-arn> --base-domain <dns-domain>
rosa create admin --cluster=<cluster_name>
W: It is recommended to add an identity provider to login to this cluster. See 'rosa create idp --help' for more information. I: Admin account has been added to cluster 'cluster_name'. It may take up to a minute for the account to become active. I: To login, run the following command: login https://api.cluster-name.t6k4.i1.organization.org:6443 \ 1 --username cluster-admin --password FWGYL-2mkJI-3ZTTZ-rINns
oc login https://api.cluster_name.t6k4.i1.organization.org:6443 \ 1 > --username cluster-admin > --password FWGYL-2mkJI-3ZTTZ-rINns Login successful. You have access to 77 projects, the list has been suppressed. You can list all projects with 'projects'
oc whoami
cluster-admin
rosa create idp --cluster=<cluster_name> --interactive
I: Interactive mode enabled. Any optional fields can be left empty and a default will be selected. ? Type of identity provider: github ? Restrict to members of: organizations ? GitHub organizations: rh-test-org ? To use GitHub as an identity provider, you must first register the application: - Open the following URL: https://github.com/organizations/rh-rosa-test-cluster/settings/applications/new?oauth_application%5Bcallback_url%5D=https%3A%2F%2Foauth-openshift.apps.rh-rosa-test-cluster.z7v0.s1.devshift.org%2Foauth2callback%2Fgithub-1&oauth_application%5Bname%5D=rh-rosa-test-cluster-stage&oauth_application%5Burl%5D=https%3A%2F%2Fconsole-openshift-console.apps.rh-rosa-test-cluster.z7v0.s1.devshift.org - Click on 'Register application'
? Client ID: <my_github_client_id> ? Client Secret: [? for help] <my_github_client_secret> ? Hostname: ? Mapping method: claim I: Configuring IDP for cluster 'rh_rosa_test_cluster' I: Identity Provider 'github-1' has been created. You need to ensure that there is a list of cluster administrators defined. See 'rosa create user --help' for more information. To login into the console, open https://console-openshift-console.apps.rh-test-org.z7v0.s1.devshift.org and click on github-1
rosa list idps --cluster=<cluster_name>
NAME TYPE AUTH URL github-1 GitHub https://oauth-openshift.apps.rh-rosa-test-cluster1.j9n4.s1.devshift.org/oauth2callback/github-1
rosa describe cluster --cluster=<cluster_name>
Name: rh-rosa-test-cluster1 ID: 1de87g7c30g75qechgh7l5b2bha6r04e External ID: 34322be7-b2a7-45c2-af39-2c684ce624e1 API URL: https://api.rh-rosa-test-cluster1.j9n4.s1.devshift.org:6443 1 Console URL: https://console-openshift-console.apps.rh-rosa-test-cluster1.j9n4.s1.devshift.org Nodes: Master: 3, Infra: 3, Compute: 4 Region: us-east-2 State: ready Created: May 27, 2020
oc login --token=z3sgOGVDk0k4vbqo_wFqBQQTnT-nA-nQLb8XEmWnw4X --server=https://api.rh-rosa-test-cluster1.j9n4.s1.devshift.org:6443 1
Logged into "https://api.rh-rosa-cluster1.j9n4.s1.devshift.org:6443" as "rh-rosa-test-user" using the token provided. 1 You have access to 67 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default".
oc version
Client Version: 4.4.0-202005231254-4a4cd75 Server Version: 4.3.18 Kubernetes Version: v1.16.2
rosa grant user cluster-admin --user=<idp_user_name> --cluster=<cluster_name>
rosa list users --cluster=<cluster_name>
GROUP NAME cluster-admins rh-rosa-test-user dedicated-admins rh-rosa-test-user
oc get all -n openshift-apiserver
NAME READY STATUS RESTARTS AGE pod/apiserver-6ndg2 1/1 Running 0 17h pod/apiserver-lrmxs 1/1 Running 0 17h pod/apiserver-tsqhz 1/1 Running 0 17h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/api ClusterIP 172.30.23.241 <none> 443/TCP 18h NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/apiserver 3 3 3 3 3 node-role.kubernetes.io/master= 18h
rosa grant user dedicated-admin --user=<idp_user_name> --cluster=<cluster_name>
oc get groups dedicated-admins
NAME USERS dedicated-admins rh-rosa-test-user
https://oauth-openshift.apps.<cluster_name>.<cluster_domain>/oauth2callback/<idp_provider_name>
https://oauth-openshift.apps.openshift-cluster.example.com/oauth2callback/github
https://oauth-openshift.apps.<cluster_name>.<cluster_domain>/oauth2callback/<idp_provider_name>
https://oauth-openshift.apps.openshift-cluster.example.com/oauth2callback/gitlab
https://oauth-openshift.apps.<cluster_name>.<cluster_domain>/oauth2callback/<idp_provider_name>
https://oauth-openshift.apps.openshift-cluster.example.com/oauth2callback/google
ldap://host:port/basedn?attribute?scope?filter
(&(<filter>)(<attribute>=<username>))
https://oauth-openshift.apps.<cluster_name>.<cluster_domain>/oauth2callback/<idp_provider_name>
https://oauth-openshift.apps.openshift-cluster.example.com/oauth2callback/openid
rosa revoke user dedicated-admin --user=<idp_user_name> --cluster=<cluster_name>
oc get groups dedicated-admins
rosa revoke user cluster-admins --user=myusername --cluster=mycluster
oc get groups cluster-admins
rosa describe cluster --cluster=<cluster_name> 1
Name: mycluster ID: 1s3v4x39lhs8sm49m90mi0822o34544a 1 Operator IAM Roles: 2 - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-machine-api-aws-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-credential-operator-cloud-crede - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-image-registry-installer-cloud-creden - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cluster-csi-drivers-ebs-cloud-credent - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-network-config-controller-cloud State: ready Private: No Created: May 13 2022 11:26:15 UTC Details Page: https://console.redhat.com/openshift/details/s/296kyEFwzoy1CREQicFRdZybrc0 OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/<oidc_config_id> 3
rosa delete cluster --cluster=<cluster_name> --watch
rosa delete oidc-provider -c <cluster_id> --mode auto 1
rosa delete operator-roles -c <cluster_id> --mode auto 1
rosa list account-roles
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION ManagedOpenShift-ControlPlane-Role Control plane arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role 4.10 ManagedOpenShift-Installer-Role Installer arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role 4.10 ManagedOpenShift-Support-Role Support arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role 4.10 ManagedOpenShift-Worker-Role Worker arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role 4.10
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION AWS Managed ManagedOpenShift-HCP-ROSA-Installer-Role Installer arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Installer-Role 4.18 Yes ManagedOpenShift-HCP-ROSA-Support-Role Support arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Support-Role 4.18 Yes ManagedOpenShift-HCP-ROSA-Worker-Role Worker arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Worker-Role 4.18 Yes
rosa delete account-roles --prefix <prefix> --mode auto 1
W: There are no classic account roles to be deleted I: Deleting hosted CP account roles ? Delete the account role 'delete-rosa-HCP-ROSA-Installer-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Installer-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Support-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Support-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Worker-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Worker-Role' I: Successfully deleted the hosted CP account roles
rosa list ocm-roles
I: Fetching ocm roles ROLE NAME ROLE ARN LINKED ADMIN AWS Managed ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> Yes Yes Yes
rosa unlink ocm-role --role-arn <arn> 1
I: Unlinking OCM role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' role from organization '<red_hat_organization_id>'? Yes I: Successfully unlinked role-arn 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' from organization account '<red_hat_organization_id>'
rosa delete ocm-role --role-arn <arn>
I: Deleting OCM role ? OCM Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> ? Delete 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' ocm role? Yes ? OCM role deletion mode: auto 1 I: Successfully deleted the OCM role
rosa list user-roles
I: Fetching user roles ROLE NAME ROLE ARN LINKED ManagedOpenShift-User-<ocm_user_name>-Role arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role Yes
rosa unlink user-role --role-arn <arn> 1
I: Unlinking user role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the current account '<ocm_user_account_id>'? Yes I: Successfully unlinked role ARN 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' from account '<ocm_user_account_id>'
rosa delete user-role --role-arn <arn>
I: Deleting user role ? User Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role ? Delete the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the AWS account? Yes ? User role deletion mode: auto 1 I: Successfully deleted the user role
{ "Version": "2012-10-17", "Statement": [ { "Action": "*", "Resource": "*", "Effect": "Allow" } ] }
aws sts get-caller-identity --output text
<aws_account_id> arn:aws:iam::<aws_account_id>:user/<username> <aws_user_id>
mv rosa /usr/local/bin/rosa
rosa
Command line tool for Red Hat OpenShift Service on AWS. For further documentation visit https://access.redhat.com/documentation/en-us/red_hat_openshift_service_on_aws Usage: rosa [command] Available Commands: completion Generates completion scripts create Create a resource from stdin delete Delete a specific resource describe Show details of a specific resource download Download necessary tools for using your cluster edit Edit a specific resource grant Grant role to a specific resource help Help about any command init Applies templates to support Red Hat OpenShift Service on AWS install Installs a resource into a cluster link Link a ocm/user role from stdin list List all resources of a specific type login Log in to your Red Hat account logout Log out logs Show installation or uninstallation logs for a cluster revoke Revoke role from a specific resource uninstall Uninstalls a resource from a cluster unlink UnLink a ocm/user role from stdin upgrade Upgrade a resource verify Verify resources are configured correctly for cluster install version Prints the version of the tool whoami Displays user account information Flags: --color string Surround certain characters with escape sequences to display them in color on the terminal. Allowed options are [auto never always] (default "auto") --debug Enable debug mode. -h, --help help for rosa Use "rosa [command] --help" for more information about a command.
rosa completion bash | sudo tee /etc/bash_completion.d/rosa
source /etc/bash_completion.d/rosa
rosa login
To login to your Red Hat account, get an offline access token at https://console.redhat.com/openshift/token/rosa ? Copy the token and paste it here: <my-offline-access-token>
I: Logged in as 'rh-rosa-user' on 'https://api.openshift.com'
rosa verify permissions
I: Validating SCP policies I: AWS SCP policies ok
rosa verify quota --region=us-west-2
I: Validating AWS quota I: AWS quota ok
rosa whoami
AWS Account ID: 000000000000 AWS Default Region: us-east-2 AWS ARN: arn:aws:iam::000000000000:user/hello OCM API: https://api.openshift.com OCM Account ID: 1DzGIdIhqEWyt8UUXQhSoWaaaaa OCM Account Name: Your Name OCM Account Username: you@domain.com OCM Account Email: you@domain.com OCM Organization ID: 1HopHfA2hcmhup5gCr2uH5aaaaa OCM Organization Name: Red Hat OCM Organization External ID: 0000000
rosa init
I: Logged in as 'rh-rosa-user' on 'https://api.openshift.com' I: Validating AWS credentials I: AWS credentials are valid! I: Validating SCP policies I: AWS SCP policies ok I: Validating AWS quota I: AWS quota ok I: Ensuring cluster administrator user 'osdCcsAdmin' I: Admin user 'osdCcsAdmin' created successfully! I: Verifying whether OpenShift command-line tool is available E: OpenShift command-line tool is not installed. Run 'rosa download oc' to download the latest version, then add it to your PATH.
rosa download oc
rosa verify oc
rosa create cluster --cluster-name=<cluster_name>
I: Creating cluster with identifier '1de87g7c30g75qechgh7l5b2bha6r04e' and name 'rh-rosa-test-cluster1' I: To view list of clusters and their status, run `rosa list clusters` I: Cluster 'rh-rosa-test-cluster1' has been created. I: Once the cluster is 'Ready' you will need to add an Identity Provider and define the list of cluster administrators. See `rosa create idp --help` and `rosa create user --help` for more information. I: To determine when your cluster is Ready, run `rosa describe cluster rh-rosa-test-cluster1`.
rosa create cluster --interactive
rosa describe cluster --cluster=<cluster_name>
Name: rh-rosa-test-cluster1 OpenShift Version: 4.6.8 DNS: *.example.com ID: uniqueidnumber External ID: uniqueexternalidnumber AWS Account: 123456789101 API URL: https://api.rh-rosa-test-cluster1.example.org:6443 Console URL: https://console-openshift-console.apps.rh-rosa-test-cluster1.example.or Nodes: Master: 3, Infra: 2, Compute: 2 Region: us-west-2 Multi-AZ: false State: ready Channel Group: stable Private: No Created: Jan 15 2021 16:30:55 UTC Details Page: https://console.redhat.com/examplename/details/idnumber
rosa logs install --cluster=<cluster_name> --watch
rosa create cluster --cluster-name=<cluster_name> --private
rosa edit cluster --cluster=<cluster_name> --private
rosa revoke user dedicated-admin --user=<idp_user_name> --cluster=<cluster_name>
oc get groups dedicated-admins
rosa revoke user cluster-admins --user=myusername --cluster=mycluster
oc get groups cluster-admins
rosa describe cluster --cluster=<cluster_name> 1
Name: mycluster ID: 1s3v4x39lhs8sm49m90mi0822o34544a 1 Operator IAM Roles: 2 - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-machine-api-aws-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-credential-operator-cloud-crede - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-image-registry-installer-cloud-creden - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cluster-csi-drivers-ebs-cloud-credent - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-network-config-controller-cloud State: ready Private: No Created: May 13 2022 11:26:15 UTC Details Page: https://console.redhat.com/openshift/details/s/296kyEFwzoy1CREQicFRdZybrc0 OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/<oidc_config_id> 3
rosa delete cluster --cluster=<cluster_name> --watch
rosa delete oidc-provider -c <cluster_id> --mode auto 1
rosa delete operator-roles -c <cluster_id> --mode auto 1
## Configures your AWS account and ensures everything is setup correctly rosa init ## Starts the cluster creation process (~30-40minutes) rosa create cluster --cluster-name=<cluster_name> ## Connect your IDP to your cluster rosa create idp --cluster=<cluster_name> --interactive ## Promotes a user from your IDP to dedicated-admin level rosa grant user dedicated-admin --user=<idp_user_name> --cluster=<cluster_name> ## Checks if your install is ready (look for State: Ready), ## and provides your Console URL to login to the web console. rosa describe cluster --cluster=<cluster_name>