Chapter 6. Configuring a shared VPC for ROSA clusters You can create Red Hat OpenShift Service on AWS (ROSA) clusters in shared, centrally-managed AWS virtual private clouds (VPCs). Important Sharing VPCs across multiple AWS accounts is currently only supported for ROSA Classic clusters using STS for authentication. Note This process requires two separate AWS accounts that belong to the same AWS organization. One account functions as the VPC-owning AWS account ( VPC Owner ), while the other account creates the cluster in the cluster-creating AWS account ( Cluster Creator ). Prerequisites for the VPC Owner You have an AWS account with the proper permissions to create roles and share resources. The Cluster Creator's AWS account is separate from the VPC Owner's AWS account. Both AWS accounts belong to the same AWS organization. You enabled resource sharing from the management account for your organization. You have access to the AWS console . Prerequisites for the Cluster Creator You installed the ROSA CLI ( rosa ) 1.2.26 or later. You created all of the required account-wide roles and policies for creating a cluster. The Cluster Creator's AWS account is separate from the VPC Owner's AWS account. Both AWS accounts belong to the same AWS organization. Note Installing a cluster in a shared VPC is supported only for OpenShift 4.12.34 and later, 4.13.10 and later, and all future 4.y-streams. 6.1. Step One - VPC Owner: Configuring a VPC to share within your AWS organization You can share subnets within a configured VPC with another AWS user account if that account is within your current AWS organization. Procedure Create or modify a VPC to your specifications in the VPC section of the AWS console . Create a custom policy file to allow for necessary shared VPC permissions that uses the name SharedVPCPolicy : USD cat <<EOF > /tmp/shared-vpc-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ChangeResourceRecordSets", "route53:ListHostedZones", "route53:ListHostedZonesByName", "route53:ListResourceRecordSets", "route53:ChangeTagsForResource", "route53:GetAccountLimit", "route53:GetChange", "route53:GetHostedZone", "route53:ListTagsForResource", "route53:UpdateHostedZoneComment", "tag:GetResources", "tag:UntagResources" ], "Resource": "*" } ] } EOF Create the policy in AWS: USD aws iam create-policy \ --policy-name SharedVPCPolicy \ --policy-document file:///tmp/shared-vpc-policy.json You will attach this policy to a role necessary for the shared VPC permissions. Create a custom trust policy file that grants permission to assume roles: USD cat <<EOF > /tmp/shared-vpc-role.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<Account-ID>:root" 1 }, "Action": "sts:AssumeRole" } ] } EOF 1 The principal will be scoped down after the Cluster Creator creates the necessary cluster roles. On creation, you must create a root user placeholder by using the Cluster Creator's AWS account ID as arn:aws:iam::{Account}:root . Create the IAM role: USD aws iam create-role --role-name <role_name> \ 1 --assume-role-policy-document file:///tmp/shared-vpc-role.json 1 Replace <role_name> with the name of the role you want to create. Attach the custom SharedVPCPolicy permissions policy: USD aws iam attach-role-policy --role-name <role_name> --policy-arn \ 1 arn:aws:iam::<AWS_account_ID>:policy/SharedVPCPolicy 2 1 Replace <role_name> with the name of the role you created. 2 Replace <AWS_account_ID> with the VPC Owner's AWS account ID. Provide the SharedVPCRole ARN to the Cluster Creator to continue configuration. Additional resources See the AWS documentation for sharing your AWS resources . 6.2. Step Two - Cluster Creator: Reserving your DNS and creating cluster operator roles After the VPC Owner creates a virtual private cloud, subnets, and an IAM role for sharing the VPC resources, reserve an openshiftapps.com DNS domain and create Operator roles to communicate back to the VPC Owner . Note For shared VPC clusters, you can choose to create the Operator roles after the cluster creation steps. The cluster will be in a waiting state until the Ingress Operator role ARN is added to the shared VPC role trusted relationships. Prerequisites You have the SharedVPCRole ARN for the IAM role from the VPC Owner . Procedure Reserve an openshiftapps.com DNS domain with the following command: USD rosa create dns-domain The command creates a reserved openshiftapps.com DNS domain. I: DNS domain '14eo.p1.openshiftapps.com' has been created. I: To view all DNS domains, run 'rosa list dns-domains' Create an OIDC configuration. Review this article for more information on the OIDC configuration process . The following command produces the OIDC configuration ID that you need: USD rosa create oidc-config You receive confirmation that the command created an OIDC configuration: I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 25tu67hq45rto1am3slpf5lq6jargg Create the Operator roles by entering the following command: USD rosa create operator-roles --oidc-config-id <oidc-config-ID> 1 --installer-role-arn <Installer_Role> 2 --shared-vpc-role-arn <Created_VPC_Role_Arn> 3 --prefix <operator-prefix> 4 1 Provide the OIDC configuration ID that you created in the step. 2 Provide your installer ARN that was created as part of the rosa create account-roles process. 3 Provide the ARN for the role that the VPC Owner created. 4 Provide a prefix for the Operator roles. Note The Installer account role and the shared VPC role must have a one-to-one relationship. If you want to create multiple shared VPC roles, you should create one set of account roles per shared VPC role. After you create the Operator roles, share the full domain name, which is created with <intended_cluster_domain_prefix>.<reserved_dns_domain> , your Ingress Operator Cloud Credentials role's ARN, and your Installer role's ARN with the VPC Owner to continue configuration. The shared information resembles these examples: my-rosa-cluster.14eo.p1.openshiftapps.com arn:aws:iam::111122223333:role/ManagedOpenShift-Installer-Role arn:aws:iam::111122223333:role/my-rosa-cluster-openshift-ingress-operator-cloud-credentials 6.3. Step Three - VPC Owner: Updating the shared VPC role and creating hosted zones After the Cluster Creator provides the DNS domain and the IAM roles, create a private hosted zone and update the trust policy on the IAM role that was created for sharing the VPC. Prerequisites You have the full domain name from the Cluster Creator . You have the Ingress Operator Cloud Credentials role's ARN from the Cluster Creator . You have the Installer role's ARN from the Cluster Creator . Procedure In the Resource Access Manager of the AWS console , create a resource share that shares the previously created public and private subnets with the Cluster Creator's AWS account ID. Update the VPC sharing IAM role and add the Installer and Ingress Operator Cloud Credentials roles to the principal section of the trust policy. { "Version": "2012-10-17", "Statement": [ { "Sid": "Statement1", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-ingress-operator-cloud-credentials", "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-Installer-Role" ] }, "Action": "sts:AssumeRole" } ] } Create a private hosted zone in the Route 53 section of the AWS console . In the hosted zone configuration, the domain name is <cluster_domain_prefix>.<reserved_dns_domain> . The private hosted zone must be associated with the created VPC. After the hosted zone is created and associated with the VPC, provide the following to the Cluster Creator to continue configuration: Hosted zone ID AWS region Subnet IDs 6.4. Step Four - Cluster Creator: Creating your cluster in a shared VPC To create a cluster in a shared VPC, complete the following steps. Note Installing a cluster in a shared VPC is supported only for OpenShift 4.12.34 and later, 4.13.10 and later, and all future 4.y-streams. Prerequisites You have the hosted zone ID from the VPC Owner . You have the AWS region from the VPC Owner . You have the subnet IDs from the VPC Owner . You have the SharedVPCRole ARN from the VPC Owner . Procedure In a terminal, enter the following command to create the shared VPC: rosa create cluster --cluster-name <cluster_name> --sts --operator-roles-prefix <prefix> --oidc-config-id <oidc_config_id> --region us-east-1 --subnet-ids <subnet_ids> --private-hosted-zone-id <hosted_zone_ID> --shared-vpc-role-arn <vpc-role-arn> --base-domain <dns-domain> Note If your cluster name is longer than 15 characters, it will contain an autogenerated domain prefix as a sub-domain for your provisioned cluster on *.openshiftapps.com . To customize the subdomain, use the --domain-prefix flag. The domain prefix cannot be longer than 15 characters, must be unique, and cannot be changed after cluster creation.

Commands:
cat <<EOF > /tmp/shared-vpc-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ChangeResourceRecordSets", "route53:ListHostedZones", "route53:ListHostedZonesByName", "route53:ListResourceRecordSets", "route53:ChangeTagsForResource", "route53:GetAccountLimit", "route53:GetChange", "route53:GetHostedZone", "route53:ListTagsForResource", "route53:UpdateHostedZoneComment", "tag:GetResources", "tag:UntagResources" ], "Resource": "*" } ] } EOF
aws iam create-policy --policy-name SharedVPCPolicy --policy-document file:///tmp/shared-vpc-policy.json
cat <<EOF > /tmp/shared-vpc-role.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<Account-ID>:root" 1 }, "Action": "sts:AssumeRole" } ] } EOF
aws iam create-role --role-name <role_name> \ 1 --assume-role-policy-document file:///tmp/shared-vpc-role.json
aws iam attach-role-policy --role-name <role_name> --policy-arn \ 1 arn:aws:iam::<AWS_account_ID>:policy/SharedVPCPolicy 2
rosa create dns-domain
I: DNS domain '14eo.p1.openshiftapps.com' has been created. I: To view all DNS domains, run 'rosa list dns-domains'
rosa create oidc-config
I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice: rosa create operator-roles --prefix <user-defined> --oidc-config-id 25tu67hq45rto1am3slpf5lq6jargg
rosa create operator-roles --oidc-config-id <oidc-config-ID> 1 --installer-role-arn <Installer_Role> 2 --shared-vpc-role-arn <Created_VPC_Role_Arn> 3 --prefix <operator-prefix> 4
{ "Version": "2012-10-17", "Statement": [ { "Sid": "Statement1", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-ingress-operator-cloud-credentials", "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-Installer-Role" ] }, "Action": "sts:AssumeRole" } ] }
rosa create cluster --cluster-name <cluster_name> --sts --operator-roles-prefix <prefix> --oidc-config-id <oidc_config_id> --region us-east-1 --subnet-ids <subnet_ids> --private-hosted-zone-id <hosted_zone_ID> --shared-vpc-role-arn <vpc-role-arn> --base-domain <dns-domain>