Chapter 12. Tutorial: Using AWS Controllers for Kubernetes on ROSA AWS Controllers for Kubernetes (ACK) lets you define and use AWS service resources directly from Red Hat OpenShift Service on AWS (ROSA). With ACK, you can take advantage of AWS-managed services for your applications without needing to define resources outside of the cluster or run services that provide supporting capabilities such as databases or message queues within the cluster. You can install various ACK Operators directly from OperatorHub. This makes it easy to get started and use the Operators with your applications. This controller is a component of the AWS Controller for Kubernetes project, which is currently in developer preview. Use this tutorial to deploy the ACK S3 Operator. You can also adapt it for any other ACK Operator in the OperatorHub of your cluster. 12.1. Prerequisites A ROSA cluster A user account with cluster-admin privileges The OpenShift CLI ( oc ) The Amazon Web Services (AWS) CLI ( aws ) 12.2. Setting up your environment Configure the following environment variables, changing the cluster name to suit your cluster: USD export CLUSTER_NAME=USD(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}USD//') USD export REGION=USD(rosa describe cluster -c USD{ROSA_CLUSTER_NAME} --output json | jq -r .region.id) USD export OIDC_ENDPOINT=USD(oc get authentication.config.openshift.io cluster -o json | jq -r .spec.serviceAccountIssuer | sed 's|^https://||') USD export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text` USD export ACK_SERVICE=s3 USD export ACK_SERVICE_ACCOUNT=ack-USD{ACK_SERVICE}-controller USD export POLICY_ARN=arn:aws:iam::aws:policy/AmazonS3FullAccess USD export AWS_PAGER="" USD export SCRATCH="/tmp/USD{ROSA_CLUSTER_NAME}/ack" USD mkdir -p USD{SCRATCH} Ensure all fields output correctly before moving to the section: USD echo "Cluster: USD{ROSA_CLUSTER_NAME}, Region: USD{REGION}, OIDC Endpoint: USD{OIDC_ENDPOINT}, AWS Account ID: USD{AWS_ACCOUNT_ID}" 12.3. Preparing your AWS Account Create an AWS Identity Access Management (IAM) trust policy for the ACK Operator: USD cat <<EOF > "USD{SCRATCH}/trust-policy.json" { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Condition": { "StringEquals" : { "USD{OIDC_ENDPOINT}:sub": "system:serviceaccount:ack-system:USD{ACK_SERVICE_ACCOUNT}" } }, "Principal": { "Federated": "arn:aws:iam::USDAWS_ACCOUNT_ID:oidc-provider/USD{OIDC_ENDPOINT}" }, "Action": "sts:AssumeRoleWithWebIdentity" } ] } EOF Create an AWS IAM role for the ACK Operator to assume with the AmazonS3FullAccess policy attached: Note You can find the recommended policy in each project's GitHub repository, for example https://github.com/aws-controllers-k8s/s3-controller/blob/main/config/iam/recommended-policy-arn . USD ROLE_ARN=USD(aws iam create-role --role-name "ack-USD{ACK_SERVICE}-controller" \ --assume-role-policy-document "file://USD{SCRATCH}/trust-policy.json" \ --query Role.Arn --output text) USD echo USDROLE_ARN USD aws iam attach-role-policy --role-name "ack-USD{ACK_SERVICE}-controller" \ --policy-arn USD{POLICY_ARN} 12.4. Installing the ACK S3 Controller Create a project to install the ACK S3 Operator into: USD oc new-project ack-system Create a file with the ACK S3 Operator configuration: Note ACK_WATCH_NAMESPACE is purposefully left blank so the controller can properly watch all namespaces in the cluster. USD cat <<EOF > "USD{SCRATCH}/config.txt" ACK_ENABLE_DEVELOPMENT_LOGGING=true ACK_LOG_LEVEL=debug ACK_WATCH_NAMESPACE= AWS_REGION=USD{REGION} AWS_ENDPOINT_URL= ACK_RESOURCE_TAGS=USD{CLUSTER_NAME} ENABLE_LEADER_ELECTION=true LEADER_ELECTION_NAMESPACE= EOF Use the file from the step to create a ConfigMap: USD oc -n ack-system create configmap \ --from-env-file=USD{SCRATCH}/config.txt ack-USD{ACK_SERVICE}-user-config Install the ACK S3 Operator from OperatorHub: USD cat << EOF | oc apply -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: ack-USD{ACK_SERVICE}-controller namespace: ack-system spec: upgradeStrategy: Default --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: ack-USD{ACK_SERVICE}-controller namespace: ack-system spec: channel: alpha installPlanApproval: Automatic name: ack-USD{ACK_SERVICE}-controller source: community-operators sourceNamespace: openshift-marketplace EOF Annotate the ACK S3 Operator service account with the AWS IAM role to assume and restart the deployment: USD oc -n ack-system annotate serviceaccount USD{ACK_SERVICE_ACCOUNT} \ eks.amazonaws.com/role-arn=USD{ROLE_ARN} && \ oc -n ack-system rollout restart deployment ack-USD{ACK_SERVICE}-controller Verify that the ACK S3 Operator is running: USD oc -n ack-system get pods Example output NAME READY STATUS RESTARTS AGE ack-s3-controller-585f6775db-s4lfz 1/1 Running 0 51s 12.5. Validating the deployment Deploy an S3 bucket resource: USD cat << EOF | oc apply -f - apiVersion: s3.services.k8s.aws/v1alpha1 kind: Bucket metadata: name: USD{CLUSTER-NAME}-bucket namespace: ack-system spec: name: USD{CLUSTER-NAME}-bucket EOF Verify the S3 bucket was created in AWS: USD aws s3 ls | grep USD{CLUSTER_NAME}-bucket Example output 2023-10-04 14:51:45 mrmc-test-maz-bucket 12.6. Cleaning up Delete the S3 bucket resource: USD oc -n ack-system delete bucket.s3.services.k8s.aws/USD{CLUSTER-NAME}-bucket Delete the ACK S3 Operator and the AWS IAM roles: USD oc -n ack-system delete subscription ack-USD{ACK_SERVICE}-controller USD aws iam detach-role-policy \ --role-name "ack-USD{ACK_SERVICE}-controller" \ --policy-arn USD{POLICY_ARN} USD aws iam delete-role \ --role-name "ack-USD{ACK_SERVICE}-controller" Delete the ack-system project: USD oc delete project ack-system

Commands:
export CLUSTER_NAME=USD(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}USD//') export REGION=USD(rosa describe cluster -c USD{ROSA_CLUSTER_NAME} --output json | jq -r .region.id) export OIDC_ENDPOINT=USD(oc get authentication.config.openshift.io cluster -o json | jq -r .spec.serviceAccountIssuer | sed 's|^https://||') export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text` export ACK_SERVICE=s3 export ACK_SERVICE_ACCOUNT=ack-USD{ACK_SERVICE}-controller export POLICY_ARN=arn:aws:iam::aws:policy/AmazonS3FullAccess export AWS_PAGER="" export SCRATCH="/tmp/USD{ROSA_CLUSTER_NAME}/ack" mkdir -p USD{SCRATCH}
echo "Cluster: USD{ROSA_CLUSTER_NAME}, Region: USD{REGION}, OIDC Endpoint: USD{OIDC_ENDPOINT}, AWS Account ID: USD{AWS_ACCOUNT_ID}"
cat <<EOF > "USD{SCRATCH}/trust-policy.json" { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Condition": { "StringEquals" : { "USD{OIDC_ENDPOINT}:sub": "system:serviceaccount:ack-system:USD{ACK_SERVICE_ACCOUNT}" } }, "Principal": { "Federated": "arn:aws:iam::USDAWS_ACCOUNT_ID:oidc-provider/USD{OIDC_ENDPOINT}" }, "Action": "sts:AssumeRoleWithWebIdentity" } ] } EOF
ROLE_ARN=USD(aws iam create-role --role-name "ack-USD{ACK_SERVICE}-controller" --assume-role-policy-document "file://USD{SCRATCH}/trust-policy.json" --query Role.Arn --output text) echo USDROLE_ARN aws iam attach-role-policy --role-name "ack-USD{ACK_SERVICE}-controller" --policy-arn USD{POLICY_ARN}
oc new-project ack-system
cat <<EOF > "USD{SCRATCH}/config.txt" ACK_ENABLE_DEVELOPMENT_LOGGING=true ACK_LOG_LEVEL=debug ACK_WATCH_NAMESPACE= AWS_REGION=USD{REGION} AWS_ENDPOINT_URL= ACK_RESOURCE_TAGS=USD{CLUSTER_NAME} ENABLE_LEADER_ELECTION=true LEADER_ELECTION_NAMESPACE= EOF
oc -n ack-system create configmap --from-env-file=USD{SCRATCH}/config.txt ack-USD{ACK_SERVICE}-user-config
cat << EOF | oc apply -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: ack-USD{ACK_SERVICE}-controller namespace: ack-system spec: upgradeStrategy: Default --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: ack-USD{ACK_SERVICE}-controller namespace: ack-system spec: channel: alpha installPlanApproval: Automatic name: ack-USD{ACK_SERVICE}-controller source: community-operators sourceNamespace: openshift-marketplace EOF
oc -n ack-system annotate serviceaccount USD{ACK_SERVICE_ACCOUNT} eks.amazonaws.com/role-arn=USD{ROLE_ARN} && oc -n ack-system rollout restart deployment ack-USD{ACK_SERVICE}-controller
oc -n ack-system get pods
NAME READY STATUS RESTARTS AGE ack-s3-controller-585f6775db-s4lfz 1/1 Running 0 51s
cat << EOF | oc apply -f - apiVersion: s3.services.k8s.aws/v1alpha1 kind: Bucket metadata: name: USD{CLUSTER-NAME}-bucket namespace: ack-system spec: name: USD{CLUSTER-NAME}-bucket EOF
aws s3 ls | grep USD{CLUSTER_NAME}-bucket
2023-10-04 14:51:45 mrmc-test-maz-bucket
oc -n ack-system delete bucket.s3.services.k8s.aws/USD{CLUSTER-NAME}-bucket
oc -n ack-system delete subscription ack-USD{ACK_SERVICE}-controller aws iam detach-role-policy --role-name "ack-USD{ACK_SERVICE}-controller" --policy-arn USD{POLICY_ARN} aws iam delete-role --role-name "ack-USD{ACK_SERVICE}-controller"
oc delete project ack-system