Chapter 13. Tutorial: Deploying the External DNS Operator on ROSA The External DNS Operator deploys and manages ExternalDNS to provide the name resolution for services and routes from the external DNS provider, like Amazon Route 53, to Red Hat OpenShift Service on AWS (ROSA) clusters. In this tutorial, we will deploy and configure the External DNS Operator with a secondary ingress controller to manage DNS records in Amazon Route 53. Important The External DNS Operator does not support STS using IAM Roles for Service Accounts (IRSA) and uses long-lived Identity Access Management (IAM) credentials instead. This tutorial will be updated when the Operator supports STS. 13.1. Prerequisites A ROSA Classic cluster Note ROSA with HCP is not supported at this time. A user account with cluster-admin privileges The OpenShift CLI ( oc ) The Amazon Web Services (AWS) CLI ( aws ) A unique domain, such as apps.example.com An Amazon Route 53 public hosted zone for the above domain 13.2. Setting up your environment Configure the following environment variables: USD export DOMAIN=<apps.example.com> 1 USD export AWS_PAGER="" USD export CLUSTER=USD(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}USD//') USD export REGION=USD(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}") USD export AWS_ACCOUNT_ID=USD(aws sts get-caller-identity --query Account --output text) USD export SCRATCH="/tmp/USD{CLUSTER}/external-dns" USD mkdir -p USD{SCRATCH} 1 Replace with the custom domain you want to use for the IngressController . Ensure all fields output correctly before moving to the section: USD echo "Cluster: USD{CLUSTER}, Region: USD{REGION}, AWS Account ID: USD{AWS_ACCOUNT_ID}" Note The "Cluster" output from the command may be the name of your cluster, the internal ID of your cluster, or the cluster's domain prefix. If you prefer to use another identifier, you can manually set this value by running the following command: USD export CLUSTER=my-custom-value 13.3. Secondary ingress controller setup Use the following procedure to deploy a secondary ingress controller using a custom domain. Prerequisites A unique domain, such as apps.example.com A wildcard or SAN TLS certificate configured with the custom domain selected above ( CN=*.apps.example.com ) Procedure Create a new TLS secret from a private key and a public certificate, where fullchain.pem is your full wildcard certificate chain (including any intermediaries) and privkey.pem is your wildcard certificate's private key: USD oc -n openshift-ingress create secret tls external-dns-tls --cert=fullchain.pem --key=privkey.pem Create a new IngressController resource: USD cat << EOF | oc apply -f - apiVersion: operator.openshift.io/v1 kind: IngressController metadata: name: external-dns-ingress namespace: openshift-ingress-operator spec: domain: USD{DOMAIN} defaultCertificate: name: external-dns-tls endpointPublishingStrategy: loadBalancer: dnsManagementPolicy: Unmanaged providerParameters: aws: type: NLB type: AWS scope: External type: LoadBalancerService EOF Warning This IngressController example will create an internet accessible Network Load Balancer (NLB) in your AWS account. To provision an internal NLB instead, set the .spec.endpointPublishingStrategy.loadBalancer.scope parameter to Internal before creating the IngressController resource. Verify that your custom domain IngressController has successfully created an external load balancer: USD oc -n openshift-ingress get service/router-external-dns-ingress Example output NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE router-external-dns-ingress LoadBalancer 172.30.71.250 a4838bb991c6748439134ab89f132a43-aeae124077b50c01.elb.us-east-1.amazonaws.com 80:32227/TCP,443:30310/TCP 43s 13.4. Preparing your AWS account Retrieve the Amazon Route 53 public hosted zone ID: USD export ZONE_ID=USD(aws route53 list-hosted-zones-by-name --output json \ --dns-name "USD{DOMAIN}." --query 'HostedZones[0]'.Id --out text | sed 's/\/hostedzone\///') Prepare a document with the necessary DNS changes to enable DNS resolution for the canonical domain of the Ingress Controller: USD NLB_HOST=USD(oc -n openshift-ingress get service/router-external-dns-ingress -ojsonpath="{.status.loadBalancer.ingress[0].hostname}") USD cat << EOF > "USD{SCRATCH}/create-cname.json" { "Comment":"Add CNAME to ingress controller canonical domain", "Changes":[{ "Action":"CREATE", "ResourceRecordSet":{ "Name": "router-external-dns-ingress.USD{DOMAIN}", "Type":"CNAME", "TTL":30, "ResourceRecords":[{ "Value": "USD{NLB_HOST}" }] } }] } EOF The External DNS Operator uses this canonical domain as the target for CNAME records. Submit your changes to Amazon Route 53 for propagation: aws route53 change-resource-record-sets \ --hosted-zone-id USD{ZONE_ID} \ --change-batch file://USD{SCRATCH}/create-cname.json Create an AWS IAM Policy document that allows the External DNS Operator to update only the custom domain public hosted zone: USD cat << EOF > "USD{SCRATCH}/external-dns-policy.json" { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ChangeResourceRecordSets" ], "Resource": [ "arn:aws:route53:::hostedzone/USD{ZONE_ID}" ] }, { "Effect": "Allow", "Action": [ "route53:ListHostedZones", "route53:ListResourceRecordSets" ], "Resource": [ "*" ] } ] } EOF Create an AWS IAM user: USD aws iam create-user --user-name "USD{CLUSTER}-external-dns-operator" Attach the policy: USD aws iam attach-user-policy --user-name "USD{CLUSTER}-external-dns-operator" --policy-arn USDPOLICY_ARN Note This will be changed to STS using IRSA in the future. Create AWS keys for the IAM user: USD SECRET_ACCESS_KEY=USD(aws iam create-access-key --user-name "USD{CLUSTER}-external-dns-operator") Create static credentials: USD cat << EOF > "USD{SCRATCH}/credentials" [default] aws_access_key_id = USD(echo USDSECRET_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId') aws_secret_access_key = USD(echo USDSECRET_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey') EOF 13.5. Installing the External DNS Operator Create a new project: USD oc new-project external-dns-operator Install the External DNS Operator from OperatorHub: USD cat << EOF | oc apply -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: external-dns-group namespace: external-dns-operator spec: targetNamespaces: - external-dns-operator --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: external-dns-operator namespace: external-dns-operator spec: channel: stable-v1.1 installPlanApproval: Automatic name: external-dns-operator source: redhat-operators sourceNamespace: openshift-marketplace EOF Wait until the External DNS Operator is running: USD oc rollout status deploy external-dns-operator --timeout=300s Create a secret from the AWS IAM user credentials: USD oc -n external-dns-operator create secret generic external-dns \ --from-file "USD{SCRATCH}/credentials" Deploy the ExternalDNS controller: USD cat << EOF | oc apply -f - apiVersion: externaldns.olm.openshift.io/v1beta1 kind: ExternalDNS metadata: name: USD{DOMAIN} spec: domains: - filterType: Include matchType: Exact name: USD{DOMAIN} provider: aws: credentials: name: external-dns type: AWS source: openshiftRouteOptions: routerName: external-dns-ingress type: OpenShiftRoute zones: - USD{ZONE_ID} EOF Wait until the controller is running: USD oc rollout status deploy external-dns-USD{DOMAIN} --timeout=300s 13.6. Deploying a sample application Now that the ExternalDNS controller is running, you can deploy a sample application to confirm that the custom domain is configured and trusted when you expose a new route. Create a new project for your sample application: USD oc new-project hello-world Deploy a hello world application: USD oc new-app -n hello-world --image=docker.io/openshift/hello-openshift Create a route for the application specifying your custom domain name: USD oc -n hello-world create route edge --service=hello-openshift hello-openshift-tls \ --hostname hello-openshift.USD{DOMAIN} Check if the DNS record was created automatically by ExternalDNS: Note It can take a few minutes for the record to appear in Amazon Route 53. USD aws route53 list-resource-record-sets --hosted-zone-id USD{ZONE_ID} \ --query "ResourceRecordSets[?Type == 'CNAME']" | grep hello-openshift Optional: You can also view the TXT records that indicate they were created by ExternalDNS: USD aws route53 list-resource-record-sets --hosted-zone-id USD{ZONE_ID} \ --query "ResourceRecordSets[?Type == 'TXT']" | grep USD{DOMAIN} Curl the newly created DNS record to your sample application to verify the hello world application is accessible: USD curl https://hello-openshift.USD{DOMAIN} Example output Hello OpenShift!

Commands:
export DOMAIN=<apps.example.com> 1 export AWS_PAGER="" export CLUSTER=USD(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}USD//') export REGION=USD(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}") export AWS_ACCOUNT_ID=USD(aws sts get-caller-identity --query Account --output text) export SCRATCH="/tmp/USD{CLUSTER}/external-dns" mkdir -p USD{SCRATCH}
echo "Cluster: USD{CLUSTER}, Region: USD{REGION}, AWS Account ID: USD{AWS_ACCOUNT_ID}"
export CLUSTER=my-custom-value
oc -n openshift-ingress create secret tls external-dns-tls --cert=fullchain.pem --key=privkey.pem
cat << EOF | oc apply -f - apiVersion: operator.openshift.io/v1 kind: IngressController metadata: name: external-dns-ingress namespace: openshift-ingress-operator spec: domain: USD{DOMAIN} defaultCertificate: name: external-dns-tls endpointPublishingStrategy: loadBalancer: dnsManagementPolicy: Unmanaged providerParameters: aws: type: NLB type: AWS scope: External type: LoadBalancerService EOF
oc -n openshift-ingress get service/router-external-dns-ingress
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE router-external-dns-ingress LoadBalancer 172.30.71.250 a4838bb991c6748439134ab89f132a43-aeae124077b50c01.elb.us-east-1.amazonaws.com 80:32227/TCP,443:30310/TCP 43s
export ZONE_ID=USD(aws route53 list-hosted-zones-by-name --output json --dns-name "USD{DOMAIN}." --query 'HostedZones[0]'.Id --out text | sed 's/\/hostedzone\///')
NLB_HOST=USD(oc -n openshift-ingress get service/router-external-dns-ingress -ojsonpath="{.status.loadBalancer.ingress[0].hostname}") cat << EOF > "USD{SCRATCH}/create-cname.json" { "Comment":"Add CNAME to ingress controller canonical domain", "Changes":[{ "Action":"CREATE", "ResourceRecordSet":{ "Name": "router-external-dns-ingress.USD{DOMAIN}", "Type":"CNAME", "TTL":30, "ResourceRecords":[{ "Value": "USD{NLB_HOST}" }] } }] } EOF
aws route53 change-resource-record-sets --hosted-zone-id USD{ZONE_ID} --change-batch file://USD{SCRATCH}/create-cname.json
cat << EOF > "USD{SCRATCH}/external-dns-policy.json" { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ChangeResourceRecordSets" ], "Resource": [ "arn:aws:route53:::hostedzone/USD{ZONE_ID}" ] }, { "Effect": "Allow", "Action": [ "route53:ListHostedZones", "route53:ListResourceRecordSets" ], "Resource": [ "*" ] } ] } EOF
aws iam create-user --user-name "USD{CLUSTER}-external-dns-operator"
aws iam attach-user-policy --user-name "USD{CLUSTER}-external-dns-operator" --policy-arn USDPOLICY_ARN
SECRET_ACCESS_KEY=USD(aws iam create-access-key --user-name "USD{CLUSTER}-external-dns-operator")
cat << EOF > "USD{SCRATCH}/credentials" [default] aws_access_key_id = USD(echo USDSECRET_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId') aws_secret_access_key = USD(echo USDSECRET_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey') EOF
oc new-project external-dns-operator
cat << EOF | oc apply -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: external-dns-group namespace: external-dns-operator spec: targetNamespaces: - external-dns-operator --- apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: external-dns-operator namespace: external-dns-operator spec: channel: stable-v1.1 installPlanApproval: Automatic name: external-dns-operator source: redhat-operators sourceNamespace: openshift-marketplace EOF
oc rollout status deploy external-dns-operator --timeout=300s
oc -n external-dns-operator create secret generic external-dns --from-file "USD{SCRATCH}/credentials"
cat << EOF | oc apply -f - apiVersion: externaldns.olm.openshift.io/v1beta1 kind: ExternalDNS metadata: name: USD{DOMAIN} spec: domains: - filterType: Include matchType: Exact name: USD{DOMAIN} provider: aws: credentials: name: external-dns type: AWS source: openshiftRouteOptions: routerName: external-dns-ingress type: OpenShiftRoute zones: - USD{ZONE_ID} EOF
oc rollout status deploy external-dns-USD{DOMAIN} --timeout=300s
oc new-project hello-world
oc new-app -n hello-world --image=docker.io/openshift/hello-openshift
oc -n hello-world create route edge --service=hello-openshift hello-openshift-tls --hostname hello-openshift.USD{DOMAIN}
aws route53 list-resource-record-sets --hosted-zone-id USD{ZONE_ID} --query "ResourceRecordSets[?Type == 'CNAME']" | grep hello-openshift
aws route53 list-resource-record-sets --hosted-zone-id USD{ZONE_ID} --query "ResourceRecordSets[?Type == 'TXT']" | grep USD{DOMAIN}
curl https://hello-openshift.USD{DOMAIN}
Hello OpenShift!