Chapter 6. Tutorial: Using AWS WAF and Amazon CloudFront to protect ROSA workloads AWS WAF is a web application firewall that lets you monitor the HTTP and HTTPS requests that are forwarded to your protected web application resources. You can use an Amazon CloudFront to add a Web Application Firewall (WAF) to your Red Hat OpenShift Service on AWS (ROSA) workloads. Using an external solution protects ROSA resources from experiencing denial of service due to handling the WAF. 6.1. Prerequisites A ROSA (HCP or Classic) cluster. You have access to the OpenShift CLI ( oc ). You have access to the AWS CLI ( aws ). 6.1.1. Environment setup Prepare the environment variables: USD export DOMAIN=apps.example.com 1 USD export AWS_PAGER="" USD export CLUSTER_NAME=USD(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}USD//') USD export REGION=USD(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}") USD export AWS_ACCOUNT_ID=USD(aws sts get-caller-identity --query Account --output text) USD export SCRATCH="/tmp/USD{CLUSTER}/cloudfront-waf" USD mkdir -p USD{SCRATCH} USD echo "Cluster: USD{CLUSTER}, Region: USD{REGION}, AWS Account ID: USD{AWS_ACCOUNT_ID}" 1 Replace with the custom domain you want to use for the IngressController . Note The "Cluster" output from the command might be the name of your cluster, the internal ID of your cluster, or the cluster's domain prefix. If you prefer to use another identifier, you can manually set this value by running the following command: USD export CLUSTER=my-custom-value 6.2. Setting up the secondary ingress controller It is necessary to configure a secondary ingress controller to segment your external WAF-protected traffic from your standard (and default) cluster ingress controller. Prerequisites A publicly trusted SAN or wildcard certificate for your custom domain, such as CN=*.apps.example.com Important Amazon CloudFront uses HTTPS to communicate with your cluster's secondary ingress controller. As explained in the Amazon CloudFront documentation , you cannot use a self-signed certificate for HTTPS communication between CloudFront and your cluster. Amazon CloudFront verifies that the certificate was issued by a trusted certificate authority. Procedure Create a new TLS secret from a private key and a public certificate, where fullchain.pem is your full wildcard certificate chain (including any intermediaries) and privkey.pem is your wildcard certificate's private key. Example USD oc -n openshift-ingress create secret tls waf-tls --cert=fullchain.pem --key=privkey.pem Create a new IngressController resource: Example waf-ingress-controller.yaml apiVersion: operator.openshift.io/v1 kind: IngressController metadata: name: cloudfront-waf namespace: openshift-ingress-operator spec: domain: apps.example.com 1 defaultCertificate: name: waf-tls endpointPublishingStrategy: loadBalancer: dnsManagementPolicy: Unmanaged providerParameters: aws: type: NLB type: AWS scope: External type: LoadBalancerService routeSelector: 2 matchLabels: route: waf 1 Replace with the custom domain you want to use for the IngressController . 2 Filters the set of routes serviced by the Ingress Controller. In this tutorial, we will use the waf route selector, but if no value was to be provided, no filtering would occur. Apply the IngressController : Example USD oc apply -f waf-ingress-controller.yaml Verify that your IngressController has successfully created an external load balancer: USD oc -n openshift-ingress get service/router-cloudfront-waf Example output NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE router-cloudfront-waf LoadBalancer 172.30.16.141 a68a838a7f26440bf8647809b61c4bc8-4225395f488830bd.elb.us-east-1.amazonaws.com 80:30606/TCP,443:31065/TCP 2m19s 6.2.1. Configure the AWS WAF The AWS WAF service is a web application firewall that lets you monitor, protect, and control the HTTP and HTTPS requests that are forwarded to your protected web application resources, like ROSA. Create a AWS WAF rules file to apply to our web ACL: USD cat << EOF > USD{SCRATCH}/waf-rules.json [ { "Name": "AWS-AWSManagedRulesCommonRuleSet", "Priority": 0, "Statement": { "ManagedRuleGroupStatement": { "VendorName": "AWS", "Name": "AWSManagedRulesCommonRuleSet" } }, "OverrideAction": { "None": {} }, "VisibilityConfig": { "SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "AWS-AWSManagedRulesCommonRuleSet" } }, { "Name": "AWS-AWSManagedRulesSQLiRuleSet", "Priority": 1, "Statement": { "ManagedRuleGroupStatement": { "VendorName": "AWS", "Name": "AWSManagedRulesSQLiRuleSet" } }, "OverrideAction": { "None": {} }, "VisibilityConfig": { "SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "AWS-AWSManagedRulesSQLiRuleSet" } } ] EOF This will enable the Core (Common) and SQL AWS Managed Rule Sets. Create an AWS WAF Web ACL using the rules we specified above: USD WAF_WACL=USD(aws wafv2 create-web-acl \ --name cloudfront-waf \ --region USD{REGION} \ --default-action Allow={} \ --scope CLOUDFRONT \ --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=USD{CLUSTER}-waf-metrics \ --rules file://USD{SCRATCH}/waf-rules.json \ --query 'Summary.Name' \ --output text) 6.3. Configure Amazon CloudFront Retrieve the newly created custom ingress controller's NLB hostname: USD NLB=USD(oc -n openshift-ingress get service router-cloudfront-waf \ -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') Import your certificate into Amazon Certificate Manager, where cert.pem is your wildcard certificate, fullchain.pem is your wildcard certificate's chain and privkey.pem is your wildcard certificate's private key. Note Regardless of what region your cluster is deployed, you must import this certificate to us-east-1 as Amazon CloudFront is a global AWS service. Example USD aws acm import-certificate --certificate file://cert.pem \ --certificate-chain file://fullchain.pem \ --private-key file://privkey.pem \ --region us-east-1 Log into the AWS console to create a CloudFront distribution. Configure the CloudFront distribution by using the following information: Note If an option is not specified in the table below, leave them the default (which may be blank). Option Value Origin domain Output from the command [1] Name rosa-waf-ingress [2] Viewer protocol policy Redirect HTTP to HTTPS Allowed HTTP methods GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE Cache policy CachingDisabled Origin request policy AllViewer Web Application Firewall (WAF) Enable security protections Use existing WAF configuration true Choose a web ACL cloudfront-waf Alternate domain name (CNAME) *.apps.example.com [3] Custom SSL certificate Select the certificate you imported from the step above [4] Run echo USD{NLB} to get the origin domain. If you have multiple clusters, ensure the origin name is unique. This should match the wildcard domain you used to create the custom ingress controller. This should match the alternate domain name entered above. Retrieve the Amazon CloudFront Distribution endpoint: USD aws cloudfront list-distributions --query "DistributionList.Items[?Origins.Items[?DomainName=='USD{NLB}']].DomainName" --output text Update the DNS of your custom wildcard domain with a CNAME to the Amazon CloudFront Distribution endpoint from the step above. Example *.apps.example.com CNAME d1b2c3d4e5f6g7.cloudfront.net 6.4. Deploy a sample application Create a new project for your sample application by running the following command: USD oc new-project hello-world Deploy a hello world application: USD oc -n hello-world new-app --image=docker.io/openshift/hello-openshift Create a route for the application specifying your custom domain name: Example USD oc -n hello-world create route edge --service=hello-openshift hello-openshift-tls \ --hostname hello-openshift.USD{DOMAIN} Label the route to admit it to your custom ingress controller: USD oc -n hello-world label route.route.openshift.io/hello-openshift-tls route=waf 6.5. Test the WAF Test that the app is accessible behind Amazon CloudFront: Example USD curl "https://hello-openshift.USD{DOMAIN}" Example output Hello OpenShift! Test that the WAF denies a bad request: Example USD curl -X POST "https://hello-openshift.USD{DOMAIN}" \ -F "user='<script><alert>Hello></alert></script>'" Example output <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"> <TITLE>ERROR: The request could not be satisfied</TITLE> </HEAD><BODY> <H1>403 ERROR</H1> <H2>The request could not be satisfied.</H2> <HR noshade size="1px"> Request blocked. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner. <BR clear="all"> If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation. <BR clear="all"> <HR noshade size="1px"> <PRE> Generated by cloudfront (CloudFront) Request ID: nFk9q2yB8jddI6FZOTjdliexzx-FwZtr8xUQUNT75HThPlrALDxbag== </PRE> <ADDRESS> </ADDRESS> </BODY></HTML> The expected result is a 403 ERROR , which means the AWS WAF is protecting your application. 6.6. Additional resources Adding Extra Security with AWS WAF, CloudFront and ROSA | Amazon Web Services on YouTube

Commands:
export DOMAIN=apps.example.com 1 export AWS_PAGER="" export CLUSTER_NAME=USD(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}USD//') export REGION=USD(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}") export AWS_ACCOUNT_ID=USD(aws sts get-caller-identity --query Account --output text) export SCRATCH="/tmp/USD{CLUSTER}/cloudfront-waf" mkdir -p USD{SCRATCH} echo "Cluster: USD{CLUSTER}, Region: USD{REGION}, AWS Account ID: USD{AWS_ACCOUNT_ID}"
export CLUSTER=my-custom-value
oc -n openshift-ingress create secret tls waf-tls --cert=fullchain.pem --key=privkey.pem
apiVersion: operator.openshift.io/v1 kind: IngressController metadata: name: cloudfront-waf namespace: openshift-ingress-operator spec: domain: apps.example.com 1 defaultCertificate: name: waf-tls endpointPublishingStrategy: loadBalancer: dnsManagementPolicy: Unmanaged providerParameters: aws: type: NLB type: AWS scope: External type: LoadBalancerService routeSelector: 2 matchLabels: route: waf
oc apply -f waf-ingress-controller.yaml
oc -n openshift-ingress get service/router-cloudfront-waf
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE router-cloudfront-waf LoadBalancer 172.30.16.141 a68a838a7f26440bf8647809b61c4bc8-4225395f488830bd.elb.us-east-1.amazonaws.com 80:30606/TCP,443:31065/TCP 2m19s
cat << EOF > USD{SCRATCH}/waf-rules.json [ { "Name": "AWS-AWSManagedRulesCommonRuleSet", "Priority": 0, "Statement": { "ManagedRuleGroupStatement": { "VendorName": "AWS", "Name": "AWSManagedRulesCommonRuleSet" } }, "OverrideAction": { "None": {} }, "VisibilityConfig": { "SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "AWS-AWSManagedRulesCommonRuleSet" } }, { "Name": "AWS-AWSManagedRulesSQLiRuleSet", "Priority": 1, "Statement": { "ManagedRuleGroupStatement": { "VendorName": "AWS", "Name": "AWSManagedRulesSQLiRuleSet" } }, "OverrideAction": { "None": {} }, "VisibilityConfig": { "SampledRequestsEnabled": true, "CloudWatchMetricsEnabled": true, "MetricName": "AWS-AWSManagedRulesSQLiRuleSet" } } ] EOF
WAF_WACL=USD(aws wafv2 create-web-acl --name cloudfront-waf --region USD{REGION} --default-action Allow={} --scope CLOUDFRONT --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=USD{CLUSTER}-waf-metrics --rules file://USD{SCRATCH}/waf-rules.json --query 'Summary.Name' --output text)
NLB=USD(oc -n openshift-ingress get service router-cloudfront-waf -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
aws acm import-certificate --certificate file://cert.pem --certificate-chain file://fullchain.pem --private-key file://privkey.pem --region us-east-1
aws cloudfront list-distributions --query "DistributionList.Items[?Origins.Items[?DomainName=='USD{NLB}']].DomainName" --output text
*.apps.example.com CNAME d1b2c3d4e5f6g7.cloudfront.net
oc new-project hello-world
oc -n hello-world new-app --image=docker.io/openshift/hello-openshift
oc -n hello-world create route edge --service=hello-openshift hello-openshift-tls --hostname hello-openshift.USD{DOMAIN}
oc -n hello-world label route.route.openshift.io/hello-openshift-tls route=waf
curl "https://hello-openshift.USD{DOMAIN}"
Hello OpenShift!
curl -X POST "https://hello-openshift.USD{DOMAIN}" -F "user='<script><alert>Hello></alert></script>'"
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"> <TITLE>ERROR: The request could not be satisfied</TITLE> </HEAD><BODY> <H1>403 ERROR</H1> <H2>The request could not be satisfied.</H2> <HR noshade size="1px"> Request blocked. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner. <BR clear="all"> If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation. <BR clear="all"> <HR noshade size="1px"> <PRE> Generated by cloudfront (CloudFront) Request ID: nFk9q2yB8jddI6FZOTjdliexzx-FwZtr8xUQUNT75HThPlrALDxbag== </PRE> <ADDRESS> </ADDRESS> </BODY></HTML>