Chapter 3. Connectivity Link platform engineer workflow This section of the walkthrough shows how as a platform engineer you can deploy a Gateway that provides secure communication and is protected and ready for use by application development teams. It also shows how to use this Gateway in multiple clusters in different geographic regions. Prerequisites See Chapter 2, Check your Connectivity Link installation and permissions . Note In multicluster environments, you must perform the following steps in each cluster individually, unless specifically excluded. 3.1. Step 1 - Set your environment variables Procedure Set the following environment variables, which are used for convenience in this guide: In this example, zid is your hosted zone ID displayed in the AWS Route 53 console. rootDomain is the top-level AWS Route 53 domain name that you will use for Connectivity Link. Note This guide uses environment variables for convenience only. If you know the environment variable values, you can set up the required .yaml files in way that suits your needs. 3.2. Step 2 - Set up a DNS provider secret The DNS provider supplies a credential to access the DNS zones that Connectivity Link can use to set up DNS configuration. You must ensure that this credential has access to only the zones that you want managed. Note You must apply the following Secret resources to each cluster. If you are adding an additional cluster, add them to the new cluster. Procedure If your Gateway namespace does not already exist, create it as follows: If the secret for your DNS provider credentials was not already created when installing Connectivity Link, create this secret in your Gateway namespace as follows: Before adding a TLS issuer, you must also create the credentials secret in the cert-manager namespace as follows: 3.3. Step 3 - Add a TLS issuer To secure communication to the Gateways, you will define a TLS issuer for TLS certificates. This example uses Let's Encrypt, but you can use any certificate issuer supported by cert-manager . Procedure Enter the following command to define a TLS issuer. This example uses Let's Encrypt, which you must also apply to all clusters: Wait for the ClusterIssuer to become ready as follows: 3.4. Step 4 - Set up a Gateway For Connectivity Link to balance traffic using DNS across two or more clusters, you must define a Gateway with a shared host. You will define this by using an HTTPS listener with a wildcard hostname based on the root domain. As mentioned earlier, you must apply these resources to all clusters. Note For now, the Gateway is set to accept an HTTPRoute from the same namespace only. This allows you to restrict who can use the Gateway until it is ready for general use. Procedure Enter the following command to create the Gateway: Check the status of your Gateway as follows: Your Gateway should be accepted and programmed, which means valid and assigned an external address. However, if you check your HTTPS listener status as follows, you will see that it is not yet programmed or ready to accept traffic due to bad TLS configuration: Connectivity Link can help with this by using a TLSPolicy, which is described in the step. 3.4.1. Optional: Configure metrics to be scraped from the Gateway instance If you have Prometheus set up in your cluster, you can configure a PodMonitor to scrape metrics directly from the Gateway pod. This configuration is required for metrics such as istio_requests_total . You must add the following configuration in the namespace where the Gateway is running: For more information on configuring metrics, see the Connectivity Link Observability Guide . 3.5. Step 5 - Configure your Gateway policies and HTTP route While your Gateway is now deployed, it has no exposed endpoints and your HTTPS listener is not programmed. , you can set up a TLSPolicy that leverages your CertificateIssuer to set up your HTTPS listener certificates. You will define an AuthPolicy that will set up a default HTTP 403 response for any unprotected endpoints, as well as a RateLimitPolicy that will set up a default artificially low global limit to further protect any endpoints exposed by this Gateway. You will also define a DNSPolicy with a load balancing strategy, and an HTTPRoute for your Gateway to communicate with your backend application API. 3.5.1. Set the TLS policy Procedure Set the TLSPolicy for your Gateway as follows: Check that your TLS policy was accepted by the controller as follows: 3.5.2. Set the Auth policy Procedure Set a default, deny-all AuthPolicy for your Gateway as follows: Check that your auth policy was accepted by the controller as follows: 3.5.3. Set the rate limit policy Procedure Set the default RateLimitPolicy for your Gateway as follows: Note It might take a few minutes for the RateLimitPolicy to be applied depending on your cluster. The limit in this example is artificially low to show it working easily. To check your rate limits have been accepted, enter the following command: 3.5.4. Set the DNS policy Procedure Set the DNSPolicy for your Gateway as follows: Note The DNSPolicy will use the DNS Provider Secret that you defined earlier. The geo in this example is EU , but you can change this to suit your requirements. Check that your DNSPolicy has been accepted as follows: 3.5.5. Create an HTTP route Note For test purposes, this section assumes that the toystore application is deployed. For more information, see Chapter 4, Connectivity Link application developer workflow . Procedure Create an HTTPRoute to test your Gateway as follows: Check your Gateway policies are enforced as follows: Check your HTTPS listener is ready as follows: 3.6. Step 6 - Test connectivity and deny all auth You can use curl to test your endpoint connectivity and auth. Procedure Enter the following command: You should see an HTTP 403 response. 3.7. Step 7 - Open up the Gateway for other namespaces Because you have configured the Gateway, secured it with Connectivity Link policies, and tested it, you can now open it up for use by other teams in other namespaces. Procedure Enter the following command: 3.8. Step 8 - Extend the Gateway to multiple clusters and configure geo-based routing Procedure To distribute this Gateway across multiple clusters, repeat this setup process for each cluster. By default, this will implement a round-robin DNS strategy to distribute traffic evenly across the different clusters. Setting up your Gateways to serve clients based on their geographic location is straightforward with your current configuration. Assuming that you have deployed Gateway instances across multiple clusters as per this guide, the step involves updating the DNS controller with the geographic regions of the visible Gateways. For instance, if you have one cluster in North America and another in the EU, you can direct traffic to these Gateways based on their location by configuring the appropriate policy. For your North American cluster, you can create a DNSPolicy and set the loadBalancing:geo field to US .

Commands:
export zid=change-to-your-DNS-zone-ID export rootDomain=demo.example.com export gatewayNS=api-gateway export gatewayName=external export devNS=toystore export AWS_ACCESS_KEY_ID=xxxx export AWS_SECRET_ACCESS_KEY=xxxx export AWS_REGION=us-east-1 export clusterIssuerName=lets-encrypt export EMAIL=foo@example.com
create ns USD{gatewayNS}
-n USD{gatewayNS} create secret generic aws-credentials --type=kuadrant.io/aws --from-literal=AWS_ACCESS_KEY_ID=USDAWS_ACCESS_KEY_ID --from-literal=AWS_SECRET_ACCESS_KEY=USDAWS_SECRET_ACCESS_KEY
-n cert-manager create secret generic aws-credentials --type=kuadrant.io/aws --from-literal=AWS_ACCESS_KEY_ID=USDAWS_ACCESS_KEY_ID --from-literal=AWS_SECRET_ACCESS_KEY=USDAWS_SECRET_ACCESS_KEY
apply -f - <<EOF apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: USD{clusterIssuerName} spec: acme: email: USD{EMAIL} privateKeySecretRef: name: le-secret server: https://acme-v02.api.letsencrypt.org/directory solvers: - dns01: route53: hostedZoneID: USD{zid} region: USD{AWS_REGION} accessKeyIDSecretRef: key: AWS_ACCESS_KEY_ID name: aws-credentials secretAccessKeySecretRef: key: AWS_SECRET_ACCESS_KEY name: aws-credentials EOF
wait clusterissuer/USD{clusterIssuerName} --for=condition=ready=true
apply -f - <<EOF apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: USD{gatewayName} namespace: USD{gatewayNS} labels: kuadrant.io/gateway: "true" spec: gatewayClassName: istio listeners: - allowedRoutes: namespaces: from: Same hostname: "*.USD{rootDomain}" name: api port: 443 protocol: HTTPS tls: certificateRefs: - group: "" kind: Secret name: api-USD{gatewayName}-tls mode: Terminate EOF
get gateway USD{gatewayName} -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}' get gateway USD{gatewayName} -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Programmed")].message}'
get gateway USD{gatewayName} -n USD{gatewayNS} -o=jsonpath='{.status.listeners[0].conditions[?(@.type=="Programmed")].message}'
apply -f - <<EOF apiVersion: monitoring.coreos.com/v1 kind: PodMonitor metadata: name: istio-proxies-monitor namespace: USD{gatewayNS} spec: selector: matchExpressions: - key: istio-prometheus-ignore operator: DoesNotExist podMetricsEndpoints: - path: /stats/prometheus interval: 30s relabelings: - action: keep sourceLabels: ["__meta_kubernetes_pod_container_name"] regex: "istio-proxy" - action: keep sourceLabels: ["__meta_kubernetes_pod_annotationpresent_prometheus_io_scrape"] - action: replace regex: (\d+);(([A-Fa-f0-9]{1,4}::?){1,7}[A-Fa-f0-9]{1,4}) replacement: "[\USD2]:\USD1" sourceLabels: [ "__meta_kubernetes_pod_annotation_prometheus_io_port", "__meta_kubernetes_pod_ip", ] targetLabel: "__address__" - action: replace regex: (\d+);((([0-9]+?)(\.|USD)){4}) replacement: "\USD2:\USD1" sourceLabels: [ "__meta_kubernetes_pod_annotation_prometheus_io_port", "__meta_kubernetes_pod_ip", ] targetLabel: "__address__" - action: labeldrop regex: "__meta_kubernetes_pod_label_(.+)" - sourceLabels: ["__meta_kubernetes_namespace"] action: replace targetLabel: namespace - sourceLabels: ["__meta_kubernetes_pod_name"] action: replace targetLabel: pod_name EOF
apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: TLSPolicy metadata: name: USD{gatewayName}-tls namespace: USD{gatewayNS} spec: targetRef: name: USD{gatewayName} group: gateway.networking.k8s.io kind: Gateway issuerRef: group: cert-manager.io kind: ClusterIssuer name: USD{clusterIssuerName} EOF
get tlspolicy USD{gatewayName}-tls -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}'
apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: AuthPolicy metadata: name: USD{gatewayName}-auth namespace: USD{gatewayNS} spec: targetRef: group: gateway.networking.k8s.io kind: Gateway name: USD{gatewayName} defaults: rules: authorization: "deny": opa: rego: "allow = false" EOF
get authpolicy USD{gatewayName}-auth -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}'
apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: RateLimitPolicy metadata: name: USD{gatewayName}-rlp namespace: USD{gatewayNS} spec: targetRef: group: gateway.networking.k8s.io kind: Gateway name: USD{gatewayName} defaults: limits: "low-limit": rates: - limit: 2 window: 10s EOF
get ratelimitpolicy USD{gatewayName}-rlp -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}'
apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: DNSPolicy metadata: name: USD{gatewayName}-dnspolicy namespace: USD{gatewayNS} spec: targetRef: name: USD{gatewayName} group: gateway.networking.k8s.io kind: Gateway providerRefs: - name: aws-credentials loadBalancing: weight: 120 geo: EU defaultGeo: true EOF
get dnspolicy USD{gatewayName}-dnspolicy -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}'
apply -f - <<EOF apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: test namespace: USD{gatewayNS} labels: service: toystore spec: parentRefs: - name: USD{gatewayName} namespace: USD{gatewayNS} hostnames: - "test.USD{rootDomain}" rules: - backendRefs: - name: toystore port: 80 EOF
get dnspolicy USD{gatewayName}-dnspolicy -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Enforced")].message}' get authpolicy USD{gatewayName}-auth -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Enforced")].message}' get ratelimitpolicy USD{gatewayName}-rlp -n USD{gatewayNS} -o=jsonpath='{.status.conditions[?(@.type=="Enforced")].message}'
get gateway USD{gatewayName} -n USD{gatewayNS} -o=jsonpath='{.status.listeners[0].conditions[?(@.type=="Programmed")].message}'
curl -w "%{http_code}" https://USD(kubectl get httproute test -n USD{gatewayNS} -o=jsonpath='{.spec.hostnames[0]}')
patch gateway USD{gatewayName} -n USD{gatewayNS} --type='json' -p='[{"op": "replace", "path": "/spec/listeners/0/allowedRoutes/namespaces/from", "value":"All"}]'