Chapter 5. Creating a router network To create a network of AMQ Interconnect routers, you define a deployment in an Interconnect Custom Resource, and then apply it. The AMQ Interconnect Operator creates the deployment by scheduling the necessary Pods and creating any needed Resources. The procedures in this section demonstrate the following router network topologies: Interior router mesh Interior router mesh with edge routers for scalability Inter-cluster router network that connects two OpenShift clusters Prerequisites The AMQ Interconnect Operator is installed in your OpenShift Container Platform project. 5.1. Creating an interior router deployment Interior routers establish connections with each other and automatically compute the lowest cost paths across the network. Procedure This procedure creates an interior router network of three routers. The routers automatically connect to each other in a mesh topology, and their connections are secured with mutual SSL/TLS authentication. Create an Interconnect Custom Resource YAML file that describes the interior router deployment. Sample router-mesh.yaml file apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: router-mesh spec: deploymentPlan: role: interior 1 size: 3 2 placement: Any 3 1 The operating mode of the routers in the deployment. The Operator will automatically connect interior routers in a mesh topology. 2 The number of routers to create. 3 Each router runs in a separate Pod. The placement defines where in the cluster the Operator should schedule and place the Pods. You can choose the following placement options: Any The Pods can run on any node in the OpenShift Container Platform cluster. Every The Operator places a router Pod on each node in the cluster. If you choose this option, the Size property is not needed - the number of routers corresponds to the number of nodes in the cluster. AntiAffinity The Operator ensures that multiple router Pods do not run on the same node in the cluster. If the size is greater than the number of nodes in the cluster, the extra Pods that cannot be scheduled will remain in a Pending state. Create the router deployment described in the YAML file. The Operator creates a deployment of interior routers in a mesh topology that uses default address semantics. It also creates a Service through which the routers can be accessed, and a Route through which you can access the web console. Verify that the router mesh was created and the Pods are running. Each router runs in a separate Pod. They connect to each other automatically using the Service that the Operator created. Review the router deployment. 1 The default address configuration. All messages sent to an address that does not match any of these prefixes are distributed in a balanced anycast pattern . 2 A router mesh of three interior routers was deployed. 3 Each interior router listens on port 45672 for connections from edge routers. 4 The interior routers connect to each other on port 55671 . These inter-router connections are secured with SSL/TLS mutual authentication. The inter-router SSL Profile contains the details of the certificates that the Operator generated. 5 Each interior router listens for connections from external clients on the following ports: 5672 - Unsecure connections from messaging applications. 5671 - Secure connections from messaging applications. 8080 - AMQ Interconnect web console access. Default user name/password security is applied. 6 Using the Red Hat Integration - AMQ Certificate Manager Operator, the Red Hat Integration - AMQ Interconnect Operator automatically creates two SSL profiles: inter-router - The Operator secures the inter-router network with mutual TLS authentication by creating a Certificate Authority (CA) and generating certificates signed by the CA for each interior router. default - The Operator creates TLS certificates for messaging applications to connect to the interior routers on port 5671 . 7 The AMQ Interconnect web console is secured with user name/password authentication. The Operator automatically generates the credentials and stores them in the router-mesh-users Secret. 5.2. Creating an edge router deployment You can efficiently scale your router network by adding an edge router deployment. Edge routers act as connection concentrators for messaging applications. Each edge router maintains a single uplink connection to an interior router, and messaging applications connect to the edge routers to send and receive messages. Prerequisites The interior router mesh is deployed. For more information, see Section 5.1, "Creating an interior router deployment" . Procedure This procedure creates an edge router on each node of the OpenShift Container Platform cluster and connects them to the previously created interior router mesh. Create an Interconnect Custom Resource YAML file that describes the edge router deployment. Sample edge-routers.yaml file apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: edge-routers spec: deploymentPlan: role: edge placement: Every 1 edgeConnectors: 2 - host: router-mesh 3 port: 45672 4 1 An edge router Pod will be deployed on each node in the OpenShift Container Platform cluster. This placement helps to balance messaging application traffic across the cluster. The Operator will create a DaemonSet to ensure that the number of Pods scheduled always corresponds to the number of nodes in the cluster. 2 Edge connectors define the connections from the edge routers to the interior routers. 3 The name of the Service that was created for the interior routers. 4 The port on which the interior routers listen for edge connections. The default is 45672 . Create the edge routers described in the YAML file: The Operator deploys an edge router on each node of the OpenShift Container Platform cluster, and connects them to the interior routers. Verify that the edge routers were created and the Pods are running. Each router runs in a separate Pod. Each edge router connects to any of the previously created interior routers. 5.3. Creating an inter-cluster router network Depending on whether you are using AMQ Certificate Manager, there are different procedures for creating an inter-cluster router network. Section 5.3.2, "Creating an inter-cluster router network using AMQ Certificate Manager" Section 5.3.1, "Creating an inter-cluster router network using a Certificate Authority" 5.3.1. Creating an inter-cluster router network using a Certificate Authority You can create a router network from routers running in different OpenShift Container Platform clusters. This enables you to connect applications running in separate clusters. Prerequisites You have already created secrets defining an existing certificate for each router. Procedure This procedure creates router deployments in two different OpenShift Container Platform clusters ( cluster1 and cluster2 ) and connects them together to form an inter-cluster router network. The connection between the router deployments is secured with SSL/TLS mutual authentication. In the first OpenShift Container Platform cluster ( cluster1 ), create an Interconnect Custom Resource YAML file that describes the interior router deployment. This example creates a single interior router with a default configuration. Sample cluster1-router-mesh.yaml file apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster1-router-mesh spec: interRouterListeners: - authenticatePeer: true 1 host: 0.0.0.0 2 port: 55672 3 saslMechanisms: EXTERNAL 4 sslProfile: inter-router-profile 5 expose: true 6 sslProfiles: - caCert: inter-router-certs-secret 7 credentials: inter-router-certs-secret 8 name: inter-router-profile 9 1 authenticatePeer must be set to true to authenticate using TLS certificates 2 listener host 3 listener port 4 SASL mechanism to authenticate, use EXTERNAL for TLS certificates 5 ssl-profile name to use for authenticating clients 6 exposes a route so that the port is accessible from outside the cluster 7 name of cluster secret or your CA containing a ca.crt name (in case you're using the same secret used in credentials, otherwise it must have a tls.crt) 8 name of cluster secret with the CA certificate containing tls.crt and tls.key files 9 ssl-profile name to use for the interRouterListener Create the router deployment described in the YAML file. The Red Hat Integration - AMQ Interconnect Operator creates an interior router with a default configuration and a listener to authenticate other routers. Log in to the second OpenShift Container Platform cluster ( cluster2 ), and switch to the project where you want to create the second router deployment. In cluster2 , create an Interconnect Custom Resource YAML file to describe the router deployment. apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster2-router-mesh spec: sslProfiles: - name: inter-router-profile 1 credentials: inter-router-certs-secret caCert: inter-router-certs-secret interRouterConnectors: - host: cluster1-router-mesh-port-55672-myproject.cluster1.openshift.com 2 port: 443 verifyHostname: false sslProfile: inter-router-profile name: cluster1 1 This SSL Profile defines the certificate needed to connect to the router deployment in cluster1 . 2 The URL of the Route for the inter-router listener on cluster1 . Create the router deployment described in the YAML file. Verify that the routers are connected. This example displays the connections from the router in cluster2 to the router in cluster1 . 5.3.2. Creating an inter-cluster router network using AMQ Certificate Manager You can create a router network from routers running in different OpenShift Container Platform clusters. This enables you to connect applications running in separate clusters. Procedure This procedure creates router deployments in two different OpenShift Container Platform clusters ( cluster1 and cluster2 ) and connects them together to form an inter-cluster router network. The connection between the router deployments is secured with SSL/TLS mutual authentication. In the first OpenShift Container Platform cluster ( cluster1 ), create an Interconnect Custom Resource YAML file that describes the interior router deployment. This example creates a single interior router with a default configuration. Sample cluster1-router-mesh.yaml file apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster1-router-mesh spec: {} Create the router deployment described in the YAML file. The Red Hat Integration - AMQ Interconnect Operator creates an interior router with a default configuration. It uses the Red Hat Integration - AMQ Certificate Manager Operator to create a Certificate Authority (CA) and generate a certificate signed by the CA. Generate an additional certificate for the router deployment in the second OpenShift Container Platform cluster ( cluster2 ). The router deployment in cluster2 requires a certificate issued by the CA of cluster1 . Create a Certificate Custom Resource YAML file to request a certificate. Sample certificate-request.yaml file apiVersion: certmanager.k8s.io/v1alpha1 kind: Certificate metadata: name: cluster2-inter-router-tls spec: commonName: cluster1-router-mesh-myproject.cluster2.openshift.com issuerRef: name: cluster1-router-mesh-inter-router-ca 1 secretName: cluster2-inter-router-tls-secret --- 1 The name of the Issuer that created the inter-router CA for cluster1 . By default, the name of the Issuer is < application-name >-inter-router-ca . Create the certificate described in the YAML file. Extract the certificate that you generated. Log in to the second OpenShift Container Platform cluster ( cluster2 ), and switch to the project where you want to create the second router deployment. In cluster2 , create a Secret containing the certificate that you generated. In cluster2 , create an Interconnect Custom Resource YAML file to describe the router deployment. apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster2-router-mesh spec: sslProfiles: - name: inter-cluster-tls 1 credentials: cluster2-inter-router-tls-secret caCert: cluster2-inter-router-tls-secret interRouterConnectors: - host: cluster1-router-mesh-port-55671-myproject.cluster1.openshift.com 2 port: 443 verifyHostname: false sslProfile: inter-cluster-tls 1 This SSL Profile defines the certificate needed to connect to the router deployment in cluster1 . 2 The URL of the Route for the inter-router listener on cluster1 . Create the router deployment described in the YAML file. Verify that the routers are connected. This example displays the connections from the router in cluster2 to the router in cluster1 .

Commands:
apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: router-mesh spec: deploymentPlan: role: interior 1 size: 3 2 placement: Any 3
oc apply -f router-mesh.yaml
oc get pods NAME READY STATUS RESTARTS AGE interconnect-operator-587f94784b-4bzdx 1/1 Running 0 52m router-mesh-6b48f89bd-588r5 1/1 Running 0 40m router-mesh-6b48f89bd-bdjc4 1/1 Running 0 40m router-mesh-6b48f89bd-h6d5r 1/1 Running 0 40m
oc get interconnect/router-mesh -o yaml apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect spec: addresses: 1 - distribution: closest prefix: closest - distribution: multicast prefix: multicast - distribution: closest prefix: unicast - distribution: closest prefix: exclusive - distribution: multicast prefix: broadcast deploymentPlan: 2 livenessPort: 8888 placement: Any resources: {} role: interior size: 3 edgeListeners: 3 - port: 45672 interRouterListeners: 4 - authenticatePeer: true expose: true port: 55671 saslMechanisms: EXTERNAL sslProfile: inter-router listeners: 5 - port: 5672 - authenticatePeer: true expose: true http: true port: 8080 - port: 5671 sslProfile: default sslProfiles: 6 - credentials: router-mesh-default-tls name: default - caCert: router-mesh-inter-router-tls credentials: router-mesh-inter-router-tls mutualAuth: true name: inter-router users: router-mesh-users 7
apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: edge-routers spec: deploymentPlan: role: edge placement: Every 1 edgeConnectors: 2 - host: router-mesh 3 port: 45672 4
oc apply -f edge-routers.yaml
oc get pods NAME READY STATUS RESTARTS AGE edge-routers-2jz5j 1/1 Running 0 33s edge-routers-fhlxv 1/1 Running 0 33s edge-routers-gg2qb 1/1 Running 0 33s edge-routers-hj72t 1/1 Running 0 33s interconnect-operator-587f94784b-4bzdx 1/1 Running 0 54m router-mesh-6b48f89bd-588r5 1/1 Running 0 42m router-mesh-6b48f89bd-bdjc4 1/1 Running 0 42m router-mesh-6b48f89bd-h6d5r 1/1 Running 0 42m
apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster1-router-mesh spec: interRouterListeners: - authenticatePeer: true 1 host: 0.0.0.0 2 port: 55672 3 saslMechanisms: EXTERNAL 4 sslProfile: inter-router-profile 5 expose: true 6 sslProfiles: - caCert: inter-router-certs-secret 7 credentials: inter-router-certs-secret 8 name: inter-router-profile 9
oc apply -f cluster1-router-mesh.yaml
apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster2-router-mesh spec: sslProfiles: - name: inter-router-profile 1 credentials: inter-router-certs-secret caCert: inter-router-certs-secret interRouterConnectors: - host: cluster1-router-mesh-port-55672-myproject.cluster1.openshift.com 2 port: 443 verifyHostname: false sslProfile: inter-router-profile name: cluster1
oc apply -f cluster2-router-mesh.yaml
oc exec cluster2-fb6bc5797-crvb6 -it -- qdstat -c Connections id host container role dir security authentication tenant ==================================================================================================================================================================================================== 1 cluster1-router-mesh-port-55672-myproject.cluster1.openshift.com:443 cluster1-router-mesh-54cffd9967-9h4vq inter-router out TLSv1/SSLv3(DHE-RSA-AES256-GCM-SHA384) x.509
apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster1-router-mesh spec: {}
oc apply -f cluster1-router-mesh.yaml
apiVersion: certmanager.k8s.io/v1alpha1 kind: Certificate metadata: name: cluster2-inter-router-tls spec: commonName: cluster1-router-mesh-myproject.cluster2.openshift.com issuerRef: name: cluster1-router-mesh-inter-router-ca 1 secretName: cluster2-inter-router-tls-secret ---
oc apply -f certificate-request.yaml
mkdir /tmp/cluster2-inter-router-tls oc extract secret/cluster2-inter-router-tls-secret --to=/tmp/cluster2-inter-router-tls
oc create secret generic cluster2-inter-router-tls-secret --from-file=/tmp/cluster2-inter-router-tls
apiVersion: interconnectedcloud.github.io/v1alpha1 kind: Interconnect metadata: name: cluster2-router-mesh spec: sslProfiles: - name: inter-cluster-tls 1 credentials: cluster2-inter-router-tls-secret caCert: cluster2-inter-router-tls-secret interRouterConnectors: - host: cluster1-router-mesh-port-55671-myproject.cluster1.openshift.com 2 port: 443 verifyHostname: false sslProfile: inter-cluster-tls
oc apply -f cluster2-router-mesh.yaml
oc exec cluster2-fb6bc5797-crvb6 -it -- qdstat -c Connections id host container role dir security authentication tenant ==================================================================================================================================================================================================== 1 cluster1-router-mesh-port-55671-myproject.cluster1.openshift.com:443 cluster1-router-mesh-54cffd9967-9h4vq inter-router out TLSv1/SSLv3(DHE-RSA-AES256-GCM-SHA384) x.509