Chapter 4. Configuring Operator-based broker deployments 4.1. How the Operator generates the broker configuration Before you use Custom Resource (CR) instances to configure your broker deployment, you should understand how the Operator generates the broker configuration. When you create an Operator-based broker deployment, a Pod for each broker runs in a StatefulSet in your OpenShift project. An application container for the broker runs within each Pod. The Operator runs a type of container called an Init Container when initializing each Pod. In OpenShift Container Platform, Init Containers are specialized containers that run before application containers. Init Containers can include utilities or setup scripts that are not present in the application image. By default, the AMQ Broker Operator uses a built-in Init Container. The Init Container uses the main CR instance for your deployment to generate the configuration used by each broker application container. If you have specified address settings in the CR, the Operator generates a default configuration and then merges or replaces that configuration with the configuration specified in the CR. This process is described in the section that follows. 4.1.1. How the Operator generates the address settings configuration If you have included an address settings configuration in the main Custom Resource (CR) instance for your deployment, the Operator generates the address settings configuration for each broker as described below. The Operator runs the Init Container before the broker application container. The Init Container generates a default address settings configuration. The default address settings configuration is shown below. <address-settings> <!-- if you define auto-create on certain queues, management has to be auto-create --> <address-setting match="activemq.management#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> </address-setting> <!-- default for catch all --> <address-setting match="#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> </address-setting> <address-settings> If you have also specified an address settings configuration in your Custom Resource (CR) instance, the Init Container processes that configuration and converts it to XML. Based on the value of the applyRule property in the CR, the Init Container merges or replaces the default address settings configuration shown above with the configuration that you have specified in the CR. The result of this merge or replacement is the final address settings configuration that the broker will use. When the Init Container has finished generating the broker configuration (including address settings), the broker application container starts. When starting, the broker container copies its configuration from the installation directory previously used by the Init Container. You can inspect the address settings configuration in the broker.xml configuration file. For a running broker, this file is located in the /home/jboss/amq-broker/etc directory. Additional resources For an example of using the applyRule property in a CR, see Section 4.2.2, "Configuring address settings" . 4.1.2. Directory structure of a broker Pod When you create an Operator-based broker deployment, a Pod for each broker runs in a StatefulSet in your OpenShift project. An application container for the broker runs within each Pod. The Operator runs a type of container called an Init Container when initializing each Pod. In OpenShift Container Platform, Init Containers are specialized containers that run before application containers. Init Containers can include utilities or setup scripts that are not present in the application image. When generating the configuration for a broker instance, the Init Container uses files contained in a default installation directory. This installation directory is on a volume that the Operator mounts to the broker Pod and which the Init Container and broker container share. The path that the Init Container uses to mount the shared volume is defined in an environment variable called CONFIG_INSTANCE_DIR . The default value of CONFIG_INSTANCE_DIR is /amq/init/config . In the documentation, this directory is referred to as <install_dir> . Note You cannot change the value of the CONFIG_INSTANCE_DIR environment variable. By default, the installation directory has the following sub-directories: Table 4.1. Pod directories Sub-directory Contents <install_dir> /bin Binaries and scripts needed to run the broker. <install_dir> /etc Configuration files. <install_dir> /data The broker journal. <install_dir> /lib JARs and libraries needed to run the broker. <install_dir> /log Broker log files. <install_dir> /tmp Temporary web application files. When the Init Container has finished generating the broker configuration, the broker application container starts. When starting, the broker container copies its configuration from the installation directory previously used by the Init Container. When the broker Pod is initialized and running, the broker configuration is located in the /home/jboss/amq-broker directory (and subdirectories) of the broker. Additional resources For more information about how the Operator chooses a container image for the built-in Init Container, see Section 2.7, "How the Operator chooses container images" . To learn how to build and specify a custom Init Container image, see Section 4.11, "Specifying a custom Init Container image" . 4.2. Configuring addresses and queues for Operator-based broker deployments 4.2.1. Configuring addresses and queues You can configure addresses and queues by using the brokerProperties attribute in the ActiveMQArtemis CR instance for your broker deployment. Or, you can configure addresses and queues in the ActiveMQArtemisAddress CR. Note The ActiveMQArtemisAddress CR is deprecated in AMQ Broker 7.12. Configuring addresses and queues by using brokerProperties You can configure addresses and queues under the brokerProperties attribute and also configure settings for each queue that you create. Prerequisites You created a basic broker deployment. For more information, see Section 3.4.1, "Deploying a basic broker instance" . Procedure Edit the ActiveMQArtemis CR instance for your broker deployment. In the spec section of the CR, add a brokerProperties attribute if it is not already in the CR. spec: ... brokerProperties: ... Configure an address in the format: - addressConfigurations.< address name >.routingTypes=< routing type > For example: spec: ... brokerProperties: - addressConfigurations.usa-news-address.routingTypes=MULTICAST ... Configure a queue for the address you created in the format: - addressConfigurations.< address name >.queueConfigs.< queue name >.address< address > Important The value of < address > for the .address setting must match the < address name > for each queue you create. If these values are different, separate addresses are created for each. In the following example, both the address name and the .address setting have the same value of usa-news-address . spec: ... brokerProperties: - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.address=usa-news-address ... Add a separate line for each setting you want to configure for a queue in the format: addressConfigurations.< address name >.queueConfigs.< queue name >.< queue setting > For example: spec: ... brokerProperties: - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.routingType=ANYCAST - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.purgeOnNoConsumers=true - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.maxConsumers=20 ... Save the CR. Check that no errors were detected in the brokerProperties configuration by reviewing the status section of the ActiveMQArtemis CR. For more information, see Section 2.4, "Configuring items not exposed in a custom resource definition (CRD)" . Configuring addresses and queues in the ActiveMQArtemisAddress CR You can configure addresses and queues in the ActiveMQArtemisAddress CR. To configure multiple addresses and/or queues in your broker deployment, you need to create separate CR instances and deploy them individually, specifying new address and/or queue names in each case. In addition, the name attribute of each CR instance must be unique. Prerequisites You installed the custom resource definition (CRD) required to create addresses and queues on your brokers by installing the AMQ Broker Operator. For information on two alternative ways to install the Operator, see: Section 3.2, "Installing the Operator using the CLI" . Section 3.3, "Installing the Operator using OperatorHub" . You are familiar with how to use a CR instance to create a basic broker deployment. For more information, see Section 3.4.1, "Deploying a basic broker instance" . Procedure Start configuring a custom resource (CR) instance to define addresses and queues for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Open the sample CR file called broker_activemqartemisaddress_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. Start a new CR instance based on the address CRD. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemisAddresss CRD. Click the Instances tab. Click Create ActiveMQArtemisAddress . Within the console, a YAML editor opens, enabling you to configure a CR instance. In the spec section of the CR, add lines to define an address, queue, and routing type. For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisAddress metadata: name: myAddressDeployment0 namespace: myProject spec: ... addressName: myAddress0 queueName: myQueue0 routingType: anycast ... The preceding configuration defines an address named myAddress0 with a queue named myQueue0 and an anycast routing type. Note In the metadata section, you need to include the namespace property and specify a value only if you are using the OpenShift Container Platform web console to create your CR instance. The value that you should specify is the name of the OpenShift project for your broker deployment. Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project for the broker deployment. Create the CR instance. Using the OpenShift web console: When you finish configuring the CR, click Create . 4.2.2. Configuring address settings You can configure groups of address settings and specify address criteria to which the settings are applied by using either of the following methods: If you configure addresses by using the brokerProperties attribute in the ActiveMQArtemis CR instance for your broker deployment, you can also configure address settings under the brokerProperties attribute. If you configure addresses in an ActiveMQArtemisAddress CR instance, you can configure address settings in the addressSettings section of the ActiveMQArtemis CR. The following examples show how to use both methods to configure a dead letter address and queue for specific address patterns. A dead letter address and queue can be used by the broker to store messages that cannot be delivered to a client to prevent infinite delivery attempts. A system administrator can later consume undelivered messages from a dead letter queue to inspect the messages. Prerequisites You created an ActiveMQArtemis CR instance to deploy a broker. For more information, see Section 3.4.1, "Deploying a basic broker instance" . You are familiar with the default address settings configuration that the Operator merges or replaces with the configuration specified in your CR instance. For more information, see Section 4.1.1, "How the Operator generates the address settings configuration" . Configuring address settings by using brokerProperties Edit the ActiveMQArtemis CR instance for your broker deployment. Create a dead letter address and queue to receive undelivered messages. For example: spec: ... brokerProperties: ... - addressConfigurations.usDeadLetter.routingTypes=MULTICAST - addressConfigurations.usDeadLetter.queueConfigs.usDeadLetter-queue.address=usDeadLetter For more information on creating addresses and queues by using brokerProperties , see, Section 4.2.1, "Configuring addresses and queues" . Add separate lines under the brokerProperties attribute in the format addressSettings.< address name >.< address setting > to: Set the dead letter address for undelivered messages to the dead letter address you created. Specify the number of delivery attempts after which a message that cannot be delivered to a matching address is sent to the dead letter address. For example: spec: ... brokerProperties: ... - addressSettings.usa-news.deadLetterAddress=usDeadLetter - addressSettings.usa-news.maxDeliveryAttempts=5 ... You can use an asterisk (*) or a number sign (#) character as wildcards to create address patterns. Matching of patterns is done at each delimiter boundary, which is represented by a period (.). The number sign character matches any sequence of zero or more words and can be used at the end of the address string. The asterisk character matches a single word and can be used anywhere within the address string. For example: spec: ... brokerProperties: ... - addressSettings."usa-news.*".deadLetterAddress=usDeadLetter - addressSettings."europe-news.#".deadLetterAddress=euDeadLetter ... In the preceding example, the following addresses are matched: The usa-news.* address pattern matches any word that follows the usa-news. string, such as usa-news.domestic and usa-news.intl , but not usa-news.domestic.politics . The europe-news.# address pattern matches any address that starts with europe-news , such as europe-news , europe-news.politics and europe-news.politics.fr . Note In brokerProperties entries, a period (.) is a reserved character. If you want to create an address pattern that contains a period, you must enclose the address in quotation marks. For example, "usa-news.*" Save the CR. Check that no errors were detected in the brokerProperties configuration by reviewing the status section of the ActiveMQArtemis CR. For more information, see Section 2.4, "Configuring items not exposed in a custom resource definition (CRD)" . Configuring addresses settings by using addressSettings in the ActiveMQArtemis CR instance If you configured the dead letter address and queue in an ActiveMQArtemisAddress CR, you can configure a setting to limit the delivery attempts in the ActiveMQArtemis CR instance for your broker deployment. Prerequisites You created an address and queue with the following details. addressName: myDeadLetterAddress queueName: myDeadLetterQueue routingType: anycast For information on creating addresses and queues, see Section 4.2.1, "Configuring addresses and queues" Procedure Edit the ActiveMQArtemis CR instance for your broker deployment. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Operators Installed Operator . Click the Red Hat Integration - AMQ Broker for RHEL 8 (Multiarch) operator. Click the AMQ Broker tab. Click the name of the ActiveMQArtemis instance name. Click the YAML tab. Within the console, a YAML editor opens, enabling you to edit the CR instance. Note In the metadata section, you need to include the namespace property and specify a value only if you are using the OpenShift Container Platform web console to create your CR instance. The value that you should specify is the name of the OpenShift project for your broker deployment. In the spec section of the CR, add a new addressSettings section that contains a single addressSetting section, as shown below. spec: ... addressSettings: addressSetting: Add a single instance of the match property to the addressSetting block. Specify an address-matching expression. For example: spec: ... addressSettings: addressSetting: - match: myAddress match Specifies the address, or set of addresses to which the broker applies the configuration that follows. In this example, the value of the match property corresponds to a single address called myAddress . Add properties related to undelivered messages and specify values. For example: spec: ... addressSettings: addressSetting: - match: myAddress deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 5 deadLetterAddress Address to which the broker sends undelivered messages. maxDeliveryAttempts Maximum number of delivery attempts that a broker makes before moving a message to the configured dead letter address. In the preceding example, if the broker makes five unsuccessful attempts to deliver a message to an address that begins with myAddress , the broker moves the message to the specified dead letter address, myDeadLetterAddress . (Optional) Apply similar configuration to another address or set of addresses. For example: spec: ... addressSettings: addressSetting: - match: myAddress deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 5 - match: 'myOtherAddresses#' deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 3 In this example, the value of the second match property includes a hash wildcard character. The wildcard character means that the preceding configuration is applied to any address that begins with the string myOtherAddresses . Note If you use a wildcard expression as a value for the match property, you must enclose the value in single quotation marks, for example, 'myOtherAddresses#' . At the beginning of the addressSettings section, add the applyRule property and specify a value. For example: spec: ... applyRule: merge_all addressSetting: - match: myAddress deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 5 - match: 'myOtherAddresses#' deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 3 The applyRule property specifies how the Operator applies the configuration that you add to the CR for each matching address or set of addresses. The values that you can specify are: merge_all For address settings specified in both the CR and the default configuration that match the same address or set of addresses: Replace any property values specified in the default configuration with those specified in the CR. Keep any property values that are specified uniquely in the CR or the default configuration. Include each of these in the final, merged configuration. For address settings specified in either the CR or the default configuration that uniquely match a particular address or set of addresses, include these in the final, merged configuration. merge_replace For address settings specified in both the CR and the default configuration that match the same address or set of addresses, include the settings specified in the CR in the final, merged configuration. Do not include any properties specified in the default configuration, even if these are not specified in the CR. For address settings specified in either the CR or the default configuration that uniquely match a particular address or set of addresses, include these in the final, merged configuration. replace_all Replace all address settings specified in the default configuration with those specified in the CR. The final, merged configuration corresponds exactly to that specified in the CR. Note If you do not explicitly include the applyRule property in your CR, the Operator uses a default value of merge_all . Save the CR instance. 4.2.2.1. Configurable address and queue settings In general, the address and queue settings that you can configure for a broker deployment on OpenShift Container Platform are fully equivalent to those of standalone broker deployments on Linux or Windows. However, you should be aware of some differences in how those settings are configured. Those differences are described in the following sub-section. To configure address and queue settings for broker deployments on OpenShift Container Platform, you add configuration to an addressSettings section of the main Custom Resource (CR) instance for the broker deployment. This contrasts with standalone deployments on Linux or Windows, for which you add configuration to an address-settings element in the broker.xml configuration file. The format used for the names of configuration items differs between OpenShift Container Platform and standalone broker deployments. For OpenShift Container Platform deployments, configuration item names are in camel case , for example, defaultQueueRoutingType . By contrast, configuration item names for standalone deployments are in lower case and use a dash ( - ) separator, for example, default-queue-routing-type . The following table shows some further examples of this naming difference. Table 4.2. Examples of differences in the names of configuration items Configuration item for standalone broker deployment Configuration item for OpenShift broker deployment address-full-policy addressFullPolicy auto-create-queues autoCreateQueues default-queue-routing-type defaultQueueRoutingType last-value-queue lastValueQueue Additional resources For examples of creating addresses and queues and matching settings for OpenShift Container Platform broker deployments, see: Creating addresses and queues for a broker deployment on OpenShift Container Platform Matching address settings to configured addresses for a broker deployment on OpenShift Container Platform To learn about all of the configuration options for addresses, queues, and address settings for OpenShift Container Platform broker deployments, see Section 8.1, "Custom Resource configuration reference" . For comprehensive information about configuring addresses, queues, and associated address settings for standalone broker deployments, see Configuring addresses and queues in Configuring AMQ Broker . You can use this information to create equivalent configurations for broker deployments on OpenShift Container Platform. 4.2.3. Deleting addresses and queues Depending on how you created addresses and queues, you can delete addresses and queues by removing brokerProperties entries in the ActiveMQArtemis CR for your broker deployment or by using the ActiveMQArtemisAddress CR. Deleting addresses and queues that were created using brokerProperties You can delete individual addresses and queues by removing the entries from under the brokerProperties attribute. Prerequisites You created a basic broker deployment. For more information, see Section 3.4.1, "Deploying a basic broker instance" . Procedure Edit the ActiveMQArtemis CR instance for your broker deployment. Add the following brokerProperties entries to allow the broker to delete any address, which is represented by the number sign (#), and associated queues that it no longer finds in the CR. spec: ... brokerProperties: - addressSettings.#.configDeleteAddresses=FORCE - addressSettings.#.configDeleteQueues=FORCE ... Under the brokerProperties attribute, delete all the lines that reference an address and queue that you want to remove. For example, delete all the lines that reference the usa-news address to remove this address and queue: spec: ... brokerProperties: - addressConfigurations.usa-news.queueConfigs.usa-news-queue.routingType=MULTICAST - addressConfigurations.usa-news.queueConfigs.usa-news-queue.purgeOnNoConsumers=true - addressConfigurations.usa-news.queueConfigs.usa-news-queue.maxConsumers=20 ... Save the CR. When the broker applies the updated configuration, it deletes addresses and queues that you removed from the CR. Deleting addresses and queues in the ActiveMQArtemisAddress CR You can delete addresses and queues in the ActiveMQArtemisAddress CR if you created the addresses and queues in the CR. Procedure Ensure that you have an address CR file with the details, for example, the name , addressName and queueName , of the address and queue you want to delete. For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisAddress metadata: name: myAddressDeployment0 namespace: myProject spec: ... addressName: myAddress0 queueName: myQueue0 routingType: anycast ... In the spec section of the address CR, add the removeFromBrokerOnDelete attribute and set it to a value of true . .. spec: addressName: myAddress1 queueName: myQueue1 routingType: anycast removeFromBrokerOnDelete: true Setting the removeFromBrokerOnDelete attribute to true causes the Operator to remove the address and any associated message for all brokers in the deployment when you delete the address CR. Apply the updated address CR to set the removeFromBrokerOnDelete attribute for the address you want to delete. Delete the address CR to delete the address from the brokers in the deployment. Additional resources To learn about all of the configuration options for addresses, queues, and address settings for OpenShift Container Platform broker deployments, see Section 8.1, "Custom Resource configuration reference" . If you installed the AMQ Broker Operator using the OpenShift command-line interface (CLI), the installation archive that you downloaded and extracted contains some additional examples of configuring address settings. In the deploy/examples folder of the installation archive, see: artemis-basic-address-settings-deployment.yaml artemis-merge-replace-address-settings-deployment.yaml artemis-replace-address-settings-deployment.yaml For comprehensive information about configuring addresses, queues, and associated address settings for standalone broker deployments, see Configuring addresses and queues in Configuring AMQ Broker . You can use this information to create equivalent configurations for broker deployments on OpenShift Container Platform. For more information about Init Containers in OpenShift Container Platform, see Using Init Containers to perform tasks before a pod is deployed in the OpenShift Container Platform documentation. 4.3. Configuring authentication and authorization By default, AMQ Broker uses a Java Authentication and Authorization Service (JAAS) properties login module to authenticate and authorize users. The configuration for the default JAAS login module is stored in a /home/jboss/amq-broker/etc/login.config file on each broker Pod and reads user and role information from the artemis-users.properties and artemis-roles.properties files in the same directory. You add the user and role information to the properties files in the default login module by updating the ActiveMQArtemisSecurity Custom Resource (CR). An alternative to updating the ActiveMQArtemisSecurity CR to add user and role information to the default properties files is to configure one or more JAAS login modules in a secret. This secret is mounted as a file on each broker Pod. Configuring JAAS login modules in a secret offers the following advantages over using the ActiveMQArtemisSecurity CR to add user and role information. If you configure a properties login module in a secret, the brokers do not need to restart each time you update the property files. For example, when you add a new user to a properties file and update the secret, the changes take effect without requiring a restart of the broker. You can configure JAAS login modules that are not defined in the ActiveMQArtemisSecurity CRD to authenticate users. For example, you can configure an LDAP login module or any other JAAS login module. Both methods of configuring authentication and authorization for AMQ Broker are described in the following sections. 4.3.1. Configuring JAAS login modules in a secret You can configure JAAS login modules in a secret to authenticate users with AMQ Broker. After you create the secret, you must add a reference to the secret in the main broker Custom Resource (CR) and also configure permissions in the CR to grant users access to AMQ Broker. Procedure Create a text file with your new JAAS login modules configuration and save the file as login.config . By saving the file as login.config , the correct key is inserted in the secret that you create from the text file. The following is an example login module configuration: After you configure JAAS login modules in a secret and add a reference to the secret in the CR, the default login module is no longer used by AMQ Broker. However, a user in the artemis-users.properties file, which is referenced in the default login module, is required by the Operator to authenticate with the broker. To ensure that the Operator can authenticate with the broker after you configure a new JAAS login module, you must either: Include the default properties login module in the new login module configuration, as shown in the example above. In the example, the default properties login module uses the artemis-users.properties and artemis-roles.properties files. If you include the default login module in the new login module configuration, you must set the baseDir to the /home/jboss/amq-broker/etc directory, which contains the default properties files on each broker Pod. Add the user and role information required by the Operator to authenticate with the broker to a properties file referenced in the new login module configuration. You can copy this information from the default artemis-users.properties and artemis-roles.properties files, which are in the /home/jboss/amq-broker/etc directory on a broker Pod. Note The properties files referenced in a login module are loaded only when the broker calls the login module for the first time. A broker calls the login modules in the order that they are listed in the login.config file until it finds the login module to authenticate a user. By placing the login module that contains the credentials used by the Operator at the end of the login.config file, all preceding login modules are called when the broker authenticates the Operator. As a result, any status message which states that property files are not visible on the broker is cleared. If the login.config file you created includes a properties login module, ensure that the users and roles files specified in the module contain user and role information. For example: new-users.properties new-roles.properties Use the oc create secret command to create a secret from the text file that you created with the new login module configuration. If the login module configuration includes a properties login module, also include the associated users and roles files in the secret. For example: Note The secret name must have a suffix of -jaas-config so the Operator can recognize that the secret contains login module configuration and propagate any updates to each broker Pod. For more information about how to create secrets, see Secrets in the Kubernetes documentation. Add the secret you created to the Custom Resource (CR) instance for your broker deployment. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR for your deployment. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Operators Installed Operator . Click the Red Hat Integration - AMQ Broker for RHEL 8 (Multiarch) operator. Click the AMQ Broker tab. Click the name of the ActiveMQArtemis instance name. Click the YAML tab. Within the console, a YAML editor opens, enabling you to configure a CR instance. Create an extraMounts element and a secrets element and add the name of the secret. The following example adds a secret named custom-jaas-config to the CR. deploymentPlan: ... extraMounts: secrets: - "custom-jaas-config" ... In the CR, grant permissions to the roles that are configured on the broker. In the spec section of the CR, add a brokerProperties element and add the permissions. You can grant a role permissions to a single address. Or, you can specify a wildcard match using the # sign to grant a role permissions to all addresses. For example: spec: ... brokerProperties: - securityRoles.#.group2.send=true - securityRoles.#.group1.consume=true - securityRoles.#.group1.createAddress=true - securityRoles.#.group1.createNonDurableQueue=true - securityRoles.#.group1.browse=true ... In the example, the group2 role is assigned send permissions to all addresses and the group1 role is assigned consume , createAddress , createNonDurableQueue and browse permissions to all addresses. Note In a Java properties file, a colon (:) is a reserved character that is used to separate a key and a value in a key/value pair. If you want to grant permissions to a fully qualified queue name (FQQN), which consists of an address name and a queue name separated by colons (::), you must use the backslash (\) character to escape the colon characters in the FQQN. For example: spec: ... brokerProperties: - 'securityRoles."my-address\:\:my-queue".group2.send=true' Save the CR. The Operator mounts the login.config file in the secret in a /amq/extra/secrets/ secret name directory on each Pod and configures the broker JVM to read the mounted login.config file instead of the default login.config file. If the login.config file contains a properties login module, the referenced users and roles properties file are also mounted on each Pod. View the status information in the CR to verify that the brokers in your deployment are using the JAAS login modules in the secret for authentication. Using the OpenShift command-line interface: Get the status conditions in the CR for your brokers. Using the OpenShift web console: In the CR, navigate to the status section. In the status information, verify that a JaasPropertiesApplied type is present, which indicates that the broker is using the JAAS login modules configured in the secret. For example: - lastTransitionTime: "2023-02-06T20:50:01Z" message: "" reason: Applied status: "True" type: JaasPropertiesApplied When you update any of the files in the secret, the value of the reason field shows OutofSync until OpenShift Container Platform propagates the latest files in the secret to each broker Pod. For example, if you add a new user to the new-users-properties file and update the secret, you see the following status information until the updated file is propagated to each Pod: - lastTransitionTime: "2023-02-06T20:55:20Z" message: 'new-users.properties status out of sync, expected: 287641156, current: 2177044732' reason: OutOfSync status: "False" type: JaasPropertiesApplied When you update user or role information in a properties file that is referenced in the secret, use the oc set data command to update the secret. You must readd all the files to the secret again, including the login.config file. For example, if you add a new user to the new-users.properties file that you created earlier in this procedure, use the following command to update the custom-jaas-config secret: Note The broker JVM reads the configuration in the login.config file only when it starts. If you change the configuration in the login.config file, for example, to add a new login module, and update the secret, the broker does not use the new configuration until the broker is restarted. Additional Resources Section 8.2, "Example JAAS login module configurations" Section 8.3, "Example: configuring AMQ Broker to use Red Hat Single Sign-On" For information about the JAAS login module format, see JAAS Login Configuration File . 4.3.2. Configuring the default JAAS login module using the Security Custom Resource (CR) You can use the ActiveMQArtemisSecurity Custom Resource (CR) to configure user and role information in the default JAAS properties login module to authenticate users with AMQ Broker. For an alternative method of configuring authentication and authorization on AMQ Broker by using secrets, see Section 4.3.1, "Configuring JAAS login modules in a secret" . Note The ActiveMQArtemisSecurity CR is deprecated starting in AMQ Broker 7.12. 4.3.2.1. Configuring the default JAAS login module using the Security Custom Resource (CR) The following procedure shows how to configure the default JAAS login module using the Security Custom Resource (CR). Prerequisites You must have already installed the AMQ Broker Operator. For information on two alternative ways to install the Operator, see: Section 3.2, "Installing the Operator using the CLI" . Section 3.3, "Installing the Operator using OperatorHub" . You should be familiar with broker security as described in Securing brokers Procedure You can deploy the security CR before or after you create a broker deployment. However, if you deploy the security CR after creating the broker deployment, the broker pod is restarted to accept the new configuration. Start configuring a Custom Resource (CR) instance to define users and associated security configuration for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR for your deployment. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Operators Installed Operator . Click the Red Hat Integration - AMQ Broker for RHEL 8 (Multiarch) operator. Click the AMQ Broker tab. Click the name of the ActiveMQArtemis instance name Click the YAML tab. Within the console, a YAML editor opens, enabling you to configure a CR instance. In the spec section of the CR, add lines to define users and roles. For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisSecurity metadata: name: ex-prop spec: loginModules: propertiesLoginModules: - name: "prop-module" users: - name: "sam" password: "samspassword" roles: - "sender" - name: "rob" password: "robspassword" roles: - "receiver" securityDomains: brokerDomain: name: "activemq" loginModules: - name: "prop-module" flag: "sufficient" securitySettings: broker: - match: "#" permissions: - operationType: "send" roles: - "sender" - operationType: "createAddress" roles: - "sender" - operationType: "createDurableQueue" roles: - "sender" - operationType: "consume" roles: - "receiver" ... Note Always specify values for the elements in the preceding example. For example, if you do not specify values for securityDomains.brokerDomain or values for roles, the resulting configuration might cause unexpected results. The preceding configuration defines two users: a propertiesLoginModule named prop-module that defines a user named sam with a role named sender . a propertiesLoginModule named prop-module that defines a user named rob with a role named receiver . The properties of these roles are defined in the brokerDomain and broker sections of the securityDomains section. For example, the send role is defined to allow users with that role to create a durable queue on any address. By default, the configuration applies to all deployed brokers defined by CRs in the current namespace. To limit the configuration to particular broker deployments, use the applyToCrNames option described in Section 8.1.3, "Security Custom Resource configuration reference" . Note In the metadata section, you need to include the namespace property and specify a value only if you are using the OpenShift Container Platform web console to create your CR instance. The value that you should specify is the name of the OpenShift project for your broker deployment. Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project for the broker deployment. Create the CR instance. Using the OpenShift web console: When you have finished configuring the CR, click Create . Additional resources Section 8.1.3, "Security Custom Resource configuration reference" Section 3.4.1, "Deploying a basic broker instance" 4.3.2.2. Storing user passwords in a secret In the Section 4.3.2.1, "Configuring the default JAAS login module using the Security Custom Resource (CR)" procedure, user passwords are stored in clear text in the ActiveMQArtemisSecurity CR. If you do not want to store passwords in clear text in the CR, you can exclude the passwords from the CR and store them in a secret. When you apply the CR, the Operator retrieves each user's password from the secret and inserts it in the artemis-users.properties file on the broker pod. Procedure Use the oc create secret command to create a secret and add each user's name and password. The secret name must follow a naming convention of security-properties- module name , where module name is the name of the login module configured in the CR. For example: In the spec section of the CR, add the user names that you specified in the secret along with the role information, but do not include each user's password. For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisSecurity metadata: name: ex-prop spec: loginModules: propertiesLoginModules: - name: "prop-module" users: - name: "sam" roles: - "sender" - name: "rob" roles: - "receiver" securityDomains: brokerDomain: name: "activemq" loginModules: - name: "prop-module" flag: "sufficient" securitySettings: broker: - match: "#" permissions: - operationType: "send" roles: - "sender" - operationType: "createAddress" roles: - "sender" - operationType: "createDurableQueue" roles: - "sender" - operationType: "consume" roles: - "receiver" ... Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project for the broker deployment. Create the CR instance. Using the OpenShift web console: When you finish configuring the CR, click Create . Additional resources For more information about secrets in OpenShift Container Platform, see Providing sensitive data to pods in the OpenShift Container Platform documentation. 4.4. Adding third-party JAR files You can make third-party JAR files available to AMQ Broker at run time. For example, if you want your broker to store messages in a JDBC database, you can configure the broker to load the required third-party JAR file for the database. You must configure the Operator to make the third-party JAR file available on a mounted volume on each broker pod and add the volume path for the JAR file to the broker's Java classpath. If a JAR file is less than 1 MB in size, you can add the JAR file to a secret or configmap and configure the Operator to mount the JAR file on each broker pod. If a JAR file is larger than the 1 MB limit for secrets and configmaps, you can configure the Operator to mount a shared volume on each broker pod and download the JAR file to that volume. 4.4.1. Using a secret or config map to mount a JAR file on broker pods If a JAR file is less than 1 MB, you can use a secret or config map to mount a third-party JAR file on each broker pod. You must also modify the broker's Java classpath to load the JAR file from the mounted location at runtime. The following procedure assume that you are using a secret to mount the JAR file. Procedure Use the oc create secret command to create a secret that contains the third-party JAR file that you want to add. For example: oc create secret generic log4j-template --from-file=log4j-layout-template-json-2.22.1.jar For more information about how to create secrets, see Secrets in the Kubernetes documentation. Edit the CR for your broker deployment and configure the Operator to mount the secret that contains the third-party JAR file on each broker pod. For example, the following configuration mounts a secret named log4j-template . deploymentPlan: ... extraMounts: secrets: - "log4j-template" ... The JAR file is mounted in a /amq/extra/secrets/ secret name directory on each broker pod. For example, /amq/extra/secrets/postgresql-driver/log4j-template.jar . Create an ARTEMIS_EXTRA_LIBS environment variable to extend the broker's Java classpath so the broker loads the JAR file from the mounted directory on each pod. For example: spec: ... env: - name: ARTEMIS_EXTRA_LIBS value: /amq/extra/secrets/log4j-template Save the CR. 4.4.2. Downloading the JAR file to a volume on each broker pod If a JAR file is larger that 1 MB, you cannot use a secret or config map to mount the JAR file on each broker pod. Instead, you can configure the Operator to download the JAR file to a persistent shared volume that the Operator mounts on each broker pod. Prerequisites A persistent shared volume is available to mount on each broker pod. Procedure Edit the ActiveMQArtemis CR for your broker deployment. In the broker CR, use the extraVolumes and extraVolumeMounts attributes to add a persistent volume and mount the volume on each broker pod. For example: deploymentPlan: ... extraVolumes: - name: extra-volume persistentVolumeClaim: claimName: extra-jars extraVolumeMounts: - name: extra-volume mountPath: /opt/extra-lib ... Use the resourceTemplates attribute to customize the StatefulSet resource for the deployment. In the customization, use an init container to mount the extra-volume volume that you created on each pod and to download the JAR file to the volume. For example: spec: ... resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: mysql-jdbc-driver-init volumeMounts: - mountPath: /opt/extra-lib name: extra-volume image: curlimages/curl:8.6.0 command: - /bin/sh args: - -c - "if ! [ -f /opt/extra-lib/mysql-connector.jar ]; then curl https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.23/mysql-connector-java-8.0.23.jar --output /opt/extra-lib/mysql-connector.jar ; fi" In the example, a curl image is used to download a mysql-connector.jar file to the mounted path of the volume, /opt/extra-lib , if the file is not already on the volume. Create an ARTEMIS_EXTRA_LIBS environment variable to extend the broker's Java classpath so the broker loads the JAR file from the shared volume. For example: spec: ... env: - name: ARTEMIS_EXTRA_LIBS value: /opt/extra-lib Save the CR. 4.5. Configuring message persistence By default, AMQ Broker does not persist (that is, store) message data. AMQ Broker has two options for persisting message data: Persisting messages in journals. This is the default method of persisting messages if you enable persistence. Journal-based persistence is a high-performance option that writes messages to journals on the file system. Persisting messages in a database. This option uses a Java Database Connectivity (JDBC) connection to persist messages to a database of your choice. Note For current information about which databases and network file systems are supported by AMQ Broker see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal. 4.5.1. Configuring journal-based persistence When you enable persistence, messages are persisted in journal files by default. Procedure Edit the ActiveMQArtemis custom resource (CR) for your broker deployment. Set the persistenceEnabled attribute to true . For example: spec: ... deploymentPlan: persistenceEnabled: true ... Save the CR. 4.5.2. Configuring database persistence You can configure AMQ Broker to persist messages in a database by using a Java Database Connectivity (JDBC) connection. When you persist message data in a database, the broker uses a Java Database Connectivity (JDBC) connection to store message and bindings data in database tables. The data in the tables is encoded using AMQ Broker journal encoding. For information about supported databases, see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal. Important An administrator might choose to store message data in a database based on the requirements of an organization's wider IT infrastructure. However, use of a database can negatively effect the performance of a messaging system. Specifically, writing messaging data to database tables via JDBC creates a significant performance overhead for a broker. Prerequisite A dedicated database for use with AMQ Broker. The required JDBC driver JAR file is available to the broker at runtime. For information on how to make a JAR file available to the broker at runtime, see Section 4.4, "Adding third-party JAR files" . The deployment has a single broker instance. To ensure that the deployment has a single broker instance, ensure that the deployment.size attribute is not in the ActiveMQArtemis custom resource (CR). When the deployment.size attribute is omitted from the CR, a single broker instance is deployed. Procedure Edit the ActiveMQArtemis custom resource (CR) for your broker deployment. Enable JDBC database persistence by using the brokerProperties attribute. For example: spec: ... brokerProperties: - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=< class name > - storeConfiguration.jdbcConnectionUrl=jdbc:< URL > - HAPolicyConfiguration=SHARED_STORE_PRIMARY ... storeConfiguration Specify a value of DATABASE to persist messages to a JDBC database. storeConfiguration.jdbcDriverClassName Fully-qualified class name of the JDBC database driver. For example, org.postgresql.Driver . For information about supported databases, see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal. storeConfiguration.jdbcConnectionUrl Full JDBC connection URL for your database server including the database name and all configuration parameters. For example: jdbc:postgresql://postgresql-service.default.svc.cluster.local:5432/postgres?user=postgres&password=postgres In the example, the database name is postgres . HAPolicyConfiguration Set to SHARED_STORE_PRIMARY to ensure that the broker uses a JDBC lease lock to protect the database tables from concurrent access by multiple brokers. If a second broker instance is deployed unintentionally, the lease lock prevents the second broker from writing to the database. (Optional) Change the default values for the following attributes, if required: storeConfiguration.jdbcNetworkTimeout JDBC network connection timeout, in milliseconds. The default value is 20000 milliseconds. storeConfiguration.jdbcLockRenewPeriod Length, in milliseconds, of the renewal period for the current JDBC lock. When this time elapses, the broker can renew the lock. Set a value that is several times smaller than the value of storeConfiguration.jdbcLockExpiration to give the broker sufficient time to extend the lease and also gives the broker time to try to renew the lock in the event of a connection problem. The default value is 2000 milliseconds. storeConfiguration.jdbcLockExpiration Time, in milliseconds, that the current JDBC lock is considered owned (that is, acquired or renewed), even if the value of storeConfiguration.jdbcLockRenewPeriod has elapsed. The broker periodically tries to renew a lock that it owns according to the value of storeConfiguration.jdbcLockRenewPeriod . If the broker fails to renew the lock, for example, due to a connection problem, the broker keeps trying to renew the lock until the value of storeConfiguration.jdbcLockExpiration has passed since the lock was last successfully acquired or renewed. An exception to the renewal behavior described above is when another broker acquires the lock. This can happen if there is a time misalignment between the Database Management System (DBMS) and the brokers, or if there is a long pause for garbage collection. In this case, the broker that originally owned the lock considers the lock lost and does not try to renew it. If the JDBC lock has not been renewed by the broker that currently owns it after the expiration time elapses, another broker can establish a JDBC lock. The default value of is 20000 milliseconds. storeConfiguration.jdbcJournalSyncPeriod Duration, in milliseconds, for which the broker journal synchronizes with JDBC. The default value is 5 milliseconds. storeConfiguration.jdbcMaxPageSizeBytes Maximum size, in bytes, of each page file when AMQ Broker persists messages to a JDBC database. The default value is 102400, which is 100KB. The value that you specify also supports byte notation such as "K" "MB", and "GB". Save the CR. 4.6. Configuring broker storage requirements To use persistent storage in an Operator-based broker deployment, you set persistenceEnabled to true in the Custom Resource (CR) instance used to create the deployment. If you do not have container-native storage in your OpenShift cluster, you need to manually provision Persistent Volumes (PVs) and ensure that these are available to be claimed by the Operator using a Persistent Volume Claim (PVC). If you want to create a cluster of two brokers with persistent storage, for example, then you need to have two PVs available. Important When you manually provision PVs in OpenShift Container Platform, ensure that you set the reclaim policy for each PV to Retain . If the reclaim policy for a PV is not set to Retain and the PVC that the Operator used to claim the PV is deleted, the PV is also deleted. Deleting a PV results in the loss of any data on the volume. For more information, about setting the reclaim policy, see Understanding persistent storage in the OpenShift Container Platform documentation. By default, a PVC obtains 2 GiB of storage for each broker from the default storage class configured for the cluster. You can override the default size and storage class requested in the PVC, but only by configuring new values in the CR before deploying the CR for the first time. 4.6.1. Configuring broker storage size and storage class The following procedure shows how to configure the Custom Resource (CR) instance for your broker deployment to specify the size and storage class of the Persistent Volume Claim (PVC) required by each broker for persistent message storage. Note If you change the storage configuration in the CR after you deploy AMQ Broker, the updated configuration is not applied retrospectively to existing Pods. However, the updated configuration is applied to new Pods that are created if you scale up the deployment. Prerequisites You should be familiar with how to use a CR instance to create a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . You must have already provisioned Persistent Volumes (PVs) and made these available to be claimed by the Operator. For example, if you want to create a cluster of two brokers with persistent storage, you need to have two PVs available. For more information about provisioning persistent storage, see Understanding persistent storage in the OpenShift Container Platform documentation. Procedure Start configuring a Custom Resource (CR) instance for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project in which you are creating the deployment. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project in which you are creating the deployment. Start a new CR instance based on the main broker CRD. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click Create ActiveMQArtemis . Within the console, a YAML editor opens, enabling you to configure a CR instance. For a basic broker deployment, a configuration might resemble that shown below. apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true Observe that in the broker_activemqartemis_cr.yaml sample CR file, the image property is set to a default value of placeholder . This value indicates that, by default, the image property does not specify a broker container image to use for the deployment. To learn how the Operator determines the appropriate broker container image to use, see Section 2.7, "How the Operator chooses container images" . To specify the broker storage size, in the deploymentPlan section of the CR, add a storage section. Add a size property and specify a value. For example: spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true storage: size: 4Gi storage.size Size, in bytes, of the Persistent Volume Claim (PVC) that each broker Pod requires for persistent storage. This property applies only when persistenceEnabled is set to true . The value that you specify must include a unit using byte notation (for example, K, M, G), or the binary equivalents (Ki, Mi, Gi). To specify the storage class that each broker Pod requires for persistent storage, in the storage section, add a storageClassName property and specify a value. For example: spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true storage: size: 4Gi storageClassName: gp3 storage.storageClassName The name of the storage class to request in the Persistent Volume Claim (PVC). Storage classes provide a way for administrators to describe and classify the available storage. For example, different storage classes might map to specific quality-of-service levels, backup policies and so on. If you do do not specify a storage class, a persistent volume with the default storage class configured for the cluster is claimed by the PVC. Note If you specify a storage class, a persistent volume is claimed by the PVC only if the volume's storage class matches the specified storage class. Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project in which you are creating the broker deployment. Create the CR instance. Using the OpenShift web console: When you have finished configuring the CR, click Create . 4.7. Configuring resource limits and requests for Operator-based broker deployments When you create an Operator-based broker deployment, the broker Pods in the deployment run in a StatefulSet on a node in your OpenShift cluster. You can configure the Custom Resource (CR) instance for the deployment to specify the host-node compute resources used by the broker container that runs in each Pod. By specifying limit and request values for CPU and memory (RAM), you can ensure satisfactory performance of the broker Pods. Important You must add configuration for limits and requests to the CR instance for your broker deployment before deploying the CR for the first time. You cannot add the configuration to a broker deployment that is already running. It is not possible for Red Hat to recommend values for limits and requests because these are based on your specific messaging system use-cases and the resulting architecture that you have implemented. However, it is recommended that you test and tune these values in a development environment before configuring them for your production environment. The Operator runs a type of container called an Init Container when initializing each broker Pod. Any resource limits and requests that you configure for each broker container also apply to each Init Container. For more information about the use of Init Containers in broker deployments, see Section 4.1, "How the Operator generates the broker configuration" . You can specify the following limit and request values: CPU limit For each broker container running in a Pod, this value is the maximum amount of host-node CPU that the container can consume. If a broker container attempts to exceed the specified CPU limit, OpenShift throttles the container. This ensures that containers have consistent performance, regardless of the number of Pods running on a node. Memory limit For each broker container running in a Pod, this value is the maximum amount of host-node memory that the container can consume. If a broker container attempts to exceed the specified memory limit, OpenShift terminates the container. The broker Pod restarts. CPU request For each broker container running in a Pod, this value is the amount of host-node CPU that the container requests. The OpenShift scheduler considers the CPU request value during Pod placement, to bind the broker Pod to a node with sufficient compute resources. The CPU request value is the minimum amount of CPU that the broker container requires to run. However, if there is no contention for CPU on the node, the container can use all available CPU. If you have specified a CPU limit, the container cannot exceed that amount of CPU usage. If there is CPU contention on the node, CPU request values provide a way for OpenShift to weigh CPU usage across all containers. Memory request For each broker container running in a Pod, this value is the amount of host-node memory that the container requests. The OpenShift scheduler considers the memory request value during Pod placement, to bind the broker Pod to a node with sufficient compute resources. The memory request value is the minimum amount of memory that the broker container requires to run. However, the container can consume as much available memory as possible. If you have specified a memory limit, the broker container cannot exceed that amount of memory usage. CPU is measured in units called millicores. Each node in an OpenShift cluster inspects the operating system to determine the number of CPU cores on the node. Then, the node multiplies that value by 1000 to express the total capacity. For example, if a node has two cores, the CPU capacity of the node is expressed as 2000m . Therefore, if you want to use one-tenth of a single core, you specify a value of 100m . Memory is measured in bytes. You can specify the value using byte notation (E, P, T, G, M, K) or the binary equivalents (Ei, Pi, Ti, Gi, Mi, Ki). The value that you specify must include a unit. 4.7.1. Configuring broker resource limits and requests The following example shows how to configure the main Custom Resource (CR) instance for your broker deployment to set limits and requests for CPU and memory for each broker container that runs in a Pod in the deployment. Important You must add configuration for limits and requests to the CR instance for your broker deployment before deploying the CR for the first time. You cannot add the configuration to a broker deployment that is already running. It is not possible for Red Hat to recommend values for limits and requests because these are based on your specific messaging system use-cases and the resulting architecture that you have implemented. However, it is recommended that you test and tune these values in a development environment before configuring them for your production environment. Prerequisites You should be familiar with how to use a CR instance to create a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . Procedure Start configuring a Custom Resource (CR) instance for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project in which you are creating the deployment. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project in which you are creating the deployment. Start a new CR instance based on the main broker CRD. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click Create ActiveMQArtemis . Within the console, a YAML editor opens, enabling you to configure a CR instance. For a basic broker deployment, a configuration might resemble that shown below. apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true Observe that in the broker_activemqartemis_cr.yaml sample CR file, the image property is set to a default value of placeholder . This value indicates that, by default, the image property does not specify a broker container image to use for the deployment. To learn how the Operator determines the appropriate broker container image to use, see Section 2.7, "How the Operator chooses container images" . In the deploymentPlan section of the CR, add a resources section. Add limits and requests sub-sections. In each sub-section, add a cpu and memory property and specify values. For example: spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true resources: limits: cpu: "500m" memory: "1024M" requests: cpu: "250m" memory: "512M" limits.cpu Each broker container running in a Pod in the deployment cannot exceed this amount of host-node CPU usage. limits.memory Each broker container running in a Pod in the deployment cannot exceed this amount of host-node memory usage. requests.cpu Each broker container running in a Pod in the deployment requests this amount of host-node CPU. This value is the minimum amount of CPU required for the broker container to run. requests.memory Each broker container running in a Pod in the deployment requests this amount of host-node memory. This value is the minimum amount of memory required for the broker container to run. Note If you specify limits for a resource, but do not specify requests , a broker container requests the configured limits values for that resource. For example, in the following configuration, a broker container requests the configured limits values of 500m cpu and 1024M memory. spec: deploymentPlan: size: 3 ... resources: limits: cpu: "500m" memory: "1024M" Important Set limits without setting requests to control the precise amount of memory and CPU requested and to ensure that the same values are requested for each broker container if there are multiple brokers in your deployment. Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project in which you are creating the broker deployment. Create the CR instance. Using the OpenShift web console: When you have finished configuring the CR, click Create . 4.8. Enabling access to AMQ Management Console Each broker pod in an Operator-based deployment hosts its own instance of AMQ Management Console at port 8161. You can enable access to the console in the custom resource (CR) instance for your broker deployment. After you enable access to the console, you can use the console to view and manage the broker in your web browser. Procedure Edit the ActiveMQArtemis (CR) instance for your broker deployment. In the spec section of the CR, add a console attribute. In the console section, add the expose attribute and set the value to true . When you expose the console, the Operator automatically creates a dedicated service and Openshift route for the console on each broker pod in the deployment. If you want to customize the host name of the routes that are exposed for the console to match the internal routing configuration on your Openshift cluster, you can do one or both of the following: Use the ingressHost attribute to replace the default host name with a custom host name for the console routes. Use the ingressDomain attribute to append a custom domain to the host name. The custom domain is also applied to all other routes, such as routes for acceptors, that are exposed by the CR configuration. To set a custom host name specifically for the console routes, add the ingressHost attribute and specify the host string. For example: spec: .. console: expose: true ingressHost: my-console-production.my-subdomain.com .. Note The ingressHost value must be unique on your Openshift cluster. If your broker cluster has multiple broker pods, you can make the ingressHost value unique by including the USD(BROKER_ORDINAL) variable in the value. The Operator replaces this variable in the route it creates for each broker pod with the ordinal number the StatefulSet assigned to the pod. For example, an ingressHost value of my-console-USD(BROKER_ORDINAL)-production.my-subdomain.com sets the host name of the route to my-console-0-production.my-subdomain.com on the first pod, my-console-1-production.my-subdomain.com on the second pod and so on. You can include any the following variables in the custom host name string for the console route: Table 4.3. Variables in a custom host name string for the console route Name Description USD(CR_NAME) The value of the metadata.name attribute in the CR. USD(CR_NAMESPACE) The namespace of the custom resource. USD(BROKER_ORDINAL) The ordinal number assigned to the broker pod by the StatefulSet. USD(ITEM_NAME) The name of the acceptor. USD(RES_TYPE) The resource type. A route has a resource type of rte . An ingress has a resource type of ing . USD(INGRESS_DOMAIN) The value of the spec.ingressDomain attribute if it is configured in the CR. To append a custom domain to the host name in routes, add a spec.ingressDomain attribute and specify a custom string. For example: spec: ... ingressDomain: my.domain.com If your organization's network policy require that you expose the console by using an ingress instead of a route, complete the following steps: Add the exposeMode attribute and set the value to ingress . spec: .. console: expose: true exposeMode: ingress .. If you want to customize the host name of the ingresses that are exposed for the console to match the internal routing configuration on your Openshift cluster, you can do one or both of the following: Use the ingressHost attribute to replace the default host name with a custom host name. Use the ingressDomain attribute to append a custom domain to the host name. The custom domain is also applied to all other ingresses, such as ingresses for acceptors, that are exposed by the CR configuration. To set a custom host name specifically for the ingresses created for the console, add the ingressHost attribute and specify the host string. For example: spec: .. console: expose: true exposeMode: ingress expose: true exposeMode: ingress ingressHost: my-console-production.my-subdomain.com ... You can include the same variables to customize an ingress host as a route host, which are described earlier in this procedure. To append a custom domain to the host name in ingresses, add a spec.ingressDomain attribute and specify a custom string. spec: ... ingressDomain: my.domain.com For the console, the default host name of an ingress is in the format <cr-name>-wconsj-<ordinal>-svc-ing-<namespace> . If, for example, you have a CR named production in the amqbroker name space, an ingressDomain value of mydomain.com gives a host value of production-wconsj-0-svc-ing-mynamespace.amqbroker.com for the ingress created on pod 0. For more information on the spec.ingressDomain attribute, see Section 8.1, "Custom Resource configuration reference" . If you want to enable secure connections to the console from clients outside of the OpenShift cluster, complete the following steps: Add the sslEnabled attribute and set the value to true . spec: .. console: expose: true exposeMode: ingress sslEnabled: true .. Add the sslSecret attribute and specify the name of a secret that contains the certificate to secure the console. For example: spec: .. console: expose: true exposeMode: ingress sslEnabled: true sslSecret: console-tls-secret .. Use the spec.env attribute to add an environment variable that configures the console to automatically load a new certificate each time the certificate is renewed. For example: spec: .. env: - name: JAVA_ARGS_APPEND value: -Dwebconfig.bindings.artemis.sslAutoReload=true .. Save the CR. Additional resources For information about how to connect to AMQ Management Console, see Chapter 5, Connecting to AMQ Management Console for an Operator-based broker deployment 4.9. Setting environment variables for the broker containers In the Custom Resource (CR) instance for your broker deployment, you can set environment variables that are passed to a AMQ Broker container. For example, you can use standard environment variables such as TZ to set the timezone or JDK_JAVA_OPTIONS to prepend arguments to the command line arguments used by the Java launcher at startup. Or, you can use a custom variable for AMQ Broker, JAVA_ARGS_APPEND , to append custom arguments to the command line arguments used by the Java launcher. Procedure Edit the Custom Resource (CR) instance for your broker deployment. Using the OpenShift command-line interface: Enter the following command: Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Operators Installed Operator . Click the Red Hat Integration - AMQ Broker for RHEL 8 (Multiarch) operator. Click the AMQ Broker tab. Click the name of the ActiveMQArtemis instance name. Click the YAML tab. Within the console, a YAML editor opens, which enables you to configure the CR instance. In the spec section of the CR, add an env element and add the environment variables that you want to set for the AMQ Broker container. For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: ... env: - name: TZ value: Europe/Vienna - name: JAVA_ARGS_APPEND value: --Hawtio.realm=console - name: JDK_JAVA_OPTIONS value: -XshowSettings:system ... In the example, the CR configuration includes the following environment variables: TZ to set the timezone of the AMQ Broker container. JAVA_ARGS_APPEND to configure AMQ Management Console to use a realm named console for authentication. JDK_JAVA_OPTIONS to set the Java -XshowSettings:system parameter, which displays system property settings for the Java Virtual Machine. Note Values configured using the JDK_JAVA_OPTIONS environment variable are prepended to the command line arguments used by the Java launcher. Values configured using the JAVA_ARGS_APPEND environment variable are appended to the arguments used by the launcher. If an argument is duplicated, the rightmost argument takes precedence. Save the CR. Note Red Hat recommends that you do not change AMQ Broker environment variables that have an AMQ_ prefix and that you exercise caution if you want to change the POD_NAMESPACE variable. Additional resources For more information about defining environment variables, see Define Environment Variables for a Container . 4.10. Overriding the default memory limit for a broker You can override the default memory limit that is set for a broker. By default, a broker is assigned half of the maximum memory that is available to the broker's Java Virtual Machine. The following procedure shows how to configure the Custom Resource (CR) instance for your broker deployment to override the default memory limit. Prerequisites You should be familiar with how to use a CR instance to create a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . Procedure Start configuring a Custom Resource (CR) instance to create a basic broker deployment. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. Start a new CR instance based on the main broker CRD. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click Create ActiveMQArtemis . Within the console, a YAML editor opens, enabling you to configure a CR instance. For example, the CR for a basic broker deployment might resemble the following: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true In the spec section of the CR, add a brokerProperties section. Within the brokerProperties section, add a globalMaxSize property and specify a memory limit. For example: spec: ... brokerProperties: - globalMaxSize=500m ... The default unit for the globalMaxSize property is bytes. To change the default unit, add a suffix of m (for MB) or g (for GB) to the value. Apply the changes to the CR. Using the OpenShift command-line interface: Save the CR file. Switch to the project for the broker deployment. Apply the CR. Using the OpenShift web console: When you finish editing the CR, click Save . (Optional) Verify that the new value you set for the globalMaxSize property overrides the default memory limit assigned to the broker. Connect to the AMQ Management Console. For more information, see Chapter 5, Connecting to AMQ Management Console for an Operator-based broker deployment . From the menu, select JMX . Select org.apache.activemq.artemis . Search for global . In the table that is displayed, confirm that the value in the Global max column is the same as the value that you configured for the globalMaxSize property. 4.11. Specifying a custom Init Container image As described in Section 4.1, "How the Operator generates the broker configuration" , the AMQ Broker Operator uses a default, built-in Init Container to generate the broker configuration. To generate the configuration, the Init Container uses the main Custom Resource (CR) instance for your deployment. In certain situations, you might need to use a custom Init Container. For example, if you want to include extra runtime dependencies, .jar files, in the broker installation directory. When you build a custom Init Container image, you must follow these important guidelines: In the build script (for example, a Docker Dockerfile or Podman Containerfile) that you create for the custom image, the FROM instruction must specify the latest version of the AMQ Broker Operator built-in Init Container as the base image. In your script, include the following line: The custom image must include a script called post-config.sh that you include in a directory called /amq/scripts . The post-config.sh script is where you can modify or add to the initial configuration that the Operator generates. When you specify a custom Init Container, the Operator runs the post-config.sh script after it uses your CR instance to generate a configuration, but before it starts the broker application container. As described in Section 4.1.2, "Directory structure of a broker Pod" , the path to the installation directory used by the Init Container is defined in an environment variable called CONFIG_INSTANCE_DIR . The post-config.sh script should use this environment variable name when referencing the installation directory (for example, USD{CONFIG_INSTANCE_DIR}/lib ) and not the actual value of this variable (for example, /amq/init/config/lib ). If you want to include additional resources (for example, .xml or .jar files) in your custom broker configuration, you must ensure that these are included in the custom image and accessible to the post-config.sh script. The following procedure describes how to specify a custom Init Container image. Prerequisites You must have built a custom Init Container image that meets the guidelines described above. For a complete example of building and specifying a custom Init Container image for the ArtemisCloud Operator, see custom Init Container image for JDBC-based persistence . To provide a custom Init Container image for the AMQ Broker Operator, you need to be able to add the image to a repository in a container registry such as the Quay container registry . You should understand how the Operator uses an Init Container to generate the broker configuration. For more information, see Section 4.1, "How the Operator generates the broker configuration" . You should be familiar with how to use a CR to create a broker deployment. For more information, see Section 3.4, "Creating Operator-based broker deployments" . Procedure Edit the CR instance for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR for your deployment. Using the OpenShift Container Platform web console: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click the instance for your broker deployment. Click the YAML tab. Within the console, a YAML editor opens, enabling you to edit the CR instance. In the deploymentPlan section of the CR, add an initImage attribute and set the value to the URL of your custom Init Container image. apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder initImage: <custom_init_container_image_url> requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true initImage Specifies the full URL for your custom Init Container image, which must be available from a container registry. Important If a CR has a custom init container image specified in the spec.deploymentPlan.initImage attribute, Red Hat recommends that you also specify the URL of the corresponding broker container image in the spec.deploymentPlan.image attribute to prevent automatic upgrades of the broker image. If you do not specify the URL of a specific broker container image in the spec.deploymentPlan.image attribute, the broker image can be automatically upgraded. After the broker image is upgraded, the versions of the broker and custom init container image are different, which might prevent the broker from running. If you have a working deployment that has a custom init container, you can prevent any further upgrades of the broker container image to eliminate the risk of a newer broker image not working with your custom init container image. For more information about preventing upgrades to the broker image, see, Section 6.6.2, "Restricting automatic upgrades of images by using image URLs" . Save the CR. Additional resources For a complete example of building and specifying a custom Init Container image for the ArtemisCloud Operator, see custom Init Container image for JDBC-based persistence . 4.12. Configuring Operator-based broker deployments for client connections 4.12.1. Configuring acceptors To enable client connections to broker pods in your OpenShift deployment, you define acceptors for your deployment. Acceptors define how a broker pod accepts connections. You define acceptors in the main Custom Resource (CR) used for your broker deployment. When you create an acceptor, you specify information such as the messaging protocols to enable on the acceptor, and the port on the broker pod to use for these protocols. Procedure Edit the ActiveMQArtemis custom resource (CR) for your broker deployment. In the acceptors attribute, add a named acceptor. Add the protocols and port attributes. Set values to specify the messaging protocols to be used by the acceptor and the port on each broker pod to expose for those protocols. For example: spec: .. acceptors: - name: my-acceptor protocols: amqp port: 5672 .. The configured acceptor exposes port 5672 to AMQP clients. The full set of values that you can specify for the protocols attribute is shown in the table. Table 4.4. Acceptor protocols Protocol Value Core Protocol core AMQP amqp OpenWire openwire MQTT mqtt STOMP stomp All supported protocols all Note For each broker pod in your deployment, the Operator also creates a default acceptor that uses port 61616. This default acceptor is required for broker clustering and has Core Protocol enabled. By default, the AMQ Broker management console uses port 8161 on the broker pod. Each broker pod in your deployment has a dedicated Service that provides access to the console. For more information, see Chapter 5, Connecting to AMQ Management Console for an Operator-based broker deployment . To use another protocol on the same acceptor, modify the protocols attribute. Specify a comma-separated list of protocols. For example: spec: .. acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 ... The configured acceptor now exposes port 5672 to AMQP and OpenWire clients. To specify the number of concurrent client connections that the acceptor allows, add the connectionsAllowed attribute and set a value. For example: spec: ... acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 ... By default, an acceptor is exposed only to clients in the same OpenShift cluster as the broker deployment. To also expose the acceptor to clients outside OpenShift, set both the expose attribute and the sslEnabled attribute to true . spec: ... acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true sslEnabled: true ... When you enable SSL (that is, Secure Sockets Layer) security on an acceptor (or connector), you can add related configuration, such as: The secret name used to store authentication credentials in your OpenShift cluster. A secret is required when you enable SSL on the acceptor. The Transport Layer Security (TLS) protocols to use for secure network communication. TLS is an updated, more secure version of SSL. You specify the TLS protocols in the enabledProtocols attribute. Whether the acceptor uses mTLS, also known as mutual authentication, between the broker and the client. You specify this by setting the value of the needClientAuth attribute to true . For more information about these tasks, see Section 4.12.2, "Securing broker-client connections" . When you expose an acceptor to clients outside OpenShift, the Operator automatically creates a dedicated service and Openshift route for the acceptor on each broker pod in the deployment. If you want to customize the host name of the route that is exposed for the acceptor on each pod to match the internal routing configuration on your Openshift cluster, you can do one or both of the following: Use the ingressHost attribute to replace the default host name with a custom host name for a specific acceptor. Use the ingressDomain attribute to append a custom domain to the host name. The custom domain is also applied to all other routes, such as routes for other acceptors and the console, that are exposed by the CR configuration. To set a custom host name for the acceptor routes, add the ingressHost attribute and specify the host string. For example: spec: ... acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true ingressHost: my-acceptor-production.my-subdomain.com ... Note The ingressHost value must be unique on your Openshift cluster. If your broker cluster has multiple broker pods, you can make the ingressHost value unique by including the USD(BROKER_ORDINAL) variable in the value. The Operator replaces this variable on each broker pod with the ordinal number the StatefulSet assigned to the pod. For example, an ingressHost value of my-acceptor-USD(BROKER_ORDINAL)-production.my-subdomain.com sets the host name of the route to my-acceptor-0-production.my-subdomain on the first pod, my-acceptor-1-production.my-subdomain on the second pod and so on. You can include any the following variables in the custom host name string for an acceptor route: Table 4.5. Variables in a custom host name string for an acceptor route Name Description USD(CR_NAME) The value of the metadata.name attribute in the CR. USD(CR_NAMESPACE) The namespace of the custom resource. USD(BROKER_ORDINAL) The ordinal number assigned to the broker pod by the StatefulSet. USD(ITEM_NAME) The name of the acceptor. USD(RES_TYPE) The resource type. A route has a resource type of rte . An ingress has a resource type of ing . USD(INGRESS_DOMAIN) The value of the spec.ingressDomain attribute if it is configured in the CR. To append a custom domain to the host name in routes, add a spec.ingressDomain attribute and specify a custom string. For example: spec: ... ingressDomain: my.domain.com If your organization's network policy require that you expose acceptors by using an ingress instead of a route, complete the following steps: Add the exposeMode attribute and set the value to ingress . spec: ... acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true exposeMode: ingress ... If you want to customize the host name of the ingresses that are exposed for the acceptor to match the internal routing configuration on your Openshift cluster, you can do one or both of the following: Use the ingressHost attribute to replace the default host name with a custom host name. Use the ingressDomain attribute to append a custom domain to the host name. The custom domain is also applied to all other ingresses, such as ingresses for other acceptors and the console, that are exposed by the CR configuration. To set a custom host name for the ingresses for the acceptor, add the ingressHost attribute and specify the host string. For example: spec: ... acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true exposeMode: ingress ingressHost: my-acceptor-production.my-subdomain.com ... You can include the same variables to customize an ingress host as a route host, which are described earlier in this procedure. To append a custom domain to the host name in ingresses, add a spec.ingressDomain attribute and specify a custom string. For example: spec: ... ingressDomain: my-subdomain.domain.com For acceptors, the default host name of an ingress is in the format of < cr-name >-< acceptor name >-< ordinal >-svc-ing-< namespace > . If, for example, you have a CR named production in the amqbroker name space, an ingressDomain value of mydomain.com gives a host value of production-wconsj-0-svc-ing-mynamespace.amqbroker.com for the ingress created on pod 0. Additional resources To learn how to configure TLS to secure broker-client connections, including generating a secret to store authentication credentials, see Section 4.12.2, "Securing broker-client connections" . For a complete Custom Resource configuration reference, including configuration of acceptors and connectors, see Section 8.1, "Custom Resource configuration reference" . 4.12.2. Securing broker-client connections If you enabled security on your acceptor or connector (that is, by setting sslEnabled to true ), you must configure Transport Layer Security (TLS) to allow certificate-based authentication between the broker and clients. TLS is an updated, more secure version of SSL. There are two primary TLS configurations: TLS Only the broker presents a certificate. The certificate is used by the client to authenticate the broker. This is the most common configuration. mTLS Both the broker and the client present certificates. This is sometimes called mutual authentication . You can use various methods to generate a TLS certificate. If the broker and clients are running on the same Openshift cluster, you can use Openshift to generate a service serving certificate for the broker. If the broker and clients are not running on the same Openshift cluster, you must generate a certificate using a method that allows you to customize the certificate. This section describes two methods that you can use to generate custom certificates: cert-manager Operator for Openshift Java Keytool utility. 4.12.2.1. Using Openshift service serving certificates If you want to secure internal connections between the broker and clients on the same Openshift cluster, you can add an annotation to the acceptor service to request that Openshift generates a service serving certificate. The generated certificate and key are in PEM format, stored in tls.crt and tls.key respectively, within a created secret. Note The service CA certificate, which issues the service certificates, is valid for 26 months and is automatically rotated when there is less than 13 months validity left. After rotation, the service CA configuration is still trusted until its expiration. This allows a grace period for all affected services to refresh their key material before the expiration. If you do not upgrade your cluster during this grace period, which restarts services and refreshes their key material, you might need to manually restart services to avoid failures after the service CA expires. Procedure Edit the ActiveMQArtemis custom resource (CR) for you broker deployment. Use the resourceTemplates attribute to annotate the service that is created for an acceptor. For example: spec: ... resourceTemplates: - selector: kind: Service name: amq-broker-myacceptor-0-svc annotations: service.beta.openshift.io/serving-cert-secret-name: myacceptor-ptls ... resourceTemplates.selector.kind Specify that the type of resource to which the customization applies is Service . resourceTemplates.selector.name Specify the name of the service to which you want to apply the annotation. An acceptor service has a name format of: < CR name >< acceptor name >< ordinal >-svc , where: < CR name > name is the value of the metadata.name attribute in the CR. < acceptor name > is the name of the acceptor. The example assumes that the name of the acceptor is myacceptor . < ordinal > is the ordinal number assigned to the broker pod by the StatefulSet. resourceTemplates.annotations Specify an annotation of service.beta.openshift.io/serving-cert-secret-name: < secret > , where < secret > is the name of the secret that Openshift creates for the service. Note The secret name must match the acceptor name and have a -ptls suffix. The specific suffix is required to allow the Operator to deploy the CR before the secret is created. In the sslSecret` attribute in the CR, specify the secret that contains the broker certificate. For example: spec: acceptors: - name: myacceptor protocols: CORE port: 61626 sslEnabled: true sslSecret: myacceptor-ptls In the brokerProperties attribute, configure the broker to automatically load a new certificate each time the certificate is renewed in Openshift. For example: spec: ... brokerProperties - "acceptorConfigurations.myacceptor.params.sslAutoReload=true" ... Add the public key of the service serving certificate to each client's trust store. If you want to configure mTLS authentication between the broker and clients, complete the following steps. Create a trust bundle that contains the certificate of each client that you want the broker to trust and add the trust bundle to a secret, for example, trusted-clients-bundle . In the acceptors configured in the broker CR, add the needClientAuth attribute and set to true to require client authentication. For example: spec: .. acceptors: - name: myacceptor protocols: all port: 62666 sslEnabled: true sslSecret: myacceptor-ptls needClientAuth: true .. In the trustSecret attribute of each acceptor, specify the secret that contains the trust bundle of client certificates. For example: spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true sslSecret: myacceptor-ptls needClientAuth: true trustSecret: trusted-clients-bundle .. Save the CR. Additional resources Securing service traffic using service serving certificate secrets . 4.12.2.2. Using cert-manager Operator for Openshift The cert-manager Operator for OpenShift is a cluster-wide service that provides application certificate lifecycle management. The cert-manager automates the management and issuance of TLS certificates from various certificate authorities. The following example procedure describes how to configure Transport Layer Security (TLS) by using a self-signed certificate. If your policy requires certificates that are signed by a recognized certificate manager, you can request the certificates by using the cert-manager Operator for OpenShift. Prerequisites cert-manager Operator for Red Hat OpenShift is installed. For more information, see Installing the cert-manager Operator for Red Hat OpenShift in the OpenShift Container Platform documentation. Trust Manager for Kubernetes is installed if you want to configure mTLS between the broker and clients. For more information, see Installing trust-manager . Procedure Create a YAML file, for example, self-signed-issuer.yaml , that defines a root self-signed issuer. An issuer is an Openshift resource that represents certificate authorities (CAs) that are able to generate signed certificates by honoring certificate signing requests. The following example yaml creates a self-signed issuer, which you can then use to create a Certification Authority(CA) certificate. Your CA certificate can be managed by the cert-manager Operator. apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: root-issuer spec: selfSigned: {} Create a YAML file, for example, root-ca.yaml , that defines a root CA certificate. In the issuerRef.name field, specify the name of the self-signed issuer, root-issuer , that you created. For example: apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: root-ca namespace: cert-manager spec: isCA: true commonName: "amq.io.root" secretName: root-ca-secret subject: organizations: - "www.amq.io" issuerRef: name: root-issuer kind: ClusterIssuer The Certificate is created in Privacy Enhanced Mail (PEM) format in a secret named root-ca-secret . Create a YAML file, for example, root-ca-issuer.yaml , that defines a CA issuer for issuing certificates that are signed by the root CA. For example: apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: root-ca-issuer spec: ca: secretName: root-ca-secret Create a YAML file, for example, broker-cert.yaml , that defines a broker certificate. In the issuerRef.Name field, specify the name of the issuer, root-ca-issuer , that you created to issue certificates that are signed by the root CA. For example: apiVersion: cert-manager.io/v1 kind: Certificate Metadata: name: broker-cert spec: isCA: false commonName: "amq.io" dnsNames: - "amq-broker-ss-0.amq-broker-svc-rte-default.cluster.local - "amq-broker-ss-1.amq-broker-svc-rte-default.cluster.local secretName: broker-cert-secret subject: organizations: - "www.amq.io" issuerRef: name: root-ca-issuer kind: ClusterIssuer Deploy the custom resources that you defined for issuers and certificates in YAML files to create the corresponding OpenShift objects. For example: USD oc create -f self-signed-issuer.yaml USD oc create -f root-ca.yaml USD oc create -f root-ca-issuer.yaml USD oc create -f broker-cert.yaml Edit the ActiveMQArtemis CR for you broker deployment. Specify the secret that contains the broker certificate in the sslSecret attribute of each acceptor that you want to secure. For example: spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true needClientAuth: false sslSecret: broker-cert-secret .. In the brokerProperties attribute, configure the broker to automatically load a new broker certificate for the acceptor each time the certificate is renewed by the cert-manager Operator for Openshift. For example: spec: ... brokerProperties - "acceptorConfigurations.new-acceptor.params.sslAutoReload=true" ... Add the root CA certificate that signed the broker certificate, which was create in a secret named root-ca-secret secret in this example procedure, to each client's trust store, so clients can trust the broker. If you want to configure mTLS authentication between the broker and clients, complete the following steps. Use Trust Manager for Kubernetes to create a trust bundle that contains the certificate of each client that you want the broker to trust and add the trust bundle to a secret, for example, trusted-clients-bundle . For information on how to create a trust bundle, see the trust-manager documentation . In the acceptors configured in the broker CR, add the needClientAuth attribute and set to true to require client authentication. For example: spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true sslSecret: broker-cert-secret needClientAuth: true .. in the trustSecret attribute of each acceptor, specify the secret that contains the trust bundle of client certificates. For example: spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true sslSecret: broker-cert-secret needClientAuth: true trustSecret: trusted-clients-bundle .. Save the CR. Additional resources cert-manager Operator for Red Hat OpenShift documentation . 4.12.2.3. Using the Java keytool utility Keytool is a certificate management utility included with Java. 4.12.2.3.1. Configuring one-way TLS The procedure in this section shows how to configure one-way Transport Layer Security (TLS) to secure a broker-client connection. In one-way TLS, only the broker presents a certificate. This certificate is used by the client to authenticate the broker. Prerequisites You should understand the requirements for broker certificate generation when clients use host name verification. For more information, see Section 4.12.2.4, "Configuring a broker certificate for host name verification" . Procedure Generate a self-signed certificate for the broker key store. USD keytool -genkey -alias broker -keyalg RSA -keystore ~/broker.ks Export the certificate from the broker key store, so that it can be shared with clients. Export the certificate in the Base64-encoded .pem format. For example: USD keytool -export -alias broker -keystore ~/broker.ks -file ~/broker_cert.pem On the client, create a client trust store that imports the broker certificate. USD keytool -import -alias broker -keystore ~/client.ts -file ~/broker_cert.pem Log in to OpenShift Container Platform as an administrator. For example: USD oc login -u system:admin Switch to the project that contains your broker deployment. For example: USD oc project <my_openshift_project> Create a secret to store the TLS credentials. For example: USD oc create secret generic my-tls-secret \ --from-file=broker.ks=~/broker.ks \ --from-file=client.ts=~/client.ks \ --from-literal=keyStorePassword= <password> \ --from-literal=trustStorePassword= <password> Note When generating a secret, OpenShift requires you to specify both a key store and a trust store. The trust store key is generically named client.ts . For one-way TLS between the broker and a client, a trust store is not actually required. However, to successfully generate the secret, you need to specify some valid store file as a value for client.ts . The preceding step provides a "dummy" value for client.ts by reusing the previously-generated broker key store file. This is sufficient to generate a secret with all of the credentials required for one-way TLS. Link the secret to the service account that you created when installing the Operator. For example: USD oc secrets link sa/amq-broker-operator secret/my-tls-secret Specify the secret name in the sslSecret parameter of your secured acceptor or connector. For example: spec: ... acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 sslEnabled: true sslSecret: my-tls-secret expose: true connectionsAllowed: 5 ... 4.12.2.3.2. Configuring two-way TLS The procedure in this section shows how to configure two-way Transport Layer Security (TLS) to secure a broker-client connection. In two-way TLS, both the broker and client presents certificates. The broker and client use these certificates to authenticate each other in a process sometimes called mutual authentication . Prerequisites You should understand the requirements for broker certificate generation when clients use host name verification. For more information, see Section 4.12.2.4, "Configuring a broker certificate for host name verification" . Procedure Generate a self-signed certificate for the broker key store. USD keytool -genkey -alias broker -keyalg RSA -keystore ~/broker.ks Export the certificate from the broker key store, so that it can be shared with clients. Export the certificate in the Base64-encoded .pem format. For example: USD keytool -export -alias broker -keystore ~/broker.ks -file ~/broker_cert.pem On the client, create a client trust store that imports the broker certificate. USD keytool -import -alias broker -keystore ~/client.ts -file ~/broker_cert.pem On the client, generate a self-signed certificate for the client key store. USD keytool -genkey -alias broker -keyalg RSA -keystore ~/client.ks On the client, export the certificate from the client key store, so that it can be shared with the broker. Export the certificate in the Base64-encoded .pem format. For example: USD keytool -export -alias broker -keystore ~/client.ks -file ~/client_cert.pem Create a broker trust store that imports the client certificate. USD keytool -import -alias broker -keystore ~/broker.ts -file ~/client_cert.pem Log in to OpenShift Container Platform as an administrator. For example: USD oc login -u system:admin Switch to the project that contains your broker deployment. For example: USD oc project <my_openshift_project> Create a secret to store the TLS credentials. For example: USD oc create secret generic my-tls-secret \ --from-file=broker.ks=~/broker.ks \ --from-file=client.ts=~/broker.ts \ --from-literal=keyStorePassword= <password> \ --from-literal=trustStorePassword= <password> Note When generating a secret, OpenShift requires you to specify both a key store and a trust store. The trust store key is generically named client.ts . For two-way TLS between the broker and a client, you must generate a secret that includes the broker trust store, because this holds the client certificate. Therefore, in the preceding step, the value that you specify for the client.ts key is actually the broker trust store file. Link the secret to the service account that you created when installing the Operator. For example: USD oc secrets link sa/amq-broker-operator secret/my-tls-secret Specify the secret name in the sslSecret parameter of your secured acceptor or connector. For example: spec: ... acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 sslEnabled: true sslSecret: my-tls-secret expose: true connectionsAllowed: 5 ... 4.12.2.4. Configuring a broker certificate for host name verification Note This section describes some requirements for the broker certificate that you must generate when configuring one-way or two-way TLS. When a client tries to connect to a broker Pod in your deployment, the verifyHost option in the client connection URL determines whether the client compares the Common Name (CN) of the broker's certificate to its host name, to verify that they match. The client performs this verification if you specify verifyHost=true or similar in the client connection URL. You might omit this verification in rare cases where you have no concerns about the security of the connection, for example, if the brokers are deployed on an OpenShift cluster in an isolated network. Otherwise, for a secure connection, it is advisable for a client to perform this verification. In this case, correct configuration of the broker key store certificate is essential to ensure successful client connections. In general, when a client is using host verification, the CN that you specify when generating the broker certificate must match the full host name for the Route on the broker Pod that the client is connecting to. For example, if you have a deployment with a single broker Pod, the CN might look like the following: To ensure that the CN can resolve to any broker Pod in a deployment with multiple brokers, you can specify an asterisk ( * ) wildcard character in place of the ordinal of the broker Pod. For example: The CN shown in the preceding example successfully resolves to any broker Pod in the my-broker-deployment deployment. In addition, the Subject Alternative Name (SAN) that you specify when generating the broker certificate must individually list all broker Pods in the deployment, as a comma-separated list. For example: 4.12.3. Networking services in your broker deployments On the Networking pane of the OpenShift Container Platform web console for your broker deployment, there are two running services; a headless service and a ping service. The default name of the headless service uses the format <custom_resource_name> -hdls-svc , for example, my-broker-deployment-hdls-svc . The default name of the ping service uses a format of <custom_resource_name> -ping-svc , for example, `my-broker-deployment-ping-svc . The headless service provides access to port 61616, which is used for internal broker clustering. The ping service is used by the brokers for discovery, and enables brokers to form a cluster within the OpenShift environment. Internally, this service exposes port 8888. 4.12.4. Connecting to the broker from internal and external clients The examples in this section show how to connect to the broker from internal clients (that is, clients in the same OpenShift cluster as the broker deployment) and external clients (that is, clients outside the OpenShift cluster). 4.12.4.1. Connecting to the broker from internal clients To connect an internal client to a broker, in the client connection details, specify the DNS resolvable name of the broker pod. For example: If the internal client is using the Core protocol and the useTopologyForLoadBalancing=false key was not set in the connection URL, after the client connects to the broker for the first time, the broker can inform the client of the addresses of all the brokers in the cluster. The client can then load balance connections across all brokers. If your brokers have durable subscription queues or request/reply queues, be aware of the caveats associated with using these queues when client connections are load balanced. For more information, see Section 4.12.4.4, "Caveats to load balancing client connections when you have durable subscription queues or reply/request queues" . 4.12.4.2. Connecting to the broker from external clients When you expose an acceptor to external clients (that is, by setting the value of the expose parameter to true ), the Operator automatically creates a dedicated service and route for each broker pod in the deployment. An external client can connect to the broker by specifying the full host name of the route created for the broker pod. You can use a basic curl command to test external access to this full host name. For example: The full host name of the route for the broker pod must resolve to the node that is hosting the OpenShift router. The OpenShift router uses the host name to determine where to send the traffic inside the OpenShift internal network. By default, the OpenShift router listens to port 80 for non-secured (that is, non-SSL) traffic and port 443 for secured (that is, SSL-encrypted) traffic. For an HTTP connection, the router automatically directs traffic to port 443 if you specify a secure connection URL (that is, https ), or to port 80 if you specify a non-secure connection URL (that is, http ). If you want external clients to load balance connections across the brokers in the cluster: Enable load balancing by configuring the haproxy.router.openshift.io/balance roundrobin option on the OpenShift route for each broker pod. If an external client uses the Core protocol, set the useTopologyForLoadBalancing=false key in the client's connection URL. Setting the useTopologyForLoadBalancing=false key prevents a client from using the AMQ Broker Pod DNS names that are in the cluster topology information provided by the broker. The Pod DNS names resolve to internal IP addresses, which an external client cannot access. If your brokers have durable subscription queues or request/reply queues, be aware of the caveats associated with using these queues when load balancing client connections. For more information, see Section 4.12.4.4, "Caveats to load balancing client connections when you have durable subscription queues or reply/request queues" . If you don't want external clients to load balance connections across the brokers in the cluster: In each client's connection URL, specify the full host name of the route for each broker pod. The client attempts to connect to the first host name in the connection URL. However, if the first host name is unavailable, the client automatically connects to the host name in the connection URL, and so on. If an external client uses the Core protocol, set the useTopologyForLoadBalancing=false key in the client's connection URL to prevent the client from using the cluster topology information provided by the broker. For non-HTTP connections: Clients must explicitly specify the port number (for example, port 443) as part of the connection URL. For one-way TLS, the client must specify the path to its trust store and the corresponding password, as part of the connection URL. For two-way TLS, the client must also specify the path to its key store and the corresponding password, as part of the connection URL. Some example client connection URLs, for supported messaging protocols, are shown below. External Core client, using one-way TLS Note The useTopologyForLoadBalancing key is explicitly set to false in the connection URL because an external Core client cannot use topology information returned by the broker. If this key is set to true or you do not specify a value, it results in a DEBUG log message. External Core client, using two-way TLS External OpenWire client, using one-way TLS External OpenWire client, using two-way TLS External AMQP client, using one-way TLS External AMQP client, using two-way TLS 4.12.4.3. Connecting to the Broker using a NodePort As an alternative to using a route, an OpenShift administrator can configure a NodePort to connect to a broker pod from a client outside OpenShift. The NodePort should map to one of the protocol-specific ports specified by the acceptors configured for the broker. By default, NodePorts are in the range 30000 to 32767, which means that a NodePort typically does not match the intended port on the broker Pod. To connect from a client outside OpenShift to the broker via a NodePort, you specify a URL in the format <protocol> :// <ocp_node_ip> : <node_port_number> . 4.12.4.4. Caveats to load balancing client connections when you have durable subscription queues or reply/request queues Durable subscriptions A durable subscription is represented as a queue on a broker and is created when a durable subscriber first connects to the broker. This queue exists and receives messages until the client unsubscribes. If the client reconnects to a different broker, another durable subscription queue is created on that broker. This can cause the following issues. Table 4.6. Issues with durable subscriptions Issue Mitigation Messages may get stranded in the original subscription queue. Enable message distribution by setting the redistributionDelay property for an address or set of addresses. You can set this property under the brokerProperties attribute in the ActiveMQArtemis CR. For example: addressSettings.< address >.redistributionDelay=5000 . In the example, the broker waits 5000 milliseconds after a queue's final consumer closes before it redistributes messages to other brokers. For more information on message redistribution, see Enabling message redistribution . Messages may be received in the wrong order as there is a window during message redistribution when other messages are still routed. None. When a client unsubscribes, it deletes the queue only on the broker it last connected to. This means that the other queues can still exist and receive messages. To delete other empty queues that may exist for a client that unsubscribed, configure both of the following properties for an address or set of addresses. You can set these properties under the brokerProperties attribute in the ActiveMQArtemis CR. addressSettings.< address >.autoDeleteQueuesMessageCount=0 addressSettings.< address >.autoDeleteQueuesDelay=5000 With the autoDeleteQueuesMessageCount property set to 0 , a queue is deleted only if there are no messages in the queue. The value of the autoDeleteQueuesDelay property is the number of milliseconds after which a queue that has no messages is deleted. For more information, see Configuring automatic creation and deletion of addresses and queues . Request/Reply queues When a JMS Producer creates a temporary reply queue, the queue is created on the broker. If the client that is consuming from the work queue and replying to the temporary queue connects to a different broker, the following issues can occur. Table 4.7. Issues with request/reply queues Issue Mitigation Since the reply queue does not exist on the broker that the client is connected to, the client may generate an error. Configure the broker to automatically create a queue when a client requests to connect to a queue that does not exist. To configure automatic queue creation, add the following property under the brokerProperties attribute in the ActiveMQArtemis CR. addressSettings.< address >.autoCreateQueues=true Messages sent to the work queue may not be distributed. Enable load balancing on demand by adding the following property under the brokerProperties attribute in the ActiveMQArtemis CR: clusterConfigurations.< cluster >.messageLoadBalancingType=ON-DEMAND . Also, enable message distribution by setting the redistributionDelay property for an address or set of addresses. You can set this property under the brokerProperties attribute in the ActiveMQArtemis CR. For example: addressSettings< address >.redistributionDelay=5000 For more information, see Enabling message redistribution . Additional resources For more information about using methods such as Routes and NodePorts for communicating from outside an OpenShift cluster with services running in the cluster, see: Configuring ingress cluster traffic overview in the OpenShift Container Platform documentation. 4.13. Securing cluster connections The internal connections between brokers in a cluster use an internal connector and acceptor, both of which are named artemis . You can enable SSL to secure the connections between the brokers in a cluster using Transport Layer Security (TLS) protocols. On the SSL-enabled acceptor, you specify a secret that contains a common TLS certificate for all the brokers in the cluster. On the SSL-enabled connector, you specify a truststore that contains the public key of the TLS certificate. The public key is required in each broker's truststore so a broker can trust the other brokers in the cluster when they establish a TLS connection. The following example procedure describes how to secure the internal connections between the brokers in a cluster by using a self-signed certificate. Procedure Generate a self-signed TLS certificate and add it to a keystore file. In the Subject Alternative Name (SAN) field of the certificate, specify a wildcard DNS name to match all of the brokers in the cluster, as shown in the following example. The example is based on using a CR named ex-aao that is deployed in a test namespace. USD keytool -storetype jks -keystore server-keystore.jks -storepass artemis -keypass artemis -alias server -genkey -keyalg "RSA" -keysize 2048 -dname "CN=AMQ Server, OU=Artemis, O=ActiveMQ, L=AMQ, S=AMQ, C=AMQ" -validity 365 -ext bc=ca:false -ext eku=sA -ext san=dns:*.ex-aao-hdls-svc.test.svc.cluster.local If the certificate does not support the use of wildcard DNS names, you can include a comma-separated list of DNS names in the SAN field of the certificate for all of the broker pods in the cluster. For example: keytool -storetype jks -keystore server-keystore.jks -storepass artemis -keypass artemis -alias server -genkey -keyalg "RSA" -keysize 2048 -dname "CN=AMQ Server, OU=Artemis, O=ActiveMQ, L=AMQ, S=AMQ, C=AMQ" -validity 365 -ext bc=ca:false -ext eku=sA -ext san=dns:ex-aao-ss-0.ex-aao-hdls-svc.test.svc.cluster.local,dns:ex-aao-ss-1.ex-aao-hdls-svc.test.svc.cluster.local If the TLS certificate does not support the use of DNS names, you must disable host verification in the ActiveMQArtemis CR, as described below. Export the public key of the TLS certificate from the keystore file so that it can be imported into a truststore file. For example: USD keytool -storetype jks -keystore server-keystore.jks -storepass artemis -alias server -exportcert -rfc > server.crt Import the public key of the TLS certificate into a truststore file so the other brokers in the clusters can trust the certificate. For example: USD keytool -storetype jks -keystore server-truststore.jks -storepass artemis -keypass artemis -importcert -alias server -file server.crt -noprompt Create a secret to store the keystore and truststore files and their associated passwords. For example: oc create secret generic artemis-ssl-secret --namespace test --from-file=broker.ks=server-keystore.jks --from-file=client.ts=server-truststore.jks --from-literal=keyStorePassword=artemis --from-literal=trustStorePassword=artemis Edit the ActiveMQArtemis CR for your broker deployment and add an internal acceptor named artemis . In the artemis acceptor, set the sslEnabled attribute to true and specify the name of the secret that you created in the sslSecret attribute. For example: spec: .. deploymentPlan: size: 2 acceptors: - name: artemis port: 61616 sslEnabled: true sslSecret: artemis-ssl-secret .. Enable SSL for the artemis connector, which is used by each broker in the cluster to connect to other brokers in the cluster. Use the brokerProperties attribute to enable SSL and specify the path and credentials of the truststore file that contains the public key of the TLS certificate. spec: .. deploymentPlan: size: 2 acceptors: - name: artemis port: 61616 sslEnabled: true sslSecret: artemis-ssl-secret brokerProperties: - 'connectorConfigurations.artemis.params.sslEnabled=true' - 'connectorConfigurations.artemis.params.trustStorePath=/etc/artemis-ssl-secret-volume/client.ts' - 'connectorConfigurations.artemis.params.trustStorePassword=artemis' .. connectorConfigurations.artemis.params.trustStorePath This value must match the location of the truststore file, client.ts on the broker pods. The truststore file and accompanying password file in the secret are mounted in a /etc/< secret name >-volume directory on each broker pod. The example specifies the location of a truststore that is in a secret named artemis-ssl-secret . If the TLS certificate does not support the use of DNS names, use the brokerProperties attribute to disable host verification. For example: spec: .. brokerProperties: .. - 'connectorConfigurations.artemis.params.verifyHost=false' .. Save the CR. 4.14. Configuring large message handling for AMQP messages Clients might send large AMQP messages that can exceed the size of the broker's internal buffer, causing unexpected errors. To prevent this situation, you can configure the broker to store messages as files when the messages are larger than a specified minimum value. Handling large messages in this way means that the broker does not hold the messages in memory. Instead, the broker stores the messages in a dedicated directory used for storing large message files. For a broker deployment on OpenShift Container Platform, the large messages directory is /opt/ <custom_resource_name> /data/large-messages on the Persistent Volume (PV) used by the broker for message storage. When the broker stores a message as a large message, the queue retains a reference to the file in the large messages directory. Note You can configure the large message size limit in the broker configuration for the AMQP protocol only. For the AMQ Core and Openwire protocols, you can configure large message size limits in the client connection configuration. For more information, see the Red Hat AMQ Clients documentation . 4.14.1. Configuring AMQP acceptors for large message handling The following procedure shows how to configure an acceptor to handle an AMQP message larger than a specified size as a large message. Prerequisites You should be familiar with how to configure acceptors for Operator-based broker deployments. See Section 4.12.1, "Configuring acceptors" . To store large AMQP messages in a dedicated large messages directory, your broker deployment must be using persistent storage (that is, persistenceEnabled is set to true in the Custom Resource (CR) instance used to create the deployment). For more information about configuring persistent storage, see: Section 2.9, "Operator deployment notes" Section 8.1, "Custom Resource configuration reference" Procedure Open the Custom Resource (CR) instance in which you previously defined an AMQP acceptor. Using the OpenShift command-line interface: USD oc edit -f <path/to/custom_resource_instance> .yaml Using the OpenShift Container Platform web console: In the left navigation menu, click Administration Custom Resource Definitions Click the ActiveMQArtemis CRD. Click the Instances tab. Locate the CR instance that corresponds to your project namespace. A previously-configured AMQP acceptor might resemble the following: spec: ... acceptors: - name: my-acceptor protocols: amqp port: 5672 connectionsAllowed: 5 expose: true sslEnabled: true ... Specify the minimum size, in bytes, of an AMQP message that the broker handles as a large message. For example: spec: ... acceptors: - name: my-acceptor protocols: amqp port: 5672 connectionsAllowed: 5 expose: true sslEnabled: true amqpMinLargeMessageSize: 204800 ... ... In the preceding example, the broker is configured to accept AMQP messages on port 5672. Based on the value of amqpMinLargeMessageSize , if the acceptor receives an AMQP message with a body larger than or equal to 204800 bytes (that is, 200 kilobytes), the broker stores the message as a large message. The broker stores the message in the large messages directory ( /opt/ <custom_resource_name> /data/large-messages , by default) on the persistent volume (PV) used by the broker for message storage. If you do not explicitly specify a value for the amqpMinLargeMessageSize property, the broker uses a default value of 102400 (that is, 100 kilobytes). If you set amqpMinLargeMessageSize to a value of -1 , large message handling for AMQP messages is disabled. 4.15. Configuring broker health checks You can configure health checks on AMQ Broker by using startup, liveness and readiness probes. A startup probe indicates whether the application within a container is started. A liveness probe determines if a container is still running. A readiness probe determines if a container is ready to accept service requests If a startup probe or a liveness probe check of a Pod fails, the probe restarts the Pod. AMQ Broker includes default readiness and liveness probes. The default liveness probe checks if the broker is running by pinging the broker's HTTP port. The default readiness probe checks if the broker can accept network traffic by opening a connection to each of the acceptor ports configured for the broker. A limitation of using the default liveness and readiness probes is that they are unable to identify underlying issues, for example, issues with the broker's file system. You can create custom liveness and readiness probes that use the broker's command-line utility, artemis , to run more comprehensive health checks. AMQ Broker does not include a default startup probe. You can configure a startup probe in the ActiveMQArtemis Custom Resource (CR). 4.15.1. Configuring a startup probe You can configure a startup probe to check if the AMQ Broker application within the broker container has started. Procedure Edit the CR instance for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR for your deployment. Using the OpenShift Container Platform web console: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click the instance for your broker deployment. Click the YAML tab. Within the console, a YAML editor opens, enabling you to edit the CR instance. In the deploymentPlan section of the CR, add a startupProbe section. For example: spec: deploymentPlan: startupProbe: exec: command: - /bin/bash - '-c' - /opt/amq/bin/artemis - 'check' - 'node' - '--up' - '--url' - 'tcp://USDHOSTNAME:61616' initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds: 3 failureThreshold: 30 command The startup probe command to run within the container. In the example, the startup probe uses the artemis check node command to verify that AMQ Broker has started in the container for a broker Pod. initialDelaySeconds The delay, in seconds, before the probe runs after the container starts. The default is 0 . periodSeconds The interval, in seconds, at which the probe runs. The default is 10 . timeoutSeconds Time, in seconds, that the startup probe command waits for a reply from the broker. If a response to the command is not received, the command is terminated. The default value is 1 . failureThreshold The minimum consecutive failures, including timeouts, of the startup probe after which the probe is deemed to have failed. When the probe is deemed to have failed, it restarts the Pod. The default value is 3 . Depending on the resources of the cluster and the size of the broker journal, you might need to increase the failure threshold to allow the broker sufficient time to start and pass the probe check. Otherwise, the broker enters a loop condition whereby the failure threshold is reached repeatedly and the broker is restarted each time by the startup probe. For example, if you set the failureThreshold to 30 and the probe runs at the default interval of 10 seconds, the broker has 300 seconds to start and pass the probe check. Save the CR. Additional resources For more information about liveness and readiness probes in OpenShift Container Platform, see Monitoring application health by using health checks in the OpenShift Container Platform documentation. 4.15.2. Configuring liveness and readiness probes The following example shows how to configure the main Custom Resource (CR) instance for your broker deployment to run health checks by using liveness and readiness probes. Prerequisites You should be familiar with how to use a CR instance to create a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . Procedure Edit the CR instance for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR for your deployment. Using the OpenShift Container Platform web console: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click the instance for your broker deployment. Click the YAML tab. To configure a liveness probe, in the deploymentPlan section of the CR, add a livenessProbe section. For example: spec: deploymentPlan: livenessProbe: initialDelaySeconds: 5 periodSeconds: 5 failureThreshold: 30 initialDelaySeconds The delay, in seconds, before the probe runs after the container starts. The default is 5 . Note If the deployment also has a startup probe configured, you can set the delay to 0 for both a liveness and a readiness probe. Both of these probes run only after the startup probe has passed. If the startup probe has already passed, it confirms that the broker has started successfully, so a delay in running the liveness and readiness probes is not required. periodSeconds The interval, in seconds, at which the probe runs. The default is 5 . failureThreshold The minimum consecutive failures, including timeouts, of the liveness probe that signify the probe has failed. When the probe fails, it restarts the Pod. The default value is 3. If your deployment does not have a startup probe configured, which verifies that the broker application is started before the liveness probe runs, you might need to increase the failure threshold to allow the broker sufficient time to start and pass the liveness probe check. Otherwise, the broker can enter a loop condition whereby the failure threshold is reached repeatedly and the broker Pod is restarted each time by the liveness probe. The time required by the broker to start and pass a liveness probe check depends on the resources of the cluster and the size of the broker journal. For example, if you set the failureThreshold to 30 and the probe runs at the default interval of 5 seconds, the broker has 150 seconds to start and pass the liveness probe check. Note If you do not configure a liveness probe or if the handler is missing from a configured probe, the AMQ Broker Operator creates a default TCP probe that has the following configuration. The default TCP probe attempts to open a socket to the broker container on the specified port. spec: deploymentPlan: livenessProbe: tcpSocket: port: 8181 initialDelaySeconds: 30 timeoutSeconds: 5 To configure a readiness probe, in the deploymentPlan section of the CR, add a readinessProbe section. For example: spec: deploymentPlan: readinessProbe: initialDelaySeconds: 5 periodSeconds: 5 If you don't configure a readiness probe, a built-in script checks if all acceptors can accept connections. If you want to configure more comprehensive health checks, add the artemis check command-line utility to the liveness or readiness probe configuration. If you want to configure a health check that creates a full client connection to the broker, in the livenessProbe or readinessProbe section, add an exec section. In the exec section, add a command section. In the command section, add the artemis check node command syntax. For example: spec: deploymentPlan: readinessProbe: exec: command: - bash - '-c' - /home/jboss/amq-broker/bin/artemis - check - node - '--silent' - '--acceptor' - < acceptor name > - '--user' - USDAMQ_USER - '--password' - USDAMQ_PASSWORD initialDelaySeconds: 30 timeoutSeconds: 5 By default, the artemis check node command uses the URI of an acceptor called artemis . If the broker has an acceptor called artemis , you can exclude the --acceptor <acceptor name> option from the command. Note USDAMQ_USER and USDAMQ_PASSWORD are environment variables that are configured by the AMQ Operator. If you want to configure a health check that produces and consumes messages, which also validates the health of the broker's file system, in the livenessProbe or readinessProbe section, add an exec section. In the exec section, add a command section. In the command section, add the artemis check queue command syntax. For example: spec: deploymentPlan: readinessProbe: exec: command: - bash - '-c' - /home/jboss/amq-broker/bin/artemis - check - queue - '--name' - livenessqueue - '--produce' - "1" - '--consume' - "1" - '--silent' - '--user' - USDAMQ_USER - '--password' - USDAMQ_PASSWORD initialDelaySeconds: 30 timeoutSeconds: 5 Note The queue name that you specify must be configured on the broker and have a routingType of anycast . For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisAddress metadata: name: livenessqueue namespace: activemq-artemis-operator spec: addressName: livenessqueue queueConfiguration: purgeOnNoConsumers: false maxConsumers: -1 durable: true enabled: true queueName: livenessqueue routingType: anycast Save the CR. Additional resources For more information about liveness and readiness probes in OpenShift Container Platform, see Monitoring application health by using health checks in the OpenShift Container Platform documentation. 4.16. Enabling message migration to support cluster scaledown If you want to be able to scale down the number of brokers in a cluster and migrate messages to remaining Pods in the cluster, you must enable message migration. When you scale down a cluster that has message migration enabled, a scaledown controller manages the message migration process. 4.16.1. Steps in message migration process The message migration process follows these steps: When a broker Pod in the deployment shuts down due to an intentional scaledown of the deployment, the Operator automatically deploys a scaledown Custom Resource to prepare for message migration. To check for Persistent Volumes (PVs) that have been orphaned, the scaledown controller looks at the ordinal on the volume claim. The controller compares the ordinal on the volume claim to that of the broker Pods that are still running in the StatefulSet (that is, the broker cluster) in the project. If the ordinal on the volume claim is higher than the ordinal on any of the broker Pods still running in the broker cluster, the scaledown controller determines that the broker Pod at that ordinal has been shut down and that messaging data must be migrated to another broker Pod. The scaledown controller starts a drainer Pod. The drainer Pod connects to one of the other live broker Pods in the cluster and migrates messages to that live broker Pod. The following figure illustrates how the scaledown controller (also known as a drain controller ) migrates messages to a running broker Pod. Figure 4.1. Message migration using the scaledown controller After the messages are migrated successfully to an operational broker Pod, the drainer Pod shuts down and the scaledown controller removes the PVC for the orphaned PV. The PV is returned to a "Released" state. Note If the reclaim policy for the PV is set to retain , the PV cannot be used by another Pod until you delete and recreate the PV. For example, if you scale the cluster up after scaling it down, the PV is not available to a Pod started until you delete and recreate the PV. Additional resources For an example of message migration when you scale down a broker deployment, see Section 4.16.2, "Enabling message migration" . 4.16.2. Enabling message migration You can enable message migration in the ActiveMQArtemis Custom Resource (CR). Prerequisites You already have a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . You understand how message migration works. For more information, see Section 4.16.1, "Steps in message migration process" . Note A scaledown controller operates only within a single OpenShift project. The controller cannot migrate messages between brokers in separate projects. If you scale a broker deployment down to 0 (zero), message migration does not occur, since there is no running broker Pod to which messaging data can be migrated. However, if you scale a deployment down to zero and then back up to a size that is smaller than the original deployment, drainer Pods are started for the brokers that remain shut down. Procedure Edit the CR instance for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR for your deployment. Using the OpenShift Container Platform web console: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click the instance for your broker deployment. Click the YAML tab. Within the console, a YAML editor opens, enabling you to edit the CR instance. In the deploymentPlan section of the CR, add a messageMigration attribute and set to true . If not already configured, add a persistenceEnabled attribute and also set to true . For example: spec: deploymentPlan: messageMigration: true persistenceEnabled: true ... These settings mean that when you later scale down the size of your clustered broker deployment, the Operator automatically starts a scaledown controller and migrates messages to a broker Pod that is still running. Save the CR. (Optional) Complete the following steps to scale down the cluster and view the message migration process. In your existing broker deployment, verify which Pods are running. USD oc get pods You see output that looks like the following. The preceding output shows that there are three Pods running; one for the broker Operator itself, and a separate Pod for each broker in the deployment. Log into each Pod and send some messages to each broker. Supposing that Pod ex-aao-ss-0 has a cluster IP address of 172.17.0.6 , run the following command: USD /opt/amq/bin/artemis producer --url tcp://172.17.0.6:61616 --user admin --password admin Supposing that Pod ex-aao-ss-1 has a cluster IP address of 172.17.0.7 , run the following command: USD /opt/amq/bin/artemis producer --url tcp://172.17.0.7:61616 --user admin --password admin The preceding commands create a queue called TEST on each broker and add 1000 messages to each queue. Scale the cluster down from two brokers to one. Open the main broker CR, broker_activemqartemis_cr.yaml . In the CR, set deploymentPlan.size to 1 . At the command line, apply the change: USD oc apply -f deploy/crs/broker_activemqartemis_cr.yaml You see that the Pod ex-aao-ss-1 starts to shut down. The scaledown controller starts a new drainer Pod of the same name. This drainer Pod also shuts down after it migrates all messages from broker Pod ex-aao-ss-1 to the other broker Pod in the cluster, ex-aao-ss-0 . When the drainer Pod is shut down, check the message count on the TEST queue of broker Pod ex-aao-ss-0 . You see that the number of messages in the queue is 2000, indicating that the drainer Pod successfully migrated 1000 messages from the broker Pod that shut down. 4.17. Controlling placement of broker pods on OpenShift Container Platform nodes You can control the placement of AMQ Broker pods on OpenShift Container Platform nodes by using node selectors, tolerations, or affinity and anti-affinity rules. Node selectors A node selector allows you to schedule a broker pod on a specific node. Tolerations A toleration enables a broker pod to be scheduled on a node if the toleration matches a taint configured for the node. Without a matching pod toleration, a taint allows a node to refuse to accept a pod. Affinity/Anti-affinity Node affinity rules control which nodes a pod can be scheduled on based on the node's labels. Pod affinity and anti-affinity rules control which nodes a pod can be scheduled on based on the pods already running on that node. 4.17.1. Placing pods on specific nodes using node selectors A node selector specifies a key-value pair that requires the broker pod to be scheduled on a node that has matching key-value pair in the node label. The following example shows how to configure a node selector to schedule a broker pod on a specific node. Prerequisites You should be familiar with how to use a CR instance to create a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . Add a label to the OpenShift Container Platform node on which you want to schedule the broker pod. For more information about adding node labels, see Using node selectors to control pod placement in the OpenShift Container Platform documentation. Procedure Create a Custom Resource (CR) instance based on the main broker CRD. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. Start a new CR instance based on the main broker CRD. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click Create ActiveMQArtemis . Within the console, a YAML editor opens, enabling you to configure a CR instance. In the deploymentPlan section of the CR, add a nodeSelector section and add the node label that you want to match to select a node for the pod. For example: spec: deploymentPlan: nodeSelector: app: broker1 In this example, the broker pod is scheduled on a node that has a app: broker1 label. Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project in which you are creating the broker deployment. Create the CR instance. Using the OpenShift web console: When you have finished configuring the CR, click Create . Additional resources For more information about node selectors in OpenShift Container Platform, see Placing pods on specific nodes using node selectors in the OpenShift Container Platform documentation. 4.17.2. Controlling pod placement using tolerations Taints and tolerations control whether pods can or cannot be scheduled on specific nodes. A taint allows a node to refuse to schedule a pod unless the pod has a matching toleration. You can use taints to exclude pods from a node so the node is reserved for specific pods, such as broker pods, that have a matching toleration. Having a matching toleration permits a broker pod to be scheduled on a node but does not guarantee that the pod is scheduled on that node. To guarantee that the broker pod is scheduled on the node that has a taint configured, you can configure affinity rules. For more information, see Section 4.17.3, "Controlling pod placement using affinity and anti-affinity rules" The following example shows how to configure a toleration to match a taint that is configured on a node. Prerequisites You should be familiar with how to use a CR instance to create a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . Apply a taint to the nodes which you want to reserve for scheduling broker pods. A taint consists of a key, value, and effect. The taint effect determines if: existing pods on the node are evicted existing pods are allowed to remain on the node but new pods cannot be scheduled unless they have a matching toleration new pods can be scheduled on the node if necessary, but preference is to not schedule new pods on the node. For more information about applying taints, see Controlling pod placement using node taints in the OpenShift Container Platform documentation. Procedure Create a Custom Resource (CR) instance based on the main broker CRD. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. Start a new CR instance based on the main broker CRD. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click Create ActiveMQArtemis . Within the console, a YAML editor opens, enabling you to configure a CR instance. In the deploymentPlan section of the CR, add a tolerations section. In the tolerations section, add a toleration for the node taint that you want to match. For example: spec: deploymentPlan: tolerations: - key: "app" value: "amq-broker" effect: "NoSchedule" In this example, the toleration matches a node taint of app=amq-broker:NoSchedule , so the pod can be scheduled on a node that has this taint configured. Note To ensure that the broker pods are scheduled correctly, do not specify a tolerationsSeconds attribute in the tolerations section of the CR. Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project in which you are creating the broker deployment. Create the CR instance. Using the OpenShift web console: When you have finished configuring the CR, click Create . Additional resources For more information about taints and tolerations in OpenShift Container Platform, see Controlling pod placement using node taints in the OpenShift Container Platform documentation. 4.17.3. Controlling pod placement using affinity and anti-affinity rules You can control pod placement using node affinity, pod affinity, or pod anti-affinity rules. Node affinity allows a pod to specify an affinity towards a group of target nodes. Pod affinity and anti-affinity allows you to specify rules about how pods can or cannot be scheduled relative to other pods that are already running on a node. 4.17.3.1. Controlling pod placement using node affinity rules Node affinity allows a broker pod to specify an affinity towards a group of nodes that it can be placed on. A broker pod can be scheduled on any node that has a label with the same key-value pair as the affinity rule that you create for a pod. The following example shows how to configure a broker to control pod placement by using node affinity rules. Prerequisites You should be familiar with how to use a CR instance to create a basic broker deployment. See Section 3.4.1, "Deploying a basic broker instance" . Assign a common label to the nodes in your OpenShift Container Platform cluster that can schedule the broker pod, for example, zone: emea . Procedure Create a Custom Resource (CR) instance based on the main broker CRD. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Open the sample CR file called broker_activemqartemis_cr.yaml that was included in the deploy/crs directory of the Operator installation archive that you downloaded and extracted. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. Start a new CR instance based on the main broker CRD. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click Create ActiveMQArtemis . Within the console, a YAML editor opens, enabling you to configure a CR instance. In the deploymentPlan section of the CR, add the following sections: affinity , nodeAffinity , requiredDuringSchedulingIgnoredDuringExecution , and nodeSelectorTerms . In the nodeSelectorTerms section, add the - matchExpressions parameter and specify the key-value string of a node label to match. For example: spec: deploymentPlan: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: zone operator: In values: - emea In this example, the affinity rule allows the pod to be scheduled on any node that has a label with a key of zone and a value of emea . Deploy the CR instance. Using the OpenShift command-line interface: Save the CR file. Switch to the project in which you are creating the broker deployment. Create the CR instance. Using the OpenShift web console: When you have finished configuring the CR, click Create . Additional resources For more information about affinity rules in OpenShift Container Platform, see Controlling pod placement on nodes using node affinity rules in the OpenShift Container Platform documentation. 4.17.3.2. Placing pods relative to other pods using anti-affinity rules Anti-affinity rules allow you to constrain which Openshift node a broker pod can be scheduled on based on the labels of pods already running on the node. You can use anti-affinity rules to ensure that multiple broker pods are not scheduled on the same Openshift node, which creates a single point of failure. Prerequisites You created two separate broker deployments. See Section 3.4.1, "Deploying a basic broker instance" . Procedure Edit the ActiveMQArtemis CR instance for your first broker deployment. In the deploymentPlan section of the CR, add a labels section. Create an identifying label for the broker so that you can create an anti-affinity rule based on the label in your second deployment. For example: spec: ... deploymentPlan: labels: name: broker1 Save the CR. Edit the ActiveMQArtemis CR instance for your second broker deployment. In the deploymentPlan section of the CR, add the following sections: affinity , podAntiAffinity , requiredDuringSchedulingIgnoredDuringExecution , and labelSelector . In the labelSelector section, add the matchExpressions parameter and specify the key-value string of the label to match. A pod in this deployment cannot be scheduled on a node that contains a pod with the matching label. spec: deploymentPlan: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: name operator: In values: - broker1 topologyKey: topology.kubernetes.io/zone In this example, the pod anti-affinity rule prevents a pod from being placed on the same node as a pod that has a label with a key of name and a value of broker1 , which is the label assigned to the broker in your first deployment. Save the CR. Additional resources For more information about affinity rules in OpenShift Container Platform, see Controlling pod placement on nodes using node affinity rules in the OpenShift Container Platform documentation. 4.18. Configuring logging for brokers AMQ Broker uses the Log4j 2 logging utility to provide message logging. When you deploy a broker, it uses a default Log4j 2 configuration. If you want to change the default configuration, you must create a new Log4j 2 configuration in either a secret or a configMap. After you add the name of the secret or configMap to the main broker Custom Resource (CR), the Operator configures each broker to use the new logging configuration, which is stored in a file that the Operator mounts on each Pod. Prerequisite You are familiar with the Log4j 2 configuration options. Procedure Prepare a file that contains the log4j 2 configuration that you want to use with AMQ Broker. The default Log4j 2 configuration file that is used by a broker is located in the /home/jboss/amq-broker/etc/log4j2.properties file on each broker Pod. You can use the contents of the default configuration file as the basis for creating a new Log4j 2 configuration in a secret or configMap. To get the contents of the default Log4j 2 configuration file, complete the following steps. Using the OpenShift Container Platform web console: Click Workloads Pods . Click the ex-aao-ss Pod. Click the Terminal tab. Use the cat command to display the contents of the /home/jboss/amq-broker/etc/log4j2.properties file on a broker Pod and copy the contents. Paste the contents into a local file, where the OpenShift Container Platform CLI is installed, and save the file as logging.properties . Using the OpenShift command-line interface: Get the name of a Pod in your deployment. Use the oc cp command to copy the log configuration file from a Pod to your local directory. Where the < name > part of the container name is the prefix before the -ss string in the Pod name. For example: Note When you create a configMap or secret from a file, the key in the configMap or secret defaults to the file name and the value defaults to the file content. By creating a secret from a file named logging.properties , the required key for the new logging configuration is inserted in the secret or configMap. Edit the logging.properties file and create the Log4j 2 configuration that you want to use with AMQ Broker. For example, with the default configuration, AMQ Broker logs messages to the console only. You might want to update the configuration so that AMQ Broker logs messages to disk also. Add the updated Log4j 2 configuration to a secret or a ConfigMap. Log in to OpenShift as a user that has privileges to create secrets or ConfigMaps in the project for the broker deployment. If you want to configure the log settings in a secret, use the oc create secret command. For example: If you want to configure the log settings in a ConfigMap, use the oc create configmap command. For example: The configMap or secret name must have a suffix of -logging-config , so the Operator can recognize that the secret contains new logging configuration. Add the secret or ConfigMap to the Custom Resource (CR) instance for your broker deployment. Using the OpenShift command-line interface: Log in to OpenShift as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR. Using the OpenShift Container Platform web console: Log in to the console as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Operators Installed Operator . Click the Red Hat Integration - AMQ Broker for RHEL 8 (Multiarch) operator. Click the AMQ Broker tab. Click the name of the ActiveMQArtemis instance name Click the YAML tab. Within the console, a YAML editor opens, enabling you to configure a CR instance. Add the secret or configMap that contains the Log4j 2 logging configuration to the CR. The following examples show a secret and a configMap added to the CR. apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: ... extraMounts: secrets: - "newlog4j-logging-config" ... apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: ... extraMounts: configMaps: - "newlog4j-logging-config" ... Save the CR. In each broker Pod, the Operator mounts a logging.properties file that contains the logging configuration in the secret or configMap that you created. In addition, the Operator configures each broker to use the mounted log configuration file instead of the default log configuration file. Note If you update the logging configuration in a configMap or secret, each broker automatically uses the updated logging configuration. 4.19. Configuring a Pod disruption budget A Pod disruption budget specifies the minimum number of Pods in a cluster that must be available simultaneously during a voluntary disruption, such as a maintenance window. Procedure Edit the CR instance for the broker deployment. Using the OpenShift command-line interface: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. Edit the CR for your deployment. Using the OpenShift Container Platform web console: Log in to OpenShift Container Platform as a user that has privileges to deploy CRs in the project for the broker deployment. In the left pane, click Administration Custom Resource Definitions . Click the ActiveMQArtemis CRD. Click the Instances tab. Click the instance for your broker deployment. Click the YAML tab. Within the console, a YAML editor opens, enabling you to edit the CR instance. In the spec section of the CR, add a podDisruptionBudget element and specify the minimum number of Pods in your deployment that must be available during a voluntary disruption. In the following example, a minimum of one Pod must be available: spec: ... podDisruptionBudget: minAvailable: 1 ... Save the CR. Additional resources For more information about Pod disruption budgets, see Understanding how to use pod disruption budgets to specify the number of pods that must be up in the OpenShift Container Platform documentation. 4.20. Configuring role-based access control for management operations Role-based access control (RBAC) is used to restrict access to the attributes and methods of MBeans. MBeans are the way the management API is exposed by AMQ Broker to support management operations. Previously, you could restrict access to MBeans by setting the RBAC configuration in the ActiveMQArtemisSecurity custom resource (CR) and restarting the broker for the changes to take effect. Starting in 7.12, you can restrict access to MBeans in the ActiveMQArtemis CR and a broker restart is not required for the changes to take effect. Procedure Edit the ActiveMQArtemis CR instance for your broker deployment. Add the following environment variable to configure the broker to use the RBAC configuration that you specify in the ActiveMQArtemis CR. spec: .. env: - name: JAVA_ARGS_APPEND value: "-Dhawtio.role=* -Djavax.management.builder.initial=org.apache.activemq.artemis.core.server.management.ArtemisRbacMBeanServerBuilder" .. In the brokerProperties attribute, add the role based access control configuration for management operations. The format of the match addresses for management operations is: mops.< resource type >.< resource name >.< operation > For example, the following configuration grants a manager role view and edit permission to an activemq.management address. The asterisk (*) in the operation position grants access to all operations. spec: .. brokerProperties: - securityRoles."mops.address.activemq.management.*".manager.view=true - securityRoles."mops.address.activemq.management.*".manager.edit=true In the following example, the number sign (#) after the mops prefix grants the amq role view and edit permissions to all MBeans. spec: .. brokerProperties: - securityRoles."mops.#".amq.view=true - securityRoles."mops.#".amq.edit=true .. Use the resourceTemplates attribute to define an init container that runs a script to remove the default RBAC configuration in the /amq/init/config/amq-broker/etc/management.xml file in each broker container, as shown in the following example. You must remove the default RBAC configuration so the broker uses the new RBAC configuration that you created in the ActiveMQArtemis CR. spec: .. resourceTemplates: - selector: kind: "StatefulSet" patch: kind: "StatefulSet" spec: template: spec: initContainers: - name: "< BROKER_NAME >-container-init" args: - '-c' - '/opt/amq/bin/launch.sh && /opt/amq-broker/script/default.sh; echo "Empty management.xml";echo "<management-context xmlns=\"http://activemq.apache.org/schema\" />" > /amq/init/config/amq-broker/etc/management.xml' Replace < BROKER_NAME > with the value of the metadata.name attribute in your CR instance. Save the CR. 4.21. Customizing Openshift resources created by the Operator An AMQ Broker deployment creates OpenShift resources, such as deployments, pods, statefulsets and service resources. These resources are managed by the AMQ Broker Operator. Only the operator that is responsible for managing a particular OpenShift resource can change that resource. Customizing an operator-managed OpenShift resource can be useful if you want to perform certain tasks, such as: Adding custom annotations that control how resources are treated by other services. Modifying attributes that are not exposed in the broker custom resource. You can use the resourceTemplates attribute to customize resources created by the AMQ Broker Operator. If you want to add an annotation or label to a resource, configure the resourceTemplates attribute to include the annotations or labels attribute. In the following example, the annotations attribute is used to add an annotation to all the services managed by the Operator. spec: .. resourceTemplates: - selector: kind: "Service" annotations: name: "amq-operator-managed" .. Note The selector attribute determines which Operator-managed resources are customized. For example, a selector value of kind: "Service" , customizes all service resources. If the selector attribute is empty, changes are applied to all Operator-managed resources. If you want to customize items other than annotations or labels for resources, you must use the patch attribute with the resourceTemplates attribute. When you specify the patch attribute, the Operator uses a strategic merge to update resources. Note If you use the patch attribute, you must populate the selector attribute to identify specific resources to update. In the following example, the patch attribute is used to change the default value of the minReadySeconds property in the StatefulSet resource. spec: .. resourceTemplates: - selector: kind: "StatefulSet" patch: kind: "StatefulSet" spec: template: spec: minReadySeconds: 10 .. Additional resources For information about strategic merges, see Use a strategic merge patch to update a Deployment . 4.22. Registering plugins with AMQ Broker You can extend the functionality of AMQ Broker by registering plugins in the brokerProperties attribute in the CR. Procedure Edit the custom resource (CR) for your broker deployment. In the brokerProperties attribute, specify the class name of the plugin and include a comma-separated string of < key >=< value > pairs that define the properties for the plugin. In the following example, the LoggingActiveMQServerPlugin plugin, which is provided with AMQ Broker, is registered. spec: ... brokerProperties: - brokerPlugins.\"org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin.class\".init=LOG_CONNECTION_EVENTS=true,LOG_SESSION_EVENTS=true,LOG_CONSUMER_EVENTS=true ... Save the CR. After an instance of the plugin is created, the init method is passed a string that contains the < key >=< value > pairs, which are used to set properties for the plugin. Note If you create a custom plugin, ensure that the JAR files for the plugin class are in the Java classpath of the broker. For more information, see Section 4.4, "Adding third-party JAR files" . 4.22.1. Segregating the brokerProperties configuration If your CR contains a brokerProperties section and the CR is at the maximum size limit of 1 MB, you can segregate the brokerProperties configuration in one or more Java properties files. You might want to also segregate the brokerProperties configuration in separate files to logically group the brokerProperties items for easier maintenance. Procedure Create a file in Java properties format that contains the brokerProperties configuration that you want to apply to the broker. Add each property in a separate line in the properties file. For example: securityRoles.address1.group2.send=true securityRoles.address2.group1.consume=true securityRoles.address2.group2.createAddress=true Save the file with a .properties extension, for example, securityRoles.properties . Create a secret that contains the .properties file you created. Note The secret name must have a suffix of -bp . When a secret has a -bp suffix, the Operator configures the broker to search for properties files in the directory where the secret is mounted on the broker pod. Add a reference to the secret in the extraMounts attribute so the Operator mounts the properties files that are in the secret on each broker pod: deploymentPlan: ... extraMounts: secrets: - "address-settings-bp" ... The Operator mounts the .properties files that are in the secret in a /amq/extra/secrets/ <secret name> directory on each broker pod. At startup, the broker searches each mounted directory for files that have a .properties extension, sorts the files alphabetically, and applies the configuration in the files one after another. Within a properties file, the broker applies the properties in the order in which they are listed. 4.23. Configuring leader-follower broker deployments for high availability A leader-follower configuration has a single broker in separate deployments. The broker in each deployment must be configured to use the same JDBC database to persist messages. High availability is achieved by the brokers competing to acquire a JDBC lock, which grants exclusive access to the database. The broker that acquires the JDBC lock becomes the leader broker, which serves client requests. The broker that fails to acquire the JDBC lock becomes a follower. A follower continuously tries to obtain the JDBC lock and, if successful, immediately becomes the leader to serve clients. Leader-follower deployments provide a faster mean time to repair (MTTR) to recover from a node failure than that provided by Openshift for a single deployment with one or more brokers. In leader-follower deployments, the brokers can be on separate clusters to protect against a cluster failure. These clusters can be in different data centers to also make the broker service resilient to a data center outage. Prerequisite You have a container image that contains the JAR file for the JDBC database you want to use with AMQ Broker. For information on creating container images, see Creating images in the Openshift documentation. In the configuration for each broker, you can specify an init container to copy the JAR file from the container image to a location that is available to the broker at runtime. Configure two ActiveMQArtemis custom resource instances to create separate broker deployments. In each custom resource, specify a unique name and ensure that the clustered and persistenceEnabled attributes are set to false . Set the size attribute to 1 to create a single broker in each deployment. For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-a spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-b spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false Note If you are configuring both broker deployments on the same Openshift cluster, ensure that the broker pods are provisioned on separate nodes on the cluster, so both brokers are not affected by a node failure. For more information on controlling the placement of pods on nodes, see Section 4.17, "Controlling placement of broker pods on OpenShift Container Platform nodes" . Add a liveness probe to each broker configuration. If you do not configure a liveness probe, a default probe is enabled to check the health of a broker. The default probe checks that the AMQ Management Console is reachable. In a leader-follower configuration, the AMQ Management Console is not reachable on the broker that is a follower at any given time, which causes the liveness probe to fail on that broker. Each time the liveness probe fails, it restarts the broker, which puts the broker in a persistent restart loop. As a result, the follower broker enters a CrashLoopBackOff state and is not available to become the leader if the current leader fails. To prevent the default liveness probe from running, you must configure a liveness probe that can run successfully when a broker is either a leader or a follower. In the following example, the liveness probe checks that the command to run the broker was executed, which is indicated by the presence of the cli.lock file. spec: .. livenessProbe: exec: command: - test - -f - /home/jboss/amq-broker/lock/cli.lock .. For more information on configuring liveness probes, see Section 4.15.2, "Configuring liveness and readiness probes" . In each broker configuration, enable JDBC database persistence by using the brokerProperties attribute. For example: spec: .. brokerProperties: - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=< class name > - storeConfiguration.jdbcConnectionUrl=jdbc:< Database URL > - HAPolicyConfiguration=SHARED_STORE_PRIMARY - storeConfiguration.jdbcLockRenewPeriodMillis=2000 - storeConfiguration.jdbcLockExpirationMillis=6000 For more information on enabling JDBC database persistence, see Section 4.5.2, "Configuring database persistence" . In each broker configuration, configure the broker to load the JAR file required to connect to the JDBC database. Use the resourceTemplates attribute to customize the StatefulSet resource for each broker. In the customization, use the patch attribute to specify an init container that copies the JAR file from the custom container image you prepared to the broker pod. Use the env attribute to create an ARTEMIS_EXTRA_LIBS environment variable to extend the broker's Java classpath to include the directory to which the JAR file for the JDBC database is copied. By extending the Java classpath, the broker can load the JAR file from the specified directory on the pod at runtime. spec: .. env: - name: ARTEMIS_EXTRA_LIBS value: '/amq/init/config/extra-libs' resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: jdbc-driver-init image: < custom container image with JAR > volumeMounts: - name: amq-cfg-dir mountPath: /amq/init/config command: - "bash" - "-c" - "mkdir -p /amq/init/config/extra-libs && cp <__JAR file_> /amq/init/config/extra-libs" For more information on customizing Openshift resources created by the Operator, see Section 4.21, "Customizing Openshift resources created by the Operator" . Save each custom resource. Example The following example shows the full configuration for leader-follower broker deployments that uses an Oracle database. apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-a spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false livenessProbe: exec: command: - test - -f - /home/jboss/amq-broker/lock/cli.lock env: - name: ARTEMIS_EXTRA_LIBS value: '/amq/init/config/extra-libs' brokerProperties: - criticalAnalyser=false - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=oracle.jdbc.OracleDriver - storeConfiguration.jdbcConnectionUrl=jdbc:< Database URL > - storeConfiguration.jdbcLockRenewPeriodMillis=2000 - storeConfiguration.jdbcLockExpirationMillis=6000 - HAPolicyConfiguration=SHARED_STORE_PRIMARY acceptors: - name: ext-acceptor protocols: CORE port: 61626 expose: true sslEnabled: true sslSecret: ext-acceptor-ssl-secret console: expose: true resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: oracle-database-jdbc-driver-init image: < custom container image with JAR > volumeMounts: - name: amq-cfg-dir mountPath: /amq/init/config command: - "bash" - "-c" - "mkdir -p /amq/init/config/extra-libs && cp < JAR file > /amq/init/config/extra-libs" apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-b spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false livenessProbe: exec: command: - test - -f - /home/jboss/amq-broker/lock/cli.lock env: - name: ARTEMIS_EXTRA_LIBS value: '/amq/init/config/extra-libs' brokerProperties: - criticalAnalyser=false - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=oracle.jdbc.OracleDriver - storeConfiguration.jdbcConnectionUrl=jdbc:< Database URL > - storeConfiguration.jdbcLockRenewPeriodMillis=2000 - storeConfiguration.jdbcLockExpirationMillis=6000 - HAPolicyConfiguration=SHARED_STORE_PRIMARY acceptors: - name: ext-acceptor protocols: CORE port: 61626 expose: true sslEnabled: true sslSecret: ext-acceptor-ssl-secret console: expose: true resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: oracle-database-jdbc-driver-init image: < custom container image with JAR > volumeMounts: - name: amq-cfg-dir mountPath: /amq/init/config command: - "bash" - "-c" - "mkdir -p /amq/init/config/extra-libs && cp < JAR file > /amq/init/config/extra-libs" 4.24. Configuring data mirroring for disaster recovery Mirroring is the process of copying data from a broker to one or more other brokers for disaster recovery. The source and target brokers in a mirror can be on separate OpenShift clusters in different data centers to protect against a data center outage. Mirroring can also be used for data backup or to create a failover broker for use during maintenance windows. Note Messages that existed before a mirror is created are not mirrored. Procedure Configure two ActiveMQArtemis custom resource (CR) instances to create a source broker and a target broker for the mirrored data. Specify a unique name for each. For example: apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: production-broker namespace: production spec: deploymentPlan: size: 1 apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: mirror-broker namespace: dr spec: deploymentPlan: size: 1 In the CR of the target broker, add an acceptor for the mirror connection. For example: metadata: name: mirror-broker namespace: dr spec: ... acceptors: - expose: true name: amqp port: 5672 protocols: amqp ... After you create the acceptor, a route is exposed for the acceptor in the following format: < broker name >-< acceptor name >-< ordinal >-svc-rte.< namespace >.< hostname > When you add the mirror configuration to the source broker later in this procedure, you use this route to create a connection to the target broker. The <ordinal> is the ordinal assigned to the broker pod by the StatefulSet. The first broker pod in a cluster is assigned an ordinal of 0. The second pod is assigned an ordinal of 1 and so on. The ordinal value of a pod is stored in a STATEFUL_SET_ORDINAL variable. You can use this variable instead of the ordinal value in the mirror connection details on the source broker. For example: < broker name >-< acceptor name >-USD{STATEFUL_SET_ORDINAL}-svc-rte.< namespace >.< hostname > By using the STATEFUL_SET_ORDINAL variable, you ensure that a source creates a mirror connection to a target that has the same ordinal if you scale up the number of brokers on the source and target clusters. If you want to send data securely over the mirror connection, configure Transport Layer Security (TLS) for the connection. Depending on your requirements, you can use various methods to generate SSL/TLS certificates. For example, you can use a trusted Certificate Authority (CA), cert-manager Operator for OpenShift or Secure Sockets Layer (SSL) tools. The following example provides a summary of the steps to configure mutual TLS authentication (mTLS) between the brokers by using SSL/TLS tools to manually generate self-signed certificates. Generate a self-signed SSL/TLS certificate for the target broker. For example: keytool -genkey -trustcacerts -alias broker -keyalg RSA -keystore broker.ks -keypass password -storepass password Export the public key of the SSL/TLS certificate you created to a file, so you can import the key into a truststore file for use on the source broker. For example: keytool -export -noprompt -alias broker -keystore broker.ks -file for_source_truststore -storepass password Import the public key of the SSL/TLS certificate that you exported into a truststore file for use on the source broker. For example: keytool -import -noprompt -trustcacerts -alias broker -keystore client.ts -file for_source_truststore -storepass password Generate a self-signed SSL/TLS certificate for the source broker. For example: keytool -genkey -trustcacerts -alias broker -keyalg RSA -keystore broker.ks -keypass password -storepass password Export the public key of the SSL/TLS certificate you created, so you can import the key into a truststore file for use on the target broker. keytool -export -noprompt -alias broker -keystore broker.ks -file for_target_truststore -storepass password Import the public key of the SSL/TLS certificate that you exported into a truststore file for use on the target broker. For example: keytool -import -noprompt -trustcacerts -alias broker -keystore client.ts -file for_target_truststore -storepass password Add the keystore and truststore files you created for the source broker to a secret in the namespace of the source broker. For example: oc create secret generic mirror --from-file=broker.ks=broker.ks --from-file=client.ts=client.ts --from-literal=keyStorePassword=password --from-literal=trustStorePassword=password Repeat this step to add the keystore and truststore files you created for the target broker to a secret in the namespace of the target broker. In the acceptor configured for the target broker, set the sslEnabled attribute to true and specify the name of the secret that you created in the namespace of the target broker. For example: metadata: name: mirror-broker namespace: dr spec: ... acceptors: - expose: true name: amqp port: 5672 protocols: amqp sslEnabled: true sslSecret: mirror ... In the CR of the source broker, add a reference to the secret you created in the namespace of the source broker to the extraMounts attribute. This step is required so the Operator mounts the keystore and truststore files in the secret on each broker pod. For example: spec: ... deploymentPlan: extraMounts: secrets: - mirror ... The keystore and truststore files in the secret are mounted in a /amq/extra/secrets/< secret name > directory on a broker pod. In the CR of the source broker, under the brokerProperties attribute, configure the mirror connection details. For the connection URI, specify the route exposed for the acceptor that you created on the target broker. If you want to use SSL/TLS to secure the mirror connection, also include the following in the URI: a port number of 443 sslEnabled=true to enable SSL/TLS the paths and credentials for the keystore and truststore files For example: spec: ... brokerProperties: - AMQPConnections.datacenter1.uri=tcp://broker-dr-amqp-USD{STATEFUL_SET_ORDINAL}-svc-rte-dr.apps.lab.redhat.com:443?;sslEnabled=true;trustStorePath=/amq/extra/secrets/mirror/client.ts;trustStorePassword=password;keyStorePath=/amq/extra/secrets/mirror/broker.ks;keyStorePassword=password - AMQPConnections.datacenter1.connectionElements.mirror.type=MIRROR ... Note If required, you can configure multiple mirror targets for the source broker. For example: spec: ... brokerProperties: - AMQPConnections.datacenter1.uri=tcp://primary-mirror-broker-amqp-USD{STATEFUL_SET_ORDINAL}-svc.dr.svc.cluster.local:61616 - AMQPConnections.datacenter1.connectionElements.mirror.type=MIRROR - AMQPConnections.datacenter2.uri=tcp://backup-mirror-broker-amqp-USD{STATEFUL_SET_ORDINAL}-svc.dr.svc.cluster.local:61616 - AMQPConnections.datacenter2.connectionElements.mirror.type=MIRROR ... In the CR of the source broker, configure additional mirror configuration properties as required. For example: - AMQPConnections.datacenter1.user=admin - AMQPConnections.datacenter1.password=admin - AMQPConnections.datacenter1.retryInterval=5000 - AMQPConnections.datacenter1.connectionElements.mirror.messageAcknowledgements=true - AMQPConnections.datacenter1.connectionElements.mirror.queueCreation=true - AMQPConnections.datacenter1.connectionElements.mirror.queueRemoval=true - AMQPConnections.datacenter1.connectionElements.mirror.addressFilter= addresses Note You can use any alphanumeric string to name the AMQP connection. In the example, the AMQP connection name is datacenter1 . AMQPConnections.< name >.user The name of the user on the target broker with the permissions to mirror the required events. AMQPConnections.< name >.password The password of the user on the target broker. AMQPConnections.< name >.retryInterval The interval, in milliseconds, between retry attempts to connect to the target broker. AMQPConnections.< name >.connectionElements.mirror.messageAcknowledgements Specifies whether message acknowledgments are mirrored. The default value is true . AMQPConnections.< name >.connectionElements.mirror.queueCreation Specifies whether queue or address creation events are mirrored. The default value is true . AMQPConnections.< name >.connectionElements.mirror.queueRemoval Specifies whether queue or address removal events are mirrored. The default value is true . AMQPConnections.< name >.connectionElements.mirror.addressFilter A filter that the source broker can use to include or exclude addresses for which events are mirrored. For example, you might want to exclude temporary queues from being mirrored. You specify the filter as a comma-separated list of addresses. If you want to specify a list of addresses to exclude, prefix each address with an exclamation mark (!). In the following example, events for addresses that start with us. and europe. are not mirrored. AMQPConnections.< name >.connectionElements.mirror.addressFilter=!us.,!europe. Note If you specify one or more addresses to include, events for all other addresses are not mirrored. If you specify one or more addresses to exclude, events for all other addresses are mirrored. In the status section of the CR for the source broker, verify that the status of the BrokerPropertiesApplied condition is true to confirm that all the properties you specified in the CR were applied. For more information, see Section 3.7, "Viewing status information for your broker deployment" . Check the logs of the source broker pod for a line similar to the following to confirm that the mirror connection was established. broker-prod-ss-0 broker-prod-container Connected on Server AMQP Connection dr on broker-dr-amqp-0-svc-rte-dr.lab.redhat.com:443 after 0 retries

Commands:
<address-settings> <!-- if you define auto-create on certain queues, management has to be auto-create --> <address-setting match="activemq.management#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> </address-setting> <!-- default for catch all --> <address-setting match="#"> <dead-letter-address>DLQ</dead-letter-address> <expiry-address>ExpiryQueue</expiry-address> <redelivery-delay>0</redelivery-delay> <!-- with -1 only the global-max-size is in use for limiting --> <max-size-bytes>-1</max-size-bytes> <message-counter-history-day-limit>10</message-counter-history-day-limit> <address-full-policy>PAGE</address-full-policy> <auto-create-queues>true</auto-create-queues> <auto-create-addresses>true</auto-create-addresses> <auto-create-jms-queues>true</auto-create-jms-queues> <auto-create-jms-topics>true</auto-create-jms-topics> </address-setting> <address-settings>
spec: brokerProperties:
spec: brokerProperties: - addressConfigurations.usa-news-address.routingTypes=MULTICAST
spec: brokerProperties: - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.address=usa-news-address
spec: brokerProperties: - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.routingType=ANYCAST - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.purgeOnNoConsumers=true - addressConfigurations.usa-news-address.queueConfigs.usa-news-queue.maxConsumers=20
login -u <user> -p <password> --server= <host:port>
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisAddress metadata: name: myAddressDeployment0 namespace: myProject spec: addressName: myAddress0 queueName: myQueue0 routingType: anycast
oc project <project_name>
oc create -f <path/to/address_custom_resource_instance> .yaml
spec: brokerProperties: - addressConfigurations.usDeadLetter.routingTypes=MULTICAST - addressConfigurations.usDeadLetter.queueConfigs.usDeadLetter-queue.address=usDeadLetter
spec: brokerProperties: - addressSettings.usa-news.deadLetterAddress=usDeadLetter - addressSettings.usa-news.maxDeliveryAttempts=5
spec: brokerProperties: - addressSettings."usa-news.*".deadLetterAddress=usDeadLetter - addressSettings."europe-news.#".deadLetterAddress=euDeadLetter
edit ActiveMQArtemis < CR instance name > -n < namespace >
spec: addressSettings: addressSetting:
spec: addressSettings: addressSetting: - match: myAddress
spec: addressSettings: addressSetting: - match: myAddress deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 5
spec: addressSettings: addressSetting: - match: myAddress deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 5 - match: 'myOtherAddresses#' deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 3
spec: applyRule: merge_all addressSetting: - match: myAddress deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 5 - match: 'myOtherAddresses#' deadLetterAddress: myDeadLetterAddress maxDeliveryAttempts: 3
spec: brokerProperties: - addressSettings.#.configDeleteAddresses=FORCE - addressSettings.#.configDeleteQueues=FORCE
spec: brokerProperties: - addressConfigurations.usa-news.queueConfigs.usa-news-queue.routingType=MULTICAST - addressConfigurations.usa-news.queueConfigs.usa-news-queue.purgeOnNoConsumers=true - addressConfigurations.usa-news.queueConfigs.usa-news-queue.maxConsumers=20
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisAddress metadata: name: myAddressDeployment0 namespace: myProject spec: addressName: myAddress0 queueName: myQueue0 routingType: anycast
.. spec: addressName: myAddress1 queueName: myQueue1 routingType: anycast removeFromBrokerOnDelete: true
oc apply -f <path/to/address_custom_resource_instance>.yaml
oc delete -f <path/to/address_custom_resource_instance>.yaml
activemq { org.apache.activemq.artemis.spi.core.security.jaas.PropertiesLoginModule sufficient reload=true org.apache.activemq.jaas.properties.user="new-users.properties" org.apache.activemq.jaas.properties.role="new-roles.properties"; org.apache.activemq.artemis.spi.core.security.jaas.PropertiesLoginModule sufficient reload=false org.apache.activemq.jaas.properties.user="artemis-users.properties" org.apache.activemq.jaas.properties.role="artemis-roles.properties" baseDir="/home/jboss/amq-broker/etc"; };
ruben=ruben01! anne=anne01! rick=rick01! bob=bob01!
admin=ruben, rick group1=bob group2=anne
create secret generic custom-jaas-config --from-file=login.config --from-file=new-users.properties --from-file=new-roles.properties
edit ActiveMQArtemis < CR instance name > -n < namespace >
deploymentPlan: extraMounts: secrets: - "custom-jaas-config"
spec: brokerProperties: - securityRoles.#.group2.send=true - securityRoles.#.group1.consume=true - securityRoles.#.group1.createAddress=true - securityRoles.#.group1.createNonDurableQueue=true - securityRoles.#.group1.browse=true
spec: brokerProperties: - 'securityRoles."my-address\:\:my-queue".group2.send=true'
oc get activemqartemis -o yaml
- lastTransitionTime: "2023-02-06T20:50:01Z" message: "" reason: Applied status: "True" type: JaasPropertiesApplied
- lastTransitionTime: "2023-02-06T20:55:20Z" message: 'new-users.properties status out of sync, expected: 287641156, current: 2177044732' reason: OutOfSync status: "False" type: JaasPropertiesApplied
set data secret/custom-jaas-config --from-file=login.config=login.config --from-file=new-users.properties=new-users.properties --from-file=new-roles.properties=new-roles.properties
login -u <user> -p <password> --server= <host:port>
edit ActiveMQArtemis < CR instance name > -n < namespace >
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisSecurity metadata: name: ex-prop spec: loginModules: propertiesLoginModules: - name: "prop-module" users: - name: "sam" password: "samspassword" roles: - "sender" - name: "rob" password: "robspassword" roles: - "receiver" securityDomains: brokerDomain: name: "activemq" loginModules: - name: "prop-module" flag: "sufficient" securitySettings: broker: - match: "#" permissions: - operationType: "send" roles: - "sender" - operationType: "createAddress" roles: - "sender" - operationType: "createDurableQueue" roles: - "sender" - operationType: "consume" roles: - "receiver"
oc project <project_name>
oc create -f <path/to/security_custom_resource_instance> .yaml
create secret generic security-properties-prop-module --from-literal=sam=samspassword --from-literal=rob=robspassword
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisSecurity metadata: name: ex-prop spec: loginModules: propertiesLoginModules: - name: "prop-module" users: - name: "sam" roles: - "sender" - name: "rob" roles: - "receiver" securityDomains: brokerDomain: name: "activemq" loginModules: - name: "prop-module" flag: "sufficient" securitySettings: broker: - match: "#" permissions: - operationType: "send" roles: - "sender" - operationType: "createAddress" roles: - "sender" - operationType: "createDurableQueue" roles: - "sender" - operationType: "consume" roles: - "receiver"
oc project <project_name>
oc create -f <path/to/address_custom_resource_instance> .yaml
create secret generic log4j-template --from-file=log4j-layout-template-json-2.22.1.jar
deploymentPlan: extraMounts: secrets: - "log4j-template"
spec: env: - name: ARTEMIS_EXTRA_LIBS value: /amq/extra/secrets/log4j-template
deploymentPlan: extraVolumes: - name: extra-volume persistentVolumeClaim: claimName: extra-jars extraVolumeMounts: - name: extra-volume mountPath: /opt/extra-lib
spec: resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: mysql-jdbc-driver-init volumeMounts: - mountPath: /opt/extra-lib name: extra-volume image: curlimages/curl:8.6.0 command: - /bin/sh args: - -c - "if ! [ -f /opt/extra-lib/mysql-connector.jar ]; then curl https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.23/mysql-connector-java-8.0.23.jar --output /opt/extra-lib/mysql-connector.jar ; fi"
spec: env: - name: ARTEMIS_EXTRA_LIBS value: /opt/extra-lib
spec: deploymentPlan: persistenceEnabled: true
spec: brokerProperties: - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=< class name > - storeConfiguration.jdbcConnectionUrl=jdbc:< URL > - HAPolicyConfiguration=SHARED_STORE_PRIMARY
login -u <user> -p <password> --server= <host:port>
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true
spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true storage: size: 4Gi
spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true storage: size: 4Gi storageClassName: gp3
oc project <project_name>
oc create -f <path/to/custom_resource_instance> .yaml
login -u <user> -p <password> --server= <host:port>
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true
spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true resources: limits: cpu: "500m" memory: "1024M" requests: cpu: "250m" memory: "512M"
spec: deploymentPlan: size: 3 resources: limits: cpu: "500m" memory: "1024M"
oc project <project_name>
oc create -f <path/to/custom_resource_instance> .yaml
spec: .. console: expose: true
spec: .. console: expose: true ingressHost: my-console-production.my-subdomain.com ..
spec: ingressDomain: my.domain.com
spec: .. console: expose: true exposeMode: ingress ..
spec: .. console: expose: true exposeMode: ingress expose: true exposeMode: ingress ingressHost: my-console-production.my-subdomain.com
spec: ingressDomain: my.domain.com
spec: .. console: expose: true exposeMode: ingress sslEnabled: true ..
spec: .. console: expose: true exposeMode: ingress sslEnabled: true sslSecret: console-tls-secret ..
spec: .. env: - name: JAVA_ARGS_APPEND value: -Dwebconfig.bindings.artemis.sslAutoReload=true ..
edit ActiveMQArtemis < CR instance name > -n < namespace >
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: env: - name: TZ value: Europe/Vienna - name: JAVA_ARGS_APPEND value: --Hawtio.realm=console - name: JDK_JAVA_OPTIONS value: -XshowSettings:system
login -u <user> -p <password> --server= <host:port>
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true
spec: brokerProperties: - globalMaxSize=500m
oc project <project_name>
oc apply -f <path/to/broker_custom_resource_instance> .yaml
FROM registry.redhat.io/amq7/amq-broker-init-rhel8:7.12
edit ActiveMQArtemis < CR instance name > -n < namespace >
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: size: 1 image: placeholder initImage: <custom_init_container_image_url> requireLogin: false persistenceEnabled: true journalType: nio messageMigration: true
spec: .. acceptors: - name: my-acceptor protocols: amqp port: 5672 ..
spec: .. acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672
spec: acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5
spec: acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true sslEnabled: true
spec: acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true ingressHost: my-acceptor-production.my-subdomain.com
spec: ingressDomain: my.domain.com
spec: acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true exposeMode: ingress
spec: acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 connectionsAllowed: 5 expose: true exposeMode: ingress ingressHost: my-acceptor-production.my-subdomain.com
spec: ingressDomain: my-subdomain.domain.com
spec: resourceTemplates: - selector: kind: Service name: amq-broker-myacceptor-0-svc annotations: service.beta.openshift.io/serving-cert-secret-name: myacceptor-ptls
spec: acceptors: - name: myacceptor protocols: CORE port: 61626 sslEnabled: true sslSecret: myacceptor-ptls
spec: brokerProperties - "acceptorConfigurations.myacceptor.params.sslAutoReload=true"
spec: .. acceptors: - name: myacceptor protocols: all port: 62666 sslEnabled: true sslSecret: myacceptor-ptls needClientAuth: true ..
spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true sslSecret: myacceptor-ptls needClientAuth: true trustSecret: trusted-clients-bundle ..
apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: root-issuer spec: selfSigned: {}
apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: root-ca namespace: cert-manager spec: isCA: true commonName: "amq.io.root" secretName: root-ca-secret subject: organizations: - "www.amq.io" issuerRef: name: root-issuer kind: ClusterIssuer
apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: root-ca-issuer spec: ca: secretName: root-ca-secret
apiVersion: cert-manager.io/v1 kind: Certificate Metadata: name: broker-cert spec: isCA: false commonName: "amq.io" dnsNames: - "amq-broker-ss-0.amq-broker-svc-rte-default.cluster.local - "amq-broker-ss-1.amq-broker-svc-rte-default.cluster.local secretName: broker-cert-secret subject: organizations: - "www.amq.io" issuerRef: name: root-ca-issuer kind: ClusterIssuer
oc create -f self-signed-issuer.yaml oc create -f root-ca.yaml oc create -f root-ca-issuer.yaml oc create -f broker-cert.yaml
spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true needClientAuth: false sslSecret: broker-cert-secret ..
spec: brokerProperties - "acceptorConfigurations.new-acceptor.params.sslAutoReload=true"
spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true sslSecret: broker-cert-secret needClientAuth: true ..
spec: .. acceptors: - name: new-acceptor protocols: all port: 62666 sslEnabled: true sslSecret: broker-cert-secret needClientAuth: true trustSecret: trusted-clients-bundle ..
keytool -genkey -alias broker -keyalg RSA -keystore ~/broker.ks
keytool -export -alias broker -keystore ~/broker.ks -file ~/broker_cert.pem
keytool -import -alias broker -keystore ~/client.ts -file ~/broker_cert.pem
oc login -u system:admin
oc project <my_openshift_project>
oc create secret generic my-tls-secret --from-file=broker.ks=~/broker.ks --from-file=client.ts=~/client.ks --from-literal=keyStorePassword= <password> --from-literal=trustStorePassword= <password>
oc secrets link sa/amq-broker-operator secret/my-tls-secret
spec: acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 sslEnabled: true sslSecret: my-tls-secret expose: true connectionsAllowed: 5
keytool -genkey -alias broker -keyalg RSA -keystore ~/broker.ks
keytool -export -alias broker -keystore ~/broker.ks -file ~/broker_cert.pem
keytool -import -alias broker -keystore ~/client.ts -file ~/broker_cert.pem
keytool -genkey -alias broker -keyalg RSA -keystore ~/client.ks
keytool -export -alias broker -keystore ~/client.ks -file ~/client_cert.pem
keytool -import -alias broker -keystore ~/broker.ts -file ~/client_cert.pem
oc login -u system:admin
oc project <my_openshift_project>
oc create secret generic my-tls-secret --from-file=broker.ks=~/broker.ks --from-file=client.ts=~/broker.ts --from-literal=keyStorePassword= <password> --from-literal=trustStorePassword= <password>
oc secrets link sa/amq-broker-operator secret/my-tls-secret
spec: acceptors: - name: my-acceptor protocols: amqp,openwire port: 5672 sslEnabled: true sslSecret: my-tls-secret expose: true connectionsAllowed: 5
CN=my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain
CN=my-broker-deployment-*-svc-rte-my-openshift-project.my-openshift-domain
"SAN=DNS:my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain,DNS:my-broker-deployment-1-svc-rte-my-openshift-project.my-openshift-domain,..."
tcp://ex-aao-ss-0:<port>
curl https://my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain
tcp://my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain:443?useTopologyForLoadBalancing=false&sslEnabled=true &trustStorePath=~/client.ts&trustStorePassword= <password>
tcp://my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain:443?useTopologyForLoadBalancing=false&sslEnabled=true &keyStorePath=~/client.ks&keyStorePassword= <password> &trustStorePath=~/client.ts&trustStorePassword= <password>
ssl://my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain:443" Also, specify the following JVM flags -Djavax.net.ssl.trustStore=~/client.ts -Djavax.net.ssl.trustStorePassword= <password>
ssl://my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain:443" Also, specify the following JVM flags -Djavax.net.ssl.keyStore=~/client.ks -Djavax.net.ssl.keyStorePassword= <password> -Djavax.net.ssl.trustStore=~/client.ts -Djavax.net.ssl.trustStorePassword= <password>
amqps://my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain:443?transport.verifyHost=true &transport.trustStoreLocation=~/client.ts&transport.trustStorePassword= <password>
amqps://my-broker-deployment-0-svc-rte-my-openshift-project.my-openshift-domain:443?transport.verifyHost=true &transport.keyStoreLocation=~/client.ks&transport.keyStorePassword= <password> &transport.trustStoreLocation=~/client.ts&transport.trustStorePassword= <password>
keytool -storetype jks -keystore server-keystore.jks -storepass artemis -keypass artemis -alias server -genkey -keyalg "RSA" -keysize 2048 -dname "CN=AMQ Server, OU=Artemis, O=ActiveMQ, L=AMQ, S=AMQ, C=AMQ" -validity 365 -ext bc=ca:false -ext eku=sA -ext san=dns:*.ex-aao-hdls-svc.test.svc.cluster.local
keytool -storetype jks -keystore server-keystore.jks -storepass artemis -keypass artemis -alias server -genkey -keyalg "RSA" -keysize 2048 -dname "CN=AMQ Server, OU=Artemis, O=ActiveMQ, L=AMQ, S=AMQ, C=AMQ" -validity 365 -ext bc=ca:false -ext eku=sA -ext san=dns:ex-aao-ss-0.ex-aao-hdls-svc.test.svc.cluster.local,dns:ex-aao-ss-1.ex-aao-hdls-svc.test.svc.cluster.local
keytool -storetype jks -keystore server-keystore.jks -storepass artemis -alias server -exportcert -rfc > server.crt
keytool -storetype jks -keystore server-truststore.jks -storepass artemis -keypass artemis -importcert -alias server -file server.crt -noprompt
create secret generic artemis-ssl-secret --namespace test --from-file=broker.ks=server-keystore.jks --from-file=client.ts=server-truststore.jks --from-literal=keyStorePassword=artemis --from-literal=trustStorePassword=artemis
spec: .. deploymentPlan: size: 2 acceptors: - name: artemis port: 61616 sslEnabled: true sslSecret: artemis-ssl-secret ..
spec: .. deploymentPlan: size: 2 acceptors: - name: artemis port: 61616 sslEnabled: true sslSecret: artemis-ssl-secret brokerProperties: - 'connectorConfigurations.artemis.params.sslEnabled=true' - 'connectorConfigurations.artemis.params.trustStorePath=/etc/artemis-ssl-secret-volume/client.ts' - 'connectorConfigurations.artemis.params.trustStorePassword=artemis' ..
spec: .. brokerProperties: .. - 'connectorConfigurations.artemis.params.verifyHost=false' ..
oc edit -f <path/to/custom_resource_instance> .yaml
spec: acceptors: - name: my-acceptor protocols: amqp port: 5672 connectionsAllowed: 5 expose: true sslEnabled: true
spec: acceptors: - name: my-acceptor protocols: amqp port: 5672 connectionsAllowed: 5 expose: true sslEnabled: true amqpMinLargeMessageSize: 204800
edit ActiveMQArtemis < CR instance name > -n < namespace >
spec: deploymentPlan: startupProbe: exec: command: - /bin/bash - '-c' - /opt/amq/bin/artemis - 'check' - 'node' - '--up' - '--url' - 'tcp://USDHOSTNAME:61616' initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds: 3 failureThreshold: 30
edit ActiveMQArtemis < CR instance name > -n < namespace >
spec: deploymentPlan: livenessProbe: initialDelaySeconds: 5 periodSeconds: 5 failureThreshold: 30
spec: deploymentPlan: livenessProbe: tcpSocket: port: 8181 initialDelaySeconds: 30 timeoutSeconds: 5
spec: deploymentPlan: readinessProbe: initialDelaySeconds: 5 periodSeconds: 5
spec: deploymentPlan: readinessProbe: exec: command: - bash - '-c' - /home/jboss/amq-broker/bin/artemis - check - node - '--silent' - '--acceptor' - < acceptor name > - '--user' - USDAMQ_USER - '--password' - USDAMQ_PASSWORD initialDelaySeconds: 30 timeoutSeconds: 5
spec: deploymentPlan: readinessProbe: exec: command: - bash - '-c' - /home/jboss/amq-broker/bin/artemis - check - queue - '--name' - livenessqueue - '--produce' - "1" - '--consume' - "1" - '--silent' - '--user' - USDAMQ_USER - '--password' - USDAMQ_PASSWORD initialDelaySeconds: 30 timeoutSeconds: 5
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemisAddress metadata: name: livenessqueue namespace: activemq-artemis-operator spec: addressName: livenessqueue queueConfiguration: purgeOnNoConsumers: false maxConsumers: -1 durable: true enabled: true queueName: livenessqueue routingType: anycast
edit ActiveMQArtemis < CR instance name > -n < namespace >
spec: deploymentPlan: messageMigration: true persistenceEnabled: true
oc get pods
activemq-artemis-operator-8566d9bf58-9g25l 1/1 Running 0 3m38s ex-aao-ss-0 1/1 Running 0 112s ex-aao-ss-1 1/1 Running 0 8s
/opt/amq/bin/artemis producer --url tcp://172.17.0.6:61616 --user admin --password admin
/opt/amq/bin/artemis producer --url tcp://172.17.0.7:61616 --user admin --password admin
oc apply -f deploy/crs/broker_activemqartemis_cr.yaml
login -u <user> -p <password> --server= <host:port>
spec: deploymentPlan: nodeSelector: app: broker1
oc project <project_name>
oc create -f <path/to/custom_resource_instance> .yaml
login -u <user> -p <password> --server= <host:port>
spec: deploymentPlan: tolerations: - key: "app" value: "amq-broker" effect: "NoSchedule"
oc project <project_name>
oc create -f <path/to/custom_resource_instance> .yaml
login -u <user> -p <password> --server= <host:port>
spec: deploymentPlan: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: zone operator: In values: - emea
oc project <project_name>
oc create -f <path/to/custom_resource_instance> .yaml
spec: deploymentPlan: labels: name: broker1
spec: deploymentPlan: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: name operator: In values: - broker1 topologyKey: topology.kubernetes.io/zone
oc get pods -o wide NAME STATUS IP amq-broker-operator-54d996c Running 10.129.2.14 ex-aao-ss-0 Running 10.129.2.15
oc cp < pod name >:/home/jboss/amq-broker/etc/log4j2.properties logging.properties -c < name >-container
oc cp ex-aao-ss-0:/home/jboss/amq-broker/etc/log4j2.properties logging.properties -c ex-aao-container
login -u <user> -p <password> --server= <host:port>
create secret generic newlog4j-logging-config --from-file=logging.properties
create configmap newlog4j-logging-config --from-file=logging.properties
login -u <user> -p <password> --server= <host:port>
edit ActiveMQArtemis < CR instance name > -n < namespace >
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: extraMounts: secrets: - "newlog4j-logging-config"
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: ex-aao spec: deploymentPlan: extraMounts: configMaps: - "newlog4j-logging-config"
edit ActiveMQArtemis < CR instance name > -n < namespace >
spec: podDisruptionBudget: minAvailable: 1
spec: .. env: - name: JAVA_ARGS_APPEND value: "-Dhawtio.role=* -Djavax.management.builder.initial=org.apache.activemq.artemis.core.server.management.ArtemisRbacMBeanServerBuilder" ..
spec: .. brokerProperties: - securityRoles."mops.address.activemq.management.*".manager.view=true - securityRoles."mops.address.activemq.management.*".manager.edit=true
spec: .. brokerProperties: - securityRoles."mops.#".amq.view=true - securityRoles."mops.#".amq.edit=true ..
spec: .. resourceTemplates: - selector: kind: "StatefulSet" patch: kind: "StatefulSet" spec: template: spec: initContainers: - name: "< BROKER_NAME >-container-init" args: - '-c' - '/opt/amq/bin/launch.sh && /opt/amq-broker/script/default.sh; echo "Empty management.xml";echo "<management-context xmlns=\"http://activemq.apache.org/schema\" />" > /amq/init/config/amq-broker/etc/management.xml'
spec: .. resourceTemplates: - selector: kind: "Service" annotations: name: "amq-operator-managed" ..
spec: .. resourceTemplates: - selector: kind: "StatefulSet" patch: kind: "StatefulSet" spec: template: spec: minReadySeconds: 10 ..
spec: brokerProperties: - brokerPlugins.\"org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin.class\".init=LOG_CONNECTION_EVENTS=true,LOG_SESSION_EVENTS=true,LOG_CONSUMER_EVENTS=true
securityRoles.address1.group2.send=true securityRoles.address2.group1.consume=true securityRoles.address2.group2.createAddress=true
create secret generic address-settings-bp --from-file=securityRoles.properties
deploymentPlan: extraMounts: secrets: - "address-settings-bp"
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-a spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-b spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false
spec: .. livenessProbe: exec: command: - test - -f - /home/jboss/amq-broker/lock/cli.lock ..
spec: .. brokerProperties: - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=< class name > - storeConfiguration.jdbcConnectionUrl=jdbc:< Database URL > - HAPolicyConfiguration=SHARED_STORE_PRIMARY - storeConfiguration.jdbcLockRenewPeriodMillis=2000 - storeConfiguration.jdbcLockExpirationMillis=6000
spec: .. env: - name: ARTEMIS_EXTRA_LIBS value: '/amq/init/config/extra-libs' resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: jdbc-driver-init image: < custom container image with JAR > volumeMounts: - name: amq-cfg-dir mountPath: /amq/init/config command: - "bash" - "-c" - "mkdir -p /amq/init/config/extra-libs && cp <__JAR file_> /amq/init/config/extra-libs"
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-a spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false livenessProbe: exec: command: - test - -f - /home/jboss/amq-broker/lock/cli.lock env: - name: ARTEMIS_EXTRA_LIBS value: '/amq/init/config/extra-libs' brokerProperties: - criticalAnalyser=false - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=oracle.jdbc.OracleDriver - storeConfiguration.jdbcConnectionUrl=jdbc:< Database URL > - storeConfiguration.jdbcLockRenewPeriodMillis=2000 - storeConfiguration.jdbcLockExpirationMillis=6000 - HAPolicyConfiguration=SHARED_STORE_PRIMARY acceptors: - name: ext-acceptor protocols: CORE port: 61626 expose: true sslEnabled: true sslSecret: ext-acceptor-ssl-secret console: expose: true resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: oracle-database-jdbc-driver-init image: < custom container image with JAR > volumeMounts: - name: amq-cfg-dir mountPath: /amq/init/config command: - "bash" - "-c" - "mkdir -p /amq/init/config/extra-libs && cp < JAR file > /amq/init/config/extra-libs"
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: peer-broker-b spec: deploymentPlan: size: 1 clustered: false persistenceEnabled: false livenessProbe: exec: command: - test - -f - /home/jboss/amq-broker/lock/cli.lock env: - name: ARTEMIS_EXTRA_LIBS value: '/amq/init/config/extra-libs' brokerProperties: - criticalAnalyser=false - storeConfiguration=DATABASE - storeConfiguration.jdbcDriverClassName=oracle.jdbc.OracleDriver - storeConfiguration.jdbcConnectionUrl=jdbc:< Database URL > - storeConfiguration.jdbcLockRenewPeriodMillis=2000 - storeConfiguration.jdbcLockExpirationMillis=6000 - HAPolicyConfiguration=SHARED_STORE_PRIMARY acceptors: - name: ext-acceptor protocols: CORE port: 61626 expose: true sslEnabled: true sslSecret: ext-acceptor-ssl-secret console: expose: true resourceTemplates: - selector: kind: StatefulSet patch: kind: StatefulSet spec: template: spec: initContainers: - name: oracle-database-jdbc-driver-init image: < custom container image with JAR > volumeMounts: - name: amq-cfg-dir mountPath: /amq/init/config command: - "bash" - "-c" - "mkdir -p /amq/init/config/extra-libs && cp < JAR file > /amq/init/config/extra-libs"
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: production-broker namespace: production spec: deploymentPlan: size: 1
apiVersion: broker.amq.io/v1beta1 kind: ActiveMQArtemis metadata: name: mirror-broker namespace: dr spec: deploymentPlan: size: 1
metadata: name: mirror-broker namespace: dr spec: acceptors: - expose: true name: amqp port: 5672 protocols: amqp
metadata: name: mirror-broker namespace: dr spec: acceptors: - expose: true name: amqp port: 5672 protocols: amqp sslEnabled: true sslSecret: mirror
spec: deploymentPlan: extraMounts: secrets: - mirror
spec: brokerProperties: - AMQPConnections.datacenter1.uri=tcp://broker-dr-amqp-USD{STATEFUL_SET_ORDINAL}-svc-rte-dr.apps.lab.redhat.com:443?;sslEnabled=true;trustStorePath=/amq/extra/secrets/mirror/client.ts;trustStorePassword=password;keyStorePath=/amq/extra/secrets/mirror/broker.ks;keyStorePassword=password - AMQPConnections.datacenter1.connectionElements.mirror.type=MIRROR
spec: brokerProperties: - AMQPConnections.datacenter1.uri=tcp://primary-mirror-broker-amqp-USD{STATEFUL_SET_ORDINAL}-svc.dr.svc.cluster.local:61616 - AMQPConnections.datacenter1.connectionElements.mirror.type=MIRROR - AMQPConnections.datacenter2.uri=tcp://backup-mirror-broker-amqp-USD{STATEFUL_SET_ORDINAL}-svc.dr.svc.cluster.local:61616 - AMQPConnections.datacenter2.connectionElements.mirror.type=MIRROR
- AMQPConnections.datacenter1.user=admin - AMQPConnections.datacenter1.password=admin - AMQPConnections.datacenter1.retryInterval=5000 - AMQPConnections.datacenter1.connectionElements.mirror.messageAcknowledgements=true - AMQPConnections.datacenter1.connectionElements.mirror.queueCreation=true - AMQPConnections.datacenter1.connectionElements.mirror.queueRemoval=true - AMQPConnections.datacenter1.connectionElements.mirror.addressFilter= addresses