Chapter 6. Persisting message data AMQ Broker has two options for persisting (that is, storing ) message data: Persisting messages in journals This is the default option. Journal-based persistence is a high-performance option that writes messages to journals on the file system. Persisting messages in a database This option uses a Java Database Connectivity (JDBC) connection to persist messages to a database of your choice. Alternatively, you can also configure the broker not to persist any message data. For more information, see Section 6.3, "Disabling persistence" . The broker uses a different solution for persisting large messages outside the message journal. See Chapter 8, Handling large messages for more information. The broker can also be configured to page messages to disk in low-memory situations. See Section 7.1, "Configuring message paging" for more information. Note For current information regarding which databases and network file systems are supported by AMQ Broker see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal. 6.1. Persisting message data in journals A broker journal is a set of append-only files on disk. Each file is pre-created to a fixed size and initially filled with padding. As messaging operations are performed on the broker, records are appended to end of the journal. Appending records allows the broker to minimize disk head movement and random access operations, which are typically the slowest operation on a disk. When one journal file is full, the broker creates a new one. The journal file size is configurable, minimizing the number of disk cylinders used by each file. Modern disk topologies are complex, however, and the broker cannot control which cylinder(s) the file is mapped to. Therefore, journal file sizing is difficult to control precisely. Other persistence-related features that the broker uses are: A garbage collection algorithm that determines whether a particular journal file is still in use. If the journal file is no longer in use, the broker can reclaim the file for reuse. A compaction algorithm that removes dead space from the journal and compresses the data. This results in the journal using fewer files on disk. Support for local transactions. Support for Extended Architecture (XA) transactions when using JMS clients. Most of the journal is written in Java. However, interaction with the actual file system is abstracted, so that you can use different, pluggable implementations. AMQ Broker includes the following implementations: NIO NIO (New I/O) uses standard Java NIO to interface with the file system. This provides extremely good performance and runs on any platform with a Java 6 or later runtime. For more information about Java NIO, see Java NIO . AIO AIO (Aynshcronous I/O) uses a thin native wrapper to talk to the Linux Asynchronous I/O Library ( libaio ). With AIO, the broker is called back after the data has made it to disk, avoiding explicit syncs altogether. By default, the broker tries to use an AIO journal, and falls back to using NIO if AIO is not available. AIO typically provides even better performance than Java NIO. To learn how to install libaio , see Section 6.1.1, "Installing the Linux Asynchronous I/O Library" . The procedures in the sub-sections that follow show how to configure the broker for journal-based persistence. 6.1.1. Installing the Linux Asynchronous I/O Library Red Hat recommends using the AIO journal (instead of NIO) for better persistence performance. Note It is not possible to use the AIO journal with other operating systems or earlier versions of the Linux kernel. To use the AIO journal, you must install the Linux Asynchronous I/O Library ( libaio ). To install libaio , use the yum command, as shown below: 6.1.2. Configuring journal-based persistence The following procedure describes how to review the default configuration that the broker uses for journal-based persistence. You can use this description to adjust your configuration as needed. Open the <broker_instance_dir> /etc/broker.xml configuration file. By default, the broker is configured to use journal-based persistence, as shown below. <configuration> <core> ... <persistence-enabled>true</persistence-enabled> <journal-type>ASYNCIO</journal-type> <bindings-directory>./data/bindings</bindings-directory> <journal-directory>./data/journal</journal-directory> <journal-datasync>true</journal-datasync> <journal-min-files>2</journal-min-files> <journal-pool-files>-1</journal-pool-files> <journal-device-block-size>4096</journal-device-block-size> <journal-file-size>10M</journal-file-size> <journal-buffer-timeout>12000</journal-buffer-timeout> <journal-max-io>4096</journal-max-io> ... </core> </configuration> persistence-enabled If the value of this parameter is set to true , the broker uses the file-based journal for message persistence. journal-type Type of journal to use. If set to ASYNCIO , the broker first attempts to use AIO. If AIO is not found, the broker uses NIO. bindings-directory File system location of the bindings journal. The default value is relative to the <broker_instance_dir> directory. journal-directory File system location of the message journal. The default value is relative to the <broker_instance_dir> directory. journal-datasync If the value of this parameter is set to true , the broker uses the fdatasync function to confirm disk writes. journal-min-files Number of journal files to initially create when the broker starts. journal-pool-files Number of files to keep after reclaiming unused files. The default value of -1 means that no files are deleted during cleanup. journal-device-block-size Maximum size, in bytes, of the data blocks used by the journal on your storage device. The default value is 4096 bytes. journal-file-size Maximum size, in bytes, of each journal file in the specified journal directory. When this limit is reached, the broker starts a new file. This parameter also supports byte notation (for example, K, M, G), or the binary equivalents (Ki, Mi, Gi). If this parameter is not explicitly specified in your configuration, the default value is 10485760 bytes (10MiB). journal-buffer-timeout Specifies how often, in nanoseconds, the broker flushes the journal buffer. AIO typically uses a higher flush rate than NIO, so the broker maintains different default values for both NIO and AIO. If this parameter not explicitly specified in your configuration, the default value for NIO is 3333333 nanoseconds (that is, 300 times per second). The default value for AIO is 50000 nanoseconds (that is, 2000 times per second). journal-max-io Maximum number of write requests that can be in the IO queue at any one time. If the queue becomes full, the broker blocks further writes until space is available. If you are using NIO, this value should always be 1 . If you are using AIO andthis parameter not explicitly specified in your configuration, the default value is 500 . Based on the preceding descriptions, adjust your persistence configuration as needed for your storage device. Additional resources To learn about all of the parameters that are available for configuring journal-based persistence, see Appendix E, Messaging Journal Configuration Elements . 6.1.3. About the bindings journal The bindings journal is used to store bindings-related data, such as the set of queues deployed on the broker and their attributes. It also stores data such as ID sequence counters. The bindings journal always uses NIO because it is typically low throughput when compared to the message journal. Files on this journal are prefixed with activemq-bindings . Each file also has an extension of .bindings and a default size of 1048576 bytes. To configure the bindings journal, include the following parameters in the core element of the <broker_instance_dir> /etc/broker.xml configuration file. bindings-directory Directory for the bindings journal. The default value is <broker_instance_dir> /data/bindings . create-bindings-dir If the value of this parameter is set to true , the broker automatically creates the bindings directory in the location specified in bindings-directory , if it does not already exist. The default value is true . 6.1.4. About the JMS journal The JMS journal stores all JMS-related data, including JMS queues, topics, and connection factories, as well as any JNDI bindings for these resources. Any JMS resources created via the management API are persisted to this journal, but any resources configured via configuration files are not. The broker creates the JMS journal only if JMS is being used. Files in the JMS journal are prefixed with activemq-jms . Each file also has an extension of .jms and a default size of 1048576 bytes. The JMS journal shares its configuration with the bindings journal. Additional resources For more information about the bindings journal, see Section 6.1.3, "About the bindings journal" . 6.1.5. Compacting journal files AMQ Broker includes a compaction algorithm that removes dead space from the journal and compresses the data so that it takes up less disk space. The following sub-sections show how to: Configure the broker to automatically compact journal files when certain criteria are met Run the compaction process manually from the command-line interface 6.1.5.1. Configuring journal file compaction The broker uses the following criteria to determine when to start compaction: The number of files created for the journal. The percentage of live data in the journal files. After the configured values for both of these criteria are reached, the compaction process parses the journal and removes all dead records. Consequently, the journal comprises fewer files. The following procedure shows how to configure the broker for journal file compaction. Procedure Open the <broker_instance_dir> /etc/broker.xml configuration file. Within the core element, add the journal-compact-min-files and journal-compact-percentage parameters and specify values. For example: <configuration> <core> ... <journal-compact-min-files>15</journal-compact-min-files> <journal-compact-percentage>25</journal-compact-percentage> ... </core> </configuration> journal-compact-min-files The minimum number of journal files that the broker has to create before compaction begins. The default value is 10 . Setting the value to 0 disables compaction. You should take care when disabling compaction, because the size of the journal can grow indefinitely. journal-compact-percentage The percentage of live data in the journal files. When less than this percentage is considered live data (and the configured value of journal-compact-min-files has also been reached), compaction begins. The default value is 30 . 6.1.5.2. Running compaction from the command-line interface The following procedure shows how to use the command-line interface (CLI) to compact journal files. Procedure As the owner of the <broker_instance_dir> directory, stop the broker. The example below shows the user amq-broker . (Optional) Run the following CLI command to get a full list of parameters for the data tool. By default, the tool uses settings found in <broker_instance_dir> /etc/broker.xml . Run the following CLI command to compact the data. After the tool has successfully compacted the data, restart the broker. Additional resources AMQ Broker includes a number of CLI commands for managing your journal files. See command-line Tools in the Appendix for more information. 6.1.6. Disabling the disk write cache Most disks contain hardware write caches. A write cache can increase the apparent performance of the disk because writes are lazily written to the disk later. By default, many systems ship with disk write cache enabled. This means that even after syncing from the operating system, there is no guarantee that the data has actually made it to disk. Therefore, if a failure occurs, critical data can be lost. Some more expensive disks have non-volatile or battery-backed write caches that do not necessarily lose data in event of failure, but you should test them. If your disk does not have such features, you should ensure that write cache is disabled. Be aware that disabling disk write cache can negatively affect performance. The following procedure shows how to disable the disk write cache on Linux on Windows. Procedure On Linux, to manage the disk write cache settings, use the tools hdparm (for IDE disks) or sdparm or sginfo (for SDSI/SATA disks). On Windows, to manage the disk writer cache settings, right-click the disk. Select Properties . 6.2. Persisting message data in a database When you persist message data in a database, the broker uses a Java Database Connectivity (JDBC) connection to store message and bindings data in database tables. The data in the tables is encoded using AMQ Broker journal encoding. For information about supported databases, see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal. Important An administrator might choose to store message data in a database based on the requirements of an organization's wider IT infrastructure. However, use of a database can negatively effect the performance of a messaging system. Specifically, writing messaging data to database tables via JDBC creates a significant performance overhead for a broker. 6.2.1. Configuring JDBC persistence The following procedure shows how to configure the broker to store messages and bindings data in database tables. Procedure Add the appropriate JDBC client libraries to the broker runtime. To do this, add the relevant .jar files to the <broker_instance_dir> /lib directory. Open the <broker_instance_dir> /etc/broker.xml configuration file. Within the core element, add a store element that contains a database-store element. <configuration> <core> <store> <database-store> </database-store> </store> </core> </configuration> Within the database-store element, add configuration parameters for JDBC persistence and specify values. For example: <configuration> <core> <store> <database-store> <jdbc-connection-url>jdbc:oracle:data/oracle/database-store;create=true</jdbc-connection-url> <jdbc-user>ENC(5493dd76567ee5ec269d11823973462f)</jdbc-user> <jdbc-password>ENC(56a0db3b71043054269d11823973462f)</jdbc-password> <bindings-table-name>BINDINGS_TABLE</bindings-table-name> <message-table-name>MESSAGE_TABLE</message-table-name> <large-message-table-name>LARGE_MESSAGES_TABLE</large-message-table-name> <page-store-table-name>PAGE_STORE_TABLE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_TABLE</node-manager-store-table-name> <jdbc-driver-class-name>oracle.jdbc.driver.OracleDriver</jdbc-driver-class-name> <jdbc-network-timeout>10000</jdbc-network-timeout> <jdbc-lock-renew-period>2000</jdbc-lock-renew-period> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-journal-sync-period>5</jdbc-journal-sync-period> </database-store> </store> </core> </configuration> jdbc-connection-url Full JDBC connection URL for your database server. The connection URL should include all configuration parameters and the database name. jdbc-user Encrypted user name for your database server. For more information about encrypting user names and passwords for use in configuration files, see Section 5.9, "Encrypting passwords in configuration files" . jdbc-password Encrypted password for your database server. For more information about encrypting user names and passwords for use in configuration files, see Section 5.9, "Encrypting passwords in configuration files" . bindings-table-name Name of the table in which bindings data is stored. Specifying a table name enables you to share a single database between multiple servers, without interference. message-table-name Name of the table in which message data is stored. Specifying this table name enables you to share a single database between multiple servers, without interference. large-message-table-name Name of the table in which large messages and related data are persisted. In addition, if a client streams a large message in chunks, the chunks are stored in this table. Specifying this table name enables you to share a single database between multiple servers, without interference. page-store-table-name Name of the table in which paged store directory information is stored. Specifying this table name enables you to share a single database between multiple servers, without interference. node-manager-store-table-name Name of the table in which the shared store high-availability (HA) locks for live and backup brokers and other HA-related data is stored on the broker server. Specifying this table name enables you to share a single database between multiple servers, without interference. Each live-backup pair that uses shared store HA must use the same table name. You cannot share the same table between multiple (and unrelated) live-backup pairs. jdbc-driver-class-name Fully-qualified class name of the JDBC database driver. For information about supported databases, see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal. jdbc-network-timeout JDBC network connection timeout, in milliseconds. The default value is 20000 milliseconds. When using a JDBC for shared store HA, it is recommended to set the timeout to a value less than or equal to jdbc-lock-expiration . jdbc-lock-renew-period Length, in milliseconds, of the renewal period for the current JDBC lock. When this time elapses, the broker can renew the lock. It is recommended to set a value that is several times smaller than the value of jdbc-lock-expiration . This gives the broker sufficient time to extend the lease and also gives the broker time to try to renew the lock in the event of a connection problem. The default value is 2000 milliseconds. jdbc-lock-expiration Time, in milliseconds, that the current JDBC lock is considered owned (that is, acquired or renewed), even if the value of jdbc-lock-renew-period has elapsed. The broker periodically tries to renew a lock that it owns according to the value of jdbc-lock-renew-period . If the broker fails to renew the lock (for example, due to a connection problem) the broker keeps trying to renew the lock until the value of jdbc-lock-expiration has passed since the lock was last successfully acquired or renewed. An exception to the renewal behavior described above is when another broker acquires the lock. This can happen if there is a time misalignment between the Database Management System (DBMS) and the brokers, or if there is a long pause for garbage collection. In this case, the broker that originally owned the lock considers the lock lost and does not try to renew it. After the expiration time elapses, if the JDBC lock has not been renewed by the broker that currently owns it, another broker can establish a JDBC lock. The default value of jdbc-lock-expiration is 20000 milliseconds. jdbc-journal-sync-period Duration, in milliseconds, for which the broker journal synchronizes with JDBC. The default value is 5 milliseconds. 6.2.2. Configuring JDBC connection pooling If you have configured the broker for JDBC persistence, the broker uses a JDBC connection to store messages and bindings data in database tables. In the event of a JDBC connection failure, and provided that there is no active connection activity (such as a database read or write) when the failure occurs, the broker stays running and tries to re-establish the database connection. To achieve this, AMQ Broker uses JDBC connection pooling . In general, a connection pool provides a set of open connections to a specified database that can be shared between multiple applications. For a broker, if the connection between a broker and the database fails, the broker attempts to reconnect to the database using a different connection from the pool. The pool tests the new connection before the broker receives it. The following example shows how to configure JDBC connection pooling. Important If you do not explicitly configure JDBC connection pooling, the broker uses connection pooling with a default configuration. The default configuration uses values from your existing JDBC configuration. For more information, see Default connection pooling configuration . Prerequisites This example builds on the example for configuring JDBC persistence. See Section 6.2.1, "Configuring JDBC persistence" To enable connection pooling, AMQ Broker uses the Apache Commons DBCP package. Before configuring JDBC connection pooling for the broker, you should be familiar with what this package provides. For more information, see: Overview of Apache Commons DBCP Apache Commons DBCP Configuration Parameters Procedure Open the <broker-instance-dir> /etc/broker.xml configuration file. Within the database-store element that you previously added for your JDBC configuration, remove the jdbc-driver-class-name , jdbc-connection-url , jdbc-user , jdbc-password , parameters. Later in this procedure, you will replace these with corresponding DBCP configuration parameters. Note If you do not explicitly remove the preceding parameters, the corresponding DBCP parameters that you add later in this procedure take precedence. Within the database-store element, add a data-source-properties element. For example: <store> <database-store> <data-source-properties> </data-source-properties> <bindings-table-name>BINDINGS</bindings-table-name> <message-table-name>MESSAGES</message-table-name> <large-message-table-name>LARGE_MESSAGES</large-message-table-name> <page-store-table-name>PAGE_STORE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_STORE</node-manager-store-table-name> <jdbc-network-timeout>10000</jdbc-network-timeout> <jdbc-lock-renew-period>2000</jdbc-lock-renew-period> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-journal-sync-period>5</jdbc-journal-sync-period> </database-store> </store> Within the new data-source-properties element, add DBCP data source properties for connection pooling. Specify key-value pairs. For example: <store> <database-store> <data-source-properties> <data-source-property key="driverClassName" value="com.mysql.jdbc.Driver" /> <data-source-property key="url" value="jdbc:mysql://localhost:3306/artemis" /> <data-source-property key="username" value="ENC(5493dd76567ee5ec269d1182397346f)"/> <data-source-property key="password" value="ENC(56a0db3b71043054269d1182397346f)"/> <data-source-property key="poolPreparedStatements" value="true" /> <data-source-property key="maxTotal" value="-1" /> </data-source-properties> <bindings-table-name>BINDINGS</bindings-table-name> <message-table-name>MESSAGES</message-table-name> <large-message-table-name>LARGE_MESSAGES</large-message-table-name> <page-store-table-name>PAGE_STORE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_STORE</node-manager-store-table-name> <jdbc-network-timeout>10000</jdbc-network-timeout> <jdbc-lock-renew-period>2000</jdbc-lock-renew-period> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-journal-sync-period>5</jdbc-journal-sync-period> </database-store> </store> driverClassName Fully-qualified class name of the JDBC database driver. url Full JDBC connection URL for your database server. username Encrypted user name for your database server. You can also specify this value as unencrypted, plain text. For more information about encrypting user names and passwords for use in configuration files, see Section 5.9, "Encrypting passwords in configuration files" . password Encrypted password for your database server. You can also specify this value as unencrypted, plain text. For more information about encrypting user names and passwords for use in configuration files, see Section 5.9, "Encrypting passwords in configuration files" . poolPreparedStatements When the value of this parameter is set to true , the pool can have an unlimited number of cached prepared statements. This reduces initialization costs. maxTotal Maximum number of connections in the pool. When the value of this parameter is set to -1 , there is no limit. If you do not explicitly configure JDBC connection pooling, the broker uses connection pooling with a default configuration. The default configuration is described in the table. Table 6.1. Default connection pooling configuration DBCP configuration parameter Default value driverClassName The value of the existing jdbc-driver-class-name parameter url The value of the existing jdbc-connection-url parameter username The value of the existing jdbc-user parameter password The value of the existing jdbc-password parameter poolPreparedStatements true maxTotal -1 Note Reconnection works only if no client is actively sending messages to the broker. If there is an attempt to write to the database tables during reconnection, the broker fails and shuts down. Additional resources For information about databases supported by AMQ Broker, see Red Hat AMQ 7 Supported Configurations on the Red Hat Customer Portal. To learn about all of the configuration options available in the Apache Commons DBCP package, see Apache Commons DBCP Configuration Parameters . 6.3. Disabling persistence In some situations, it might be a requirement that a messaging system does not store any data. In these situations you can disable persistence on the broker. The following procedure shows how to disable persistence. Procedure Open the <broker_instance_dir> /etc/broker.xml configuration file. Within the core element, set the value of the persistence-enabled parameter to false . <configuration> <core> ... <persistence-enabled>false</persistence-enabled> ... </core> </configuration> No message data, bindings data, large message data, duplicate ID cache, or paging data is persisted.

Commands:
install libaio
<configuration> <core> <persistence-enabled>true</persistence-enabled> <journal-type>ASYNCIO</journal-type> <bindings-directory>./data/bindings</bindings-directory> <journal-directory>./data/journal</journal-directory> <journal-datasync>true</journal-datasync> <journal-min-files>2</journal-min-files> <journal-pool-files>-1</journal-pool-files> <journal-device-block-size>4096</journal-device-block-size> <journal-file-size>10M</journal-file-size> <journal-buffer-timeout>12000</journal-buffer-timeout> <journal-max-io>4096</journal-max-io> </core> </configuration>
<configuration> <core> <journal-compact-min-files>15</journal-compact-min-files> <journal-compact-percentage>25</journal-compact-percentage> </core> </configuration>
su - amq-broker cd <broker_instance_dir> /bin ./artemis stop
./artemis help data compact.
./artemis data compact.
./artemis run
<configuration> <core> <store> <database-store> </database-store> </store> </core> </configuration>
<configuration> <core> <store> <database-store> <jdbc-connection-url>jdbc:oracle:data/oracle/database-store;create=true</jdbc-connection-url> <jdbc-user>ENC(5493dd76567ee5ec269d11823973462f)</jdbc-user> <jdbc-password>ENC(56a0db3b71043054269d11823973462f)</jdbc-password> <bindings-table-name>BINDINGS_TABLE</bindings-table-name> <message-table-name>MESSAGE_TABLE</message-table-name> <large-message-table-name>LARGE_MESSAGES_TABLE</large-message-table-name> <page-store-table-name>PAGE_STORE_TABLE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_TABLE</node-manager-store-table-name> <jdbc-driver-class-name>oracle.jdbc.driver.OracleDriver</jdbc-driver-class-name> <jdbc-network-timeout>10000</jdbc-network-timeout> <jdbc-lock-renew-period>2000</jdbc-lock-renew-period> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-journal-sync-period>5</jdbc-journal-sync-period> </database-store> </store> </core> </configuration>
<store> <database-store> <data-source-properties> </data-source-properties> <bindings-table-name>BINDINGS</bindings-table-name> <message-table-name>MESSAGES</message-table-name> <large-message-table-name>LARGE_MESSAGES</large-message-table-name> <page-store-table-name>PAGE_STORE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_STORE</node-manager-store-table-name> <jdbc-network-timeout>10000</jdbc-network-timeout> <jdbc-lock-renew-period>2000</jdbc-lock-renew-period> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-journal-sync-period>5</jdbc-journal-sync-period> </database-store> </store>
<store> <database-store> <data-source-properties> <data-source-property key="driverClassName" value="com.mysql.jdbc.Driver" /> <data-source-property key="url" value="jdbc:mysql://localhost:3306/artemis" /> <data-source-property key="username" value="ENC(5493dd76567ee5ec269d1182397346f)"/> <data-source-property key="password" value="ENC(56a0db3b71043054269d1182397346f)"/> <data-source-property key="poolPreparedStatements" value="true" /> <data-source-property key="maxTotal" value="-1" /> </data-source-properties> <bindings-table-name>BINDINGS</bindings-table-name> <message-table-name>MESSAGES</message-table-name> <large-message-table-name>LARGE_MESSAGES</large-message-table-name> <page-store-table-name>PAGE_STORE</page-store-table-name> <node-manager-store-table-name>NODE_MANAGER_STORE</node-manager-store-table-name> <jdbc-network-timeout>10000</jdbc-network-timeout> <jdbc-lock-renew-period>2000</jdbc-lock-renew-period> <jdbc-lock-expiration>20000</jdbc-lock-expiration> <jdbc-journal-sync-period>5</jdbc-journal-sync-period> </database-store> </store>
<configuration> <core> <persistence-enabled>false</persistence-enabled> </core> </configuration>