Chapter 8. Message Persistence and Paging AMQ Broker 7 provides persistence through either a message journal or a JDBC store. The method by which the broker stores messages and pages them to disk is different than AMQ 6, and the configuration properties you use to configure message persistence are changed. 8.1. Message Persistence Changes AMQ Broker 7 uses a different type of message journal than AMQ 6, and it does not use a journal index. AMQ 6 used KahaDB for a message store, and it maintained a message journal index to track the position of each message inside the journal. This index enabled the broker to pull paged messages from its journal in batches and place them in its cache. By default, AMQ Broker 7 uses an in-memory message journal from which the broker can dispatch messages. Therefore, AMQ Broker 7 does not use a message journal index. If a broker instance runs out of memory, messages are paged as they arrive at the broker, but before they are queued. These message page files are stored on disk sequentially in the same order in which they arrived. Then, when memory is freed on the broker, the messages are moved from the page file to the journal on the broker. Because the journal is read sequentially, there is no need to keep an index of messages in the journal. In addition, AMQ Broker 7 also offers a different JDBC-based message journal option that was not available in AMQ 6. The AMQ Broker 7 message journal supports the following shared file systems: NFSv4 GFS2 Related Information For more information about the default in-memory message journal, see About Journal-Based Persistence in Configuring AMQ Broker . For more information about the new JDBC-based persistence option, see Configuring JDBC Persistence in Configuring AMQ Broker . 8.2. How Message Persistence is Configured You use the BROKER_INSTANCE_DIR /etc/broker.xml configuration file to configure the broker instance's message journal. The broker.xml configuration file contains the following default message journal configuration properties: <core> <name>0.0.0.0</name> <persistence-enabled>true</persistence-enabled> <journal-type>ASYNCIO</journal-type> <paging-directory>./data/paging</paging-directory> <bindings-directory>./data/bindings</bindings-directory> <journal-directory>./data/journal</journal-directory> <large-messages-directory>./data/large-messages</large-messages-directory> <journal-datasync>true</journal-datasync> <journal-min-files>2</journal-min-files> <journal-pool-files>-1</journal-pool-files> <journal-buffer-timeout>744000</journal-buffer-timeout> <disk-scan-period>5000</disk-scan-period> <max-disk-usage>90</max-disk-usage> <global-max-size>104857600</global-max-size> ... </core> To configure the message journal, you can change the default values for any of the journal configuration properties. You can also add additional configuration properties. 8.3. Message Persistence Configuration Property Changes AMQ 6 and AMQ Broker 7 both offer a number of configuration properties to control how the broker persists messages. This section compares the configuration properties in the AMQ 6 KahaDB journal to the equivalent properties in the AMQ Broker 7 in-memory message journal. For complete details on each message persistence configuration property for the in-memory message journal, see the following: The Bindings Journal in Configuring AMQ Broker Messaging Journal Configuration Elements in Configuring AMQ Broker 8.3.1. Journal Size and Management The following table compares the journal size and management configuration properties in AMQ 6 to the equivalent properties in AMQ Broker 7: To set... In AMQ 6 In AMQ Broker 7 The time interval between cleaning up data logs that are no longer used cleanupInterval The default is 30000 ms. No equivalent. In AMQ Broker 7, journal files that exceed the pool size are no longer used. The number of message store GC cycles that must be completed without cleaning up other files before compaction is triggered compactAcksAfterNoGC No equivalent. In AMQ Broker 7, compaction is not related to particular record types. Whether compaction should be run when the message store is still growing, or if it should only occur when it has stopped growing compactAcksIgnoresStoreGrowth The default is false . No equivalent. The minimum number of journal files that can be stored on the broker before it will compact them No equivalent. <journal-compact-min-files> The default is 10. If you set this value to 0, compaction will be deactivated. The threshold to reach before compaction starts No equivalent. <journal-compact-percentage> The default is 30%. When less than this percentage is considered to be live data, compaction will start. The path to the top-level folder that holds the message store's data files directory AMQ Broker 7 has a separate directory for each type of journal: <journal-directory> - The default is /data/journal . <bindings-directory> - The default is /data/bindings . <paging-directory> - The default is /data/paging . <large-message-directory> - The default is /data/large-messages . Whether the bindings directory should be created automatically if it does not already exist No equivalent. <create-bindings-dir> The default is true . Whether the journal directory should be created automatically if it does not already exist No equivalent. <create-journal-dir> The default is true . Whether the message store should periodically compact older journal log files that contain only message acknowledgements enableAckCompaction No equivalent. The maximum size of the data log files journalMaxFileLength The default is 32 MB. <journal-file-size> The default is 10485760 bytes (10 MiB). The policy that the broker should use to preallocate the journal files when a new journal file is needed preallocationStrategy The default is sparse_file . No equivalent. By default, preallocated journal files are typically filled with zeroes, but it can vary depending on the file system. The policy the broker should use to preallocate the journal files preallocationScope The default is entire_journal . AMQ Broker 7 automatically preallocates the journal files specified by <journal-min-files> when the broker instance is started. The journal type (either NIO or AIO) No equivalent. <journal-type> You can choose either NIO (Java NIO journal), or ASYNCIO (Linux asynchronous I/O journal). The minimum number of files that the journal should maintain No equivalent. <journal-min-files> The number of journal files the broker should keep when reclaiming files No equivalent. <journal-pool-files> The default is -1, which means the broker instance will never delete files on the journal once created. 8.3.2. Write Boundaries The following table compares the write boundary configuration properties in AMQ 6 to the equivalent properties in AMQ Broker 7: To set... In AMQ 6 In AMQ Broker 7 The time interval between writing the metadata cache to disk checkpointInterval The default is 5000 ms. No equivalent. Whether the message store should dispatch queue messages to clients concurrently with message storage concurrentStoreAndDispatchQueues The default is true . No equivalent. Whether the message store should dispatch topic messages to interested clients concurrently with message storage concurrentStoreAndDispatchTopics The default is false . No equivalent. Whether a disk sync should be performed after each non-transactional journal write enableJournalDiskSyncs The default is true . <journal-sync-transactional> Flushes transaction data to disk whenever a transaction boundary is reached (commit, prepare, and rollback). The default is true . <journal-sync-nontransactional> Flushes non-transactional message data to disk (sends and acknowledgements). The default is true . When to flush the entire journal buffer No equivalent. <journal-buffer-timeout> The default for NIO is 3,333,333 nanoseconds, and the default for AIO is 500,000 nanoseconds. The amount of data to buffer between journal disk writes journalMaxWriteBatchSize The default is 4000 bytes. No equivalent. The size of the task queue used to buffer the journal's write requests maxAsyncJobs The default is 10000. <journal-max-io> This property controls the maximum number of write requests that can be in the I/O queue at any given point. The default for NIO is 1, and the default for AIO is 500. Whether to use fdatasync on journal writes No equivalent. <journal-datasync> The default is true . 8.3.3. Index Configuration AMQ 6 has a number of configuration properties for configuring the journal index. Because AMQ Broker 7 does not use journal indexes, you do not need to configure any of these properties for your broker instance. 8.3.4. Journal Archival AMQ 6 has several configuration properties for controlling which files are archived and where the archives are stored. In AMQ Broker 7, however, when old journal files are no longer needed, the broker reuses them instead of archiving them. Therefore, you do not need to configure any journal archival properties for your broker instance. 8.3.5. Journal Recovery AMQ 6 has several configuration properties for controlling how the broker checks for corrupted journal files and what to do when it encounters a missing journal file. In AMQ Broker 7, however, you do not need to configure any journal recovery properties for your broker instance. Journal files have a different format in AMQ Broker 7, which should prevent a corrupted entry in the journal from corrupting the entire journal file. Even if the journal is partially damaged, the broker should still be able to extract data from the undamaged entries.

Commands:
<core> <name>0.0.0.0</name> <persistence-enabled>true</persistence-enabled> <journal-type>ASYNCIO</journal-type> <paging-directory>./data/paging</paging-directory> <bindings-directory>./data/bindings</bindings-directory> <journal-directory>./data/journal</journal-directory> <large-messages-directory>./data/large-messages</large-messages-directory> <journal-datasync>true</journal-datasync> <journal-min-files>2</journal-min-files> <journal-pool-files>-1</journal-pool-files> <journal-buffer-timeout>744000</journal-buffer-timeout> <disk-scan-period>5000</disk-scan-period> <max-disk-usage>90</max-disk-usage> <global-max-size>104857600</global-max-size> </core>