Project: Apache Storm (App Definition and Development / Streaming & Messaging)
File: storm-metricstore.md

title: Storm Metricstore layout: documentation documentation: true A metric store () interface was added to Nimbus to allow storing metric information () to a database. The default implementation () is using RocksDB, a key-value store. As metrics are stored in RocksDB, their string values (for topology ID and executor ID, etc.) are converted to unique integer IDs, and these strings are also stored to the database as metadata indexed by the integer ID. When a metric is stored, it is also aggregated with any existing metric within the same 1, 10, and 60 minute timeframe. The class provides an interface to select which options can be used to scan the metrics. The following configuation options exist: ```yaml storm.metricstore.class: "org.apache.storm.metricstore.rocksdb.RocksDbStore" storm.metricprocessor.class: "org.apache.storm.metricstore.NimbusMetricProcessor" storm.metricstore.rocksdb.location: "storm_rocks" storm.metricstore.rocksdb.createifmissing: true storm.metricstore.rocksdb.metadatastringcache_capacity: 4000 storm.metricstore.rocksdb.retention_hours: 240 ``` storm.metricstore.class is the class that implements the (). storm.metricprocessor.class is the class that implements the (). storm.metricstore.rocksdb.location provides to location of the RocksDB database on Nimbus storm.metricstore.rocksdb.createifmissing permits creating a RocksDB database if missing storm.metricstore.rocksdb.metadatastringcache_capacity controls the number of metadata strings cached in memory. storm.metricstore.rocksdb.retention_hours sets the length of time metrics will remain active. The RocksDB Key (represented by ) fields are as follows: | Field | Size | Offset | Description | |-||--|--| | Type | 1 | 0 | The type maps to the KeyType enum, specifying a metric or various types of metadata strings | | Aggregation Level | 1 | 1 | The aggregation level for a metric (see AggLevel enum). Set to 0 for metadata. | | Topology Id | 4 | 2 | The metadata string Id representing a topologyId for a metric, or the unique string Id for a metadata string | | Timestamp | 8 | 6 | The timestamp for a metric, unused for metadata | | Metric Id | 4 | 14 | The metadata string Id for the metric name | | Component Id | 4 | 18 | The metadata string Id for the component Id | | Executor Id | 4 | 22 | The metadata string Id for the executor Id | | Host Id | 4 | 26 | The metadata string Id for the host Id | | Port | 4 | 30 | The port number | | Stream Id | 4 | 34 | The metadata string Id for the stream Id | The RocksDB Value fields for metadata strings (represented by ) are as follows: | Field | Size | Offset | Description | |--||--|-| | Version | 1 | 0 | The current metadata version - allows migrating if the format changes in the future | | Timestamp | 8 | 1 | The time when the metadata was last used by a metric. Allows deleting of old metadata. | | Metadata String | any | 9 | The metadata string | RocksDB Value fields for metric data are as follows: | Field | Size | Offset | Description | |||--|--| | Version | 1 | 0 | The current metric version - allows migrating if the format changes in the future | | Value | 8 | 1 | The metric value | | Count | 8 | 9 | The metric count | | Min | 8 | 17 | The minimum metric value | | Max | 8 | 25 | The maximum metric value | | Sum | 8 | 33 | The sum of the metric values |